Data Processing Method, Apparatus, Server, and Storage Medium

Abstract

The present application discloses a data processing method, apparatus, server, and storage medium. The solution of the present application is applied to a business service platform that runs a main process and at least one slave process. The method includes: obtaining a data processing request sent by a client; when the data processing request includes a read request, determining a first data identifier of the data requested to be read by the read request; if the first data identifier belongs to the data identifiers in the unsynchronized records cached, forwarding the read request to the main process for processing, where the unsynchronized records record the data identifiers corresponding to the data that has not been synchronized from the main process to the slave process; if the first data identifier does not belong to the data identifiers in the unsynchronized records, forwarding the read request to the slave process for processing. The solution of the present application can improve the data request processing efficiency while reducing the situation of data reading errors or abnormalities caused by data inconsistency between the main and slave processes.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a data processing method, apparatus, server, and storage medium. Background Art

[0002] In business scenarios involving a large amount of data and high requirements for data reading and writing performance, the business service platform generally caches data through processes and processes data reading and writing requests to achieve more efficient data reading and writing processing.

[0003] Currently, there are two common ways for the business service platform to process data reading and writing requests through processes: one is the single-process method, and the other is the master-slave process method. Among them, the single-process method means that a single process simultaneously processes read requests and write requests. The advantage of the single-process method is that no data synchronization is required, and there is no situation of data inconsistency. However, due to the large read and write pressure of the single process, the client request delay is relatively long. In the master-slave process method, the business service platform runs a master process and at least one slave process. Among them, the master process is responsible for processing the client's write requests and synchronizing the data to the slave processes. On this basis, the slave processes are responsible for processing the client's read requests, which can reduce the data reading and writing pressure of the master process, is conducive to improving the data reading and writing efficiency, and reducing the request processing delay. However, in the master-slave process method, it may be due to the master process not synchronizing data to the slave processes in a timely manner, resulting in data inconsistency between the master process and the slave processes, so that the data fed back by the slave processes to the client is incorrect or the data reading is abnormal. It can be seen that how to reduce the situation of data reading errors or abnormalities caused by data inconsistency while reducing the request processing delay is a technical problem that those skilled in the art urgently need to solve. Summary of the Invention

[0004] In view of this, this application provides a data processing method, apparatus, server, and storage medium to improve the data request processing efficiency, reduce the data request processing delay, and at the same time, be able to reduce the situation of data reading errors or abnormalities caused by data inconsistency between the master and slave processes.

[0005] To achieve the above object, this application provides the following technical solutions:

[0006] On the one hand, this application provides a data processing method, which is applied to a business service platform that runs a master process and at least one slave process. The method includes:

[0007] Obtain a data processing request sent by a client;

[0008] In the case where the data processing request includes a read request, determine a first data identifier of the data requested to be read by the read request;

[0009] If the first data identifier belongs to the data identifiers in the cached unsynchronized records, forward the read request to the master process for processing. The unsynchronized records record the data identifiers corresponding to the data that has not been synchronized from the master process to the slave process yet;

[0010] If the first data identifier does not belong to the data identifiers in the unsynchronized records, forward the read request to the slave process for processing.

[0011] In a possible implementation, the method further includes:

[0012] When the data processing request includes a write request, determine a second data identifier of the data requested to be written by the write request;

[0013] Forward the write request to the master process for processing, and add the second data identifier to the unsynchronized records.

[0014] In another possible implementation, it further includes:

[0015] Obtain a synchronization completion indication fed back by the master process. The synchronization completion indication carries a third data identifier, and the synchronization completion indication is used to indicate that the master process has synchronized the data represented by the third data identifier to the slave process;

[0016] Delete the third data identifier in the unsynchronized records.

[0017] In another possible implementation, multiple slave processes are running on the service platform;

[0018] The method further includes:

[0019] When it is confirmed that the master process fails, query the synchronization sequence numbers stored by each of the multiple slave processes. Among them, the synchronization sequence number represents the total number of data synchronized from the master process to the slave process;

[0020] Based on the synchronization sequence numbers stored by each of the multiple slave processes, determine a target slave process that has obtained the largest total amount of synchronized data from the master process;

[0021] Determine the target slave process as the master process.

[0022] In another aspect, the present application also provides a data processing device, which is applied to a service platform. The service platform runs a master process and at least one slave process. The device includes:

[0023] A request acquisition unit, configured to acquire a data processing request sent by a client;

[0024] A first identifier determination unit, configured to determine a first data identifier of data requested to be read by the read request when the data processing request includes a read request;

[0025] A first request processing unit, configured to forward the read request to the main process for processing if the first data identifier belongs to the data identifiers in the unsynchronized records cached, where the unsynchronized records record the data identifiers corresponding to the data that has not been synchronized from the main process to the slave process yet;

[0026] A second request processing unit, configured to forward the read request to the slave process for processing if the first data identifier does not belong to the data identifiers in the unsynchronized records.

[0027] In another aspect, the present application further provides a server, including a memory and a processor;

[0028] Wherein, the memory is used to store a program;

[0029] The processor is configured to execute the program, and when the program is executed, it is specifically configured to implement the data processing method according to any one of claims 1 to 6.

[0030] In another aspect, the present application further provides a storage medium, configured to store a program, and when the program is executed, it is used to implement the data processing method as described above.

[0031] As can be seen from the above, the business service platform of the present application caches unsynchronized records, and the unsynchronized records store the data identifiers corresponding to the data that has not been completed synchronizing from the main process to the slave process. Based on this, after receiving a read request from the client, the present application will only forward the read request to the slave process for processing when the unsynchronized records do not include the data identifier requested by the read request, that is, when it is confirmed that the data requested by the read request has been synchronized from the main process to the slave process; and if the unsynchronized records include the data identifier requested by the read request, the read request will still be forwarded to the main process for processing, thereby reducing the number of read requests processed by the main process to the greatest extent, reducing data reading errors or exceptions caused by data inconsistency between the main process and the slave process, and further realizing reducing data reading errors or exceptions while improving the request processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0033] Figure 1 The figure shows a schematic diagram of the composition architecture of a scenario applicable to the present application;

[0034] Figure 2 The figure shows a schematic flowchart of an embodiment of the data processing method of the present application;

[0035] Figure 3 The figure shows a schematic diagram of a principle framework for implementing the data processing method of the present application;

[0036] Figure 4 The figure shows a schematic diagram of the process interaction of another embodiment of the data processing method of the present application;

[0037] Figure 5 The figure shows a schematic flowchart of a process in which the routing process selects a master process from multiple slave processes in the present application;

[0038] Figure 6 The figure shows a schematic example diagram of the routing process selecting a master process;

[0039] Figure 7 The figure shows a schematic diagram of a composition structure of the data processing device of the present application;

[0040] Figure 8 The figure shows a schematic diagram of a composition architecture of the server of the present application. Detailed implementation manners

[0041] The data processing method of the present application is applicable to business service platforms in a variety of business scenarios. For example, in the case where the business scenario is a game business scenario, the business service platform is a game service platform. Correspondingly, the game client can request to read or write game-related data from the game service platform. Another example is that in the case where the business scenario is a financial transaction business, the business service platform is a transaction service platform. In this scenario, when the client conducts operations such as payment or transfer, it may involve requesting to read or write data from the business service platform. Of course, there may be other possibilities for the business scenarios applicable to the present application, and no limitation is imposed thereon.

[0042] In the present application, the business service platform can be an independent server, or a cloud platform, a server cluster, or a distributed system composed of multiple servers, etc.

[0043] In the embodiments of the present application, a master process for processing read or write requests and at least one slave process for assisting the master process in processing read requests are running on the business service platform.

[0044] In the embodiments of the present application, after receiving a data processing request from a client, the business service platform can reasonably allocate the data processing request to the main process or the slave process in combination with the type of the data processing request and the data synchronization status between the main process and the slave process, so as to reduce data reading errors or reading exceptions while ensuring the processing efficiency of the data processing request.

[0045] In a possible implementation, the business service platform also runs a routing process, which is a process independent of the main process and the slave process, and is responsible for managing the main process and the slave process and allocating read requests or write requests to the main process and the slave process.

[0046] Among them, in the case where the business service platform includes multiple servers, the routing process, the main process, and at least one slave process can run on the same server or on different servers.

[0047] Optionally, in order to improve disaster tolerance (for example, to avoid a single server failure affecting multiple processes in the main process and the slave process at the same time), or to ensure the running performance of each process, the routing process, the main process, and each slave process can run on different servers respectively.

[0048] For ease of understanding, the following takes the business service platform as a cloud platform as an example to illustrate a scenario applicable to the solution of the present application, as Figure 1 shown.

[0049] As Figure 1 can be seen, this scenario includes: Cloud platform 10, and the cloud platform can include multiple cloud servers 101.

[0050] Among them, the cloud platform is also called a cloud computing platform, which is a network platform built based on cloud technology. Among them, cloud technology is a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to realize data calculation, storage, processing, and sharing.

[0051] Cloud technology is the general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, be used on demand, and is flexible and convenient. The background services of the technical network system require a large amount of computing and storage resources, such as image storage and encoding, etc. With the high development and application of the Internet industry, in the future, each item may have its own identification mark and needs to be transmitted to the background system for logical processing. Data at different levels will be processed separately, and various industry data requires a powerful system support, which can only be achieved through cloud computing.

[0052] Among them, cloud computing is a computing model that distributes computing tasks across a resource pool composed of a large number of computing devices, enabling various application systems to obtain computing power, storage space, and information services as needed. The network that provides resources is called the "cloud". The resources in the "cloud" seem to users to be infinitely expandable, and can be obtained at any time, used on demand, expanded at any time, and paid according to usage.

[0053] As a basic capability provider of cloud computing, a cloud computing resource pool (abbreviated as cloud platform, generally called IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to select and use. The cloud computing resource pool mainly includes: computing devices (virtual machines containing operating systems), storage devices, and network devices.

[0054] Such as Figure 1 As shown, there is a main process and at least one slave process running on the cloud platform 10. In addition, a routing process other than the main process and the slave process is also running on the cloud platform. Among them, the main process, the routing process, and at least one slave process can run on different cloud servers of the cloud platform respectively.

[0055] In Figure 1 this scenario, there is also at least one client 20. The client can send a data processing request to the cloud platform, and the data processing request can be a write request or a read request.

[0056] Correspondingly, the cloud platform responds to the data processing request through the routing process, and determines the process for processing the data processing request from the main process and the slave process.

[0057] It can be understood that Figure 1 this is taking the business service platform as the cloud platform as an example. In actual applications, the same is true for the business service platform being a server cluster or other situations, which will not be elaborated here.

[0058] Combining the above content, the image processing method of this application will be introduced below in conjunction with the flowchart.

[0059] Such as Figure 2 As shown, it shows a schematic flowchart of a data processing method in an embodiment. The method of this embodiment can be applied to a business service platform. For example, it can be specifically executed by the routing process of the business service platform. As can be seen from the previous content, the business service platform runs a main process and at least one slave process.

[0060] The method of this embodiment may include:

[0061] S201, obtain the data processing request sent by the client.

[0062] Among them, the data processing request may include one or both of a write request and a read request. Among them, the write request is used to request writing data; the read request is used to request reading data. It can be understood that in most business scenarios, the data processing request is generally one of a write request and a read request.

[0063] S202. When the data processing request includes a read request, determine a first data identifier of the data requested to be read by the read request.

[0064] It can be understood that read requests and write requests will have different identification methods. Therefore, by parsing the data processing request, it can be analyzed whether the data processing request includes a read request.

[0065] Among them, the data identifier of the data is used to uniquely identify the data. Among them, in different business scenarios, the specific form or composition structure of the data identifier that uniquely identifies the data will also vary, and can be specifically set according to needs.

[0066] For example, taking the business scenario as a game scenario, a data can be uniquely identified by the data type and number of the data. For example, taking the gang game scenario as an example, in the gang game, an organization composed of multiple players can be called a gang. Therefore, for any game data generated by a gang, it can be uniquely identified by the gang data type and the in-gang identifier.

[0067] In the embodiment of the present application, for the convenience of distinction, the data identifier corresponding to the data requested by the read request is referred to as the first data identifier.

[0068] It can be understood that in order for the business service platform to confirm the data requested to be read by the read request, the read request will contain or carry relevant information for characterizing the data identifier of the data. Therefore, by parsing the read request, the first data identifier for characterizing the data to be read can be parsed out.

[0069] S203. If the first data identifier belongs to the data identifier in the unsynchronized records cached, forward the read request to the main process for processing.

[0070] Among them, the unsynchronized records record the data identifiers corresponding to the data that has not been synchronized from the main process to the slave process.

[0071] Among them, there is no restriction on the specific form of the unsynchronized record and the data identification recorded in the unsynchronized record. For example, the data identification can be directly stored in the unsynchronized record; for another example, the unsynchronized record can include records corresponding to different data identifications. Each record corresponding to a data identification not only includes the data identification, but also can include the moment when the client requests to write the data corresponding to the data identification, and other data related to the data identification can also be recorded as needed.

[0072] Among them, the business service platform can cache the unsynchronized process in a fixed cache space. For example, in the case where the business service platform manages the main process and the slave process through a routing process, the unsynchronized record can be cached in the memory space corresponding to the routing process.

[0073] Among them, there are various ways for the business service platform to determine the unsynchronized record. For the convenience of understanding, several possible situations will be introduced.

[0074] In one possible situation, the business service platform can query the data identification of the data that has not been synchronized to the slave process from the main process according to a polling cycle or irregularly. Correspondingly, the business service platform can record the data identification corresponding to the data that has not been synchronized from the main process to the slave process in the unsynchronized record; or, based on the queried data identification, update the unsynchronized record.

[0075] However, in this situation, the business service platform needs to continuously query the situation of the main process synchronizing data to the slave process, which may increase the occupation of bandwidth resources between the business service platform and the main process, and may also occupy the data processing resources of the main process.

[0076] In order to reduce bandwidth occupation and avoid occupying the data processing resources of the main process, in another possible situation, when the business service platform receives a write request, it can determine the second data identification of the data requested to be written by the write request and add the second data identification to the unsynchronized record. Of course, when adding the second data identification to the unsynchronized record or after that, the business service platform will also forward the write request to the main process for processing.

[0077] It can be understood that when the business service platform receives a write request, since the data requested to be written by the write request has not been written by the main process, therefore, the data corresponding to the second data identification has naturally not been synchronized from the main process to the slave process. It can be seen that after the business service platform receives the write request, it can determine the data requested to be written by the write request as the data that has not been synchronized from the main process to the slave process, and store the second data identification in the write request in the unsynchronized record, so that there is no need to query the data identification corresponding to the unsynchronized data from the main process again.

[0078] The above takes two ways of generating unsynchronized records by the business service platform as examples. Other ways of determining the data identifiers corresponding to the data that may not have been synchronized yet are also applicable to this embodiment, and there is no limitation thereto.

[0079] It can be understood that in order to ensure that the data that the main process has not yet synchronized to the slave process is consistent with the data represented by the data identifiers recorded in the unsynchronized records cached by the business service platform, the business service platform can also continuously update the unsynchronized records to delete the data identifiers that have been synchronized in the unsynchronized records.

[0080] For example, in a possible implementation manner, the business service platform can periodically send a synchronized information request to the main process. The synchronized information request is used to request the main process to feedback the data identifiers corresponding to the data that has been successfully synchronized to the slave process during the current period. Among them, after the main process successfully synchronizes data to the slave process, it will store the data identifiers corresponding to the synchronized data. Correspondingly, after the main process receives the synchronized information request, the main process sends the information of the data identifiers corresponding to the data that has been synchronized in the most recent period to the business service platform. On this basis, the business service platform deletes the corresponding data identifiers in the unsynchronized records based on the information of the data identifiers feedback by the main process.

[0081] This implementation manner is actually that the business service platform periodically updates the unsynchronized records. Therefore, there may be a situation where the unsynchronized records are not updated in a timely manner, resulting in a situation where a certain data has been synchronized from the main process to the slave process, but the unsynchronized records of the business service platform still indicate that the data has not been synchronized to the slave process, and further forwards the read request for this data to the main process for processing, making the number of read requests processed by the main process relatively large.

[0082] In order to minimize the number of read requests processed by the main process to the greatest extent, reduce the pressure on the main process to the greatest extent, further improve the processing efficiency of data processing requests, and reduce the latency of client data processing requests, the present application can also actively report the data identifiers corresponding to the synchronized data to the business service platform after the main process synchronizes the data to the slave process.

[0083] Specifically, the business service platform can obtain a synchronization completion indication feedback by the main process. The synchronization completion indication carries a third data identifier, and the synchronization completion indication is used to represent that the main process has synchronized the data represented by the third data identifier to the slave process.

[0084] Among them, the synchronization completion indication is generated after the main process confirms that the data with the third data identifier has been synchronized to the slave process. It can be understood that, for the convenience of distinction, the data identifier corresponding to the data synchronized from the main process to the slave process is called the third data identifier. Correspondingly, the service platform can delete the third data identifier in the unsynchronized record based on the synchronization completion indication.

[0085] In the second method of updating the unsynchronized record, each time the main process finishes synchronizing data to the slave process, it will notify the service platform. As a result, the data that has not been synchronized determined based on the unsynchronized record is consistent with the data that has not been synchronized to the slave process in the main process. This is beneficial for forwarding all read requests corresponding to the data that has been synchronized to the slave process for processing, minimizing the number of read requests that the main process needs to handle and reducing the data processing pressure on the main process.

[0086] It can be understood that if there is a data identifier of a certain data in the unsynchronized record, it means that the data has not been synchronized from the main process to the slave process yet, that is, the data is only cached in the memory space corresponding to the main process. Therefore, when the first data identifier carried in the read request belongs to the data identifier in the unsynchronized record, in order to avoid data read or write errors or abnormalities caused by the inconsistency of the data represented by the first data identifier in the main process and the slave process, the read request will be assigned to the main process for processing.

[0087] S204, if the first data identifier does not belong to the data identifier in the unsynchronized record, forward the read request to the slave process for processing.

[0088] It can be understood that if the first data identifier of the data requested to be read by the read request does not belong to the unsynchronized record, then it can be determined that the data has been synchronized from the main process to the slave process. Therefore, in order to avoid increasing the pressure on the main process, the read request can be sent to the slave process for processing. Moreover, since the data has been synchronized between the main process and the slave process, there will be no data read errors or abnormalities caused by the inconsistency of the data in the main process and the slave process.

[0089] It can be understood that when there are multiple slave processes running on the service platform, the service platform can select one slave process from multiple slave processes and forward the read request to the selected slave process. Among them, there are various ways for the service server platform to select the slave process for processing the read request. For example, it can select a slave process with a lower load based on the load status of the slave process to process the read request, etc., and there is no restriction on this.

[0090] It can be seen that since the business service platform of the present application caches unsynchronized records, and the unsynchronized records store data identifiers corresponding to data that has not yet been completed synchronizing from the main process to the slave process. Based on this, after the present application receives a read request from the client, only when the unsynchronized records do not include the data identifier requested by the read request, that is, when it is confirmed that the data requested by the read request has been synchronized from the main process to the slave process, will the read request be forwarded to the slave process for processing; if the unsynchronized records include the data identifier requested by the read request, the read request will still be forwarded to the main process for processing, thereby minimizing the number of read requests processed by the main process and reducing data reading errors or anomalies caused by data inconsistency between the main process and the slave process, and further achieving the reduction of data reading errors or anomalies while improving the request processing efficiency.

[0091] For ease of understanding, the following takes the business service platform running a routing process and scheduling data processing requests to the main process and the slave process through the routing process as an example for illustration.

[0092] As Figure 3 shown, it shows a schematic framework diagram of an implementation principle of the data processing method of the present application.

[0093] In Figure 3 the business service platform 300 runs a routing process 301, a main process 302, and at least one slave process 303.

[0094] As Figure 3 shown, the routing process caches unsynchronized records.

[0095] Among them, after the client 310 sends a read request or a write request to the business service platform, the routing process of the business service platform will obtain the read request or the write request sent by the client.

[0096] Among them, for the obtained write request, the routing process 301 can directly send the write request to the main process.

[0097] For the obtained read request, the routing process will detect whether it hits the cached unsynchronized records, that is, whether the data identifier carried by the read request belongs to the data identifier in the unsynchronized records. Correspondingly, the routing process will send the read request that does not hit the unsynchronized records to the slave process; if the read request hits the unsynchronized records, the read request will be sent to the main process for processing.

[0098] Next, based on the principle framework diagram in Figure 3 and combined with Figure 4 , the data processing method of the present application will be introduced in detail from the interaction process among the routing process, the main process, and the slave process.

[0099] As Figure 4The data processing method shown may include:

[0100] S401, the client sends a read request to the business service platform so that the routing process obtains the read request.

[0101] It can be understood that, in this embodiment, the data processing request sent by the client to the business service platform can be one of a read request or a write request. If the data processing request sent by the client to the business service platform includes a read request and a write request, then the routing process can separate these two requests and perform corresponding processing for the read request and the write request respectively.

[0102] S402, the routing process obtains the first data identifier carried by the read request.

[0103] Wherein, the first data identifier is used to represent the data requested to be read by the read request.

[0104] S403, if the routing process confirms that the first data identifier belongs to the data identifier in the cached unsynchronized records, the routing process forwards the read request to the main process for processing.

[0105] It can be understood that after receiving the read request, the main process can query the corresponding data from the cached business data based on the read request, and then return the queried data to the client. There is no limitation on the specific implementation of the main process returning the corresponding data to the client based on the read request, and this part of the content is not shown either. Figure 4 in

[0106] S404, if the routing process confirms that the first data identifier does not belong to the data identifier in the cached unsynchronized records, the routing process selects a slave process for processing the read request from at least one slave process, and forwards the read request to the selected slave process.

[0107] Similar to the main process processing the read request, since each data synchronized by the main process is also cached by the slave process, the slave process can respond to the read request, query the data corresponding to the read request from the cached data, and return the queried data to the client.

[0108] S405, the client sends a write request to the business service platform so that the routing process obtains the write request sent by the client.

[0109] S406, the routing process confirms the second data identifier of the data requested to be written by the write request, and adds the second data identifier to the cached unsynchronized records.

[0110] It can be understood that before the write request is forwarded to the main process, the main process has not written the data corresponding to the write request. Therefore, this data naturally does not belong to the data that has been synchronized to the slave process. Thus, the routing process can determine that this data is the data that has not been synchronized from the main process to the slave process, and then store the second data identifier of this data in the unsynchronized record.

[0111] As an optional method, adding the second data identifier to the unsynchronized record can be to generate a record corresponding to the second data identifier. For example, the second data identifier is stored in the record corresponding to the second data identifier. Among them, other data related to the second data identifier can also be stored in the record corresponding to the second data identifier.

[0112] It can be understood that in any of the above embodiments of the present application, considering that the space for caching unsynchronized records in the business service platform is relatively limited. For example, the memory space allocated to the routing process by the business service platform is limited. Therefore, if the number of data processing requests processed by the business service platform is relatively large, the number of data identifiers that may need to be stored in the unsynchronized record will also be relatively large.

[0113] In this case, when the business service platform confirms that the cache space is insufficient, the present application can also delete the data identifiers that have not been used for a long time in the unsynchronized record. Among them, the data identifiers that have not been used for a long time in the unsynchronized record refer to the data identifiers of the data that has not been requested to be read for a relatively long time.

[0114] Specifically, after the routing process obtains a read request for the second data identifier (that is, a read request for requesting to read the data corresponding to the second data identifier), if the second data identifier belongs to the unsynchronized record, the routing process will also determine the moment when the read request is obtained as the access moment of the second data identifier, and store the access moment in the record corresponding to the second data identifier.

[0115] Among them, when the access moment is stored in the record of the second data identifier, the routing process will update the access moment in the record of the second data identifier with the most recently determined access moment, so that the access moment stored in the record of the second data identifier is the most recent access moment of the second data identifier.

[0116] Correspondingly, if the routing process confirms that the cache space is insufficient, it can delete the records of at least one data identifier whose access moment is relatively long from the current moment in the unsynchronized record. For example, delete the records of each data identifier whose access moment exceeds the set duration from the current moment, or delete the record of one or more data identifiers whose access moment is the longest from the current moment.

[0117] S407. The routing process forwards the write request to the main process for processing.

[0118] It should be noted that the order of the two steps of adding the second data identifier to the unsynchronized records in the cache and forwarding the write request to the main process is not limited. For example, these two steps can also be executed simultaneously.

[0119] S408, the main process performs a data write operation based on the write request and caches the write operation result.

[0120] The main process responds to the write request to perform a write operation, thereby implementing the data corresponding to the cache write operation.

[0121] S409, the main process sends at least one piece of written but unsynchronized data to each slave process.

[0122] Among them, the specific manner in which the main process sends the data to be synchronized to the slave process can be unrestricted. For example, after the main process finishes the write operation corresponding to the write request, the main process can send the written data to each slave process to synchronize the data to the slave process. Another example is that in order to avoid frequent data synchronization from occupying too many processing resources of the main process, the main process can also regularly (such as according to a set time interval or cycle, etc.) send a data synchronization indication to the main process, and the data synchronization indication can carry all the data that has been written into the cache space of the main process but not synchronized to the slave process. Of course, the main process can also have other ways to synchronize data with the slave process, which will not be elaborated here.

[0123] S410, after the main process confirms that the at least one piece of data has been successfully synchronized to the slave process, the main process confirms the third data identifier of each of the successfully synchronized at least one piece of data and feeds back a synchronization completion indication corresponding to the at least one piece of data to the routing process.

[0124] Among them, there can also be various ways for the main process to confirm whether the data has been successfully synchronized to the slave process. For example, after the slave process completes data synchronization based on the data sent by the main process, the slave process can feed back a success indication to the main process, and the success indication represents that the slave process has completed the synchronization of the corresponding data. For example, the success indication carries the data identifier of the data with successful synchronization. Of course, the main process can actively query the synchronization status of the slave process to detect whether the data synchronization is successful.

[0125] S411, the routing process deletes the at least one third data identifier in the unsynchronized records based on the at least one third data identifier carried in the synchronization completion indication.

[0126] After obtaining the synchronization completion indication fed back by the main process, the routing process can determine that the data represented by the third data identifier carried in the synchronization completion indication has completed data synchronization. On this basis, the routing process can delete the third data identifier from the unsynchronized records. Correspondingly, after receiving a read request carrying the third data identifier subsequently, the routing process can forward the read request to the slave process for processing.

[0127] It can be understood that if the main process in the business service platform fails, it may affect the processing of read and write requests from the client by the business service platform. Therefore, after determining that the main process has failed, a corresponding recovery mechanism is required. However, if the main process is directly restarted, the complexity is high and the time consumption is long. To solve this problem, after determining that the main process in the business service platform of the present application has failed, a slave process will be reselected from multiple slave processes as the main process.

[0128] As Figure 5 shown, it shows a schematic flow diagram of the routing process of the present application for selecting a main process from multiple slave processes. This embodiment may include:

[0129] S501, when the heartbeat packet sent by the main process is not detected for more than a set duration, the routing process queries the synchronization sequence numbers stored in each slave process.

[0130] Among them, the main process can send a heartbeat packet to the routing process according to a set first heartbeat period. Correspondingly, the slave process can send a heartbeat packet to the routing process according to a set second heartbeat period. Among them, the first heartbeat period and the second heartbeat period can be the same or different.

[0131] For example, the main process or the slave process can send a heartbeat packet to the business service platform at the heartbeat sending moment according to their respective corresponding heartbeat periods.

[0132] It can be understood that since both the main process and the slave process regularly send heartbeat packets to the routing process, the routing process can analyze whether the main process is abnormal based on the situation of the heartbeat packet sent by the main process; correspondingly, the routing process can also analyze whether the slave process is abnormal based on the situation of the heartbeat packet sent by the slave process.

[0133] If the routing process does not receive the heartbeat packet of the main process for more than a set duration, the routing process confirms that the main process has failed.

[0134] It can be understood that this embodiment analyzes whether the main process fails based on the situation of the main process sending a heartbeat packet to the routing process. However, it can be understood that in actual applications, the routing process can also discover the failure of the main process through other means. For example, the routing process can regularly send a detection packet to the main process. If the feedback from the main process for this detection packet is not received for more than a set duration, or if no feedback is received for consecutive multiple detection reports, it is confirmed that the main process has failed.

[0135] Among them, the synchronization sequence number represents the total quantity of data synchronized from the master process to the slave processes. Among them, the total quantity of data synchronized from the master process to the slave processes can also be regarded as the total number of rounds of data synchronization initiated by the master process to multiple slave processes. Among them, each round of data synchronization corresponds to a round of data synchronization indication initiated by the master process for a piece of data to be synchronized to multiple slave processes.

[0136] In a possible implementation, before the master process synchronizes data to the slave processes each time, it updates the synchronization sequence number to increase the total quantity of data synchronized from the master process to the slave processes represented by the synchronization sequence number. For example, if the synchronization sequence number can be the quantity value of data synchronized from the master process to the slave processes, then each time the synchronization sequence number is updated, the quantity value is incremented by one. Correspondingly, each time the master process can send the data to be synchronized and the latest updated synchronization sequence number to each slave process so that the slave process stores the synchronization sequence number.

[0137] In another possible implementation, each time the master process receives a write request, it updates the synchronization sequence number to increase the total quantity of data synchronized from the master process to the slave processes represented by the synchronization sequence number. Correspondingly, when the master process synchronizes data to the slave processes, it can send the data to be synchronized and the currently determined synchronization sequence number to each slave process.

[0138] For example, after the master process receives the write request for the first time, it can set the synchronization sequence number to "0001". Then, after the master process completes the write operation corresponding to the write request, the master process can send the data corresponding to the write operation and the synchronization sequence number "0001" to each slave process. After receiving the write request for the second time, the synchronization sequence number will be changed to "0002". In this way, after the master process completes the write operation of the second received write request, when synchronizing the data to each slave process, the synchronization sequence number "0002" will also be sent to each process.

[0139] It can be understood that since the synchronization sequence number can reflect the total quantity of data synchronized from the master process to the slave processes, therefore, if the master process fails and causes some data in the master process to be synchronized to only some of the slave processes, then based on the synchronization sequence numbers stored in the slave processes, it can be determined which slave processes have relatively complete synchronized data.

[0140] For example, as described above, when the master process sends synchronized data to each synchronization process for the second time, it carries the synchronization sequence number "0002". Then, if the slave process has received the second synchronized data from the master process, the synchronization sequence number stored in the slave process should be "0002". However, if the synchronization sequence number stored in the slave process is "0001", it means that the slave process has not completed the second data synchronization but has received the data synchronized by the master process for the first time.

[0141] S502. Determine a target slave process that has obtained the largest total number of synchronization data from the master process based on the synchronization sequence numbers stored by each of the multiple slave processes.

[0142] Among them, the target slave process refers to the slave process with the largest total number represented by the synchronization sequence numbers stored among the multiple slave processes. It can be understood that in the case where there are at least two slave processes with the largest total number of represented synchronization data, one slave process can be randomly selected as the target slave process; or it can be selected according to the load situation from among the at least two slave processes.

[0143] For example, in the above example, the slave process with the stored synchronization sequence number "0002" has obtained a larger total number of synchronization data from the master process than the slave process with the stored synchronization sequence number "0001".

[0144] S503. Determine the target slave process as the master process.

[0145] As can be seen from the above introduction, the data in the target slave process has the highest consistency with the data in the master process. Therefore, determining the target slave process as the new master process is beneficial to reducing data reading anomalies caused by data inconsistency.

[0146] Among them, the operations required to determine the target slave process as the master process may include: storing information that the target slave process is the master process; it may also include: broadcasting information that the target slave process is the master process to the target slave process and other slave processes.

[0147] It can be understood that after the target slave process becomes the new master process, the target slave process can synchronize its own stored data with other slave processes to ensure data consistency.

[0148] For the sake of easy understanding, taking the synchronization sequence number as an example of the total number of data synchronized from the master process to the slave process, for example, see Figure 6 , which shows a schematic diagram of the principle of the routing process selecting a master process from multiple slave processes.

[0149] When Figure 6 the master process crashes, the synchronization sequence number stored in the master process is 112, which means that the master process has synchronized 112 data to the slave process (or initiated 112 rounds of data synchronization). However, since the master process crashed during the synchronization of the 112th data, only some of the slave processes completed the synchronization of the 112th data.

[0150] Such as Figure 6In this case, if the synchronization sequence number in Process 1 is 112, it indicates that Process 1 has obtained the 112th data synchronized by the master process to this slave process 1. However, the synchronization requirements in Process 2 and Process 3 are still 111. Therefore, since the synchronization sequence number stored in slave process 1 is the largest among all slave processes, it shows that the data consistency between slave process 1 and the failed master process is the highest. Thus, this slave process can be determined as the new master process.

[0151] Correspondingly, this new master process can synchronize its own data to other slave processes to ensure data consistency between each master process and this new master process.

[0152] Corresponding to a data processing method of the present application, the present application also provides a data processing device.

[0153] As Figure 7 shown, it shows a schematic structural diagram of a composition of an embodiment of a data processing device of the present application. The device of this embodiment is applied to a service platform, and the service platform runs a master process and at least one slave process. The device includes:

[0154] A request acquisition unit 701, configured to acquire a data processing request sent by a client;

[0155] A first identifier determination unit 702, configured to determine a first data identifier of the data requested to be read by the read request when the data processing request includes a read request;

[0156] A first request processing unit 703, configured to forward the read request to the master process for processing if the first data identifier belongs to the data identifiers in the unsynchronized records cached, and the unsynchronized records record the data identifiers corresponding to the data that has not been synchronized from the master process to the slave process;

[0157] A second request processing unit 704, configured to forward the read request to the slave process for processing if the first data identifier does not belong to the data identifiers in the unsynchronized records.

[0158] In a possible implementation manner, the device may further include:

[0159] A second identifier determination unit, configured to determine a second data identifier of the data requested to be written by the write request when the data processing request includes a write request;

[0160] A third request processing unit, configured to forward the write request to the master process for processing and add the second data identifier to the unsynchronized records.

[0161] Optionally, the device further includes:

[0162] An indication acquisition unit, configured to acquire a synchronization completion indication fed back by the main process, where the synchronization completion indication carries a third data identifier, and the synchronization completion indication is used to indicate that the main process has synchronized the data represented by the third data identifier to the slave process;

[0163] An identifier deletion unit, configured to delete the third data identifier in the unsynchronized record.

[0164] Optionally, when adding the second data identifier to the unsynchronized record, the third request processing unit is specifically configured to add a record of the second data identifier in the unsynchronized record;

[0165] Correspondingly, the apparatus may further include:

[0166] A time recording unit, configured to, after acquiring a read request for the second data identifier, determine the time when the read request is acquired as the access time of the second data identifier, and store the access time of the second data identifier in the record of the second data identifier;

[0167] A record deletion unit, configured to, when the cache space is insufficient, based on the addition time of each data identifier in the unsynchronized record, delete the records of at least one data identifier in the unsynchronized record whose duration from the access time to the current time is relatively long.

[0168] In a possible implementation manner, multiple slave processes are running on the service platform;

[0169] The apparatus of the present application may further include:

[0170] A sequence number query unit, configured to query the synchronization sequence numbers stored by each of the multiple slave processes when it is confirmed that the main process fails, where the synchronization sequence number represents the total number of data synchronized by the main process to the slave process;

[0171] A process determination unit, configured to determine a target slave process that has obtained the largest total amount of synchronized data from the main process based on the synchronization sequence numbers stored by each of the multiple slave processes;

[0172] A main process replacement unit, configured to determine the target slave process as the main process.

[0173] Optionally, when it is confirmed that the main process fails, the sequence number query unit is specifically: when a heartbeat packet sent by the main process is not detected for more than a set duration, it is confirmed that the main process fails, where the main process sends a heartbeat packet to the service platform at the heartbeat sending time according to a set heartbeat period.

[0174] In another aspect, the present application further provides a server, and the server may be an independent server, a server in a server cluster, or a node in a cloud platform, etc. AsFigure 8 , which shows a schematic diagram of a composition architecture of the server provided in this application. In Figure 8 , the server 800 may include: a processor 801 and a memory 802.

[0175] Optionally, the server may further include: a communication interface 803, an input unit 804, a display 805, and a communication bus 806.

[0176] Among them, the processor 801, the memory 802, the communication interface 803, the input unit 804, and the display 805 all complete mutual communication through the communication bus 806.

[0177] In the embodiments of this application, the processor 801 may be a central processing unit, a specific application integrated circuit, etc.

[0178] The processor may call the program stored in the memory 802. Specifically, the processor may execute the operations performed on the cloud server side in the above embodiments.

[0179] The memory 802 is used to store one or more programs. The program may include program code, and the program code includes computer operation instructions. In the embodiments of this application, the memory stores at least a program for implementing the data processing method in any one of the above embodiments.

[0180] In a possible implementation manner, the memory 802 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, the above-mentioned program, and application programs required for functions such as image playback, etc.; the data storage area may store data created during the use of the server.

[0181] The communication interface 803 may be an interface of a communication module.

[0182] This application may further include an input unit 804, and the input unit may include a touch sensing unit, a keyboard, etc.

[0183] The display 805 includes a display panel, such as a touch display panel, etc.

[0184] Of course, Figure 8 the server structure shown does not limit the server in the embodiments of this application. In actual applications, the server may include more or fewer components than Figure 8 shown, or combine some components.

[0185] On the other hand, this application also provides a storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the data processing method in any one of the above embodiments is implemented.

[0186] According to one aspect of the present application, there is provided a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in any one of the above data processing method embodiments.

[0187] It should be noted that the various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the parts that are the same or similar among the various embodiments, reference can be made to each other. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for the relevant content.

[0188] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0189] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0190] The above are only the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A data processing method, characterized in that, Applied to a business service platform that runs a main process and multiple slave processes, the method includes: Obtain a data processing request sent by a client; When the data processing request includes a read request, determine a first data identifier of the data requested to be read by the read request; If the first data identifier belongs to the data identifiers in the cached unsynchronized records, forward the read request to the main process for processing. The unsynchronized records record the data identifiers corresponding to the data that has not been synchronized from the main process to the slave processes yet; If the first data identifier does not belong to the data identifiers in the unsynchronized records, forward the read request to the slave process for processing; Obtain a synchronization completion indication feedback by the main process after confirming that the data has been successfully synchronized to the slave process. The synchronization completion indication is used to indicate that the main process has synchronized the data represented by a third data identifier to the slave process. The main process is used to confirm that the data has been successfully synchronized to the slave process according to the success indication feedback by the slave process after completing data synchronization, or confirm that the data has been successfully synchronized to the slave process by querying the synchronization status at the slave process; Delete the third data identifier in the unsynchronized records based on the synchronization completion indication; When it is confirmed that the main process fails, query the synchronization sequence numbers stored by each of the multiple slave processes. The synchronization sequence number represents the total number of data synchronized from the main process to the slave processes; Based on the synchronization sequence numbers stored by each of the multiple slave processes, determine a target slave process that has obtained the most total amount of synchronized data from the main process; Determine the target slave process as the new main process, and the new main process is used to synchronize its own data to other slave processes.

2. The method according to claim 1, characterized in that, It further includes: When the data processing request includes a write request, determine a second data identifier of the data requested to be written by the write request; Forward the write request to the main process for processing, and add the second data identifier to the unsynchronized records.

3. The method according to claim 2, wherein The adding the second data identifier to the unsynchronized records includes: Add a record of the second data identifier in the unsynchronized records; The method further includes: After obtaining a read request for the second data identifier, determine the moment when the read request is obtained as the access moment of the second data identifier, and store the access moment of the second data identifier in the record of the second data identifier; When the cache space is insufficient, based on the addition moments of the data identifiers in the unsynchronized records, delete the records of at least one data identifier in the unsynchronized records whose duration from the access moment to the current moment is longer.

4. The method according to claim 1, wherein The confirming that the main process fails includes: When no heartbeat packet sent by the main process is detected for more than a set duration, confirm that the main process fails. Wherein, the main process sends a heartbeat packet to the business service platform at the heartbeat sending moment according to a set heartbeat period.

5. A data processing device, characterized in that, Applied to a business service platform that runs a main process and multiple slave processes, the device includes: A request obtaining unit, configured to obtain a data processing request sent by a client; A first identifier determination unit, configured to determine a first data identifier of data requested to be read by the read request when the data processing request includes a read request; A first request processing unit, configured to forward the read request to the main process for processing if the first data identifier belongs to a data identifier in the unsynchronized records cached, where the unsynchronized records record data identifiers corresponding to data that has not been synchronized from the main process to the slave process yet; A second request processing unit, configured to forward the read request to the slave process for processing if the first data identifier does not belong to a data identifier in the unsynchronized records; An indication obtaining unit, configured to obtain a synchronization completion indication fed back after the main process confirms that data has been successfully synchronized to the slave process, where the synchronization completion indication is used to represent that the main process has synchronized data represented by a third data identifier to the slave process; the main process is configured to confirm that data has been successfully synchronized to the slave process according to a success indication fed back after the slave process completes data synchronization, or confirm that data has been successfully synchronized to the slave process by querying the synchronization status at the slave process; An identifier deletion unit, configured to delete the third data identifier in the unsynchronized records based on the synchronization completion indication; A sequence number query unit, configured to query the synchronization sequence numbers stored by each of the multiple slave processes when it is confirmed that the main process fails, where the synchronization sequence number represents the total quantity of data synchronized from the main process to the slave process; A process determination unit, configured to determine a target slave process that has obtained the largest total quantity of synchronized data from the main process based on the synchronization sequence numbers stored by each of the multiple slave processes; A main process replacement unit, configured to determine the target slave process as a new main process, and the new main process is configured to synchronize its own data to other slave processes.

6. The device according to claim 5, characterized in that, Further included: A second identifier determination unit, configured to determine a second data identifier of data requested to be written by the write request when the data processing request includes a write request; A third request processing unit, configured to forward the write request to the main process for processing and add the second data identifier to the unsynchronized records.

7. The device according to claim 6, characterized in that When adding the second data identifier to the unsynchronized records, the third request processing unit is specifically configured to add a record of the second data identifier in the unsynchronized records; The apparatus further includes: A time recording unit, configured to, after obtaining a read request for the second data identifier, determine the time when the read request is obtained as the access time of the second data identifier, and store the access time of the second data identifier in the record of the second data identifier; A record deletion unit, configured to, when the cache space is insufficient, delete records of at least one data identifier in the unsynchronized records whose duration from the access time to the current time is relatively long based on the addition time of each data identifier in the unsynchronized records.

8. The device according to claim 5, characterized in that, When confirming that the main process fails, the serial number query unit is specifically configured to confirm that the main process fails when no heartbeat packet sent by the main process is detected after a set duration. The main process sends a heartbeat packet to the service platform at the heartbeat sending moment according to a set heartbeat period.

9. A server, characterized in that, It includes a memory and a processor; wherein, the memory is used for storing programs; the processor is used for executing the program, and when the program is executed, it is specifically configured to implement the data processing method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, It is used for storing programs, and when the programs are executed, they are used to implement the data processing method according to any one of claims 1 to 4.

11. A computer program product, characterized in that, It includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method according to any one of claims 1 to 4.

Images