Data processing method and device, and time service system

Abstract

The embodiment of the application provides a kind of data processing method, device and time service system, it is related to computer technical field.The method comprises: receiving multiple time service requests sent by multiple clients;Multiple time service requests are merged, at least one time service request set is generated, and at least one time service request set is forwarded to the master node, so that the master node processes each time service request set to obtain at least one corresponding response result set;At least one response result set returned by the master node is received, and multiple response results in at least one response result set are returned to the corresponding client respectively.The embodiment of the application improves the processing capacity and efficiency of time service without changing the consistency model or upper application, can adapt to large-scale and / or high-load time service object, can adapt to different use scenarios, greatly reduces migration and development cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular, the present application relates to a data processing method and device and time service system. BACKGROUND

[0002] In a distributed system, how to accurately and quickly sequence the start and end of an event is an important problem. The transaction under the distributed system, MVCC (Multiversion Concurrency Control) data storage and other functions cannot be separated from the basic ability of event sequencing.

[0003] TSO (Timestamp Oracle) is a linearly consistent, continuously increasing sequence number service. In a distributed system, TSO service nodes often serve as a single point to provide time service to the entire cluster, which is used to define the transaction start and commit order in the distributed system, and provides a stable and reliable linear consistency basis for distributed event sequencing.

[0004] However, the expansion of the cluster size, the increase of TSO service objects and the improvement of the cluster load will test the processing capacity of the TSO time service. In the prior art, the optimization of the time service is often realized by modifying the consistency model or the specific use mechanism of the upper application, but this scheme reduces the universality and scalability of the TSO service, cannot adapt to different use scenarios, and also increases the cost of application development and technical maintenance.

[0005] Therefore, under the premise of not changing the consistency model, how to improve the processing capacity of the TSO time service is a problem to be solved. SUMMARY

[0006] The embodiments of the present application provide a data processing method, device and time service system, which can solve the problem of low processing capacity of the TSO time service under the premise of not changing the consistency model in the prior art.

[0007] The technical solution is as follows:

[0008] According to an aspect of the embodiments of the present application, a data processing method is provided, which is applied to a proxy node set including at least one proxy node, and includes:

[0009] receiving a plurality of time service requests sent by a plurality of clients;

[0010] merge the plurality of timing service requests to generate at least one timing service request set, and forward the at least one timing service request set to a master node for processing each timing service request set to obtain a corresponding at least one response result set;

[0011] receive the at least one response result set returned by the master node, and return a plurality of response results in the at least one response result set to corresponding clients respectively.

[0012] Optionally, the merging the plurality of timing service requests to generate at least one timing service request set, and forwarding the at least one timing service request set to a master node comprises:

[0013] merging a plurality of timing service requests in a preset time interval according to a receiving time of each timing service request to obtain a timing service request set corresponding to the time interval;

[0014] forwarding the generated at least one timing service request set to the master node in sequence.

[0015] Optionally, the forwarding the generated at least one timing service request set to the master node in sequence comprises:

[0016] forwarding a current timing service request set to the master node after receiving a response result set corresponding to a previous timing service request set.

[0017] Optionally, the returning a plurality of response results in the at least one response result set to corresponding clients respectively comprises:

[0018] decomposing the response result set to obtain a plurality of response results included in the response result set for each response result set;

[0019] determining a time sequence of the plurality of response results;

[0020] returning the plurality of response results to corresponding clients in sequence according to the time sequence.

[0021] Optionally, the receiving a plurality of timing service requests sent by a plurality of clients comprises:

[0022] receiving, by each proxy node, a preset number of timing service requests sent by at least one corresponding client.

[0023] Optionally, the preset number is determined based on a configuration and / or real-time running state of the proxy node.

[0024] Optionally, before the receiving, the method further comprises:

[0025] sending the real-time running state of the at least one proxy node to the at least one client, so that the at least one client determines a forwarding weight of each of the corresponding proxy nodes based on the real-time running state, and determines the preset number based on the forwarding weight of each of the corresponding proxy nodes.

[0026] According to another aspect of the embodiments of the present application, a time service system is provided, which comprises a master node, a plurality of clients, and a proxy node set, wherein the proxy node set comprises at least one proxy node.

[0027] Each of the proxy nodes in the proxy node set is in communication connection with the master node, and each of the proxy nodes in the proxy node set is in communication connection with at least one corresponding client.

[0028] The proxy node set performs the data processing method provided in the above embodiments.

[0029] According to another aspect of the embodiments of the present application, a data processing apparatus is provided, which comprises:

[0030] a receiving module configured to receive a plurality of time service requests sent by a plurality of clients;

[0031] a merging and forwarding module configured to merge the plurality of time service requests to generate at least one time service request set, and forward the at least one time service request set to a master node, so that the master node processes each time service request set to obtain at least one corresponding response result set;

[0032] a returning module configured to receive the at least one response result set returned by the master node, and return a plurality of response results in the at least one response result set to corresponding clients respectively.

[0033] According to another aspect of the embodiments of the present application, an electronic device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of any of the above data processing methods when executing the program.

[0034] According to still another aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, and the computer program is executable on a processor to implement the steps of any of the above data processing methods.

[0035] The technical scheme provided by the embodiments of the present application has the beneficial effects that:

[0036] The multiple time service requests are forwarded to the master node through the proxy node set, the pressure of the master node for processing the request connections is shared by the proxy node set, the number of request connections of the master node is reduced, the resources of the CPU of the master node for processing the connection switching are reduced, the master node can have sufficient CPU resources to process the time service, the processing capacity and efficiency of the time service are improved without changing the consistency model or the upper application, and the time service can adapt to large-scale and / or high-load time service objects. Meanwhile, the consistency semantics and specific interfaces provided externally are not changed, the universality and scalability of the TSO service are reserved, the time service can well adapt to different use scenarios, additional development work is not needed for different use scenarios, and the migration and development costs are greatly reduced.

[0037] Further, the multiple time service requests are forwarded by the proxy node set, the number of the time service request set is far less than the number of the time service requests, and the number of request connections of the master node is further reduced.

[0038] In addition, the processing capacity of the time service request forwarding and calculation can be improved by vertically expanding the CPU of a single proxy node or master node, the number of request connections that can be carried by the time service as a whole can be improved by horizontally expanding the number of proxy nodes, and the overall throughput of the time service system is improved. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced.

[0040] Figure 1 It is a time service system architecture diagram in the prior art;

[0041] Figure 2 It is a system architecture diagram of a time service system provided by the embodiments of the present application;

[0042] Figure 3 It is a flowchart of a data processing method provided by the embodiments of the present application;

[0043] Figure 4 It is a schematic diagram of a request merging process provided by the embodiments of the present application;

[0044] Figure 5 It is a schematic diagram of a request distribution process provided by the embodiments of the present application;

[0045] Figure 6 It is a flowchart of a data processing method provided by another embodiment of the present application

[0046] Figure 7A structural schematic diagram of a data processing device provided by an embodiment of the present application is shown in FIG. 1.

[0047] Figure 8 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 1. DETAILED DESCRIPTION

[0048] Embodiments of the present application will be described below with reference to the accompanying drawings. It should be understood that the embodiments described below in conjunction with the drawings are exemplary descriptions of the technical solutions of the embodiments of the present application, and do not limit the technical solutions of the embodiments of the present application.

[0049] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an" and "the" used herein can also include the plural forms. It should be further understood that the terms "comprise" and "include" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the present technology. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or can mean that the element and the other element are connected through an intermediate element. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The term "and / or" used herein means that at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or as "B", or as "A and B".

[0050] In order to make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0051] Time is a collection of a series of past, present and future events, and the order of events determines causality. In a distributed system, how to accurately and quickly sequence the start and end of events is an important problem.

[0052] Transaction in a distributed system, MVCC data storage and other functions cannot be separated from the basic ability of event sequencing. How to provide stable and reliable linear consistency for distributed event sequencing, obtain or simulate such an abstract time is a complex thing. Currently, atomic clock synchronization, timestamp time service and HLC (Hybrid Logical Clocks, hybrid logical clock) synchronization and other technical solutions have been proposed to provide time service and sequencing service that meet the above requirements.

[0053] TSO represents a linear consistent, continuously increasing sequence number service. In a distributed system, TSO service nodes often provide time service as a single point to the entire cluster to define the transaction start and commit order in the distributed system, and provide stable and reliable linear consistency basis for distributed event sequencing.

[0054] The principle of TSO time service is relatively simple and reliable, but due to its working principle, it has strong single-point properties and can only work as a global or regional single service point in a distributed system. Figure 1 The prior art time service system architecture diagram is shown in Figure 1 The system includes a master node 101 and a plurality of clients 102, the master node is in communication connection with the plurality of clients, the plurality of clients respectively send a plurality of time service requests to the master node, the master node receives the plurality of time service requests and processes the plurality of time service requests to obtain a plurality of corresponding response results, and finally returns the plurality of response results to the corresponding clients.

[0055] The single-point property of TSO can cause cluster-level unavailability once a problem occurs, and cannot achieve good availability. At the same time, the single-point property makes it difficult to have the ability of horizontal expansion of the distributed system, so that the single-point property is more likely to become the bottleneck of the entire distributed system.

[0056] With the expansion of the cluster size, the increase of TSO service objects and the improvement of cluster load, the allocation speed and response speed of TSO time service are greatly tested. In the prior art, the optimization of time service is often realized by modifying the consistency model or the specific use mechanism of the upper application, such as adding TSO cache on the TSO request instance, weakening the consistency model, sacrificing the consistency requirement, etc. to achieve performance improvement, so that it is no longer strongly dependent on the single-point processing capability of TSO service, and even other types of consistency algorithms such as HLC are used. Such a scheme reduces the universality and scalability of TSO service, couples it with specific business use scenarios, cannot adapt to different use scenarios well, and also increases the application development and technical maintenance cost.

[0057] Therefore, under the premise of not replacing the TSO time model, how to improve the processing capability of TSO time service is a problem to be solved.

[0058] The data processing method, device and time service system provided by the present application aim to solve the above technical problems of the prior art.

[0059] The technical solutions of the embodiments of the present application and the technical effects generated by the technical solutions of the present application are described below through the description of several exemplary embodiments. It should be noted that the following embodiments can be mutually referenced, borrowed or combined. For the same terms, similar features and similar implementation steps in different embodiments, they are not described repeatedly.

[0060] Figure 2 The system architecture diagram of the time service system provided by the embodiments of the present application is shown in Figure 2 The system includes a master node 201, a plurality of client nodes 202 and a proxy node set. The proxy node set includes at least one proxy node 203. Each proxy node in the proxy node set is in communication connection with the master node, and each proxy node is in communication connection with at least one corresponding client node.

[0061] The plurality of client nodes can send a plurality of time service requests to the proxy node set. After receiving the plurality of time service requests, the proxy node set merges the plurality of time service requests to generate at least one time service request set, and forwards the at least one time service request set to the master node. The master node processes each received time service request set to obtain at least one corresponding response result set, and sends the at least one response result set to the proxy node set. After receiving the at least one response result set, the proxy node set returns the plurality of response results in the at least one response result set to the corresponding client nodes.

[0062] The proxy node, the master node and the client node can be a server or a terminal. The server can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., which is not limited by the present application.

[0063] It should be noted that Figure 2 The number of proxy nodes in the proxy node set, the number of client nodes, and the connection mode of the proxy node set and the master node, and the connection mode of the proxy node set and the plurality of client nodes in the system architecture provided by the embodiments of the present application are not limited.

[0064] Figure 3 The flowchart of the data processing method provided by the embodiments of the present application is shown in Figure 3 The method includes the following steps:

[0065] Step S101, receiving a plurality of time service requests sent by a plurality of client nodes.

[0066] Specifically, the set of proxy nodes can be a set including at least one proxy node, and the proxy node can be a node that processes on behalf of the master node. The proxy node can be a worker, specifically a terminal or a server.

[0067] The set of proxy nodes receives a plurality of time service requests from a plurality of clients. The time service request can be a request for time service for an event. In the embodiment of the application, the plurality of time service requests sent by the plurality of clients are calculated and processed by a master node, that is, a single-point service attribute, thereby ensuring linear consistency of the time service.

[0068] The client can send at least one time service request to one proxy node, the client can send at least two time service requests to at least two proxy nodes respectively, and at least two clients can send at least two time service requests to one proxy node. The embodiment of the application does not limit the distribution mode of the time service request.

[0069] In step S102, the plurality of time service requests are merged to generate at least one time service request set, and the at least one time service request set is forwarded to the master node for processing each time service request set to obtain at least one corresponding response result set.

[0070] Specifically, for one of the proxy nodes in the set of proxy nodes, after receiving the plurality of time service requests sent by the client, the proxy node can merge the plurality of time service requests to generate at least one time service request set. The time service request set can be a set including at least two time service requests.

[0071] The proxy node can merge the plurality of time service requests into one time service request set, or can merge the plurality of time service requests into at least two time service request sets. The embodiment of the application does not limit this.

[0072] Then, the proxy node forwards the at least one time service request set generated by merging to the master node. After receiving the at least one time service request set, the master node processes each time service request set to obtain a response result set corresponding to each time service request set. The response result set can be a set including a plurality of response results, and one response result corresponds to one time service request.

[0073] Optionally, the processing process of the master node can be calculating the timestamp of the event corresponding to each time service request in the time service request set, and taking the obtained timestamp as the response result corresponding to the request to form the response result set. The calculated timestamp can be applied in a database, for example, transaction sequencing, MVCC, etc.

[0074] It should be noted that the above process is performed by at least part of the proxy node set, that is, all proxy nodes in the proxy node set can perform the above operation, or part of the proxy nodes in the proxy node set perform the above operation, and the embodiments of the present application do not limit this.

[0075] In the embodiments of the present application, the proxy node set forwards multiple time service requests to the master node, that is, the proxy node is used as a relay between the client and the master node, which saves a lot of connection switching overhead in frequent network request transmission, and the pressure of the master node for processing request connections is shared by the proxy node set, the number of request connections of the master node is reduced, the resources of the CPU (Central Processing Unit) of the master node for processing connection switching are reduced, so that the master node has sufficient CPU resources to process time service, thereby improving the processing capacity and efficiency of the time service, and the time service object can adapt to large-scale and / or high-load.

[0076] For example, when the number of clients is 20 and the number of proxy nodes is 4, the master node only needs to communicate with the 4 proxy nodes, such as HTTP long connection, RPC Stream connection, etc., without maintaining connection with each of the 20 clients, which greatly reduces the number of request connections of the master node.

[0077] Further, the proxy node set merges multiple time service requests to generate at least one time service request set, and forwards the at least one time service request set to the master node, so that the number of time service request sets obtained is much smaller than the number of time service requests, thereby further reducing the number of request connections that the master node needs to process.

[0078] Step S103, receiving at least one response result set returned by the master node, and returning multiple response results in the at least one response result set to corresponding clients respectively.

[0079] Specifically, after the master node generates at least one response result set, the master node returns the at least one response result set to the proxy node set. For each proxy node in the proxy node set, after the proxy node receives the at least one response result set, for each response result set, the proxy node decomposes multiple response results in the response result set, and returns the multiple response results to corresponding clients respectively.

[0080] For each response result, the identifier of the client corresponding to the response result can be determined according to the identifier of the response result, the client corresponding to the response result can be determined according to the identifier of the client, and then the response result is returned to the client.

[0081] The method provided in this application forwards multiple time synchronization service requests to the master node through a proxy node set. By distributing the pressure on the master node to handle request connections with the help of the proxy node set, the number of request connections on the master node is reduced, and the CPU resources of the master node used to handle connection switching are reduced. This allows the master node to have sufficient CPU resources to handle the time synchronization service. Without changing the consistency model or upper-layer applications, the processing capacity and efficiency of the time synchronization service are improved, and it can adapt to large-scale and / or high-load time synchronization service objects.

[0082] Moreover, the embodiments of this application improve the internal mechanism of the time synchronization service without changing the consistent semantics and specific interfaces provided to the outside world, thus preserving the universality and scalability of the TSO service. It can adapt well to different use cases without requiring additional development work for different use cases, which greatly reduces migration and development costs.

[0083] Furthermore, by merging and forwarding multiple time synchronization service requests through a set of proxy nodes, the number of time synchronization service request sets is much smaller than the number of time synchronization service requests, thereby further reducing the number of request connections for the master node.

[0084] Furthermore, vertically scaling the CPU of a single proxy node or master node can improve the processing capacity for forwarding and computing requests in the time synchronization service; horizontally scaling the number of proxy nodes can increase the overall number of request connections that the time synchronization service can handle, thereby improving the overall throughput of the time synchronization service system.

[0085] As an optional implementation, the proxy node can be a disaster recovery node. A disaster recovery node refers to a node in the system used for disaster recovery and backup. Disaster recovery nodes are typically used to provide sufficient protection for the system in the face of disasters caused by various uncertainties, enabling the system to resume normal operation as quickly as possible. Therefore, disaster recovery nodes are idle for most of the system's normal operating time, and their CPU utilization is at a low level. By using the disaster recovery node as a proxy node, the CPU utilization of the disaster recovery node is improved, making full use of idle resources in the system, thereby improving the overall resource utilization of the system.

[0086] As an optional embodiment, step S102 merges multiple timing service requests to generate at least one timing service request set, and forwards the at least one timing service request set to the master node, including:

[0087] According to the reception time of each time synchronization service request, multiple time synchronization service requests within a preset time interval are merged to obtain a time synchronization service request set corresponding to a time interval.

[0088] Forward at least one set of time synchronization service requests generated sequentially to the master node.

[0089] Specifically, in order to ensure linear consistency in time service, the proxy node set can perform ordered merging when merging multiple time service requests. For a proxy node in the proxy node set, the proxy node can merge multiple time service requests received in a preset time interval into a time service request set according to the reception time of each received time service request, to obtain a time interval corresponding time service request set; and forward the generated at least one time service request set to the master node in order of reception time. Here, the time service request set can be sent in streaming, that is, the time service request set is sent immediately after being generated.

[0090] The preset time interval can be set according to the specific application scenario, and the preset time interval can be in the order of milliseconds, for example, 2 milliseconds. Different preset time intervals can be set for different proxy nodes; the sending order of the time service request set can be determined based on the reception time of the time service request in the time service request set, or based on the generation time of the time service request, which is not limited in the embodiments of the present application.

[0091] It should be noted that, considering the influence of network transmission and other factors, the number of time service requests received in each time interval can be different, for example, when the network transmission speed is fast, more time service requests are received in a time interval; when the network transmission speed is slow, fewer time service requests are received in a time interval. Therefore, the number of time service requests included in each time service request set is also different.

[0092] Figure 4 The schematic diagram of the request merging process provided by the embodiments of the present application is shown in Figure 4 As shown, multiple clients continuously send time service requests to the proxy node set, the preset time interval is set to 2 milliseconds, for a proxy node, the proxy node merges 2 time service requests received between 10:20:30:456 (representing 10:20:30:456 milliseconds) and 10:20:30:458 into a time service request set, and immediately forwards the time service request set to the master node; at the same time, the proxy node merges 4 time service requests received between 10:20:30:460 and 10:20:30:462 into a time service request set, and immediately forwards the time service request set to the master node.

[0093] In the embodiments of the present application, by real-time merging and forwarding of multiple time service requests, the number of request connections of the master node and the number of request connections that need to be maintained by the entire time service system are further reduced on the basis of ensuring linear consistency in time service.

[0094] The logical clock order and the physical clock order of the time service request may not be consistent. For example, the logical clock order of the read data request is before the write data request, but when affected by the network and other factors, the response result of the write data request is received first and the response result of the read data request is received later, that is, the problem of out-of-order occurs.

[0095] To solve the above problem, as an optional embodiment, the step of forwarding the generated at least one time service request set to the master node in sequence comprises:

[0096] After receiving the response result set corresponding to the previous time service request set, the current time service request set is forwarded to the master node.

[0097] Specifically, to avoid the problem of out-of-order, the proxy node set can use a synchronous blocking sending mode when forwarding the at least one time service request set, that is, the current request is not sent until the return result of the previous request is not received.

[0098] For a proxy node, the received at least one time service request set is sorted according to the time sequence. The time service request set can be sorted according to the receiving time of the time service request with the earliest receiving time in the time service request set, or sorted according to the generation time of the time service request set.

[0099] The time service request before the current time service request set in time is regarded as the previous time service request set. The proxy node receives the response result set corresponding to the previous time service request set, and then forwards the current time service request set to the master node.

[0100] The time service request set is generated based on the receiving time sequence of the time service request, so that the time sequence of the generation of the time service request set can be consistent with the logical sequence of the time service request. Through the synchronous blocking sending mode, the time sequence of the return of the response result corresponding to the time service request set can be consistent with the logical sequence, avoiding the out-of-order problem caused by the network and other factors, and ensuring the strict linear consistency of the time service.

[0101] It should be noted that the synchronous blocking sending mode in the embodiments of the present application can be applied not only to the communication process between the proxy node and the master node, but also to the communication process between the client and the proxy node.

[0102] As an optional embodiment, the step S103 of returning the multiple response results in the at least one response result set to the corresponding client respectively comprises:

[0103] For each response result set, the response result set is decomposed to obtain a plurality of response results included in the response result set;

[0104] The time sequence of the plurality of response results is determined;

[0105] According to the time sequence, the plurality of response results are returned to the corresponding client in turn.

[0106] Specifically, for a response result set, the response result set is decomposed to obtain a plurality of response results included in the response result set. Considering the influence of network and other factors, the time sequence of the plurality of response results returned may not be consistent with the logical sequence of the plurality of response results, for example, a response result returned later in the logical sequence. Therefore, the time sequence of the plurality of response results needs to be determined, and here, the time sequence of each response result can be determined according to the receiving time of the time service request corresponding to each response result.

[0107] For example, the plurality of time service requests can be sequentially numbered according to the receiving time of the plurality of time service requests, so that the serial number sequence of the plurality of time service requests is consistent with the receiving time sequence. The receiving time sequence of the time service request is usually consistent with the logical sequence, so the serial number sequence of the plurality of time service requests is also consistent with the logical sequence. For a time service request, the serial number corresponding to the request can be bound to the request as an attribute, and the response result corresponding to the request is also bound. According to the size sequence of the serial numbers corresponding to the plurality of response results, the time sequence of the plurality of response results can be determined.

[0108] Then, the plurality of response results are returned to the corresponding client in turn according to the time sequence, so that the client that sends the request first can receive the returned response result first, thereby ensuring the linear consistency in time service.

[0109] As an optional embodiment, step S101 receives a plurality of time service requests sent by a plurality of clients, including:

[0110] Through each proxy node, a preset number of time service requests sent by at least one corresponding client are received.

[0111] Specifically, to realize load balancing of multiple proxy nodes in the proxy node cluster, multiple clients can send multiple time service requests according to a preset distribution rule. The preset distribution rule can be a random distribution algorithm, for example, a reservoir algorithm can be used to randomly and fairly distribute the requests among all proxy nodes, or a normal distribution or Zipf distribution can be used to non-equally distribute the requests among all proxy nodes, or each proxy node can be assigned a corresponding weight and the distribution can be performed according to the corresponding weight of each proxy node. The corresponding weight of each proxy node can be determined based on the load level of each proxy node, that is, the higher the load level, the greater the load weight; the lower the load level, the smaller the load weight. The distribution rule can be set adaptively according to system configuration and / or application scenario, or can be designed by a user according to specific scenarios or requirements, and the embodiments of the present application do not limit this.

[0112] It should be noted that multiple clients can follow the same distribution rule, or multiple clients can follow different distribution rules respectively, and the embodiments of the present application do not limit this.

[0113] For each client, the client can determine at least one corresponding proxy node and the number of requests forwarded by each proxy node in the at least one proxy node according to the preset distribution rule.

[0114] For each proxy node, the proxy node can receive a preset number of time service requests sent by at least one corresponding client. The preset number can be the sum of the number of time service requests sent by each client in the at least one corresponding client to the proxy node, which is determined by the distribution rule.

[0115] Figure 5 A schematic diagram of a request distribution process provided by the embodiments of the present application is shown in FIG. 1, which shows that the system includes a client A and a client B, and a proxy node a and a proxy node b. The preset distribution rule is to distribute the requests to the proxy node a and the proxy node b in a ratio of 2:1, and the client A and the client B both follow the above distribution rule. Figure 5

[0116] When the number of time service requests sent by the client A is 60, 40 time service requests are sent to the proxy node a and 20 time service requests are sent to the proxy node b; when the number of time service requests sent by the client B is 30, 20 time service requests are sent to the proxy node a and 10 time service requests are sent to the proxy node b.

[0117] ​For the proxy node a, the clients corresponding thereto include client A and client B, and the preset quantity is 60; for the proxy node b, the clients corresponding thereto include client A and client B, and the preset quantity is 30.

[0118] In the embodiments of the present application, the preset distribution rule is set to determine the quantity of the timing service requests received by each proxy node, thereby realizing the load balancing of the plurality of proxy nodes in the proxy node set; different distribution rules are adaptively set according to different application scenarios and / or requirements, thereby being applicable to different application scenarios and meeting different requirements.

[0119] As an optional embodiment, the preset quantity is determined based on the configuration situation and / or real-time running state of the proxy node.

[0120] Specifically, for each proxy node, the preset quantity of the timing service requests received by the proxy node can be determined according to at least one of the configuration situation and the real-time running state of the proxy node.

[0121] The configuration situation can include information such as hardware, software and network configuration of the proxy node, and the real-time running state can include dynamic information actually running by the proxy node, and the real-time running state can be represented by a parameter reflecting the real-time running of the proxy node, such as the usage rate of the CPU of the proxy node.

[0122] Optionally, for each client, the client can determine the forwarding weight of the proxy node according to the configuration situation of the proxy node. The higher the configuration of the proxy node, the stronger the load capacity of the proxy node, and the more the quantity of the requests that can be forwarded. A higher forwarding weight can be set for the proxy node with high configuration, and a lower forwarding weight can be set for the proxy node with low configuration.

[0123] By setting the forwarding weight of the proxy node according to the configuration situation of the proxy node, different forwarding weights are adaptively set for different load levels of different proxy nodes, so that the proxy node with high configuration is responsible for a larger quantity of forwarding, and the proxy node with low configuration is responsible for a smaller quantity of forwarding, thereby reducing the processing pressure of the proxy node with low configuration, fully utilizing the processing capacity of the proxy node with high configuration, and improving the resource utilization rate of the entire proxy node set.

[0124] Optionally, for each client, the client can determine the forwarding weight of the proxy node according to the real-time running state of the proxy node. When the actual running load of the proxy node is large, the idle resource of the proxy node is small, and the quantity of the requests that can be processed and forwarded is small. A lower forwarding weight can be set for the proxy node with high load running, and a higher forwarding weight can be set for the proxy node with low load running.

[0125] The forwarding weight of the proxy node is set according to the real-time running state of the proxy node, so that the forwarding weight of the proxy node can be dynamically adjusted according to the actual running state of the proxy node, and dynamic load balancing of the proxy node is realized.

[0126] Optionally, for each client, the client can determine the forwarding weight of the proxy node in combination with the configuration condition and the real-time running state of the proxy node. The configuration condition and the real-time running state of the proxy node can be respectively set with corresponding weights, the load level of the proxy node is obtained by weighting the configuration condition and the real-time running state of the proxy node, and then the forwarding weight of the proxy node is determined according to the load level of the proxy node.

[0127] The forwarding weight of the proxy node is determined in combination with the configuration condition and the real-time running state of the proxy node, the static characteristics such as the hardware and software configuration of the proxy node and the dynamic characteristics of the real-time running state are comprehensively considered, so that the calculated forwarding weight can more truly reflect the processing capability of the proxy node.

[0128] After determining the forwarding weight of each proxy node, the preset number can be determined according to the forwarding weight of each proxy node and the total number of the multiple time service requests sent by the multiple clients. For example, the product of the forwarding weight of each proxy node and the total number is taken as the preset number.

[0129] As an optional embodiment, before receiving the preset number of time service requests sent by the corresponding at least one client, the method further comprises:

[0130] The real-time running state of the at least one proxy node is sent to the at least one client, so that the at least one client determines the forwarding weight of each corresponding proxy node based on the real-time running state, and determines the preset number based on the forwarding weight of each corresponding proxy node.

[0131] Specifically, for each proxy node, the real-time running state of the proxy node can be fed back in real time by the proxy node. Optionally, the proxy node can feed back the real-time running state to the corresponding client separately, for example, a feedback time interval can be preset, and feedback is performed once every time interval; the proxy node can also return the real-time running state and the response result to the corresponding client together, and the embodiments of the present application do not limit this.

[0132] The client corresponding to the proxy node can determine the forwarding weight of each corresponding proxy node based on the real-time running state after receiving the real-time running state, and determine the preset number based on the forwarding weight of each corresponding proxy node. The above process can be referred to the corresponding description in the foregoing, and the embodiments of the present application will not be repeated here.

[0133] In the embodiments of the present application, the forwarding weight of the proxy node is dynamically adjusted through the real-time running state of real-time feedback, and the dynamic changes of the actual running conditions of each proxy node in the proxy node cluster are fully considered, so that the dynamic load balancing of each proxy node is realized.

[0134] As an optional embodiment, Figure 6 A flowchart of a data processing method provided by the embodiments of the present application is shown in the figure. The data processing method provided by the embodiments of the present application is applied to a TSO time service system, which includes a master node, multiple client nodes and multiple backup nodes. In the system, if the TSO Proxy is turned on, the client node sends a TSO request to the backup node; if the TSO Proxy is not turned on, the client node directly sends a TSO request to the master node.

[0135] As shown in the figure, the method includes the following steps: Figure 6

[0136] Step S201, the client node sends multiple TSO requests to the backup node according to a preset random distribution algorithm;

[0137] Step S202, the backup node continuously receives the TSO requests sent by the client node, and combines multiple TSO requests received within a preset time interval into a TSO request set through a queue channel;

[0138] Step S203, the backup node sends the TSO request set generated by combination to the master node in a streaming manner, that is, the TSO request set is sent immediately after the TSO request set is generated;

[0139] Step S204, the master node processes the received TSO request set to obtain a corresponding response result set;

[0140] Step S205, the master node sends the response result set to the backup node;

[0141] Step S206, after receiving the response result set, the backup node decomposes the response result set to obtain multiple response results, and sequentially numbers the multiple response results according to the receiving time of the TSO requests corresponding to the multiple response results;

[0142] Step S207, the backup node returns the multiple response results to the corresponding client nodes in sequence according to the sequence number of the multiple response results.

[0143] ​In the embodiments of the present application, the internal mechanism of the distributed TSO service is improved without changing the consistency semantics and specific interfaces provided externally, so that the optimization is more universal, the migration and use cost is extremely low, and almost no user matching is required for any business modification.

[0144] The method provided by the embodiments of the present application fully considers the basic properties, use characteristics and actual scene optimization requirements of the distributed database time service, and proposes a real-time forwarding and merging, request quantity optimization and rich load balancing mechanism to fully utilize the characteristics of the time service and the characteristics of the use scene. Through forwarding and merging, the TSO service cluster can have higher resource utilization, fully utilize the once-idle redundant backup node CPU, and enable the TSO service to have stronger vertical and horizontal expansion capabilities: the CPU of a single backup node or master node can be vertically expanded to improve the processing performance of TSO forwarding and calculation, and the number of backup nodes can be horizontally expanded to improve the overall request connection quantity that can be carried by the TSO service and improve the throughput. The introduction of the merging algorithm further reduces the required request connection quantity within the cluster, and does not damage the consistency of the TSO itself; the rich load balancing mechanism greatly reduces the deployment requirements of the TSO service, and can help users reduce costs. By setting different random distribution algorithms, requests can be sent to different backup nodes according to a specific random distribution and then forwarded to the master node. In complex scenarios such as heterogeneous deployment, the forwarding server resource allocation can be flexibly arranged according to the requirements, so as to more efficiently meet the server resource expansion requirements.

[0145] Figure 7 A structural schematic diagram of a data processing device provided by the embodiments of the present application is shown in FIG. 1, and the device of the embodiments can include: Figure 7

[0146] The receiving module 301 is configured to receive a plurality of time service requests sent by a plurality of clients.

[0147] The merging and forwarding module 302 is configured to merge the plurality of time service requests, generate at least one time service request set, and forward the at least one time service request set to a master node, so that the master node processes each time service request set to obtain at least one corresponding response result set.

[0148] The returning module 303 is configured to receive the at least one response result set returned by the master node, and return a plurality of response results in the at least one response result set to corresponding clients respectively.

[0149] As an optional embodiment, the merging and forwarding module 302 includes:

[0150] ​The merging sub-module is configured to merge a plurality of timing service requests in a preset time interval according to a receiving time of each timing service request, and obtain a timing service request set corresponding to the time interval;

[0151] The forwarding sub-module is configured to sequentially forward the generated at least one timing service request set to the master node.

[0152] As an optional embodiment, the forwarding sub-module is configured to:

[0153] forward the current timing service request set to the master node after receiving a response result set corresponding to a previous timing service request set.

[0154] As an optional embodiment, the returning module 303 is configured to:

[0155] decompose each response result set to obtain a plurality of response results included in the response result set;

[0156] determine a time sequence of the plurality of response results;

[0157] sequentially return the plurality of response results to corresponding clients according to the time sequence.

[0158] As an optional embodiment, the receiving module 301 is configured to:

[0159] receive, by each proxy node, a preset number of timing service requests sent by a corresponding at least one client.

[0160] As an optional embodiment, the preset number in the device is determined based on a configuration condition and / or a real-time running state of the proxy node.

[0161] As an optional embodiment, the device further comprises:

[0162] a feedback module configured to send a real-time running state of the at least one proxy node to the at least one client, so that the at least one client determines a forwarding weight of each corresponding proxy node based on the real-time running state, and determines the preset number based on the forwarding weight of each corresponding proxy node.

[0163] The device of the embodiments of the present application can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the device of the embodiments of the present application are corresponding to the steps in the method of the embodiments of the present application. For the detailed function description of each module of the device, refer to the description of the corresponding method in the foregoing description, which will not be repeated here.

[0164] Compared with the related art, the electronic device provided in the embodiments of the present application can forward multiple time service requests to a master node through a set of proxy nodes, share the pressure of processing request connections of the master node by means of the set of proxy nodes, reduce the number of request connections of the master node, reduce the resources of the CPU of the master node for processing connection switching, enable the master node to have sufficient CPU resources to process time service, improve the processing capacity and efficiency of the time service without changing the consistency model or upper-layer application, and adapt to large-scale and / or high-load time service objects. Meanwhile, without changing the consistency semantics and specific interfaces provided externally, the TSO service is kept universal and scalable, can be well adapted to different use scenarios, does not need additional development work for different use scenarios, greatly reduces the migration and development cost. Further, the set of time service requests is far smaller than the number of time service requests, so that the number of request connections of the master node is further reduced. In addition, the processing capacity of time service request forwarding and calculation can be improved by vertically expanding the CPU of a single proxy node or master node, and the overall bearable request connection quantity of the time service can be improved by horizontally expanding the number of proxy nodes, thereby improving the overall throughput of the time service system.

[0165] In an optional embodiment, an electronic device is provided, as shown in Figure 8 Figure 8 The electronic device 4000 shown in the figure includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, through a bus 4002. Optionally, the electronic device 4000 can also include a transceiver 4004, which can be used for data interaction, such as data sending and / or data receiving, between the electronic device and other electronic devices. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0166] ​The processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in connection with the disclosure. The processor 4001 can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0167] The bus 4002 can include a path for transmitting information between the above-mentioned components. The bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 8 Only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0168] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, an optical disk storage (including a compact disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium, other magnetic storage device, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation.

[0169] The memory 4003 is configured to store a computer program for implementing the embodiments of the present application, and the processor 4001 is configured to control the execution of the computer program stored in the memory 4003. The processor 4001 is configured to execute the computer program stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0170] The embodiments of the present application provide a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps and corresponding contents of the foregoing method embodiments.

[0171] The terms "first", "second", "third", "fourth", "1", "2", and the like (if any) in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described.

[0172] It should be understood that, although the flowcharts of the embodiments of the present application indicate the respective operation steps by arrows, the implementation order of the steps is not limited to the order indicated by the arrows. Unless otherwise specified herein, in some implementation scenarios of the embodiments of the present application, the implementation steps in each flowchart can be executed in other orders as required. In addition, part or all of the steps in each flowchart can include multiple sub-steps or multiple stages based on the actual implementation scenario. Part or all of these sub-steps or stages can be executed at the same time, and each of these sub-steps or stages can also be executed at different times. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of the present application do not limit this.

[0173] The above is only an optional implementation of some implementation scenarios of the present application. It should be pointed out that, for those skilled in the art, other similar implementation means based on the technical concept of the present application can also be adopted without departing from the technical concept of the present application, and such implementation means also belong to the protection scope of the embodiments of the present application.

Claims

1. A data processing method, characterized in that, Applied to a set of proxy nodes, the set of proxy nodes including at least two proxy nodes, including: Receive multiple time synchronization service requests from multiple clients; The multiple time synchronization service requests are merged to generate at least one time synchronization service request set, and the at least one time synchronization service request set is forwarded to the master node using a synchronous blocking sending method. The master node calculates the timestamp of each event corresponding to each time synchronization service request in the time synchronization service request set, and uses the timestamp of each time synchronization service request as the response result set corresponding to the time synchronization service request set. Receive the at least one response result set returned by the master node, and return multiple response results in the at least one response result set to the corresponding client respectively; The step of merging the multiple time synchronization service requests to generate at least one time synchronization service request set, and forwarding the at least one time synchronization service request set to the master node using a synchronous blocking transmission method, includes: At each preset time interval, according to the receiving time of each time synchronization service request, multiple time synchronization service requests within the preset time interval are merged to obtain a time synchronization service request set corresponding to a time interval. The generation time order of each time synchronization service request set is determined; the generation time order of each time synchronization service request set is consistent with the logical order of each time synchronization service request set. For each time synchronization service request set, based on the time order in which each time synchronization service request set is generated, the time synchronization service request set preceding this time synchronization service request set is taken as the previous time synchronization service request set; after forwarding the previous time synchronization service request set to the master node, and after receiving the response result set corresponding to the previous time synchronization service request set, the time synchronization service request set is forwarded to the master node. If no response result set corresponding to the previous timing service request set is received, then the timing service request set will not be forwarded to the master node.

2. The data processing method according to claim 1, characterized in that, The step of returning multiple response results from the at least one response result set to their respective clients includes: For each response result set, the response result set is decomposed to obtain multiple response results included in the response result set; Determine the temporal order of the multiple response results; The multiple response results are returned to the corresponding clients in chronological order.

3. The data processing method according to claim 1 or 2, characterized in that, The receiving of multiple time synchronization service requests from multiple clients includes: Each proxy node receives a preset number of time synchronization service requests from at least one corresponding client.

4. The data processing method according to claim 3, characterized in that, The preset number is determined based on the configuration and / or real-time operating status of the agent nodes.

5. The data processing method according to claim 3, characterized in that, Before receiving a preset number of time synchronization service requests sent by at least one corresponding client, the method further includes: The real-time operating status of the at least one proxy node is sent to the at least one client, so that the at least one client can determine the forwarding weight of each proxy node based on the real-time operating status, and determine the preset quantity based on the forwarding weight of each proxy node.

6. A time synchronization service system, characterized in that, It includes a master node, multiple clients, and a set of proxy nodes, wherein the set of proxy nodes includes at least two proxy nodes; Each of the proxy nodes in the proxy node set is communicatively connected to the master node, and each of the proxy nodes in the proxy node set is communicatively connected to at least one corresponding client. The proxy node set performs the data processing method as described in any one of claims 1 to 5.

7. A data processing apparatus, characterized in that, Applied to a set of proxy nodes, the set of proxy nodes including at least two proxy nodes, including: The receiving module is used to receive multiple time synchronization service requests sent by multiple clients; The merging and forwarding module is used to merge the multiple time synchronization service requests to generate at least one time synchronization service request set, and to forward the at least one time synchronization service request set to the master node using a synchronous blocking sending method. The master node can then calculate the timestamp of the event corresponding to each time synchronization service request in the set, and use the timestamp of each time synchronization service request as the response result set corresponding to the time synchronization service request set. The return module is used to receive the at least one response result set returned by the master node, and return multiple response results in the at least one response result set to the corresponding client respectively; When the merging and forwarding module merges the multiple time synchronization service requests to generate at least one time synchronization service request set, and forwards the at least one time synchronization service request set to the master node using a synchronous blocking sending method, it is specifically used for: At each preset time interval, according to the receiving time of each time synchronization service request, multiple time synchronization service requests within the preset time interval are merged to obtain a time synchronization service request set corresponding to a time interval. The generation time order of each time synchronization service request set is determined; the generation time order of each time synchronization service request set is consistent with the logical order of each time synchronization service request set. For each time synchronization service request set, based on the time order in which each time synchronization service request set is generated, the time synchronization service request set preceding this time synchronization service request set is taken as the previous time synchronization service request set; after forwarding the previous time synchronization service request set to the master node and receiving the response result set corresponding to the previous time synchronization service request set, the time synchronization service request set is forwarded to the master node. If no response result set corresponding to the previous timing service request set is received, then the timing service request set will not be forwarded to the master node.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images