Timestamp generation method and apparatus, and electronic device, and non-volatile storage medium

Abstract

Disclosed in the present application are a timestamp generation method and apparatus, and an electronic device and a non-volatile computer-readable storage medium. The method comprises: receiving a timestamp request transaction sent by a database client, and determining a target number of timestamp request transactions received within a time period corresponding to a target time window; on the basis of the target number, sending a timestamp acquisition request to a first timestamp processing node, wherein the timestamp acquisition request is used for acquiring the target number of timestamps, one first timestamp processing node corresponding to a plurality of second timestamp processing nodes; and receiving a target timestamp returned by the first timestamp processing node in response to the timestamp acquisition request, and sending the target timestamp to a corresponding database client, wherein the target timestamp is a timestamp in a target time window that is acquired by the first timestamp processing node in a memory.

Description

Timestamp generation method, device, electronic device and non-volatile storage medium

[0001] Related applications

[0002] This application claims priority to Chinese patent application number 202311609299.6, filed on November 28, 2023, entitled “Timestamp Generation Method, Device, Electronic Device and Non-volatile Storage Medium,” the entire text of which is hereby incorporated by reference. Technical Field

[0003] The present application relates to the technical field of big data storage and processing, and in particular to a timestamp generation method, device, electronic device, and non-volatile computer-readable storage medium. Background Art

[0004] Timestamp Order (TSO) is crucial for achieving global consistency in distributed databases. It can help store and distinguish snapshots of different distributed databases, coordinate the order of transaction submissions between different host nodes, and maintain time-based data visibility in distributed databases.

[0005] However, in response to the massive connection requests in big data scenarios, the single TSO service in related technologies is limited by performance bottlenecks and can no longer meet the needs of millions of connections and high concurrency requirements.

[0006] To address the above-mentioned problems, no effective solutions have been proposed so far.

[0007] Summary of the Invention

[0008] The embodiments of the present application provide a timestamp generation method, device, electronic device, and non-volatile computer-readable storage medium to at least solve the technical problem that related technologies cannot meet the needs of millions of connections and high concurrency requirements for big data cluster application scenarios when obtaining timestamps.

[0009] In a first aspect, the present application provides a timestamp generation method, comprising: receiving a timestamp request transaction sent by a database client, and determining a target number of timestamp request transactions received within a time period corresponding to a target time window; sending a timestamp acquisition request to a first timestamp processing node based on the target number, the timestamp acquisition request being used to acquire a target number of timestamps, one first timestamp processing node corresponding to multiple second timestamp processing nodes; receiving a target timestamp returned by the first timestamp processing node in response to the timestamp acquisition request, and sending the target timestamp to the corresponding database client, the target timestamp being the timestamp within the target time window acquired by the first timestamp processing node in memory.

[0010] In some embodiments, the target timestamp includes: a physical clock part and a logical clock part. The physical clock part is used to represent the world time corresponding to the operation in the database system and the real world. The logical clock part is a monotonically increasing integer value with a preset number of bits, which is used to represent the sequence of various operations in the database system.

[0011] In some embodiments, the target time window is a time period of a preset window length immediately before the target moment, wherein the target moment is the moment corresponding to the maximum value of the physical clock part of the generated target timestamp stored in the memory within the target time window.

[0012] In some of these embodiments, when the current time is not less than the target moment stored in the consistent key-value storage system, the first timestamp processing node is used to determine the moment that is after the target moment and within a preset window length from the target moment as the new target moment, and update the new target moment to the consistent key-value storage system, and generate a target timestamp within a time period of a preset window length immediately before the new target moment and store it in the memory.

[0013] In some embodiments, the target timestamp stored in the memory is used to be directly assigned by the first timestamp processing node to the second timestamp processing node when the first timestamp processing node receives a timestamp acquisition request sent by the second timestamp processing node.

[0014] In some embodiments, each second timestamp processing node corresponds to multiple database clients; receiving a timestamp request transaction sent by a database client includes: determining a hash value corresponding to a client identifier of the database client; determining the second timestamp processing node corresponding to the database client based on the hash value and the number of second timestamp processing nodes, and receiving the timestamp request transaction sent by the database client by the second timestamp processing node corresponding to the database client.

[0015] In some embodiments, the first timestamp processing node includes: a leader node and a follower node; the leader node is used to receive and process the timestamp acquisition request sent by the second timestamp processing node, and the follower node is used to synchronize the data information of the leader node, and in the event of an abnormality in the leader node, replace the leader node to receive and process the timestamp acquisition request sent by the second timestamp processing node.

[0016] In a second aspect, the present application also provides a timestamp generation device, including: a client request receiving module, used to receive a timestamp request transaction sent by a database client, and determine a target number of timestamp request transactions received within a time period corresponding to a target time window; a timestamp request module, used to send a timestamp acquisition request to a first timestamp processing node based on the target number, the timestamp acquisition request being used to obtain a target number of timestamps, one first timestamp processing node corresponding to multiple second timestamp processing nodes; a request response module, used to receive a target timestamp returned by the first timestamp processing node in response to the timestamp acquisition request, and send the target timestamp to the corresponding database client, the target timestamp being the timestamp within the target time window obtained by the first timestamp processing node in memory.

[0017] In some embodiments, the target timestamp includes: a physical clock part and a logical clock part. The physical clock part is used to represent the world time corresponding to the operation in the database system and the real world. The logical clock part is a monotonically increasing integer value with a preset number of bits, which is used to represent the sequence of various operations in the database system.

[0018] In some embodiments, the target time window is a time period of a preset window length immediately before the target moment, and the target moment is the moment corresponding to the maximum value of the physical clock part of the generated target timestamp stored in the memory within the target time window.

[0019] In some of these embodiments, when the current time is not less than the target moment stored in the consistent key-value storage system, the first timestamp processing node is used to determine the moment that is after the target moment and within a preset window length from the target moment as the new target moment, and update the new target moment to the consistent key-value storage system, and generate a target timestamp within a time period of a preset window length immediately before the new target moment and store it in the memory.

[0020] In some embodiments, the target timestamp stored in the memory is used to be directly assigned by the first timestamp processing node to the second timestamp processing node when the first timestamp processing node receives a timestamp acquisition request sent by the second timestamp processing node.

[0021] In some embodiments, each second timestamp processing node corresponds to a plurality of database clients, and the client request receiving module is further configured to:

[0022] Determine the hash value corresponding to the client identifier of the database client;

[0023] The second timestamp processing node corresponding to the database client is determined according to the hash value and the number of the second timestamp processing nodes, and the second timestamp processing node corresponding to the database client receives the timestamp request transaction sent by the database client.

[0024] In some embodiments, the first timestamp processing node includes: a leader node and a follower node; the leader node is used to receive and process the timestamp acquisition request sent by the second timestamp processing node, and the follower node is used to synchronize the data information of the leader node, and in the event of an abnormality in the leader node, replace the leader node to receive and process the timestamp acquisition request sent by the second timestamp processing node.

[0025] In a third aspect, the present application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program, the timestamp generation method in any one of the embodiments of the first aspect is implemented.

[0026] In a fourth aspect, the present application further provides a non-volatile computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes the timestamp generation method according to any one of the embodiments of the first aspect.

[0027] According to the technical solution provided by the embodiment of the present application, by receiving the timestamp request transaction sent by the database client and determining the target number of timestamp request transactions received within the time period corresponding to the target time window; based on the target number, sending a timestamp acquisition request to the first timestamp processing node, the timestamp acquisition request is used to obtain the target number of timestamps, and one first timestamp processing node corresponds to multiple second timestamp processing nodes; receiving the target timestamp returned by the first timestamp processing node in response to the timestamp acquisition request, and sending the target timestamp to the corresponding database client, the target timestamp is the timestamp within the target time window obtained in the memory by the first timestamp processing node. In this way, the purpose of multi-level TSO generation and acquisition in a million-level distributed database is achieved, and the technical problem that the related technology cannot meet the million-level connection requirements and high concurrency requirements for big data cluster application scenarios when acquiring timestamps is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technology descriptions. Obviously, the drawings described below are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be derived from these drawings without inventive effort.

[0029] FIG1 is a hardware structure block diagram of a computer device (or electronic terminal) for implementing a timestamp generation method according to an embodiment of the present application.

[0030] FIG2 is a flow chart of a timestamp generation method according to an embodiment of the present application.

[0031] FIG3 is a schematic diagram of the structure of a target timestamp provided according to an embodiment of the present application.

[0032] FIG4 is a schematic diagram of a multi-stage TSO synchronization method provided according to an embodiment of the present application.

[0033] FIG5 is a schematic structural diagram of a timestamp generation device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] To facilitate those skilled in the art to better understand the embodiments of the present application, some technical terms or nouns involved in the embodiments of the present application are explained as follows:

[0037] Timestamp Order (TSO): Used to mark the time point of each event in a distributed system, confirm the order of their occurrence, and ensure consistency.

[0038] SqlDB database: is a relational database that uses SQL language for data management and query.

[0039] KVDB database: It is a key-value database in which data is stored and queried in the form of key-value pairs.

[0040] TiKV database: is a distributed key-value store database used to store large-scale data and provide high availability and scalability.

[0041] PD cluster (Placement Driver cluster): Responsible for storing cluster metadata, load balancing, failover and other functions in a distributed database system.

[0042] etcd: is an open-source distributed key-value store for reliable data storage and access. It is commonly used in distributed systems for configuration management, service discovery, and storage sharing.

[0043] The consistent timestamp (TSO) of a distributed database is crucial to achieving global consistency of the database. Its main functions are as follows:

[0044] 1) TSO can help store and differentiate snapshots of different distributed databases (e.g., SQLDB or KVDB). This helps maintain historical data, protect data, and mitigate the impact of major management errors based on the different snapshots.

[0045] 2) Distributed databases use a two-phase commit method to maintain the atomicity of transactions. TSO can help coordinate the order of transaction submission between different host nodes, ensuring that the order of parallel transaction execution is exactly the same as the effect of serial execution of transactions according to timestamps, thereby reducing conflicts in distributed databases.

[0046] 3) TSO helps maintain time-based data visibility in distributed databases, which solves the problem of reading expired data or writing conflicts that may occur in distributed systems.

[0047] However, facing the massive number of connection requests in big data scenarios, the single TSO service in related technologies is limited by performance bottlenecks and can no longer meet the needs of millions of connections and high concurrency requirements. Taking the open source distributed database TiKV as an example, the current number of connections in the TiKV database is only about 100,000, but big data clusters require millions of connections. The current open source distributed database can no longer meet the connection needs of big data clusters. In addition, the PD cluster in TiKV can theoretically generate 262,144,000 globally consistent timestamps TSO per second, which cannot meet the concurrency requirement of 50 million per second. In actual use, the TSO that can be obtained is far less than the theoretical value, accompanied by a large delay time, which has a significant impact on actual production applications.

[0048] In order to solve the above problems, the technical solutions provided in this application will be described in detail through the following embodiments in conjunction with the accompanying drawings.

[0049] The timestamp generation method provided in the embodiments of the present application can be executed in a mobile terminal, a computer device or a similar computing device. Figure 1 shows a hardware structure block diagram of a computer device (or mobile terminal) for implementing the timestamp generation method. As shown in Figure 1, the computer device 10 (or mobile terminal) may include one or more (as shown in 102a, 102b, ..., 102n in Figure 1) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor (MCU) or a programmable logic device (FPGA)), a memory 104 for storing data, and a transmission device 106 for communication. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be used as a port of a bus (BUS)), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that the structure shown in Figure 1 is only illustrative and does not limit the structure of the above-mentioned electronic device. For example, the computer device 10 may also include more or fewer components than shown in Figure 1, or have a configuration different from that shown in Figure 1.

[0050] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer device 10 (or mobile terminal). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0051] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the timestamp generation method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned timestamp generation method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer device 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0053] The display may be, for example, a touch screen liquid crystal display (LCD), which may enable a user to interact with a user interface of the computer device 10 (or mobile terminal).

[0054] It should be noted that, in some alternative embodiments, the computer device shown in FIG1 may include hardware components (including circuits), software components (including computer code stored on a computer-readable medium), or a combination of hardware and software components. It should be noted that FIG1 is merely a specific embodiment, which is intended to illustrate the types of components that may be present in the computer device.

[0055] In the above operating environment, the embodiments of the present application provide a timestamp generation method. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed by a set of computer-executable instructions, such as in a computer system, and although the flowcharts show a logical order, in some cases, the steps shown or described can be executed in a different order than shown here.

[0056] FIG2 is a flowchart of a timestamp generation method according to an embodiment of the present application. As shown in FIG2 , the method includes the following steps S202 to S206 .

[0057] Step S202: receiving a timestamp request transaction sent by a database client, and determining a target number of timestamp request transactions received within a time period corresponding to a target time window;

[0058] In step S202, the target time window is a TSO allocation interval within a time range. The first timestamp processing node pre-appoints a target time window upon startup or periodic triggering. This time window starts at the current time and ends at the current time plus a fixed interval (i.e., the preset window duration), such as 3 seconds. The purpose of this target time window is to pre-allocate a target timestamp TSO within a time range, eliminating the need to generate a new TSO in real time for each timestamp request transaction.

[0059] Step S204: sending a timestamp acquisition request to the first timestamp processing node according to the target number, wherein the timestamp acquisition request is used to acquire the target number of timestamps, and one first timestamp processing node corresponds to multiple second timestamp processing nodes.

[0060] In the above step S204, the second timestamp processing node pulls a batch of consistent target timestamps from the first timestamp processing node according to the number of requests sent by the connected clients within the target time window limit (i.e., the above target number), and then distributes them to each database client for use, so as to reduce the overhead of remote procedure calls (RPCs).

[0061] Step S206: Receive the target timestamp returned by the first timestamp processing node in response to the timestamp acquisition request, and send the target timestamp to the corresponding database client, wherein the target timestamp is the timestamp within the target time window acquired by the first timestamp processing node in memory.

[0062] Through the above steps, by combining two layers of TSO to obtain a globally consistent timestamp and distributing different batches of TSO according to the number of secondary TSO nodes, multi-level TSO generation and acquisition in a million-level distributed database is achieved, which reduces the waiting time of TSO and greatly increases the number of requests that the database can respond to per second. This solves the technical problem that related technologies cannot meet the needs of millions of connections and high concurrency requirements for big data cluster application scenarios when obtaining timestamps.

[0063] The timestamp generation method in steps S202 to S206 is further described below.

[0064] The target timestamp generated in the embodiments of the present application includes a physical clock component and a logical clock component. Figure 3 is a schematic diagram of the structure of a target timestamp provided in accordance with an embodiment of the present application. As shown in Figure 3, the physical clock component is used to represent the real-world time corresponding to operations in the database system. The logical clock component is a monotonically increasing integer value with a preset number of bits, which is used to represent the sequence of operations in the database system.

[0065] For example, in this embodiment, the target timestamp is a monotonically increasing 64-bit integer value, consisting of a 32-bit physical clock portion and a 32-bit logical portion. The upper 32 bits of the target timestamp are the physical clock portion, which represents milliseconds in Unix time (Unix time is a time representation method that represents the number of seconds elapsed from 00:00:00 on January 1, 1970, to the present, and is commonly used in computer systems); the lower 32 bits are the logical clock portion, which is a numerical counter.

[0066] Combining the above two parts, we can get a complete target timestamp. Theoretically, this solution can generate 4294967296 (2 to the power of 32) consistent timestamps per millisecond and can ensure the monotonically increasing transaction version number.

[0067] To reduce access pressure on the database server, the present embodiment adds a buffer layer node, namely a second timestamp processing node, to the TSO service, forming a multi-layer TSO structure, thereby achieving multi-level TSO allocation for a million-level distributed database. The following is a further introduction to the multi-level TSO allocation process. Figure 4 is a schematic diagram of a multi-level TSO synchronization method provided according to an embodiment of the present application, as shown in Figure 4.

[0068] In order to solve the problem of the limited number of connections of the original TSO processing node, in the embodiment of the present application, a secondary TSO node (i.e., the above-mentioned second timestamp processing node) is added after the first-level TSO cluster (i.e., the above-mentioned first timestamp processing node) as a buffer layer. In this embodiment, each buffer layer node (second timestamp processing node) can connect to 50,000 to 100,000 database clients, and the total number of connectable clients is far more than the TSO services in other distributed databases in related technologies.

[0069] In addition, in order to achieve load balancing of the buffer layer nodes, the consistent hashing method can be used in this embodiment to select the corresponding buffer layer node (second timestamp processing node) for the database client to reduce the impact of the buffer layer node expansion and contraction on the specific client. The specific steps are as follows.

[0070] In some embodiments of the present application, each second timestamp processing node corresponds to multiple database clients; receiving a timestamp request transaction sent by a database client includes the following steps: determining a hash value corresponding to a client identifier of the database client; determining the second timestamp processing node corresponding to the database client based on the hash value and the number of second timestamp processing nodes, and receiving the timestamp request transaction sent by the database client by the second timestamp processing node corresponding to the database client.

[0071] The following further describes the process of allocating target timestamps in a multi-layer TSO structure.

[0072] First, the second timestamp processing node receives the timestamp request transactions sent by the database client, and determines the target number of timestamp request transactions received within the time period corresponding to the target time window. Then, based on the target number, a batch of consistent timestamps are uniformly pulled from the first timestamp processing node.

[0073] The first timestamp processing node obtains the last saved target time from the consistent key-value storage system (such as etcd). If the current time is less than the target time, the first timestamp node can directly use the target timestamps within the current target time window in memory and distribute them in batches to different second timestamp processing nodes for use, as follows.

[0074] In the timestamp generation method provided in an embodiment of the present application, the target timestamp stored in the memory is used to be directly assigned by the first timestamp processing node to the second timestamp processing node when the first timestamp processing node receives a timestamp acquisition request sent by the second timestamp processing node.

[0075] Specifically, the first timestamp processing node stores the target timestamps within the allocatable target time window in memory, so when the second timestamp processing node requests it, it can directly calculate and return the corresponding batch timestamp.

[0076] The target time window is a period of time of a preset window length immediately before the target time, wherein the target time is the time corresponding to the maximum value of the physical clock part of the generated target timestamp stored in the memory within the target time window.

[0077] In addition, the first timestamp node will update the target time window when the current time is not less than the last saved target time, and will also update the target timestamp in the target time window saved in the memory. The specific steps are as follows.

[0078] In the timestamp generation method provided in an embodiment of the present application, when the current time is not less than the target moment stored in the consistency key-value storage system, the first timestamp processing node is used to determine the moment that is after the target moment and within a preset window length from the target moment as the new target moment, and update the new target moment to the consistency key-value storage system, and generate a target timestamp within a time period of a preset window length immediately before the new target moment and store it in the memory.

[0079] For example, assuming the current target time is t1, and the maximum time window that can be applied for each time is tw (i.e., the preset window length mentioned above), the first timestamp processing node will save the time value of t1+tw to etcd, that is, the updated target time. It can then directly use the target timestamps in the target time window from t1 to t1+tw in memory and distribute them to different buffer layer nodes in batches. When the current time t2 is greater than the target time saved in etcd (i.e., t1+tw), the target time stored in etcd is updated to t2+tw or t1+tw+tw, and a batch of target timestamps in the target time window from t2 to t2+tw are updated and stored in memory.

[0080] As an optional implementation, in the timestamp generation method provided in the embodiment of the present application, the first timestamp processing node includes: a leader node (i.e., the TSO leader shown in FIG4 ) and a follower node (i.e., the TSO follower shown in FIG4 ). The leader node is used to receive and process the timestamp acquisition request sent by the second timestamp processing node. The follower node is used to synchronize the data information of the leader node, and in the event of an abnormality in the leader node, replaces the leader node to receive and process the timestamp acquisition request sent by the second timestamp processing node. This ensures the reliability and stability of the multi-level TSO architecture in this embodiment.

[0081] This application scheme implements a TSO engine design for a million-level distributed database, increases the number of bits in the TSO logic part, and increases the number of TSOs generated per second to tens of millions, meeting the response requirements of million-level production scenarios. At the same time, a multi-layer TSO buffer structure is proposed, including a first-level TSO cluster for generating TSOs and a second-level TSO buffer layer. The buffer layer has multiple machine nodes that can be elastically expanded and contracted, which expands the number of client connections, reduces the waiting time for TSO requests, and greatly speeds up the response to client requests for TSOs.

[0082] According to an embodiment of the present application, a timestamp generation device is also provided. FIG5 is a schematic diagram of the structure of a timestamp generation device provided according to an embodiment of the present application. As shown in FIG5 , the device includes:

[0083] The client request receiving module 502 is configured to receive a timestamp request transaction sent by a database client and determine a target number of timestamp request transactions received within a time period corresponding to a target time window.

[0084] The timestamp request module 504 is configured to send a timestamp acquisition request to the first timestamp processing node according to the target number, wherein the timestamp acquisition request is used to acquire the target number of timestamps, and one first timestamp processing node corresponds to multiple second timestamp processing nodes.

[0085] The request response module 506 is used to receive the target timestamp returned by the first timestamp processing node in response to the timestamp acquisition request, and send the target timestamp to the corresponding database client, wherein the target timestamp is the timestamp within the target time window obtained by the first timestamp processing node in the memory.

[0086] In some embodiments, the target timestamp may include: a physical clock part and a logical clock part, wherein the physical clock part is used to represent the world time corresponding to the operation in the database system and the real world, and the logical clock part is a monotonically increasing integer value with a preset number of bits, which is used to represent the sequence of various operations in the database system.

[0087] In some embodiments, the target time window is a time period of a preset window length immediately before the target moment, wherein the target moment is the moment corresponding to the maximum value of the physical clock part of the generated target timestamp stored in the memory within the target time window.

[0088] In some embodiments, when the current time is not less than the target moment stored in the consistent key-value storage system, the first timestamp processing node is used to determine the moment that is after the target moment and within a preset window length from the target moment as the new target moment, and update the new target moment to the consistent key-value storage system, and generate a target timestamp within a time period of a preset window length immediately before the new target moment and store it in the memory.

[0089] In some embodiments, the target timestamp stored in the memory is used to be directly assigned by the first timestamp processing node to the second timestamp processing node when the first timestamp processing node receives a timestamp acquisition request sent by the second timestamp processing node.

[0090] In some embodiments, each second timestamp processing node corresponds to multiple database clients; the client request receiving module 502 is further used to: determine the hash value corresponding to the client identifier of the database client; determine the second timestamp processing node corresponding to the database client based on the hash value and the number of second timestamp processing nodes, and the second timestamp processing node corresponding to the database client receives the timestamp request transaction sent by the database client.

[0091] In some embodiments, the first timestamp processing node includes: a leader node and a follower node; the leader node is used to receive and process the timestamp acquisition request sent by the second timestamp processing node, and the follower node is used to synchronize the data information of the leader node, and in the event of an abnormality in the leader node, replace the leader node to receive and process the timestamp acquisition request sent by the second timestamp processing node.

[0092] Through the client request receiving module 502, timestamp request module 504, and request response module 506 in the above-mentioned timestamp generation device, the purpose of generating and obtaining multi-level TSO in a million-level distributed database is achieved, thereby solving the technical problem that the relevant technology cannot meet the million-level connection requirements and high concurrency requirements for big data cluster application scenarios when obtaining timestamps.

[0093] It should be noted that the various modules in the above-mentioned timestamp generation device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0094] It should be noted that the timestamp generation device provided in this embodiment is used to execute the timestamp generation method shown in FIG2 , so the relevant explanations in the above timestamp generation method are also applicable to the timestamp generation device and will not be repeated here.

[0095] The present application also provides a non-volatile computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program can execute the following timestamp generation method: receiving a timestamp request transaction sent by a database client and determining a target number of timestamp request transactions received within a time period corresponding to a target time window; sending a timestamp acquisition request to a first timestamp processing node based on the target number, wherein the timestamp acquisition request is used to acquire a target number of timestamps, and one first timestamp processing node corresponds to multiple second timestamp processing nodes; receiving a target timestamp returned by the first timestamp processing node in response to the timestamp acquisition request, and sending the target timestamp to the corresponding database client, wherein the target timestamp is the timestamp within the target time window acquired by the first timestamp processing node in memory.

[0096] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0097] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0098] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0099] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0100] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0101] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0102] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0103] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0104] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A timestamp generation method, comprising: Receive a timestamp request transaction sent by a database client, and determine a target number of the timestamp request transactions received within a time period corresponding to a target time window; According to the target number, a timestamp acquisition request is sent to a first timestamp processing node, wherein the timestamp acquisition request is used to acquire the target number of timestamps, and one first timestamp processing node corresponds to multiple second timestamp processing nodes; as well as Receive the target timestamp returned by the first timestamp processing node in response to the timestamp acquisition request, and send the target timestamp to the corresponding database client, where the target timestamp is the timestamp within the target time window acquired by the first timestamp processing node in the memory.

2. The timestamp generation method according to claim 1, wherein the target timestamp comprises: A physical clock part and a logical clock part, wherein the physical clock part is used to represent the world time corresponding to the operation in the database system and the real world, and the logical clock part is a monotonically increasing integer value with a preset number of bits, which is used to represent the sequence of various operations in the database system.

3. The timestamp generation method according to claim 2, wherein the target time window is a time period of a preset window length immediately before the target moment, and the target moment is the moment corresponding to the maximum value of the physical clock part of the generated target timestamp stored in the memory within the target time window.

4. The timestamp generation method according to claim 3, wherein when the current time is not less than the target moment stored in the consistency key-value storage system, the first timestamp processing node is used to determine the moment that is after the target moment and within the preset window length from the target moment as the new target moment, and update the new target moment to the consistency key-value storage system, and generate the target timestamp within a time period of the preset window length immediately before the new target moment and store it in the memory.

5. The timestamp generation method according to claim 4, wherein the target timestamp stored in the memory is used to be directly assigned by the first timestamp processing node to the second timestamp processing node when the first timestamp processing node receives the timestamp acquisition request sent by the second timestamp processing node.

6. The timestamp generation method according to claim 1, wherein each of the second timestamp processing nodes corresponds to a plurality of the database clients; The transaction for receiving a timestamp request sent by a database client includes: Determine a hash value corresponding to the client identifier of the database client; Determine the database client corresponding to the hash value and the number of the second timestamp processing nodes. A second timestamp processing node is configured to receive the timestamp request transaction sent by the database client.

7. The timestamp generation method according to claim 1, wherein the first timestamp processing node comprises: Leader nodes and follower nodes; The leading node is used to receive and process the timestamp acquisition request sent by the second timestamp processing node, and the follower node is used to synchronize the data information of the leading node, and in the event of an abnormality in the leading node, replace the leading node to receive and process the timestamp acquisition request sent by the second timestamp processing node.

8. A timestamp generating device, comprising: A client request receiving module, configured to receive a timestamp request transaction sent by a database client, and determine a target number of the timestamp request transactions received within a time period corresponding to a target time window; A timestamp request module, used for sending a timestamp acquisition request to a first timestamp processing node according to the target number, wherein the timestamp acquisition request is used for acquiring the target number of timestamps, and one first timestamp processing node corresponds to multiple second timestamp processing nodes; as well as A request response module is used to receive the target timestamp returned by the first timestamp processing node in response to the timestamp acquisition request, and send the target timestamp to the corresponding database client. The target timestamp is the timestamp within the target time window obtained by the first timestamp processing node in the memory.

9. The timestamp generating device according to claim 8, wherein the target timestamp comprises: A physical clock part and a logical clock part, wherein the physical clock part is used to represent the world time corresponding to the operation in the database system and the real world, and the logical clock part is a monotonically increasing integer value with a preset number of bits, which is used to represent the sequence of various operations in the database system.

10. A timestamp generating device according to claim 9, wherein the target time window is a time period of a preset window length immediately before the target moment, and the target moment is the moment corresponding to the maximum value of the physical clock part of the generated target timestamp stored in the memory within the target time window.

11. The timestamp generation device according to claim 10, wherein when the current time is not less than the target moment stored in the consistency key-value storage system, the first timestamp processing node is used to determine the moment that is after the target moment and within the preset window length from the target moment as the new target moment, and update the new target moment to the consistency key-value storage system, and generate the target timestamp within a time period of the preset window length immediately before the new target moment and store it in the memory.

12. A timestamp generation device according to claim 11, wherein the target timestamp stored in the memory is used to be directly assigned by the first timestamp processing node to the second timestamp processing node when the first timestamp processing node receives the timestamp acquisition request sent by the second timestamp processing node.

13. The timestamp generating device according to claim 8, wherein each of the second timestamp processing nodes corresponds to a plurality of the database clients, and the client request receiving module is further used for: Determine a hash value corresponding to the client identifier of the database client; The second timestamp processing node corresponding to the database client is determined according to the hash value and the number of the second timestamp processing nodes, and the second timestamp processing node corresponding to the database client receives the timestamp request transaction sent by the database client.

14. The timestamp generating device according to claim 8, wherein the first timestamp processing node comprises: Leader nodes and follower nodes; The leading node is used to receive and process the timestamp acquisition request sent by the second timestamp processing node, and the follower node is used to synchronize the data information of the leading node, and in the event of an abnormality in the leading node, replace the leading node to receive and process the timestamp acquisition request sent by the second timestamp processing node.

15. An electronic device, comprising: A memory and a processor, wherein the memory stores a computer program, and the processor executes the timestamp generation method according to any one of claims 1 to 7 when running the computer program.

16. A non-volatile computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, executes the timestamp generation method according to any one of claims 1 to 7.

Images