Primary and standby time delay query method and device, equipment and medium

Abstract

The invention discloses a main and standby time delay query method and device, equipment and a medium. The method comprises the following steps: after executing a transaction, sending a corresponding log file to a standby machine to indicate the standby machine to calculate main and standby time delay information according to the log file, and returning the main and standby time delay information to a host; local lock-free hash table storage is carried out on the main and standby time delay information returned by different standby machines; and in response to a time delay query request initiated by a client, obtaining pre-stored main and standby time delay information of each standby machine, and returning the main and standby time delay information to the client. According to the embodiment of the invention, the query efficiency of the main-standby time delay can be improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance technology, and in particular to a method, device, equipment and medium for querying master-slave delay. Background Art

[0002] As distributed database systems are increasingly used in a wide range of application scenarios, database operation and maintenance are becoming more and more complicated. In particular, querying the master-slave latency becomes more complex and difficult in scenarios with multiple shards and multiple backup machines, which seriously affects the operation and maintenance and daily monitoring of the database.

[0003] The current master-slave delay requires querying the timestamps of log files on the master and slave machines respectively, synchronizing the delay timestamps and replaying the delay timestamps, and then calculating the difference to obtain the delay information. There is a lack of merging and summarizing of delay information, and a simpler way to obtain delay information is urgently needed. Summary of the invention

[0004] The present invention provides a method, device, equipment and medium for querying the primary and standby time delays, so as to improve the query efficiency of the primary and standby time delays.

[0005] According to one aspect of the present invention, a method for querying active / standby delay is provided, comprising:

[0006] After executing the transaction, the corresponding log file is sent to the standby machine, which is used to instruct the standby machine to calculate the master-slave delay information according to the log file, and return the master-slave delay information to the host;

[0007] The master-slave delay information returned by different backup machines is stored in a local lock-free hash table;

[0008] In response to a latency query request initiated by a client, pre-stored master-slave latency information of each backup machine is acquired, and the master-slave latency information is returned to the client.

[0009] According to another aspect of the present invention, a device for querying a master-slave delay is provided, comprising:

[0010] A sending module, used to send a corresponding log file to the standby machine after executing a transaction, to instruct the standby machine to calculate the master-slave delay information according to the log file, and return the master-slave delay information to the host;

[0011] A storage module is used to store the master-slave delay information returned by different backup machines in a local lock-free hash table;

[0012] The query module is used to respond to the delay query request initiated by the client, obtain the pre-stored master-slave delay information of each backup machine, and return the master-slave delay information to the client.

[0013] According to another aspect of the present invention, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method for querying the active / standby delay according to any embodiment of the present invention is implemented.

[0014] According to another aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for querying the master-slave delay described in any embodiment of the present invention.

[0015] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for querying the active-standby delay described in any embodiment of the present invention when executed.

[0016] The embodiment of the present invention extends the network communication protocol, records the master-slave delay information in the network packet sent by the backup machine to the master machine, and stores this information in the master machine without locks, and the master machine summarizes and displays the relevant information. There is no need to log in to the backup machine for query, which reduces the complexity of operation and maintenance operations and improves query efficiency.

[0017] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 This is a flow chart of a method for querying active / standby delay provided according to an embodiment of the present invention;

[0020] Figure 2 is a flow chart of a method for querying active / standby delays provided according to another embodiment of the present invention;

[0021] Figure 3 is a structural schematic diagram of a query device for active / standby delay provided according to another embodiment of the present invention;

[0022] Figure 4It is a schematic diagram of the structure of an electronic device implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0025] Figure 1 This is a flow chart of a method for querying the master-slave delay provided by an embodiment of the present invention. This embodiment can be applied to the situation where the master-slave delay information of the distributed database system needs to be queried. The method can be executed by a query device for the master-slave delay. The device can be implemented in the form of hardware and / or software. The device can be configured in an electronic device with corresponding data processing capabilities, such as a host in a distributed database system. Figure 1 As shown, the method includes:

[0026] S110. After executing the transaction, a corresponding log file is sent to the standby machine, so as to instruct the standby machine to calculate the master-slave delay information according to the log file, and return the master-slave delay information to the host.

[0027] S120: The master-slave delay information returned by different backup machines is respectively stored in a local lock-free hash table.

[0028] S130 . In response to a latency query request initiated by the client, obtain pre-stored master-slave latency information of each backup machine, and return the master-slave latency information to the client.

[0029] Among them, the master-slave delay information includes the master-slave synchronization delay and the master-slave playback delay. The master-slave synchronization delay refers to the time spent on the data synchronization process between the master and the slave, and the master-slave playback delay refers to the time required for the slave to replay the received log (i.e., execute the operations recorded in the log) to the local state. The lock-free hash table is a special data structure that allows concurrent operations without using traditional locking mechanisms. In a multi-threaded environment, the key values ​​and count values ​​involved in multi-threaded operations in the lock-free hash table use atomic variables, and the modification method is also modified through atomic operations to ensure the consistency and integrity of the hash table data in multi-threaded operations, avoiding lock competition and performance impact. The lock-free hash table uses a lock-free probing mechanism to handle hash conflicts. The pin operation ensures that the modified object is in a fixed position in the memory, avoiding errors caused by operations such as memory reallocation in multi-threading, and also ensures that the memory object will not be modified by other threads until the pin is released, so it can replace the effect of the mutex lock. The lock-free hash table uses atomic operations and other methods to allow threads to search in parallel instead of locking the hash table. This can avoid thread waiting / blocking caused by locking, thereby improving efficiency and reducing lock contention.

[0030] Specifically, the host executes the transaction, and after the transaction is completed, the generated log file (binlog) is sent to the standby machine. The standby machine finds the timestamp representing the specific transaction in the log file, compares the difference between the timestamp of the transaction on the host and the timestamp when the standby machine receives and processes the same transaction, and uses it as the synchronization delay. The standby machine determines the time point when it receives the log, records the time point when it completes the log playback, and calculates the difference between the two time points as the playback delay.

[0031] In the previous latency query solution, the master and standby machines respectively stored their own timestamp information related to the master and standby latency. If you want to know the latency information, you need to log in to the master first, and then log in to the standby machine from the master. Not only does it require two logins, the process is cumbersome. It also affects the normal data interaction between the master and standby machines.

[0032] Therefore, in the present invention, after the standby machine calculates the delay information, it additionally returns the calculated master-slave delay information to the host. A new structure (SLAVE_DELAY_INFO) is added on the host to store the standby machine information and the corresponding master-slave delay information returned by the standby machine. Since there are multiple standby machines on a host, there are multi-threaded operations and thread safety issues. If thread safety is guaranteed by locking, frequent locking and unlocking will lead to performance degradation, and there will be lock competition leading to insufficient resources and even deadlock problems. Therefore, a lock-free hashing method is used to store the structure.

[0033] When the latency information needs to be queried later, the user can initiate a latency query request for the standby machine through the client, and the client will send the request to the host. There is no need to log in to the standby machine for query. The host uses the show statement to display the master-slave latency information of all standby machines connected to the host and reply to the client. The client then displays the target latency information replied by the host to the user. In addition, after receiving the latency query request, the host also actively queries the relevant information of the standby machine and returns it to the client for the client to display to the user. This information includes the identifier of the shard to which the target standby machine belongs, IP, master-slave role information, io thread status, and sql thread status.

[0034] The embodiment of the present invention extends the network communication protocol, records the master-slave delay information in the network packet sent by the backup machine to the master machine, and stores this information in the master machine without locks, and the master machine summarizes and displays the relevant information. There is no need to log in to the backup machine for query, which reduces the complexity of operation and maintenance operations and improves query efficiency.

[0035] Based on the above embodiment, optionally, the master-slave delay information is returned to the host as an extended byte of an existing network packet.

[0036] Specifically, when the standby machine returns the master-slave delay information to the host, it puts the master-slave delay information into the existing network packet as the extension byte of the existing network packet. The existing network packet retains the original content and additionally carries the master-slave delay information. There is no need for the standby machine to separately feedback the master-slave delay information to the standby machine based on the original interaction process, and there is no additional thread resource overhead.

[0037] Based on the above embodiment, optionally, before sending the corresponding log file to the standby machine after executing the transaction, the method further includes:

[0038] Set the switch of whether the master-slave communication protocol extension carries delay information in the dynamic configuration item to on.

[0039] Specifically, dynamic configuration items are set in the master in advance, and global control of whether the standby machine sends delay information to the master is achieved through the dynamic configuration items. The content of the dynamic configuration items is shown in Table 1 below:

[0040] Table 1

[0041]

[0042] Figure 2 This is a flowchart of a method for querying the master-slave delay provided by another embodiment of the present invention. This embodiment is optimized and improved on the basis of the above embodiment. Figure 2 As shown, the method includes:

[0043] S210. After executing the transaction, a corresponding log file is sent to the standby machine, so as to instruct the standby machine to calculate the master-slave delay information according to the log file, and return the master-slave delay information to the host.

[0044] S220: For each standby machine, obtain the latency information returned by the standby machine, and calculate the hash value of the identifier of the standby machine; based on the hash value, insert the latency information into a lock-free hash table through an atomic operation.

[0045] Specifically, when the host receives the master-slave latency information to be stored, it uses a predefined hash function to calculate the hash value of the standby machine identification (ID). This hash value will be used to determine which slot in the hash table the latency information should be stored in. According to the calculated hash value, the corresponding target bucket is located in the hash table, and then the target bucket is searched to see if there is a key that matches the hash value. If a matching key is found, the master-slave latency information updates the value of the key. If no matching key is found, a new key-value pair is created with the hash value as the key and the master-slave latency information as the value. Use atomic operations (such as CAS, Compare-And-Swap) to try to insert a new key-value pair. This is the core part of lock-free programming, ensuring that even if multiple threads try to modify the same location at the same time, it will not cause data contention or destroy the consistency of the data structure.

[0046] S230: In response to the latency query request initiated by the client, obtain pre-stored master-slave latency information of each backup machine, and return the master-slave latency information to the client.

[0047] The embodiment of the present invention adds a lock-free hash table to store and manage the master-slave delay information, realizes lock-free information interaction, and reduces the impact of lock contention on performance when a host is equipped with multiple backup machines.

[0048] Figure 3 A schematic diagram of the structure of a query device for active / standby delay provided by another embodiment of the present invention. Figure 3 As shown, the device comprises:

[0049] A sending module 310 is used to send a corresponding log file to the standby machine after executing a transaction, and to instruct the standby machine to calculate the master-slave delay information according to the log file, and return the master-slave delay information to the host;

[0050] The storage module 320 is used to store the master-slave delay information returned by different backup machines in a local lock-free hash table;

[0051] The query module 330 is used to respond to the latency query request initiated by the client, obtain the pre-stored master-slave latency information of each backup machine, and return the master-slave latency information to the client.

[0052] The active-standby delay query device provided in the embodiment of the present invention can execute the active-standby delay query method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0053] Optionally, the storage module includes:

[0054] A calculation unit, configured to obtain, for each standby machine, the delay information returned by the standby machine, and calculate a hash value of the identifier of the standby machine;

[0055] A storage unit is used to insert the delay information into a lock-free hash table through an atomic operation based on the hash value.

[0056] Optionally, the master-slave delay information is returned to the host as an extended byte of an existing network packet.

[0057] Optionally, the device further comprises:

[0058] The enabling module is used to set the switch of whether the master-slave machine communication protocol extension carries delay information in the dynamic configuration item to the enabling state.

[0059] Optionally, the master-slave delay information includes synchronization delay of the master and standby machines and playback delay of the master and standby machines.

[0060] The query device for the active / standby delay further described can also execute the query method for the active / standby delay provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0061] Figure 4 A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0062] like Figure 4As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0063] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0064] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as a query method for the active-standby delay.

[0065] In some embodiments, the query method of the active-standby delay may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the query method of the active-standby delay described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to execute the query method of the active-standby delay in any other appropriate manner (e.g., by means of firmware).

[0066] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0067] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0068] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0069] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0070] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0071] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0072] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0073] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for querying the master-slave delay, characterized in that: Applied to a host, the method comprises: After executing the transaction, the corresponding log file is sent to the standby machine, which is used to instruct the standby machine to calculate the master-slave delay information according to the log file, and return the master-slave delay information to the host; The master-slave delay information returned by different backup machines is stored in a local lock-free hash table; In response to a latency query request initiated by a client, pre-stored master-slave latency information of each backup machine is acquired, and the master-slave latency information is returned to the client.

2. The method according to claim 1, characterized in that The storing of the master-slave delay information returned by different backup machines in local lock-free hash tables respectively includes: For each standby machine, obtain the delay information returned by the standby machine, and calculate the hash value of the identifier of the standby machine; Based on the hash value, the delay information is inserted into a lock-free hash table through an atomic operation.

3. The method according to claim 1, characterized in that The master-slave delay information is returned to the host as an extension byte of an existing network packet.

4. The method according to claim 1, characterized in that: After executing a transaction, before sending the corresponding log file to the standby machine, it also includes: Set the switch of whether the master-slave communication protocol extension carries delay information in the dynamic configuration item to on.

5. The method according to claim 4, characterized in that The master-slave delay information includes the synchronization delay of the master and the standby machines and the playback delay of the master and the standby machines.

6. A query device for active / standby delay, characterized in that: Deployed on a host, the device includes: A sending module, used to send a corresponding log file to the standby machine after executing a transaction, to instruct the standby machine to calculate the master-slave delay information according to the log file, and return the master-slave delay information to the host; A storage module is used to store the master-slave delay information returned by different backup machines in a local lock-free hash table; The query module is used to respond to the delay query request initiated by the client, obtain the pre-stored master-slave delay information of each backup machine, and return the master-slave delay information to the client.

7. The device according to claim 6, characterized in that The storage module comprises: A calculation unit, configured to obtain, for each standby machine, the delay information returned by the standby machine, and calculate a hash value of the identifier of the standby machine; A storage unit is used to insert the delay information into a lock-free hash table through an atomic operation based on the hash value.

8. The device according to claim 6, characterized in that The master-slave delay information is returned to the host as an extension byte of an existing network packet.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the query method for the active / standby delay according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the query method for the active-standby delay according to any one of claims 1 to 5 when executed.

Images