Storage node detection method and device of distributed database, equipment and medium

Abstract

The embodiment of the invention discloses a storage node detection method and device for a distributed database, equipment and a medium, and the method comprises the steps: generating a corresponding node detection task based on the link state information of each storage node when a storage node detection instruction of a target distributed database is received, all node detection tasks are added into a task queue; for each storage node, when the node detection task of the current storage node in the task queue meets a condition, processing the node detection task, and monitoring task feedback information; when task feedback information of the node detection task is monitored, generating a result processing task corresponding to the current storage node based on the task feedback information, and adding the result processing task to a task queue; and when the result processing task of the current storage node in the task queue meets the condition, the result processing task is processed, and the node detection process of the current storage node is ended when the processing is completed, so that the real-time performance and the detection efficiency of the fault detection of the storage node are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a storage node detection method, device, equipment and medium for a distributed database. Background Art

[0002] Distributed databases are widely used in the computer field due to their high reliability and high security. Distributed databases use an extensible system structure to store data in multiple independent devices, that is, distributed databases include multiple storage nodes, which improves the reliability, availability and access efficiency of the system. In order to ensure the stable operation of the distributed database, it is necessary to perform fault detection on each storage node of the distributed database.

[0003] At present, the storage node detection method of distributed databases is usually as follows: the computing node thread periodically detects the faults of the storage nodes in the cluster. When the detection period arrives, the thread serially calls the detection synchronization interface to detect whether each storage node has a fault in turn.

[0004] However, in large-scale distributed database cluster scenarios, the number of storage nodes may reach hundreds or thousands, and it takes a long time to complete a round of fault detection. When a large-scale network failure occurs, multiple storage nodes will time out at the same time, which will cause a longer time to complete a round of fault detection. Therefore, there are problems such as long node fault detection time, low efficiency, and poor real-time performance. Summary of the invention

[0005] The present invention provides a storage node detection method, device, equipment and medium for a distributed database, so as to avoid the waiting time in the fault detection of a single storage node. Even in fault scenarios such as large-scale clusters or network failures, the fault detection of all storage nodes in the cluster can be completed within a timeout period, thereby improving the real-time performance and detection efficiency of storage node fault detection.

[0006] In a first aspect, an embodiment of the present invention provides a storage node detection method for a distributed database, wherein the distributed database includes a plurality of storage nodes, and the method includes:

[0007] When receiving a storage node detection instruction of a target distributed database, generating a corresponding node detection task based on the link status information of each storage node, and adding all the node detection tasks to a task queue;

[0008] For each of the storage nodes, when the node detection task corresponding to the current storage node in the task queue meets the first preset condition, the node detection task is processed and task feedback information is monitored;

[0009] When task feedback information of the node detection task is monitored, a result processing task corresponding to the current storage node is generated based on the task feedback information, and the result processing task is added to the task queue;

[0010] When the result processing task corresponding to the current storage node in the task queue meets the second preset condition, the result processing task is processed until the node detection process of the current storage node is ended when the result processing task is completed.

[0011] In a second aspect, an embodiment of the present invention further provides a storage node detection device for a distributed database, wherein the distributed database includes a plurality of storage nodes, and the device includes:

[0012] A detection task generation module is used to generate corresponding node detection tasks based on the link status information of each storage node when receiving a storage node detection instruction of a target distributed database, and add all the node detection tasks to a task queue;

[0013] A task feedback monitoring module is used to process the node detection task for each storage node and monitor task feedback information when the node detection task corresponding to the current storage node in the task queue meets the first preset condition;

[0014] A result task generation module is used to generate a result processing task corresponding to the current storage node based on the task feedback information when monitoring the task feedback information of the node detection task, and add the result processing task to the task queue;

[0015] The result task processing module is used to process the result processing task corresponding to the current storage node in the task queue when the result processing task meets the second preset condition, and end the node detection process of the current storage node when the result processing task is completed.

[0016] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0017] one or more processors;

[0018] A storage device is used to store one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement a storage node detection method for a distributed database as described in any of the embodiments of the present invention.

[0019] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute a storage node detection method for a distributed database as described in any one of the embodiments of the present invention.

[0020] The technical solution of the embodiment of the present invention, when receiving the storage node detection instruction of the target distributed database, generates the corresponding node detection task based on the link status information of each storage node, and adds all the node detection tasks to the task queue, so that for each storage node, when the node detection task corresponding to the current storage node in the task queue meets the first preset condition, the node detection task is processed, and the task feedback information is listened to, further, when the task feedback information of the node detection task is listened to, the result processing task corresponding to the current storage node is generated based on the task feedback information, and the result processing task is added to the task queue, so that when the result processing task corresponding to the current storage node in the task queue meets the second preset condition, the result processing task is processed, and the node detection process of the current storage node is ended when the result processing task is completed. The technical solution provided in this embodiment uses an asynchronous interface to process the node detection task of each storage node through a task queue, and monitors the task feedback information of the node detection task. When the task feedback information is monitored, the result processing task is generated and processed. The waiting time is avoided in the fault detection of a single storage node, and the detection time of the storage node is greatly shortened. Even in failure scenarios such as large-scale clusters or network failures, the fault detection of all storage nodes in the cluster can be completed within a timeout period, thereby improving the real-time performance and detection efficiency of storage node fault detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required to describe the embodiments. Obviously, the drawings introduced are only drawings of a part of the embodiments to be described in the present invention, rather than all the drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of a flow chart of a storage node detection method for a distributed database provided by an embodiment of the present invention;

[0023] Figure 2 For a schematic diagram of the implementation flow of the storage node detection method of the distributed database provided in this embodiment, see FIG.

[0024] Figure 3 This is a schematic diagram of the work thread jump logic involved in this embodiment;

[0025] Figure 4 A schematic diagram of the structure of a storage node detection device for a distributed database provided by an embodiment of the present invention;

[0026] Figure 5The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] Before introducing this solution, the application scenario of the embodiment of the present invention can be described first. The technical solution provided by this embodiment can be applied to any scenario where fault detection is required for each data storage node in a distributed database. A distributed database refers to the use of a high-speed computer network to connect multiple physically dispersed data storage units to form a logically unified database. A distributed database includes multiple storage nodes, and each connected data storage unit is called a storage node. Distributed database middleware is a middleware for managing and operating distributed databases, located between applications and underlying databases. Generally speaking, distributed database middleware (referred to as middleware) is connected to the client upstream and to multiple storage nodes of the distributed database downstream (nodes responsible for data storage persistence in the distributed database). The distributed database middleware needs to have the ability to detect faults in each storage node.

[0028] Currently, when the distributed database middleware performs fault detection on multiple storage nodes of the distributed database, the middleware needs to send a detection request to each storage node. If the middleware receives a correct response from the storage node, it means that the storage node is in normal state. If the middleware receives an incorrect response from the storage node or does not receive a response, it means that the storage node has a fault. The middleware needs to detect all related storage nodes in sequence to complete a round of fault detection. When using the synchronous serial model for fault detection, the time to complete a round of fault detection is:

[0029] Total detection time = detection request response time (the total time from sending a request to receiving a response) × the number of storage nodes.

[0030] In a large-scale distributed database cluster scenario, the number of storage nodes may reach hundreds or even thousands. The more storage nodes there are, the longer it takes for the middleware to complete a round of fault detection. In particular, when a large-scale network failure occurs, multiple storage nodes will time out at the same time. In this scenario, the time to complete a round of fault detection becomes:

[0031] Total detection time = detection timeout × number of faulty nodes + detection request response time × number of normal nodes.

[0032] Generally speaking, the detection timeout is generally much longer than the time taken to respond to a single detection request, so the time taken to detect faults in a fault scenario will be greatly increased. It can be seen that quickly and accurately completing the fault detection of storage nodes so that the detection time is not affected by the cluster size and the number of faulty nodes is a key problem that needs to be solved by distributed database middleware. The technical solution provided by the embodiment of the present invention can be integrated into the distributed database middleware, so that the storage node detection method can be used to quickly detect faults in a large number of data storage nodes contained in the distributed database.

[0033] Embodiment 1

[0034] Figure 1 A flow chart of a storage node detection method for a distributed database provided in an embodiment of the present invention. This embodiment can be applied to any scenario that requires fault detection of each data storage node in distributed data. The method can be executed by a storage node detection device for a distributed database, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC or a server.

[0035] like Figure 1 As shown, the storage node detection method of the distributed database includes:

[0036] S110. When a storage node detection instruction of a target distributed database is received, a corresponding node detection task is generated based on the link status information of each storage node, and all node detection tasks are added to a task queue.

[0037] The target distributed database is a distributed database that is about to perform fault detection on each storage node contained therein. The storage node detection instruction refers to a control instruction generated when it is necessary to perform fault detection on each storage node in the target distributed database. For example, the storage node detection instruction can be triggered by a predefined scheduled task or can be triggered according to a preconfigured detection control.

[0038] The link status information is information that characterizes the link connectivity status between a storage node and the middleware. For any storage node, the corresponding link status information includes link connectivity status information or link disconnection status information. The link connectivity status information means that there is a link between the middleware and a storage node, that is, the middleware and the storage node are in a network connectivity state. The link disconnection status information means that there is no link between the middleware and a storage node, that is, the middleware and the storage node are in a network disconnection state.

[0039] Among them, the node detection task refers to an object event that requires node detection on a storage node. The node detection task is used to represent what kind of processing is about to be done on a storage node. The task queue is a container that stores tasks during the transmission process of the task. The task queue is used to temporarily store the task information corresponding to each storage node.

[0040] Specifically, the technical solution provided by this embodiment can be integrated into the middleware. Predefine a scheduled task. For example, the scheduled task can be to perform a fault detection on each storage node in the target distributed database every preset time (for example, the preset time can be 6 hours, 12 hours, 24 hours, etc.). Thus, whenever the current moment reaches the node fault detection triggering moment, a storage node detection instruction can be generated. Or, a target control for triggering a storage node detection instruction can be pre-set. When the user triggers the target control, a storage node detection instruction can be generated. Based on this, the middleware

[0041] When receiving the storage node detection instruction of the target distributed database, the link state information of each storage node associated with the target distributed database can be obtained. Furthermore, for each storage node, a corresponding node detection task can be generated according to the difference in link state information. After the node detection tasks of all storage nodes are generated, these node detection tasks can be added to the task queue at one time, or each time a node detection task is generated, it can be added to the task queue.

[0042] On the basis of the above embodiments, optionally, a specific implementation method of generating a corresponding node detection task based on the link status information of each storage node may include: for each storage node, if the link identification information of the current storage node is link connectivity status information, a node detection task of a network packet detection type is generated; if the link identification information of the current storage node is link disconnection status information, a node detection task of a link reconstruction type is generated.

[0043] The node detection task of the network packet detection type specifically refers to: the middleware can send a network detection request to a storage node, and then wait for the corresponding feedback information of the storage node. If the feedback information obtained is normal, it indicates that there is no node failure in the storage node. If no feedback information is received within the specified time or abnormal feedback information is received, it indicates that there is a node failure in the storage node.

[0044] Specifically, the node detection task of the link reconstruction type refers to: the middleware can send a link establishment request to a storage node, and then wait for the corresponding link establishment feedback information of the storage node. If the link is successfully established between the middleware and the storage node, it indicates that there is no node failure in the storage node. If the link is not successfully established within the specified time, or the link establishment between the middleware and the storage node fails, it indicates that there is a node failure in the storage node.

[0045] In this embodiment, the processing method for each storage node is the same. Here, any one of the storage nodes is used as the current storage node, and the current storage node is used as an example for exemplary description. If the link identification information of the current storage node is the link connection status information, this situation indicates that there is already a connection link between the middleware and the storage node at the current moment. At this time, it is only necessary to send a network detection request to the storage node, and judge whether the node has a fault according to the feedback information corresponding to the storage node, so a node detection task of the network packet detection type can be generated. If the link identification information of the current storage node is the link disconnection status information, this situation indicates that there is no link between the middleware and the storage node at the current moment or the link has been disconnected. At this time, if the middleware can successfully establish a link with the current storage node, it can be indicated that the current storage node does not have a fault; otherwise, it indicates that the current storage node has a fault. It should be particularly noted that if the middleware can successfully establish a link with the current storage node, it can be indicated that the current storage node does not have a fault, and the middleware does not need to send a network detection request to the current storage node.

[0046] Exemplarily, the target distributed database includes 1000 storage nodes, namely storage node 1, storage node 2, storage node 3, ..., storage node 1000. When the middleware receives the storage node detection instruction of the target distributed database, the link status information corresponding to storage node 1 is the link connectivity status information, then a node detection task 1 of the network packet detection type corresponding to storage node 1 can be generated; the link status information corresponding to storage node 2 is the link disconnection status information, then a node detection task 2 of the link reconstruction type corresponding to storage node 2 can be generated...; the link status information corresponding to storage node 1000 is the link connectivity status information, then a node detection task 1000 of the network packet detection type corresponding to storage node 1000 can be generated. Based on this, these 1000 node detection tasks can be added to the task queue.

[0047] S120. For each storage node, when the node detection task corresponding to the current storage node in the task queue meets a first preset condition, the node detection task is processed and task feedback information is monitored.

[0048] The first preset condition is a pre-set processing rule, which is used to control whether the node detection task corresponding to the current storage node can be processed. The current storage node refers to the storage node currently being processed. Task feedback information refers to the result information fed back by the storage node after the middleware sends a request to a storage node according to the node detection task.

[0049] In this embodiment, the processing method for each storage node is the same. Here, any one of the storage nodes is used as the current storage node, and the current storage node is used as an example for exemplary description. Specifically, when the node detection task corresponding to the current storage node in the task queue meets the first preset condition, for example, when the node detection task corresponding to the current storage node is at the top of the task queue, the pre-configured work thread can be called, and the work thread can execute the node detection task corresponding to the current storage node. In the process of the work thread executing the node detection task, the middleware sends a network packet detection request or a link establishment request to the current storage node. After sending the request, the work thread is released and can execute the node detection tasks corresponding to other storage nodes. The node detection task corresponding to the current storage node is monitored by a special processing unit to monitor the corresponding task feedback results.

[0050] Optionally, when the node detection task corresponding to the current storage node in the task queue meets the first preset condition, the specific implementation method of processing the node detection task may include: when the node detection task corresponding to the current storage node is at the top of the task queue and there is a preset processing thread in an idle state, the node detection task corresponding to the current storage node is processed based on the preset processing thread in an idle state, and the node detection task is monitored by the multiplexer to determine the task feedback information.

[0051] Among them, the preset processing threads are multiple pre-configured working threads used to perform fault detection tasks corresponding to each storage node. A multiplexer refers to a data processing unit designed with IO multiplexing technology (Input / Output Multiplexing). IO multiplexing technology is a technology for managing multiple input / output channels in a single thread. It allows a thread to listen to multiple input streams (such as network sockets, file descriptors, etc.) at the same time, and perform corresponding processing when there is data to read or write, without creating a separate thread for each channel. This technology registers multiple IO channels to an event manager, waits for events to occur in a blocking manner, and once an event occurs (such as data to read or write), the thread will be awakened, and then specific events can be processed.

[0052] In this embodiment, multiple preset processing threads can be preconfigured, and multiple preset processing threads can execute data processing tasks at the same time. When the node detection task corresponding to the current storage node is at the top of the task queue, and multiple preset processing threads store idle preset processing threads, the idle preset processing thread can obtain the node detection task corresponding to the current storage node from the task queue, and execute the corresponding processing flow according to the task type corresponding to the node detection task. At the same time, the multiplexer can monitor the node detection task corresponding to the current storage node, and when the file descriptor corresponding to the node detection task is monitored, the task feedback information can be determined.

[0053] Based on the above embodiments, the specific steps of processing the node detection task corresponding to the current storage node based on the preset processing thread in the idle state may include: if the node detection task corresponding to the current storage node corresponds to the network packet detection type, a network packet detection request is sent to the current storage node through the preset processing thread; if the node detection task corresponding to the current storage node corresponds to the link reconstruction type, a link establishment request is sent to the current storage node through the preset processing thread.

[0054] Specifically, if the node detection task corresponding to the current storage node corresponds to the network packet detection type, at this time, the middleware can send a network packet detection request to the current storage node through a preset processing thread. For example, the network packet detection request can be a ping request. If the node detection task corresponding to the current storage node corresponds to the link reconstruction type, at this time, the middleware can send a link establishment request to the current storage node through a preset processing thread.

[0055] S130. When task feedback information of the node detection task is monitored, a result processing task corresponding to the current storage node is generated based on the task feedback information, and the result processing task is added to the task queue.

[0056] The result processing task refers to the object event that the middleware should handle next after receiving the task feedback information of the current storage node. For example, the result processing task may include the result processing task of reporting that the current storage node is a node with no abnormal attributes and the result processing task of disconnecting the corresponding link of the current storage node.

[0057] Specifically, the task feedback information may include success attribute information and failure attribute information. Success attribute information refers to the normal response information fed back by the current storage node after receiving the request sent by the middleware. Failure attribute information refers to the abnormal response information fed back by the current storage node after receiving the request sent by the middleware. In specific applications, if the task feedback information is different, the subsequent execution logic is different, so it is necessary to generate a result processing task corresponding to the task feedback information. Further, the result processing task corresponding to the current storage node is re-added to the task queue.

[0058] Optionally, the specific implementation method of generating a result processing task corresponding to the current storage node based on the task feedback information may include: if the task feedback information is success attribute information, then generating a result processing task to report the current storage node as a node with no abnormal attributes; if the task feedback information is failure attribute information, then generating a result processing task to disconnect the link corresponding to the current storage node.

[0059] In this embodiment, if the task feedback information is success attribute information, it indicates that there is no fault in the current storage node. A result processing task can be generated to report that the current storage node is a node with no abnormal attributes, so that the middleware can obtain a fault-free detection result for the node corresponding to the current storage node.

[0060] If the task feedback information is failure attribute information, it indicates that the storage node is faulty. In the target distributed database, the current storage unit cannot be used temporarily to avoid data loss. At this time, the link between the middleware and the current storage node can be temporarily disconnected. Therefore, a result processing task for disconnecting the link corresponding to the current storage node can be generated. It should be noted that since the link between the middleware and the current storage node has been disconnected during the node fault detection, the link status information corresponding to the current storage node will be the link disconnection status information when the target distributed database is detected for the next node fault. A node detection task of the link reconstruction type needs to be generated next time.

[0061] S140: When the result processing task corresponding to the current storage node in the task queue meets the second preset condition, the result processing task is processed until the result processing task is completed, and the node detection process of the current storage node is ended.

[0062] The second preset condition is a preset processing rule, which is used to control whether the result processing task corresponding to the current storage node can be started.

[0063] Specifically, when the result processing task corresponding to the current storage node in the task queue meets the second preset condition, for example, when the result processing task corresponding to the current storage node is at the top of the task queue, the preset working thread can be called, and the working thread can execute the result processing task corresponding to the current storage node. When the working thread completes the result processing task, it indicates that the node detection process of the current storage node has been completed, the fault detection result corresponding to the current storage node has been obtained, and the post-fault detection processing measures have been performed on the current storage node (such as reporting the current storage node as a node with no abnormal attributes, or disconnecting the link corresponding to the current storage node).

[0064] More specifically, when the result processing task corresponding to the current storage node in the task queue meets the second preset condition, the result processing task is processed until the result processing task is completed and the node detection process of the current storage node is ended. The specific implementation method may include: when the result processing task of the current storage node is at the first place in the task queue and there is a preset processing thread in an idle state, the result processing task is processed based on the preset processing thread in an idle state; when the end identification information corresponding to the result processing task is received, the node detection process of the current storage node is ended.

[0065] The end identification information refers to identification information that can be fed back when the result processing task is completed.

[0066] In this embodiment, multiple preset processing threads can be preconfigured, and multiple preset processing threads can execute data processing tasks at the same time. When the result processing task corresponding to the current storage node is at the top of the task queue, and a preset processing thread in an idle state is stored in multiple preset processing threads, the preset processing thread in an idle state can obtain the result processing task corresponding to the current storage node from the task queue, and perform the corresponding processing flow according to the specific content of the task corresponding to the result processing task. Further, when the end identification information corresponding to the result processing task is received, it indicates that the result processing task has been executed at this time, and the node detection process of the current storage node can be ended.

[0067] Based on the above embodiment, when the end identification information corresponding to all storage nodes is received, the target storage node whose task feedback information is failure attribute information is determined to generate a database storage node detection report based on the node identification information of the target storage node.

[0068] The target storage node is a storage node with suspected faults. The node identification information is a unique identification code corresponding to each storage node. The database storage node detection report is used to indicate which storage nodes in the target distributed database are suspected of having faults.

[0069] In this embodiment, the above S120-S140 elaborates on how to execute the corresponding storage node detection method for any storage node, and the specific implementation process of the remaining storage nodes is the same as the implementation process of the current storage node, which will not be repeated here. When all storage nodes have completed the corresponding node detection tasks and result processing tasks, the end identification information corresponding to all storage nodes can be obtained. At this time, the task feedback information can be summarized as the target storage node of the failure attribute information. Furthermore, the node identification information of these target storage nodes can be used to generate a database storage node detection report. These storage nodes with suspected faults need further inspection and confirmation, so that the operation and maintenance personnel can check and repair the storage nodes with suspected faults according to the node identification information in the database storage node detection report.

[0070] The technical solution of the embodiment of the present invention, when receiving the storage node detection instruction of the target distributed database, generates the corresponding node detection task based on the link status information of each storage node, and adds all the node detection tasks to the task queue, so that for each storage node, when the node detection task corresponding to the current storage node in the task queue meets the first preset condition, the node detection task is processed, and the task feedback information is listened to, further, when the task feedback information of the node detection task is listened to, the result processing task corresponding to the current storage node is generated based on the task feedback information, and the result processing task is added to the task queue, so that when the result processing task corresponding to the current storage node in the task queue meets the second preset condition, the result processing task is processed, and the node detection process of the current storage node is ended when the result processing task is completed. The technical solution provided in this embodiment uses an asynchronous interface to process the node detection task of each storage node through a task queue, and monitors the task feedback information of the node detection task. When the task feedback information is monitored, the result processing task is generated and processed. The waiting time is avoided in the fault detection of a single storage node, and the detection time is greatly shortened. Even in failure scenarios such as large-scale clusters or network failures, the fault detection of all storage nodes in the cluster can be completed within a timeout period, thereby improving the real-time performance and detection efficiency of storage node fault detection.

[0071] Embodiment 2

[0072] In the embodiment of the present invention, a storage node detection method of a distributed database provided by the present embodiment is introduced in a specific implementation manner. The storage node detection method of the distributed database can be integrated into a distributed database middleware associated with the target distributed database. The implementation flow diagram of the storage node detection method is shown in FIG. Figure 2 ,like Figure 2As shown, the middleware can use an asynchronous interface, IO multiplexer, management thread, listener thread and multiple worker threads to cooperate with a group of threads to complete the fault detection of storage nodes. The specific working logic is as follows:

[0073] 1. Management thread: The management thread periodically drives a round of fault detection for each storage node through a timer. For example, when the timer is triggered, the management thread can build a node detection task for each storage node being detected. After all tasks are built, the management thread adds the batch of node detection tasks to the task queue of the working thread.

[0074] 2. Multiple worker threads: Each worker thread obtains tasks from the task queue in a competitive manner for processing. During the task processing, the worker thread executes different logics according to the different task status flags. (For example, the node detection task of the packet detection type, the node detection task of the link reconstruction type, and the result processing task can correspond to different task status flags respectively), which may include:

[0075] (1) When executing a node detection task, if a network packet detection type node detection task needs to be executed, a network packet detection request (i.e., a ping request) can be sent to a storage node; if a link reconstruction type node detection task needs to be executed, a link establishment request can be sent to a storage node, and the file descriptor of the link corresponding to the storage node is added to the IO multiplexer to monitor the task feedback information corresponding to the storage node based on the IO multiplexer.

[0076] (2) When executing the result processing task, that is, when the ping request returns, or the link establishment request returns, the working thread can process the result processing task corresponding to the request return information. For example, if the task feedback information is success attribute information, a result processing task is generated to report that the current storage node is a node with no abnormal attributes; if the task feedback information is failure attribute information, a result processing task is generated to disconnect the link corresponding to the current storage node.

[0077] Optionally, in actual applications, the worker thread can be driven by a state machine. For a schematic diagram of the worker thread jump logic, see Figure 3 , the working thread can include three working states, namely ping state, connect state or close state. The conversion logic between the three working states can be:

[0078] For any storage node, when a work thread implements the corresponding node detection process, the work thread can determine which state to jump to based on the task type information of the node detection task in the initial state. If the node detection task corresponds to the network packet detection type, it indicates that the link between the storage node and the middleware is normal. At this time, the state of the work thread changes to the ping state, and sends a ping request to the storage node asynchronously; if the node detection task corresponds to the link reconstruction type, it indicates that the link between the storage node and the middleware is abnormal. At this time, the state of the work thread changes to the connect state, and sends a link establishment request to the storage node asynchronously. When ping / connect is successful, the processing flow ends. Otherwise, jump to the close state, close the faulty link, and the processing flow ends.

[0079] The connect state or ping state is divided into three sub-states: start sub-state, wait sub-state, and end sub-state. For the jump logic diagram of these three sub-states, see Figure 3 , specifically including:

[0080] Start state: call the asynchronous start() interface to send a request to the server. If waiting is required, add the handle (file descriptor of the link) to the multiplexer and the state jumps to wait. If waiting is not required, jump to the end state.

[0081] Wait state: The wait state is the state of waiting for the asynchronous interface to return. When the IO multiplexer returns, the task is added to the work queue of the worker thread again. After the worker thread wakes up, it calls the cont() interface to determine whether the asynchronous process has returned. If so, it jumps to end, otherwise the state remains in wait. On this basis, when the total waiting time exceeds the set timeout time, the timeout error flag is set in the task and jumps to the end state.

[0082] End state: Process the result returned by the asynchronous interface. If the detection is normal, the state machine ends. If a failure occurs, the link is closed.

[0083] 3. IO multiplexer: used to monitor the task feedback information corresponding to each storage node at the same time. For example, its specific implementation method can be to monitor the file descriptor of the link corresponding to each storage node (i.e. Figure 2 The file descriptor is used to distinguish which storage node the currently monitored task feedback information corresponds to.

[0084] 4. Listening thread: The listening thread processes the return of the IO multiplexer and is responsible for adding the result processing task corresponding to the returned result to the task queue of the working thread again, which will be processed by the working thread later.

[0085] Based on the above node fault detection process, ping requests or connect requests can be sent to all storage nodes in parallel. The detection time is not affected by the total number of nodes and the number of faulty nodes. The fault detection of all storage nodes in the cluster can be completed within a timeout period, thereby improving the detection efficiency of distributed database storage nodes.

[0086] Embodiment 3

[0087] Figure 4 A structural diagram of a storage node detection device for a distributed database provided in an embodiment of the present invention, the distributed database includes multiple storage nodes, and the device includes: a detection task generation module 310, a task feedback monitoring module 320, a result task generation module 330 and a result task processing module 340.

[0088] Among them, the detection task generation module 310 is used to generate corresponding node detection tasks based on the link status information of each storage node when receiving the storage node detection instruction of the target distributed database, and add all the node detection tasks to the task queue;

[0089] The task feedback monitoring module 320 is used to process the node detection task for each storage node and monitor the task feedback information when the node detection task corresponding to the current storage node in the task queue meets the first preset condition;

[0090] A result task generation module 330 is used to generate a result processing task corresponding to the current storage node based on the task feedback information when monitoring the task feedback information of the node detection task, and add the result processing task to the task queue;

[0091] The result task processing module 340 is used to process the result processing task when the result processing task corresponding to the current storage node in the task queue meets the second preset condition, and end the node detection process of the current storage node when the result processing task is completed.

[0092] Based on the above device, optionally, the link status information includes link connection status information or link disconnection status information, and the detection task generation module 310 includes:

[0093] A network packet detection task generating unit is used to generate a node detection task of a network packet detection type for each storage node if the link identification information of the current storage node is link connectivity status information;

[0094] The link reconstruction task generating unit is used to generate a node detection task of the link reconstruction type if the link identification information of the current storage node is link disconnection status information.

[0095] On the basis of the above-mentioned device, optionally, a task feedback monitoring module 320 is specifically used to process the node detection task corresponding to the current storage node based on the preset processing thread in the idle state when the node detection task corresponding to the current storage node is at the top of the task queue and there is a preset processing thread in the idle state, and the multiplexer performs task monitoring on the node detection task to determine the task feedback information.

[0096] Based on the above device, optionally, the task feedback monitoring module 320 includes:

[0097] A network packet detection task processing unit, configured to send a network packet detection request to the current storage node through a preset processing thread if the node detection task corresponding to the current storage node corresponds to the network packet detection type;

[0098] The link reconstruction task processing unit is used to send a link establishment request to the current storage node through a preset processing thread if the node detection task corresponding to the current storage node corresponds to the link reconstruction type.

[0099] Based on the above device, optionally, the result task generation module 330 includes:

[0100] A successful result task processing unit, used for generating a result processing task for reporting that the current storage node is a node with no abnormal attributes if the task feedback information is successful attribute information;

[0101] The failure result task processing unit is used to generate a result processing task of disconnecting the link corresponding to the current storage node if the task feedback information is failure attribute information.

[0102] Based on the above-mentioned device, optionally, a result task processing module 340 is specifically used to process the result processing task based on the preset processing thread in the idle state when the result processing task of the current storage node is at the top of the task queue and there is a preset processing thread in the idle state; when the end identification information corresponding to the result processing task is received, the node detection process of the current storage node is ended.

[0103] Based on the above device, optionally, the storage node detection device of the distributed database further includes: a detection report determination module;

[0104] Based on the above device, a detection report determination module is used to determine the target storage node whose task feedback information is failure attribute information when the end identification information corresponding to all storage nodes is received, so as to generate a database storage node detection report based on the node identification information of the target storage node.

[0105] The technical solution of the embodiment of the present invention, when receiving the storage node detection instruction of the target distributed database, generates the corresponding node detection task based on the link status information of each storage node, and adds all the node detection tasks to the task queue, so that for each storage node, when the node detection task corresponding to the current storage node in the task queue meets the first preset condition, the node detection task is processed, and the task feedback information is listened to, further, when the task feedback information of the node detection task is listened to, the result processing task corresponding to the current storage node is generated based on the task feedback information, and the result processing task is added to the task queue, so that when the result processing task corresponding to the current storage node in the task queue meets the second preset condition, the result processing task is processed, and the node detection process of the current storage node is ended when the result processing task is completed. The technical solution provided in this embodiment uses an asynchronous interface to process the node detection task of each storage node through a task queue, and monitors the task feedback information of the node detection task. When the task feedback information is monitored, the result processing task is generated and processed. The waiting time is avoided in the fault detection of a single storage node, and the detection time is greatly shortened. Even in failure scenarios such as large-scale clusters or network failures, the fault detection of all storage nodes in the cluster can be completed within a timeout period, thereby improving the real-time performance and detection efficiency of storage node fault detection.

[0106] The storage node detection device for a distributed database provided in an embodiment of the present invention can execute the storage node detection method for a distributed database provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0107] It is worth noting that the various units and modules included in the above system are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0108] Embodiment 4

[0109] Figure 5 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 5 A block diagram of an exemplary electronic device 40 suitable for implementing exemplary embodiments of the present invention is shown. Figure 5 The electronic device 40 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0110] like Figure 5As shown, the electronic device 40 is in the form of a general computing device. The components of the electronic device 40 may include but are not limited to: one or more processors or processing units 401, a system memory 402, and a bus 403 connecting different system components (including the system memory 402 and the processing unit 401).

[0111] Bus 403 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0112] The electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 40, including volatile and non-volatile media, removable and non-removable media.

[0113] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data medium interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.

[0114] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402, such program modules 407 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 407 generally perform the functions and / or methods of the embodiments described herein.

[0115] The electronic device 40 may also communicate with one or more external devices 409 (e.g., keyboards, pointing devices, displays 810, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 40, and / or may communicate with any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 411. Furthermore, the electronic device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 412. As shown, the network adapter 412 communicates with other modules of the electronic device 40 via the bus 403. It should be understood that although Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 40, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0116] The processing unit 401 executes various functional applications and page processing by running the programs stored in the system memory 402, for example, implementing the storage node detection method of the distributed database provided in the embodiment of the present invention.

[0117] Embodiment 5

[0118] An embodiment of the present invention further provides a storage medium including computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute a storage node detection method for a distributed database, wherein the distributed database includes a plurality of storage nodes. The method includes:

[0119] When receiving a storage node detection instruction of a target distributed database, generating a corresponding node detection task based on the link status information of each storage node, and adding all the node detection tasks to a task queue;

[0120] For each of the storage nodes, when the node detection task corresponding to the current storage node in the task queue meets the first preset condition, the node detection task is processed and task feedback information is monitored;

[0121] When task feedback information of the node detection task is monitored, a result processing task corresponding to the current storage node is generated based on the task feedback information, and the result processing task is added to the task queue;

[0122] When the result processing task corresponding to the current storage node in the task queue meets the second preset condition, the result processing task is processed until the node detection process of the current storage node is ended when the result processing task is completed.

[0123] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0124] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0125] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0126] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A storage node detection method for a distributed database, wherein the distributed database includes a plurality of storage nodes, characterized in that: include: When receiving a storage node detection instruction of a target distributed database, generating a corresponding node detection task based on the link status information of each storage node, and adding all the node detection tasks to a task queue; For each of the storage nodes, when the node detection task corresponding to the current storage node in the task queue meets the first preset condition, the node detection task is processed and task feedback information is monitored; When task feedback information of the node detection task is monitored, a result processing task corresponding to the current storage node is generated based on the task feedback information, and the result processing task is added to the task queue; When the result processing task corresponding to the current storage node in the task queue meets the second preset condition, the result processing task is processed until the node detection process of the current storage node is ended when the result processing task is completed.

2. The method according to claim 1, characterized in that The link status information includes link connection status information or link disconnection status information. The generating of the corresponding node detection task based on the link status information of each storage node includes: For each of the storage nodes, if the link identification information of the current storage node is link connectivity status information, a node detection task of a network packet detection type is generated; If the link identification information of the current storage node is link disconnection status information, a node detection task of the link reconstruction type is generated.

3. The method according to claim 1, characterized in that: When the node detection task corresponding to the current storage node in the task queue meets the first preset condition, processing the node detection task includes: When the node detection task corresponding to the current storage node is at the first place in the task queue and there is a preset processing thread in an idle state, the node detection task corresponding to the current storage node is processed based on the preset processing thread in the idle state, and the node detection task is monitored by the multiplexer to determine the task feedback information.

4. The method according to claim 3, characterized in that The processing of the node detection task corresponding to the current storage node based on the preset processing thread in the idle state includes: If the node detection task corresponding to the current storage node corresponds to a network packet detection type, sending a network packet detection request to the current storage node through the preset processing thread; If the node detection task corresponding to the current storage node corresponds to a link reconstruction type, a link establishment request is sent to the current storage node through the preset processing thread.

5. The method according to claim 1, characterized in that The task feedback information includes success attribute information and failure attribute information, and the generating of the result processing task corresponding to the current storage node based on the task feedback information includes: If the task feedback information is success attribute information, a result processing task is generated to report that the current storage node is a node with no abnormal attributes; If the task feedback information is failure attribute information, a result processing task of disconnecting the link corresponding to the current storage node is generated.

6. The method according to claim 1, characterized in that When the result processing task corresponding to the current storage node in the task queue meets the second preset condition, the result processing task is processed until the result processing task is processed and the node detection process of the current storage node is ended, including: When the result processing task of the current storage node is at the top of the task queue and there is a preset processing thread in an idle state, processing the result processing task based on the preset processing thread in an idle state; When the end identification information corresponding to the result processing task is received, the node detection process of the current storage node is ended.

7. The method according to claim 6, characterized in that Also includes: When the end identification information corresponding to all storage nodes is received, the target storage node for which the task feedback information is failure attribute information is determined, so as to generate a database storage node detection report based on the node identification information of the target storage node.

8. A storage node detection device for a distributed database, the distributed database comprising a plurality of storage nodes, characterized in that: The device includes: A detection task generation module is used to generate corresponding node detection tasks based on the link status information of each storage node when receiving a storage node detection instruction of a target distributed database, and add all the node detection tasks to a task queue; A task feedback monitoring module is used to process the node detection task for each storage node and monitor task feedback information when the node detection task corresponding to the current storage node in the task queue meets the first preset condition; A result task generation module is used to generate a result processing task corresponding to the current storage node based on the task feedback information when monitoring the task feedback information of the node detection task, and add the result processing task to the task queue; The result task processing module is used to process the result processing task corresponding to the current storage node in the task queue when the result processing task meets the second preset condition, and end the node detection process of the current storage node when the result processing task is completed.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and A memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the storage node detection method for a distributed database as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the storage node detection method for a distributed database according to any one of claims 1 to 7 when executed.

Images