Distributed data operation and maintenance method, system, equipment and program product

Abstract

The invention discloses a distributed data operation and maintenance method, system, device and program product, and belongs to the technical field of databases, the method comprises the following steps: performing configuration processing on a distributed database to obtain metadata of the distributed database and attribute data of each node in the distributed database, the metadata indicating a configuration mode of the distributed database, and the attribute data indicating the attribute data of each node in the distributed database; the attribute data indicates a connection process of the node and the distributed database; distributing corresponding operation and maintenance instructions to the nodes according to the metadata and the attribute data of the nodes; in response to the node selection instruction, determining at least one node corresponding to the node selection instruction as a target node, controlling the at least one target node to execute a corresponding operation and maintenance instruction, obtaining an operation and maintenance execution result of the at least one target node, and storing the operation and maintenance execution result in a log form; and analyzing and processing the database log of each node to obtain a log analysis report of the distributed database, and visualizing the log analysis report. The operation and maintenance precision and the operation and maintenance efficiency of the distributed database are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, and particularly relates to a distributed data operation and maintenance method, system, device and program product. BACKGROUND

[0002] A distributed database is a logical unified database system formed by connecting multiple physical nodes through a computer network, and the physical nodes are usually located in different geographical locations or servers. With the development of the big data era, the distributed database gradually replaces the traditional centralized database due to its high data processing performance. At present, the operation and maintenance of the distributed database usually depends on manual methods. However, the method depends on manual operation and cannot efficiently process the complex architecture of the distributed database, and has problems of low fault tolerance and high operation and maintenance cost. Therefore, the operation and maintenance efficiency and operation and maintenance accuracy of the related technology need to be improved. SUMMARY

[0003] The main purpose of the embodiment of the present application is to provide a distributed data operation and maintenance method, system, device and program product, which aims to effectively improve the operation and maintenance efficiency and operation and maintenance accuracy of the distributed database.

[0004] To achieve the above-mentioned purpose, one aspect of the embodiment of the present application provides a distributed data operation and maintenance method, which comprises the following steps: Performing configuration processing on a distributed database to obtain metadata of the distributed database and attribute data of each node in the distributed database; wherein the metadata is used to indicate the configuration mode of the distributed database, and the attribute data is used to indicate the connection process of the node and the distributed database; According to the metadata and the attribute data of each node, a corresponding operation and maintenance instruction is allocated to each node; In response to a node selection instruction, at least one node corresponding to the node selection instruction is determined as a target node, at least one target node is controlled to execute a corresponding operation and maintenance instruction, an operation and maintenance execution result of at least one target node is obtained and stored in a log form; Performing analysis processing on the database log of each node to obtain a log analysis report of the distributed database and perform visualization.

[0005] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application provides a distributed database operation and maintenance system, which comprises: A cluster metadata collector is configured to process a distributed database to obtain metadata of the distributed database and attribute data of each node in the distributed database, wherein the metadata is used to indicate a configuration mode of the distributed database, and the attribute data is used to indicate a connection process of the node and the distributed database. An automated operation instruction storage is configured to allocate a corresponding operation instruction to each node according to the metadata and the attribute data of each node. A distributed request execution scheduler is configured to determine at least one target node corresponding to the node selection instruction in response to a node selection instruction, control the at least one target node to execute a corresponding operation instruction, obtain an operation execution result of the at least one target node, and store the operation execution result in a log form. A log collection analyzer is configured to analyze database logs of each node to obtain a log analysis report of the distributed database and visualize the log analysis report.

[0006] To achieve the above object, another aspect of the embodiments of the present application provides an electronic device, which comprises: at least one processor; at least one memory configured to store at least one program; when the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned distributed data operation method.

[0007] To achieve the above object, another aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above-mentioned distributed data operation method.

[0008] To achieve the above object, another aspect of the embodiments of the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the above-mentioned distributed data operation method.

[0009] According to the distributed data operation and maintenance method, system, device and program product provided by the embodiment of the present application, the distributed database is configured and processed to obtain the metadata of the distributed database and the attribute data of each node in the distributed database. The metadata indicates the configuration mode of the distributed database, and the attribute data indicates the connection process of the node and the distributed database. The corresponding operation and maintenance instruction is allocated to each node according to the metadata and the attribute data of each node. At least one node corresponding to the node selection instruction is determined as a target node in response to the node selection instruction, and the corresponding operation and maintenance instruction is executed by the at least one target node to obtain the operation and maintenance execution result of the at least one target node and store it in the form of a log. The database log of each node is analyzed and processed to obtain a log analysis report of the distributed database and visualize it. According to the technical solution of the embodiment of the present application, not only the operation and maintenance demand of the distributed database can be quickly responded, but also the node required for operation and maintenance can be targetedly managed and operated, the operation and maintenance quality of the distributed database is improved, and then the operation and maintenance efficiency and operation and maintenance precision of the distributed database are effectively improved.

[0010] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and attained by the structure particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a flowchart of a distributed database operation and maintenance method provided by the present application; Figure 2 is Figure 1 is a flowchart of step S101 in Figure 3 is an example diagram of a configuration processing interface provided by the present application; Figure 4 is Figure 1 is a flowchart of step S102 in Figure 5 is an example diagram of an instruction preset interface provided by the present application; Figure 6 is an example diagram of a task setting window provided by the present application; Figure 7 is Figure 1 is a flowchart of step S103 in Figure 8 is an example diagram of an operation and maintenance execution interface provided by the present application; Figure 9 is an example diagram of an execution window provided by the present application; Figure 10 is Figure 1 is a flowchart of step S104 in Figure 11 is an example diagram of a log analysis interface provided by the present application; Figure 12 is a structural diagram of a distributed database operation and maintenance device provided by the present application; Figure 13 is an example diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0012] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementation manners described in the following exemplary embodiments do not represent all the implementation manners consistent with the embodiments of the present application, but are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0013] It can be understood that the terms "first", "second", and the like used in the present application can be used herein to describe various concepts, but unless specifically stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0014] The terms "at least one", "multiple", "each", "any", and the like used in the present application include one, two or more than two, multiple includes two or more than two, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0016] Before the embodiments of the present application are described in detail, first, some nouns and terms involved in the embodiments of the present application are described, and the nouns and terms involved in the embodiments of the present application are applicable to the following explanations.

[0017] 1) DDB: Distributed Data Base, a logical unified database system formed by connecting multiple physical nodes through a computer network, which are usually located in different geographical locations or servers. Compared with centralized databases, distributed databases are more suitable for large-scale data storage, high-concurrency access, and cross-regional business scenarios.

[0018] 2) GTM: Global Time Manager, a core component for implementing consistency in distributed databases, used to ensure global transaction consistency and maintain global transactions of distributed databases through global timestamps.

[0019] 3) CN: Coordinating Node, located in the distributed database, used to generate execution plans to forward business requests and maintain database statistics.

[0020] 4) DN: Data Node, located in the distributed database, used to store business data. Usually, data is stored in multiple data nodes, which can be queried through the coordinating node.

[0021] 5) SQL: Structured Query Language, a database query and programming language used to access data and query, update and manage relational database systems. Structured Query Language is a high-level non-procedural programming language that allows users to work on high-level data structures. It does not require users to specify the storage method of data, nor does it require users to understand the specific data storage method, so different database systems with completely different underlying structures can use the same structured query language as the interface for data input and management. Structured Query Language statements can be nested, which makes it extremely flexible and powerful.

[0022] 6) Shell: is the command line interpreter for user interaction with the operating system, the core function is to parse and execute the commands input by the user, and support task automation through scripts.

[0023] 7) SSH: Secure Shell, a secure network protocol for remotely controlling computers or executing commands through encrypted network connections, which provides users with encrypted remote login, command execution and data transmission capabilities in an insecure network environment.

[0024] 8) SCP: Secure Copy Protocol, is a file transfer protocol based on SSH, which provides an encrypted connection, allowing users to securely copy files and directories between computers on a network, ensuring that data is not accessed or tampered with by unauthorized users during transmission.

[0025] 9) ARM: Advanced RISC Machine, is a Reduced Instruction Set Computing (RISC) based processor architecture, known for its low power consumption and high efficiency, widely used in embedded devices and mobile devices.

[0026] 10) X86: is a Complex Instruction Set Computing (CISC) based processor architecture, known for its high performance and wide compatibility, mainly used in personal computers, servers and workstations.

[0027] With the development of the big data era, distributed databases gradually replace traditional centralized databases due to their high data processing performance. From the perspective of technical architecture, compared with traditional centralized databases, the technical architecture of distributed databases is more complex. Specifically, centralized databases usually only contain data nodes, while distributed databases not only contain data nodes, but also contain coordination nodes, global clock management nodes and other nodes, and distributed databases use virtual Ethernet addresses, which provide support for the horizontal expansion of distributed databases and service continuity in fault exception scenarios. From the perspective of database management and operation and maintenance, when facing centralized databases, operation and maintenance personnel only need to focus on the current node to complete all management and operation and maintenance work, without considering the data processing and database instruction execution between multiple nodes. However, distributed databases often involve multiple database instance nodes, and database instructions can run in different execution states on multiple nodes. For operation and maintenance personnel, the difficulty of managing and operating distributed databases is much higher than that of centralized databases. At present, it is usually necessary to rely on manual methods to operate and maintain distributed databases. However, this method relies on manual operation and cannot efficiently handle the complex architecture of distributed databases, and has problems such as low fault tolerance and high operation and maintenance cost. Therefore, the operation and maintenance efficiency and accuracy of related technologies for distributed databases need to be improved.

[0028] Therefore, the embodiment of the present application provides a distributed database operation and maintenance method, system, device and program product. The scheme first determines the metadata of the distributed database and the attribute data of each node in the distributed database through configuration processing of the distributed database, and accordingly allocates corresponding operation and maintenance instructions to each node. Then, in response to the node selection instruction, the node required for operation and maintenance, i.e., the target node, is selected, and the selected node is controlled to execute the operation and maintenance instruction to obtain the corresponding operation and maintenance execution result and store it in the form of a log. Finally, the database logs of each node are analyzed and processed, thereby realizing the automatic management and operation and maintenance processing of the distributed database. In this way, not only can the operation and maintenance requirements of the distributed database be quickly responded, especially large-scale operation and maintenance requirements, but also the nodes required for operation and maintenance can be ensured to be managed and operated in a targeted manner, thereby improving the operation and maintenance quality of the distributed database and effectively improving the operation and maintenance efficiency and operation and maintenance accuracy of the distributed database. The embodiment of the present application can be deployed on any network-reachable server device and is compatible with X86 or ARM server architecture.

[0029] It should be noted that the type of the distributed database is not specifically limited in the embodiment of the present application. For example, the distributed database can be a TeleDB-X database, or other distributed databases using GTM, CN, DN architecture and supporting standard SQL syntax, such as Tbase database, but is not limited thereto.

[0030] A distributed database operation and maintenance method provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0031] The distributed database operation and maintenance method provided by the embodiment of the present application can be applied to a terminal, a server, or software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, etc., but is not limited thereto. The server can be configured as a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms, etc. The server can also be a node server in a blockchain network. The software can be an application that implements the distributed database operation and maintenance method provided by the embodiment of the present application, etc., but is not limited to the above forms.

[0032] The application is operable with numerous general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that can be suitable for use with the application include personal computers, server computers, handheld or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. The application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0033] With reference to Figure 1 , Figure 1 is a flowchart of a distributed database operation and maintenance method provided by the application. The distributed database operation and maintenance method can include the following steps S101-S104.

[0034] S101, configuration processing is performed on the distributed database to obtain metadata of the distributed database and attribute data of each node in the distributed database; wherein the metadata is used to indicate a configuration mode of the distributed database, and the attribute data is used to indicate a connection process of the node and the distributed database.

[0035] In this step, configuration processing is performed on the distributed database to persist the connection information of the distributed database into the configuration, based on which the metadata information of the distributed database cluster and the attribute data information of all nodes in the distributed database cluster can be automatically parsed and obtained. The metadata and the configuration mode of the distributed database are highly related, and are usually used to define information such as structure, attribute, relationship and management of the distributed database; the attribute data and the connection process of the node and the distributed database are highly related, and are usually used to define parameters involved in the connection process.

[0036] Optionally, the metadata can be set according to actual conditions, and embodiments of the application do not make specific limitations thereon. For example, the metadata can include information at the database level such as database name, version, creation time, information at the table level such as table name, field name, data type, constraint condition, index information, information at the user level such as username, permission, but is not limited thereto.

[0037] Optionally, the attribute data can be set according to actual conditions, and embodiments of the application do not make specific limitations thereon. For example, the attribute data can include node version, connection address, port, storage path and the like information used to connect the database, but is not limited thereto.

[0038] S102, according to the metadata and the attribute data of each node, each node is allocated a corresponding operation and maintenance instruction.

[0039] In this step, a plurality of operation and maintenance instructions are pre-configured and stored. After obtaining the metadata and the attribute data of each node, the operation and maintenance instructions are published and persisted to each node of the distributed database cluster according to these data, so that each node is allocated a corresponding operation and maintenance instruction.

[0040] S103, in response to the node selection instruction, at least one node corresponding to the node selection instruction is determined as a target node, at least one target node is controlled to execute a corresponding operation and maintenance instruction, and operation and maintenance execution results of at least one target node are obtained and stored in a log form.

[0041] In this step, first, the node selection instruction is obtained and responded to, so as to determine at least one node corresponding to the node selection instruction as a target node, that is, a node required for operation and maintenance; then, at least one target node is controlled to execute a corresponding operation and maintenance instruction, so as to obtain operation and maintenance execution results of at least one target node and store them in a log form, thereby realizing targeted operation and maintenance operation on the node required for operation and maintenance.

[0042] S104, the database logs of each node are analyzed and processed, a log analysis report of the distributed database is obtained and visualized.

[0043] In this step, the database logs of each node are collected to a deployment end, the deployment end refers to a client or server where the distributed database is deployed, by analyzing the database logs of each node, a log analysis report of the distributed database can be obtained, and the log analysis report is visualized, so that the user can quickly know the actual running status of the distributed database.

[0044] As can be seen, the embodiments of the present application first determine the metadata of the distributed database and the attribute data of each node in the distributed database through configuration processing of the distributed database, and allocate a corresponding operation and maintenance instruction to each node, then in response to the node selection instruction, select a node required for operation and maintenance, that is, a target node, and control the selected node to execute the operation and maintenance instruction, obtain a corresponding operation and maintenance execution result and store it in a log form, and finally analyze and process the database logs of each node, thereby realizing automatic management and operation and maintenance processing of the distributed database. In this way, not only can the operation and maintenance demand of the distributed database be quickly responded, especially large-scale operation and maintenance demand, but also the node required for operation and maintenance can be ensured to be managed and operated in a targeted manner, the operation and maintenance quality of the distributed database is improved, and the operation and maintenance efficiency and operation and maintenance precision of the distributed database are effectively improved.

[0045] The above steps will be further described below.

[0046] In some embodiments, referring to Figure 2 The configuration process of the distributed database in step S101 can include the following steps A01-A02: A01, obtain the configuration file of the seed node, and connect the seed node to the distributed database; A02, configure the corresponding attribute data for each node according to the configuration file, and configure the metadata for the distributed database.

[0047] This embodiment is a link for configuring the distributed database cluster, collecting the metadata of the distributed database cluster, and parsing the attribute data of each node in the distributed database cluster: according to the prompt information in the configuration file of the distributed database, configure the information such as connection address, port, user information, etc. of the distributed database cluster, and automatically identify and parse the metadata of the distributed database cluster and the attribute data of each node in the distributed database cluster. The process of this configuration process only needs to configure the information of a specified node or any node in the distributed database cluster, and the node is the seed node. By configuring the information of the seed node, the metadata of the distributed database and the attribute data of each node in the distributed database cluster can be automatically obtained and parsed.

[0048] In specific implementation, first, a node is selected from multiple nodes as a seed node, and the configuration file of the seed node is obtained, which is a pre-set file. Then, the seed node is connected to the distributed database cluster using the configuration file of the seed node. After being connected to the distributed database cluster, the attribute data of each node and the metadata of the distributed database cluster are automatically obtained by executing SQL statements according to the configuration file, such as node version, connection address, port, storage path, etc. for connecting the database attribute data, and metadata such as username, database version, database list, table list, etc. to configure the corresponding attribute data for each node and the metadata for the distributed database, which comprehensively feedbacks the overall status of the distributed database cluster.

[0049] Therefore, on the one hand, the embodiment realizes the automatic configuration processing by modifying the configuration file and persisting the related information of the distributed database into the configuration, which simplifies the configuration process, reduces the influence of human intervention on the operation and maintenance process, and improves the reliability of the configuration processing and the automation level of the distributed database operation and maintenance. On the other hand, the embodiment realizes the automatic configuration processing by taking the configuration file of the seed node as the configuration entry, which enables the configuration process to only set the configuration file of the seed node without setting the configuration files of other nodes in the distributed database cluster, thereby effectively improving the accuracy of the configuration processing, and when the distributed database cluster expands, the attribute data of each node and the metadata of the distributed database cluster can be updated only by the configuration file of the seed node without setting the configuration file for the newly added node again, which effectively reduces the information amount of the configuration processing and speeds up the configuration processing. In summary, the embodiment can effectively improve the accuracy and efficiency of the configuration processing, which helps to improve the operation and maintenance efficiency and accuracy of the distributed database.

[0050] Optionally, the selection manner of the seed node can be set according to actual conditions, and the embodiment does not make specific limitation thereto.

[0051] Exemplarily, one node is randomly selected from the plurality of nodes as the seed node, so that the seed node can be quickly determined by the random selection manner.

[0052] Exemplarily, a plurality of nodes within a preset range of the distributed database are selected as candidate nodes; for each candidate node, a ratio of a current idle bandwidth of the candidate node to a maximum bandwidth of the candidate node is calculated as a bandwidth score of the candidate node, an inverse of a mean value of a round-trip time (RTT) of the candidate node and each node is calculated as a topology score of the candidate node, and the topology score and the bandwidth score are weighted and summed to obtain a performance score of the candidate node; and the candidate node with the highest performance score is selected as the seed node. Here, the embodiment first selects the nodes close to the distributed database as the candidate nodes based on the network topology priority principle, then measures the communication quality of the candidate nodes by the bandwidth score, measures the contribution of the candidate nodes to the distributed database cluster by the topology score, determines the performance quality of the candidate nodes in combination with the two scores, and finally selects the candidate node with the best performance quality as the seed node, so that the selection accuracy of the seed node can be improved, and the accuracy of the configuration processing can be ensured. The weights of the topology score and the bandwidth score can be flexibly set according to actual conditions, for example, the weight of the topology score is 0.3 and the weight of the bandwidth score is 0.7, but the embodiment is not limited thereto.

[0053] In one example, the database operation and maintenance manner of the related technology is often limited to professional operation and maintenance personnel, which requires high professional knowledge of the user, and non-operation and maintenance personnel is difficult to manage and maintain the distributed database, thereby leading to low convenience and high difficulty of the operation and maintenance of the distributed database. To this end, the present embodiment introduces an interactive display technology to realize the configuration processing link in the operation and maintenance process of the distributed database. A database operation and maintenance interface is displayed in the terminal, and the database operation and maintenance interface can include a configuration control 110. In response to a trigger instruction of the configuration control 110, a configuration processing interface is displayed, as shown in Figure 3 The configuration processing interface can include a configuration file acquisition control 120, a configuration processing control 130, and a configuration result display area 140. In response to a trigger instruction of the configuration file acquisition control 120, the configuration file of the seed node is acquired. In response to a trigger instruction of the configuration processing control 130, the corresponding attribute data of each node is configured according to the configuration file, and the metadata of the distributed database is configured, and the attribute data of each node and the metadata of the distributed database cluster are displayed in the configuration result display area 140.

[0054] Here, the user can enter the configuration processing interface by triggering the configuration control 110. In the configuration processing interface, the user can acquire the configuration file of the seed node by triggering the configuration file acquisition control 120, and acquire the corresponding attribute data of each node and the metadata of the distributed database by triggering the configuration processing control 130. The user can view the acquired data in the configuration result display area 140. In this way, the present embodiment introduces an interactive display technology to realize the configuration processing link in the operation and maintenance process of the distributed database, so that the configuration processing link is intuitive, transparent, traceable, and what you see is what you get. This not only provides rich detailed information of the distributed database, but also reduces the learning cost and professional knowledge requirement of the user, so that non-operation and maintenance personnel can also perform configuration processing on the distributed database, thereby reducing the difficulty of configuration processing and effectively improving the efficiency of configuration processing. This helps to reduce the operation and maintenance difficulty of the distributed database and improve the operation and maintenance efficiency of the distributed database.

[0055] In some embodiments, referring to Figure 4 In step S102, the corresponding operation and maintenance instructions of each node are allocated according to the metadata and the attribute data of each node, which can include steps B01-B02: B01, at least one operation and maintenance instruction corresponding to each node is determined according to the metadata and the attribute data of each node; B02, at least one operation and maintenance instruction corresponding to each node is issued to each node.

[0056] It should be noted that each node can be pre-provisioned with one or more operation and maintenance instructions for automatic operation and maintenance call.

[0057] In this embodiment, after the metadata of the distributed database cluster and the attribute data of each node are parsed and obtained, the automatic operation and maintenance instruction presetting link is entered: the metadata and the attribute data obtained in the foregoing steps are used to persist the built-in automatic operation and maintenance instructions in all nodes, so as to call the automatic operation and maintenance instructions on each node and uniformly manage them in subsequent steps. In specific implementation, a plurality of preset operation and maintenance instructions are pre-set, and the operation and maintenance tasks corresponding to different preset operation and maintenance instructions are different. Through the foregoing steps, the metadata of the distributed database cluster and the attribute data of each node in the distributed database cluster can be obtained. According to the attribute data of each node, one or more preset operation and maintenance instructions matched with the operation and maintenance requirements of the node are selected as the operation and maintenance instructions of the node according to the metadata and the attribute data of the node; at least one operation and maintenance instruction corresponding to each node is determined by traversing the plurality of nodes. Subsequently, for each node, at least one operation and maintenance instruction corresponding to the node is issued to the node, including a view, a stored procedure, a custom function, a Shell instruction, and the like.

[0058] As can be seen, the embodiment provides a rich set of operation and maintenance instructions, and uses the metadata of the distributed database cluster and the attribute data of each node as a benchmark to accurately issue at least one corresponding operation and maintenance instruction for each node, so that each node is pre-set with an operation and maintenance instruction matched with the operation and maintenance task of the node. Through the storage and issuance of the automatic operation and maintenance instructions, the operation and maintenance consistency and traceability of each node can be effectively improved, the influence of human intervention factors on the operation and maintenance process can be reduced, the preset accuracy of the operation and maintenance instructions can be improved, the operation and maintenance workload of the distributed database can be effectively reduced, and the preset efficiency of the operation and maintenance instructions can be improved, which helps to improve the operation and maintenance efficiency and accuracy of the distributed database.

[0059] Optionally, the determination manner of the operation and maintenance instructions of each node can be set according to actual conditions, and the embodiment does not make a specific limitation thereto.

[0060] Exemplarily, for each node, the operation and maintenance task type corresponding to the node is determined according to the metadata and the attribute data of the node in combination with a machine learning method, and the preset operation and maintenance instruction corresponding to the operation and maintenance task type is determined as the operation and maintenance instruction of the node. Thus, by introducing the machine learning method, the operation and maintenance task required by the node can be accurately identified, and appropriate operation and maintenance instructions are allocated to the node accordingly, so that the preset accuracy of the operation and maintenance instructions can be further improved, which helps to improve the operation and maintenance accuracy of the distributed database. The machine learning method can be flexibly set according to actual conditions, for example, the machine learning method can be a support vector machine, a logistic regression, or the like, but is not limited thereto.

[0061] Further exemplarily, the database operation and maintenance manner of the related art is often limited to professional operation and maintenance personnel, and the professional knowledge of the user is required, and the non-operation and maintenance personnel is difficult to manage and maintain the distributed database, and thus the operation and maintenance convenience of the distributed database is low, and the operation and maintenance difficulty is high. In this regard, the embodiment introduces the interactive display technology to realize the operation and maintenance instruction preset link in the operation and maintenance process of the distributed database. The database operation and maintenance interface is displayed in the terminal, and the database operation and maintenance interface can include the instruction preset control 210. In response to the trigger instruction of the instruction preset control 210, the instruction preset interface is displayed, as shown in Figure 5 The instruction preset interface can include the node list area 220 and the instruction allocation control 230, the node list area 220 is used to display each node in the distributed cluster, and each node is provided with a corresponding task setting control 260. For each node, in response to the trigger instruction of the task setting control 260 of the node, the task setting window of the node is displayed, as shown in Figure 6 The task setting window can include the determination control 240 and a plurality of task selection controls 250, the task selection control 250 corresponds to one operation and maintenance task, in response to the trigger instruction of at least one task selection control 250, the operation and maintenance task corresponding to the trigger instruction of the task selection control 250 is determined as the operation and maintenance task of the node, and in response to the trigger instruction of the determination control 240, it returns to the instruction preset interface as shown in Figure 5 In response to the trigger instruction of the instruction allocation control 230, at least one operation and maintenance instruction matched with the operation and maintenance task of each node is allocated to each node.

[0062] Here, the user can enter the instruction preset interface by triggering the instruction preset control 210. In the instruction preset interface, when the user wants to set the operation and maintenance task of a node, the user enters the task setting window of the node by triggering the task setting control 260 of the node. In the task setting window, the user selects one or more operation and maintenance tasks for the node by triggering one or more task selection controls 250, and triggers the determination control 240 when the selection is completed, so as to return to the instruction preset interface. After the user sets the operation and maintenance task of one or more nodes, the user allocates the operation and maintenance instruction suitable for the operation and maintenance task of each node by triggering the instruction allocation control 230, so as to realize the automatic presetting of the operation and maintenance instruction. In this way, the embodiment introduces the interactive display technology to realize the operation and maintenance instruction presetting link in the distributed database operation and maintenance process, so that the operation and maintenance instruction presetting link is intuitive, transparent, traceable, and what you see is what you get. Therefore, not only can rich details of the distributed database be provided, but also the learning cost and professional knowledge requirement of the user can be reduced, so that non-operation and maintenance personnel can also preset the operation and maintenance instruction of the distributed database, the difficulty of the operation and maintenance instruction presetting is reduced, and the efficiency of the operation and maintenance instruction presetting is effectively improved. This helps to reduce the operation and maintenance difficulty of the distributed database and improve the operation and maintenance efficiency of the distributed database.

[0063] In some embodiments, with reference to Figure 7 In the step S103, in response to the node selection instruction, at least one node corresponding to the node selection instruction is determined as a target node, the corresponding operation and maintenance instruction is executed on the at least one target node, the operation and maintenance execution result of the at least one target node is obtained and stored in the form of a log, which can include the following steps C01-C04: C01, classifying each node to obtain a plurality of node category groups; wherein each node category group includes at least one node belonging to the category corresponding to the node category group; C02, in response to a selection instruction of at least one node category group, all nodes in the node category group corresponding to the selection instruction are determined as target nodes; C03, controlling each target node to execute the corresponding operation and maintenance instruction; C04, receiving and processing the operation and maintenance execution result of each target node, and storing the operation and maintenance execution result of each target node in the database log of each target node.

[0064] The embodiment is a link for executing operation and maintenance instructions by each node in a distributed database cluster: the operation and maintenance instructions are published to a plurality of specified nodes, and the returned results are collected, checked, judged, and converted. In the specific implementation, first, each node is automatically classified according to the category of each node, thereby obtaining a plurality of node category groups, and each node category group includes at least one node belonging to the category corresponding to the node category group. Then, the selected instructions of at least one node category group are obtained and responded, all nodes in the node category group corresponding to the selected instructions are determined as target nodes, and the target nodes are nodes required for operation and maintenance. Then, for each target node, the target node executes the operation and maintenance instruction corresponding to the target node, and returns the corresponding operation and maintenance execution result, which indicates the execution condition of the target node executing the operation and maintenance instruction. Finally, the operation and maintenance execution results of all target nodes are received, and the results are processed accordingly, and the processed results are stored in the database log of each target node.

[0065] As can be seen, the embodiment first encapsulates the complex nodes into easily understood categories through classification processing to obtain a plurality of node category groups, so as to quickly determine the nodes required for operation and maintenance through the categories of the nodes, then selects all nodes in one or more node category groups as target nodes, and controls the target nodes to execute the corresponding operation and maintenance instructions, finally receives the operation and maintenance execution results of all target nodes and processes them accordingly, and stores the processed results in the database log of each target node, thereby realizing the automatic operation and maintenance of multiple nodes, ensuring that the nodes required for operation and maintenance are managed and maintained in a targeted manner, improving the operation and maintenance quality of the distributed database, optimizing the utilization of operation and maintenance resources, reducing the operation and maintenance workload of the distributed database, and thereby effectively improving the operation and maintenance precision and efficiency of the distributed database.

[0066] Optionally, the category corresponding to the node category group can be set according to actual conditions, and the embodiment does not make specific limitations thereon. For example, the category corresponding to the node category group can include any one of a SQL command type, a SQL file type, or a Shell command type, but is not limited thereto.

[0067] Optionally, the classification processing method can be set according to actual conditions, and the embodiment does not make specific limitations thereon.

[0068] Exemplarily, the classification processing is performed according to the type of the operation and maintenance instruction, that is, the instruction transmission parameter of each node is obtained, for each node, the category corresponding to the instruction transmission parameter of the node is determined as the category of the node, the category of each node can be obtained by traversing the plurality of nodes, nodes belonging to the same category are classified into the same node category group, and then a plurality of node category groups are obtained, so that the nodes can be quickly divided into corresponding groups through the parameter matching mode, which facilitates the selection of one or more nodes of the command type to execute the operation and maintenance operation in the subsequent steps, thereby realizing targeted management and operation. The category of the node can include any one of the SQL command type, the SQL file type or the Shell command type. For example, the category of the node with the instruction transmission parameter f is the SQL file type, the category of the node with the instruction transmission parameter c is the SQL command type, and the category of the node with the instruction transmission parameter s is the Shell command type, but not limited thereto.

[0069] Exemplarily, the classification processing is performed according to the hierarchical relationship of the nodes, that is, each primary node usually has one or more secondary nodes, for example, a node with a larger storage space or storing more data is a primary node, and a node with a smaller storage space or storing less data is a secondary node, but not limited thereto. Accordingly, if the node has one or more secondary nodes, the node is determined as a master node type; if the node does not have one or more secondary nodes and has a corresponding primary node, the node is determined as a slave node type; all nodes of the master node type are classified into a master node category group, all nodes of the slave node type are classified into a slave node type, and all nodes are respectively classified into corresponding individual category groups, and the individual category group only contains one node, so that the nodes can be quickly divided into corresponding groups through the hierarchical relationship of the nodes, which facilitates the selection of one or more primary nodes or secondary nodes to execute the operation and maintenance operation in the subsequent steps, thereby realizing targeted management and operation. For example, the primary nodes are master DN1 and master DN2, the secondary node of master DN1 is slave DN1, and the secondary node of master DN2 is slave DN2, so that these nodes are classified to obtain the master node category group including master DN1 and master DN2, the slave node category group including slave DN1 and slave DN2, the individual category group of master DN1, the individual category group of master DN2, the individual category group of slave DN1 and the individual category group of slave DN2, but not limited thereto.

[0070] Optionally, the order of executing the operation and maintenance instruction by each node can be set according to actual conditions, which is not limited in the embodiment. For example, each node can execute the corresponding operation and maintenance instruction simultaneously; for example, according to the execution order of each node, each node executes the corresponding operation instruction in turn, but not limited thereto.

[0071] In one example, the database operation and maintenance manner of the related technology is often limited to professional operation and maintenance personnel, which requires high professional knowledge of the user, and non-operation and maintenance personnel is difficult to manage and maintain the distributed database accordingly, thereby leading to low operation and maintenance convenience and high operation and maintenance difficulty of the distributed database. In this regard, the present embodiment introduces an interactive display technology to implement the operation and maintenance instruction execution link in the distributed database operation and maintenance process. A database operation and maintenance interface is displayed in the terminal, and the database operation and maintenance interface can include an operation and maintenance execution control 310. In response to a trigger instruction of the operation and maintenance execution control 310, an operation and maintenance execution interface is displayed, as shown in Figure 8 The operation and maintenance execution interface can include a classification control 320, a category group list area 330, an execution control 340, and a result display area 350. The category group list area 330 can include a plurality of category group selection controls, and the category group selection controls and the node category groups are one-to-one corresponding. In response to a trigger instruction of the classification control 320, each node is classified and processed to obtain a plurality of node category groups and display them in the category group list area 330. In response to a selection instruction of at least one category group selection control, all nodes in the node category group corresponding to the selection instruction are determined as target nodes. In response to a trigger instruction of the execution control 340, each target node executes the corresponding operation and maintenance instruction, the operation and maintenance execution results of each target node are received and processed, and displayed in the result display area 350. The operation and maintenance execution results of each target node are stored in the database log of each target node.

[0072] Here, the user can enter the operation and maintenance execution interface by triggering the operation and maintenance execution control 310. In the operation and maintenance execution interface, the user triggers the classification control 320 to realize the classification processing of each node, so as to obtain a plurality of node category groups and display them in the category group list area 330. When the user wants to select the nodes of the required operation and maintenance, the user triggers one or more category group selection controls displayed in the category group list area 330 to select one or more category groups, so as to select one or more nodes of the required operation and maintenance. After selecting the nodes of the required operation and maintenance, the user triggers the execution control 340 to control the selected nodes to execute the corresponding operation and maintenance operation. The operation and maintenance execution results of these selected nodes are synchronously displayed in the result display area 350, so as to realize the targeted operation and maintenance and management of a certain type of node. In this way, the embodiment introduces interactive display technology to realize the operation and maintenance instruction execution link in the distributed database operation and maintenance process, so that the operation and maintenance instruction execution link is intuitive, transparent, traceable, and what you see is what you get. Therefore, not only can rich detailed information of the distributed database be provided, but also the learning cost and professional knowledge requirement of the user can be reduced, so that non-operation and maintenance personnel can also perform operation and maintenance instruction execution processing (for example, targeted operation and maintenance and management of a certain type of node) on the distributed database, the difficulty of operation and maintenance instruction execution processing is reduced, and the efficiency of operation and maintenance instruction execution processing is effectively improved. This helps to reduce the operation and maintenance difficulty of the distributed database and improve the operation and maintenance efficiency of the distributed database.

[0073] Further, in response to the triggering instruction of the execution control 340, an execution window is entered, as shown in Figure 9 The execution window can include a parallel execution selection control 360, a serial execution selection control 370, a serial order setting area 380, and a determination execution control 390. In response to the triggering instruction of the parallel execution selection control 360, the parallel execution mode is determined as the instruction execution mode. The parallel execution mode is to control all target nodes to execute the corresponding operation and maintenance instruction at the same time. Alternatively, in response to the triggering instruction of the serial execution selection control 370, the serial execution mode is determined as the instruction execution mode. In response to the setting instruction of the serial order setting area 380, the execution order of each target node is set. The serial execution mode is to control each target node to execute the corresponding operation and maintenance instruction in sequence according to the execution order of each target node. In response to the triggering instruction of the determination execution control 390, each target node executes the corresponding operation and maintenance instruction according to the instruction execution mode.

[0074] Here, the user can enter the execution window by triggering the execution control 340, in which the user can determine the parallel execution mode as the instruction execution mode by triggering the parallel execution selection control 360 when the user wants to concurrently execute the operation and maintenance operation, or determine the serial execution mode as the instruction execution mode by triggering the serial execution selection control 370 when the user wants to serially execute the operation and maintenance operation, and set the execution order of each target node in the serial order setting area 380. After setting the instruction execution mode, the user can control each target node to execute the corresponding operation and maintenance instruction according to the instruction execution mode by triggering the determination execution control 390. In this way, the concurrent execution mode or the serial execution mode of the operation and maintenance instruction can be determined through the interactive display technology, so that non-operation and maintenance personnel can also set the required execution mode according to the actual demand, effectively improving the convenience of operation and maintenance instruction execution processing, which helps to reduce the operation and maintenance difficulty of the distributed database and improve the operation and maintenance efficiency of the distributed database.

[0075] In some embodiments, in the step C02, the control of the target node to execute the corresponding operation and maintenance instruction can include the following steps: If the operation and maintenance instruction of the target node is of the structured query language command type, the structured query language import method is used to control the target node to execute the corresponding operation and maintenance instruction. Alternatively, if the operation and maintenance instruction of the target node is of the structured query language file type, the file loading method is used to control the target node to execute the corresponding operation and maintenance instruction. Alternatively, if the operation and maintenance instruction of the target node is of the command line interpreter command type, the secure shell protocol method is used to control the target node to execute the corresponding operation and maintenance instruction.

[0076] In the embodiment, for each node, when the target node is controlled to execute the operation and maintenance instruction, the type of the operation and maintenance instruction pre-set by the node is first detected, which can include the SQL command type, the SQL file type and the Shell command type, and then a method corresponding to the detected type is used to control the target node to execute the operation and maintenance instruction. In specific implementation, if the operation and maintenance instruction is detected as the SQL command type, the SQL import method is used to execute the operation and maintenance operation; if the operation and maintenance instruction is detected as the SQL file type, the file loading method is used to execute the operation and maintenance operation; and if the operation and maintenance instruction is detected as the Shell command type, the SHH method is used to execute the operation and maintenance operation. In this way, the appropriate operation and maintenance method is determined according to the type of the operation and maintenance instruction of the target node, and the operation and maintenance operation is implemented accordingly, which can effectively improve the operation and maintenance quality of the distributed database and further improve the operation and maintenance accuracy of the distributed database.

[0077] In some embodiments, in the step C04, the operation and maintenance execution result of each target node is received and processed, including at least one of the following: The operation execution result is formatted; The operation execution result is data-converted; The operation execution result is highlighted and visualized.

[0078] In this embodiment, the receiving processing can include at least one of the formatting processing, the data conversion processing, and the highlighting and visualizing processing. The formatting processing refers to processing the line breaks, spaces, and the like in the returned operation execution result, and adding dividing contents such as tab characters to achieve standard data of the data. The data conversion processing can include, but is not limited to, separator conversion, data encoding conversion, case conversion, data truncation processing, data precision conversion, and the like. The highlighting and visualizing processing can be highlighting the returned operation execution result, or highlighting the abnormal information in the returned operation execution result, but is not limited thereto. In this way, by receiving and processing the operation execution result, the standardized output of the operation execution result can be achieved, and the readability of the operation execution result is improved, thereby helping to improve the operation convenience of the distributed database.

[0079] In some embodiments, with reference to Figure 10 In the step S104, the database logs of the nodes are analyzed to obtain the log analysis report of the distributed database and to visualize the log analysis report, which can include the following steps D01-D03: D01, obtaining the database logs of the nodes; D02, performing format recognition processing on the database logs of the nodes to obtain the formats of the database logs of the nodes; D03, performing analysis processing on the database logs of the nodes according to the formats of the database logs of the nodes to obtain the log analysis report of the distributed database and to visualize the log analysis report.

[0080] This embodiment is the log analysis link of the distributed database: the database logs of the nodes are uniformly collected, and are converted into formats such as Hyper Text Markup Language (HTML) and the like through automatic processing and are visualized, so that the user can more intuitively understand the overall status of the distributed database.

[0081] In a specific implementation, first, the SCP file transfer service is invoked to transfer the database logs of all nodes to the deployment end (i.e., the server) of the embodiment, so as to collect the original log files stored in multiple remote targets. Then, the database logs of each node are subjected to format identification processing to determine the format of the database logs of each node, which can include any of a table (csv) format or a log (log) format. Subsequently, according to the format of the database logs of each node, the database logs of each node are automatically generated into an HTML format log analysis report, i.e., an HTML file, by using character processing. Finally, the generated HTML file is visualized, thereby realizing log analysis processing of the distributed database.

[0082] As can be seen, the embodiment unifies the collection of log information of each node in the distributed database cluster, analyzes and processes the log information by using character processing, outputs an HTML format log analysis report, and visualizes the report. This can provide a more intuitive and easy-to-understand log analysis report, which enables users to quickly know the actual running status of the distributed database and helps to quickly achieve purposes such as locating problems of the distributed database and speeding up fault recovery of the distributed database.

[0083] In one example, the database operation and maintenance manner of the related art is often limited to professional operation and maintenance personnel, which requires high professional knowledge of users, and non-operation and maintenance personnel are difficult to manage and maintain the distributed database, thereby resulting in low convenience and high difficulty of operation and maintenance of the distributed database. To this end, the embodiment introduces interactive display technology to realize the log analysis link in the operation and maintenance process of the distributed database. The database operation and maintenance interface is displayed in the terminal, and the database operation and maintenance interface can include the log analysis control 410. In response to a triggering instruction of the log analysis control 410, the log analysis interface is displayed, as shown in Figure 11 The log analysis interface displays the HTML format log analysis report. Here, the user enters the log analysis interface by triggering the log analysis control 410, and the log analysis interface displays the HTML format log analysis report. In this way, the embodiment introduces interactive display technology to realize the log analysis link in the operation and maintenance process of the distributed database, so that the log analysis report is intuitive, transparent, traceable, and what you see is what you get. This not only provides rich log analysis detail information, but also reduces the learning cost and professional knowledge requirement of users, so that non-operation and maintenance personnel can also analyze the log of the distributed database, reduces the difficulty of log analysis, and effectively improves the efficiency of log analysis. This helps to reduce the operation and maintenance difficulty of the distributed database and improve the operation and maintenance efficiency of the distributed database.

[0084] In addition, with reference toFigure 12 The embodiment of the present application further provides a distributed database operation and maintenance system, which can comprise: a cluster metadata collector (CMC) configured to perform configuration processing on the distributed database to obtain metadata of the distributed database and attribute data of each node in the distributed database, wherein the metadata is used to indicate a configuration mode of the distributed database, and the attribute data is used to indicate a connection process of the node and the distributed database; an automatic operation and maintenance instruction storage (ACS) configured to allocate corresponding operation and maintenance instructions to each node according to the metadata and the attribute data of each node; a distributed request execution scheduler (DRES) configured to determine at least one node corresponding to the node selection instruction as a target node in response to the node selection instruction, control the at least one target node to execute the corresponding operation and maintenance instruction, obtain operation and maintenance execution results of the at least one target node and store the operation and maintenance execution results in a log form; a distributed database log collection analyzer (DDLCA) configured to analyze and process database logs of each node to obtain a log analysis report of the distributed database and perform visualization.

[0085] The contents in the method embodiments are all applicable to the system embodiments, the system embodiments specifically implement the functions same as those of the method embodiments, and achieve the same beneficial effects as those of the method embodiments.

[0086] In addition, the embodiment of the present application further provides an electronic device, which can comprise: at least one processor; at least one memory configured to store at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the distributed data operation and maintenance method.

[0087] Reference Figure 13 , Figure 13 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device comprises: The processor 1301 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing relevant programs to implement the technical solutions provided by the embodiments of the present application. The memory 1302 can be implemented by a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 1302 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, relevant program codes are stored in the memory 1302 and are called and executed by the processor 1301 to implement the above-mentioned methods of the embodiments of the present application. The input / output interface 1303 is configured to implement information input and output. The communication interface 1304 is configured to implement the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.). The bus 1305 is configured to transmit information between various components (for example, the processor 1301, the memory 1302, the input / output interface 1303, and the communication interface 1304) of the device. The processor 1301, the memory 1302, the input / output interface 1303, and the communication interface 1304 are connected to each other through the bus 1305 to realize the communication connection between them in the device.

[0088] It can be understood that the contents in the above-mentioned method embodiments are all applicable to the device embodiments. The device embodiments specifically implement the same functions as the above-mentioned method embodiments, and achieve the same beneficial effects as the above-mentioned method embodiments.

[0089] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the above-mentioned distributed database operation and maintenance method.

[0090] It can be understood that the contents in the above-mentioned method embodiments are all applicable to the storage medium embodiments. The storage medium embodiments specifically implement the same functions as the above-mentioned method embodiments, and achieve the same beneficial effects as the above-mentioned method embodiments.

[0091] The embodiment of the application further provides a computer program product comprising a computer program which, when executed by a processor, implements the method described above.

[0092] It can be understood that the contents in the method embodiments described above are applicable to the program product embodiments, the program product embodiments specifically implement the functions same as those of the method embodiments described above, and achieve the beneficial effects same as those of the method embodiments described above.

[0093] The principle of the distributed database operation and maintenance method, system, device and program product provided by the embodiment of the application will be explained by an example. The distributed database involved in the example is a TeleDB-X database.

[0094] With reference to Figure 12 The example provides a distributed database operation and maintenance system, in which a cluster metadata collector is configured to automatically collect and parse metadata information and node information of a TeleDB-X database cluster through a configuration file. An automatic operation and maintenance instruction storage is configured to store automatic operation and maintenance instructions set by a user, and automatically publish the operation and maintenance instructions to each node of the TeleDB-X database for subsequent calling according to the cluster metadata and the node information. A distributed request execution scheduler is configured to automatically encapsulate and display the metadata information and the node information to the user, and automatically prompt the user of the quality of the operation and maintenance instructions in the operation and maintenance instruction storage, so as to realize quick operation and maintenance operation of multiple nodes. A distributed database log collection analyzer is configured to process database logs of multiple nodes in the TeleDB-X database cluster through text processing, so as to automatically generate an HTML format log analysis report from the numerous logs.

[0095] The specific process of the example for operation and maintenance and management of the TeleDB-X database is as follows: S301, the cluster metadata collector selects one node as a seed node from multiple nodes, and obtains a configuration file of the seed node, the configuration file being a pre-set file, and then connects the seed node to the TeleDB-X database cluster by using the configuration file of the seed node. After being connected to the TeleDB-X database cluster, the cluster metadata collector automatically obtains attribute data of each node and metadata of the TeleDB-X database cluster, such as node version, connection address, port, storage path, attribute data for connecting the database, and metadata such as username, database version, database list, table list, according to the configuration file by executing an SQL statement, so as to configure corresponding attribute data for each node and configure metadata for the TeleDB-X database.

[0096] S302, the automation operation instruction storage determines at least one operation instruction corresponding to each node according to the metadata and the attribute data of each node, and issues the at least one operation instruction corresponding to each node to each node, including view, stored procedure, custom function, Shell instruction, etc.

[0097] S303, the distributed request execution scheduler classifies and processes each node to obtain a plurality of node category groups, and determines all nodes in a node category group corresponding to a selected instruction as target nodes in response to the selected instruction. For example, the primary nodes are primary DN1 and primary DN2, the secondary nodes of primary DN1 are slave DN1, and the secondary nodes of primary DN2 are slave DN2. Accordingly, these nodes are classified to obtain a primary node category group including primary DN1 and primary DN2, a slave node category group including slave DN1 and slave DN2, a separate category group of primary DN1, a separate category group of primary DN2, a separate category group of slave DN1, and a separate category group of slave DN2, but not limited thereto. Subsequently, the distributed request execution scheduler distributes operation instructions to each target node and controls each target node to execute corresponding operation instructions. In specific implementation, if it is detected that the operation instruction is of a SQL command type, a SQL import method is used to execute the operation; if it is detected that the operation instruction is of a SQL file type, a file load method is used to execute the operation; if it is detected that the operation instruction is of a Shell command type, an SHH method is used to execute the operation. Subsequently, the distributed request execution scheduler obtains operation execution results returned by each target node and performs processing such as format processing, data conversion processing, and highlight visualization processing, and stores the processed results into a database log of each target node.

[0098] S304, the TeleDB-X database log collection analyzer calls an SCP file transmission service to transmit the database logs of all nodes to the deployment end (i.e., server) of the embodiment, and then performs format identification processing on the database logs of each node to determine the format of the database logs of each node, which can include any one of table format or log format, and finally uses character processing to automatically generate an HTML file from the database logs of each node through quote, and visualizes the generated HTML file.

[0099] The distributed database operation and maintenance method, system, device and program product provided by the application first determine the metadata of the distributed database and the attribute data of each node in the distributed database through configuration processing of the distributed database, and allocate corresponding operation and maintenance instructions to each node according to the metadata and the attribute data, then select the node required to be operated and maintained, that is, the target node, in response to the node selection instruction, and control the selected node to execute the operation and maintenance instruction to obtain the corresponding operation and maintenance execution result and store it in the form of a log, finally analyze and process the database log of each node, thereby realizing the automatic management and operation and maintenance processing of the distributed database. In this way, not only can the operation and maintenance demand of the distributed database be quickly responded, especially the large-scale operation and maintenance demand, but also the node required to be operated and maintained can be ensured to be managed and operated in a targeted manner, the operation and maintenance quality of the distributed database is improved, and the operation and maintenance efficiency and operation and maintenance precision of the distributed database are effectively improved.

[0100] The embodiments described in the embodiments of the application are used to more clearly illustrate the technical solutions of the embodiments of the application, and do not constitute a limitation on the technical solutions provided by the embodiments of the application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the application are also applicable to similar technical problems.

[0101] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the application, and can include more or fewer steps than the figures, or combine certain steps or different steps.

[0102] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the application.

[0103] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the function modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0104] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of orderly or chronological mundane operation, reverse order operation, based on circuitry availability, based on stated preference or the like, and that "default" or other orderings are thus permissible. Further, the terms "comprise", "comprising", "include", "including", and the like, are specifically intended to be open-ended. That is, references to individual steps and the like do not suhstantially exclude the presence of two or more of a given step or its integral presence in the process, method, system, article, or apparatus having been made with a wider scope. The use of notation such as "first", "second", "third", etc. does not generally limit the areas, but is used to connect like elements or to distinguish one claim from another. These terms can be used interchangeably when appropriate. Terms concerning the relative position of elements can be interpreted such that their use adheres to their normal meaning, but they can also be interpreted to mean the opposite according to specific claims.

[0105] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are only A, only B, and A and B at the same time. Where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and back associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c, can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0106] In several embodiments provided by the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the above-described device embodiments are only illustrative, for example, the division of the above-mentioned units is only a logical functional division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0107] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the application.

[0108] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0109] If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0110] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and the scope of the rights of the embodiments of the present application is not limited thereto. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A distributed data operation and maintenance method, characterized in that, The method comprises the following steps: configuring a distributed database to obtain metadata of the distributed database and attribute data of each node in the distributed database; the metadata is used to indicate a configuration mode of the distributed database, and the attribute data is used to indicate a connection process of the node and the distributed database; allocating corresponding operation and maintenance instructions for each node according to the metadata and the attribute data of each node; in response to a node selection instruction, determining at least one node corresponding to the node selection instruction as a target node, controlling at least one target node to execute corresponding operation and maintenance instructions, obtaining operation and maintenance execution results of at least one target node and storing the operation and maintenance execution results in a log form; analyzing database logs of each node to obtain a log analysis report of the distributed database and visualizing the log analysis report.

2. The method of claim 1, wherein, The configuration of the distributed database to obtain the metadata of the distributed database and the attribute data of each node in the distributed database comprises: acquiring a configuration file of a seed node and connecting the seed node to the distributed database; configuring corresponding attribute data for each node according to the configuration file, and configuring the metadata for the distributed database.

3. The method of claim 1, wherein, The allocation of the corresponding operation and maintenance instructions for each node according to the metadata and the attribute data of each node comprises: determining at least one operation and maintenance instruction corresponding to each node according to the metadata and the attribute data of each node; downloading at least one operation and maintenance instruction corresponding to each node to each node.

4. The method of claim 1, wherein, The response to the node selection instruction to determine at least one node corresponding to the node selection instruction as a target node, control at least one target node to execute corresponding operation and maintenance instructions, obtain operation and maintenance execution results of at least one target node and store the operation and maintenance execution results in a log form comprises: classifying each node to obtain a plurality of node category groups; wherein the node category group comprises at least one node belonging to a category corresponding to the node category group; in response to a selection instruction of at least one node category group, determining all nodes in the node category group corresponding to the selection instruction as the target node; controlling each target node to execute corresponding operation and maintenance instructions; receiving and processing operation and maintenance execution results of each target node, and storing the operation and maintenance execution results of each target node in a database log of each target node.

5. The method of claim 4, wherein, The control of each target node to execute corresponding operation and maintenance instructions comprises: if the operation and maintenance instruction of the target node is a structured query language command type, the target node is controlled to execute the corresponding operation and maintenance instruction by a structured query language import method; or, if the operation and maintenance instruction of the target node is a structured query language file type, the target node is controlled to execute the corresponding operation and maintenance instruction by a file loading method; or, if the operation and maintenance instruction of the target node is a command line interpreter command type, the target node is controlled to execute the corresponding operation and maintenance instruction by a secure shell protocol method.

6. The method of claim 4, wherein, The receiving processing of the operation and maintenance execution result of each target node includes at least one of the following: The operation and maintenance execution result is formatted; The operation and maintenance execution result is data converted; The operation and maintenance execution result is highlighted and visualized.

7. The method of claim 1, wherein, The analysis processing of the database log of each node obtains the log analysis report of the distributed database and visualizes it, including: Obtain the database log of each node; Format identification processing is performed on the database log of each node to obtain the format of the database log of each node; According to the format of the database log of each node, the database log of each node is analyzed to obtain the log analysis report of the distributed database and visualized.

8. A distributed database operation and maintenance system, characterized in that, It includes: The cluster metadata collector is used for configuring the distributed database to obtain the metadata of the distributed database and the attribute data of each node in the distributed database; wherein the metadata is used to indicate the configuration mode of the distributed database, and the attribute data is used to indicate the connection process of the node and the distributed database; The automatic operation and maintenance instruction storage is used to allocate corresponding operation and maintenance instructions for each node according to the metadata and the attribute data of each node; The distributed request execution scheduler is used to determine at least one target node corresponding to the node selection instruction in response to the node selection instruction, control at least one target node to execute the corresponding operation and maintenance instruction, obtain the operation and maintenance execution result of at least one target node and store it in the form of log; The log collection analyzer is used for analyzing the database log of each node to obtain the log analysis report of the distributed database and visualizing it.

9. An electronic device, comprising: It includes: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a distributed data operation and maintenance method as claimed in any one of claims 1-7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement a distributed data operation and maintenance method as claimed in any one of claims 1 to 7.

Images