Online Data Processing Method, Device and Program Product of Database

Abstract

An embodiment of the present application provides a method, an apparatus, and a program product for processing online data of a database, relating to databases and data queries. In this method, a proxy service is set for each node of the database, and the management server processes data processing requests. The data processing requests are split into one or more tasks to be processed, each task to be processed corresponds to a node, and the tasks to be processed are sent to the proxy services of the corresponding nodes. The proxy services of each node execute the tasks to be processed to obtain the online data of the database and return it to the management server, thereby realizing the management of the online data of the database system.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of databases, and in particular, to a method, an apparatus, and a program product for processing online data of a database. Background Art

[0002] A database management system (DMS) is a large software for manipulating and managing databases, used to uniformly manage and control databases to ensure the security and integrity of the databases.

[0003] In the prior art, Apache Calcite can be used for database management. Apache Calcite is an open-source data management framework widely used in various data processing projects and tools. Apache Calcite provides a standard Structured Query Language (SQL), various query optimizations, and a basic framework for connecting various data sources, allowing users to easily access various data and implement querying data using SQL.

[0004] However, existing data management methods can usually only manage offline data of databases and cannot efficiently manage online data of databases. Summary of the Invention

[0005] Embodiments of the present application provide a method, an apparatus, and a program product for processing online data of a database, which can dynamically process online data of a database in a unified manner.

[0006] In a first aspect, embodiments of the present application provide a method for processing online data of a database, which is applied to a management server. The management server is used to manage at least one database. Each database includes at least one node, and each node is provided with a proxy service. The proxy service is used to control the corresponding node. The method includes:

[0007] Determine a to-be-processed task corresponding to the data processing request and a scheduling node associated with the to-be-processed task according to the data processing request sent by the client. The data processing request is used to request processing of online data of the database;

[0008] Send the to-be-processed task to the proxy service corresponding to the scheduling node. The proxy service corresponding to the scheduling node is used to communicate with the scheduling node according to the to-be-processed task and obtain an execution result of the to-be-processed task;

[0009] Receive the execution result returned by the proxy service corresponding to the scheduling node;

[0010] Determine the processing result corresponding to the data processing request according to the execution result corresponding to the to-be-processed task, and send the processing result to the client.

[0011] In a second aspect, an embodiment of the present application provides a method for processing online data of a database. The database includes at least one node, and each node is provided with an agent service for controlling the corresponding node. The method includes:

[0012] The agent service receives the to-be-processed task sent by the management server;

[0013] The agent service communicates with the node corresponding to the agent service according to the to-be-processed task, and obtains the execution result of the to-be-processed task;

[0014] The agent service sends the execution result to the management server.

[0015] In a third aspect, an embodiment of the present application provides a device for processing online data of a database. The device is applied to a management server, and the management server is used to manage at least one database. Each database includes at least one node, and each node is provided with an agent service for controlling the corresponding node. The device includes:

[0016] A determination module, configured to determine the to-be-processed task corresponding to the data processing request and the scheduling node associated with the to-be-processed task according to the data processing request sent by the client. The data processing request is used to request processing of the online data of the database;

[0017] A distribution module, configured to send the to-be-processed task to the agent service corresponding to the scheduling node. The agent service corresponding to the scheduling node is used to communicate with the scheduling node according to the to-be-processed task and obtain the execution result of the to-be-processed task;

[0018] A receiving module, configured to receive the execution result returned by the agent service corresponding to the scheduling node;

[0019] A sending module, configured to determine the processing result corresponding to the data processing request according to the execution result corresponding to the to-be-processed task, and send the processing result to the client.

[0020] In a fourth aspect, an embodiment of the present application provides a device for processing online data of a database. The database includes at least one node, and each node is provided with an agent service for controlling the corresponding node. The device is applied to the agent service, and the device includes:

[0021] A receiving module, configured to receive a to-be-processed task sent by a management server;

[0022] A processing module, configured to communicate with a node corresponding to the proxy service according to the to-be-processed task, and obtain an execution result of the to-be-processed task;

[0023] A sending module, configured to send the execution result to the management server.

[0024] In a fifth aspect, an embodiment of the present application provides a management server, which includes: a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method described in the first aspect above.

[0025] In a sixth aspect, an embodiment of the present application provides a server, which includes: a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method described in the second aspect above.

[0026] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program, and the computer program enables a computer to execute the method described in the first aspect or the second aspect above.

[0027] In an eighth aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method described in the first aspect or the second aspect above.

[0028] The technical solution provided by the embodiment of the present application is as follows: by setting a proxy service for each node of the database, the management server processes the data processing request, splits the data processing request into one or more to-be-processed tasks, each to-be-processed task corresponds to a node, and distributes the to-be-processed task to the proxy service of the corresponding node. The proxy services of each node execute the to-be-processed task to obtain the online data of the database and return it to the management server, thereby realizing the management of the online data of the database system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0030] Figure 1 It is a schematic diagram of an application scenario applicable to an embodiment of the present application;

[0031] Figure 2 Another schematic diagram of an application scenario applicable to the embodiments of the present application;

[0032] Figure 3 Another schematic diagram of an application scenario applicable to the embodiments of the present application;

[0033] Figure 4 Flowchart of the method for processing online data of the database provided in Embodiment 1 of the present application;

[0034] Figure 5 Flowchart of the method for processing online data of the database provided in Embodiment 2 of the present application;

[0035] Figure 6 Schematic diagram of the process for node shutdown;

[0036] Figure 7 Signaling flowchart of the method for processing online data of the database provided in Embodiment 3 of the present application;

[0037] Figure 8 Schematic diagram of a functional module of the management server;

[0038] Figure 9 Schematic diagram of the structure of the device for processing online data of the database provided in Embodiment 4 of the present application;

[0039] Figure 10 Schematic diagram of the structure of the device for processing online data of the database provided in Embodiment 5 of the present application;

[0040] Figure 11 Schematic diagram of a structure of the server provided in the embodiments of the present application. Detailed implementation manners

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0042] It should be noted that in the description of the present invention, the terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0043] It can be understood that in the specific embodiments of the present application, when it comes to data related to user information, etc., when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0044] To facilitate the understanding of the embodiments of the present application, before describing each embodiment of the present application, some concepts involved in all embodiments of the present application will be appropriately elaborated first.

[0045] A database, in short, can be regarded as an electronic filing cabinet - a place for storing electronic files, where users can perform operations such as adding, querying, updating, and deleting data in the files. The so-called "database" is a collection of data stored together in a certain way, shared by multiple users, with as little redundancy as possible, and independent of application programs.

[0046] A database management system (DBMS for short) is a computer software system designed to manage a database, generally having basic functions such as storage, interception, security guarantee, and backup. The database management system can be classified according to the database model it supports, such as relational, XML (Extensible Markup Language); or according to the type of computer it supports, such as server clusters, mobile phones; or according to the query language it uses, such as SQL (Structured Query Language), XQuery; or according to the key points of performance impulse, such as maximum scale, highest running speed; or other classification methods. No matter which classification method is used, some DBMS can cross categories. For example, they can support multiple query languages at the same time.

[0047] MySQL is an open-source relational database management system (RDBMS for short). Relational databases store data in different tables, which increases query speed and flexibility compared to storing all data in a large warehouse. The SQL language used by MySQL is the most commonly used standardized language for accessing databases.

[0048] In a database system, the task of the optimizer (also known as the query optimizer) is to select the most effective execution plan to execute the query submitted by the user. A database system is also called a database cluster. Common types of query optimizers include:

[0049] Rule-Based Optimizer (RBO), which performs query optimization based on a set of predefined heuristic rules. According to these rules, it selects one of the many possible execution plans of the SQL as the final execution plan of the SQL.

[0050] Cost-Based Optimizer (CBO for short), which evaluates the costs of different execution plans of the SQL based on a cost model and statistical information, and selects the plan with the lowest cost as the final execution plan of the SQL.

[0051] In the prior art, the management of a database system is usually the management of offline data of the database. Offline data can be understood as static data, which is usually stored in a fixed location or data table. When a user initiates a query or other processing of the offline data, the management server of the database system reads the corresponding offline data from the fixed location or data table according to the data processing request.

[0052] This offline data is collected and stored in the fixed location or data table through some services. This offline data can be the on-off state, log data, running state, etc. of nodes (i.e., devices) in the database system. This offline data is not the current real-time data of the database system, but some historical data.

[0053] However, the devices and data in the database system are dynamically changing. Obviously, the management of offline data cannot meet the user's needs. Users need to obtain the online data of the database system in real time. This online data is also called dynamic data. This online data is not stored in a fixed location or table and needs to be obtained in real time through a certain method.

[0054] The embodiments of the present application provide a method for processing dynamic data of a database, which can manage the online data of the database. Of course, this method can also manage the offline data of the database.

[0055] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0056] For example, Figure 1 A schematic diagram of an application scenario applicable to the embodiment of the present application is shown in FIG. Figure 1 As shown, the application scenario includes: a client 10, a management server 20 and a database system 30. The client 10 and the management server 20 are connected via a communication network, and the management server 20 and the database system 30 are connected via a communication network.

[0057] The client 10 is a user-oriented interface and a front-end application of the database system. The user can log in or access the management server 20 through the client 10, access the database system 30 through the management server 20, and manage the database system 30. For example, the user initiates a data processing request on the client 10, and the data processing request is used to request processing of online data of the database system 30.

[0058] The device (eg, computer) where the client 10 is located is called a client device or a client computer. In this application scenario, the number of clients 10 can be multiple or large. For example, in a big data scenario, a large number of clients 10 access the database system 30 at the same time.

[0059] The management server 20 is also called a management device, which is used to receive the data processing request sent by the client 10, parse the data processing request 10, determine the pending task corresponding to the data processing request and the scheduling node associated with the pending task, and send the pending task to the corresponding scheduling node, which is a node in the database system 30.

[0060] The database system 30 includes at least one node, each of which is a server (or database server). Each node is provided with an agent service, and each agent service controls its corresponding node. The agent service can be a software or applet that can run independently, and the agent service can run on each node. Figure 3 In the application scenario shown, an agent service is installed on each node of the database system 30. Optionally, the agent service can also be run on a server alone, that is, the agent service does not run on the node.

[0061] The agent service is the actual execution entity for the tasks to be processed. The agent service executes or processes the tasks to be processed sent by the management server 20, and obtains the execution results of the tasks to be processed. The data in the execution results is the online data of the database system 30, so that the online data of the database system 30 can be obtained through the agent service.

[0062] The agent service returns the execution results of the tasks to be processed to the management server 20. The management server 20 determines the processing results corresponding to the data processing requests according to the execution results returned by each node, and returns the processing results to the client for display.

[0063] The database system 30 can be a relational database, a non-relational database, or a file system, etc. Common relational databases include MySQL, Oracle, SQLServer, DB2, Hive, HBase, Spark, etc.

[0064] It should be noted that the above management server 20 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud security, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0065] Figure 2 Another schematic diagram of an application scenario applicable to the embodiments of the present application. Comparing Figure 1 and Figure 2 , in Figure 2 In the application scenario shown, the agent services of each node of the database system 30 run on a separate proxy server 40, and the agent services of each node are respectively communicatively connected to their corresponding nodes.

[0066] Figure 2 In the scenario shown, the agent services of each node of the database system 30 run on a proxy server 40. It can be understood that the agent services of each node of the database system 30 can also run on multiple proxy servers 40, or the agent service of each node runs on a proxy server 40 respectively. The embodiments of the present application do not limit this.

[0067] Figure 3 Another schematic diagram of an application scenario applicable to the embodiments of the present application. Comparing Figure 1 and Figure 3 , in Figure 3In the application scenario shown, there are two database systems: database system 30 and database system 50. Both database system 30 and database system 50 can include multiple nodes. Database system 30 and database system 50 can be two databases of the same type or two databases of different types. For example, database system 30 is a MySQL database and database system 50 is a non-relational database. Thus, it can support referencing multiple different types of data sources in a single query, enabling the system to store across multiple data sources and meet the requirements of complex queries.

[0068] It should be noted that the above database system 30 or database system 50 can adopt a distributed system.

[0069] Based on Figures 1-3 the application scenario shown, the technical solution of this application will be elaborated in detail below.

[0070] Figure 4 The flowchart of the method for processing online data of the database provided in the first embodiment of this application. The method of this embodiment is executed by the management server, as Figure 4 shown, the method provided in this embodiment includes the following steps:

[0071] S101. According to the data processing request sent by the client, determine the to-be-processed task corresponding to the data processing request and the scheduling node associated with the to-be-processed task. This data processing request is used to request the processing of the online data of the database system.

[0072] The management server receives the data processing request sent by the client. This data processing request is used to request the processing of the online data of the database system. This online data is also referred to as dynamic data or real-time data.

[0073] The processing method provided in the embodiment of this application is still compatible with the existing management of offline data. The management server can execute different processing flows according to the different data types requested by the data processing request.

[0074] In one implementation, the data processing request sent by the client includes a data type identifier, which is used to identify whether the requested data type is online data or offline data. For example, the identifier of this data type is 1 and 0. When the identifier of this data type is 1, it indicates that the data requested by this data processing request is online data. When the identifier of this data type is 0, it indicates that the data requested by this data processing request is offline data.

[0075] In another implementation, the data processing request message or format is used to distinguish whether the data requested by the data processing request is online data or offline data.

[0076] When the management server determines that the data requested by the data processing request is offline data through the above two methods, it reads the corresponding data from the storage location or data table of the offline data and returns it to the client. When the management server determines that the data requested by the data processing request is online data through the above two methods, it executes the method provided in the embodiment of the present application to communicate with the nodes of the database in real time to obtain the online data.

[0077] The client and the management server communicate using a fixed database protocol, which is also called a data query language. Exemplarily, the database protocol is the MySQL protocol, the ES (Elasticsearch) protocol, etc. The MySQL protocol refers to the data query language used by the MySQL database, and the ES protocol refers to the data query language used by the ES system. When the management server uses the MySQL protocol, the data processing request is an SQL request (or called an SQL statement).

[0078] When the management server determines that the data processing request is used to request online data processing of the database system, the management server parses the data processing request and parses it into a query logic representation for internal use; according to the query logic representation, it determines the processing operation corresponding to the data processing request and the nodes to be accessed by the processing operation; a to-be-processed task is generated according to the processing operation corresponding to the data processing request, and the nodes to be accessed by the processing operation are determined as the scheduling nodes associated with the to-be-processed task.

[0079] The query logic representation is used to describe the logical structure of the query, including the tables queried, join conditions, selection conditions, operators, etc. It is usually represented in a tree structure. For example, the query logic representation can be an Abstract Syntax Tree (AST for short) or a Query Tree.

[0080] AST is a data structure commonly used in programming language processors to represent the syntax structure of source code. AST is a tree structure, where each node represents a syntax structure in the source code, and the child nodes represent the components of that structure. AST is very useful for parsing, analyzing, and transforming source code.

[0081] The SQL query tree refers to converting an SQL statement into an internal representation form that the database system can understand. In a relational database, logical operators are usually used to construct the query tree, such as SELECT, FROM, WHERE, etc.

[0082] Taking the SQL query tree as an example, the SQL statement is parsed into a query tree, which is completed by a series of operators and operands. The operands are tables in the database. A query tree usually includes multiple nodes, where the parent nodes are operators and the leaf nodes are operands. The operators are also called logical operators, and the operators include: select, update, from, where, etc.

[0083] The processing operations corresponding to the data processing requests are, for example, query operations (select), delete operations (delete), update operations (update), statistical operations (count), etc. The processing operations can be determined according to the operators in the query logic representation. The objects to be accessed by the processing operations can be nodes or the tables on the nodes. To access the tables on the nodes, the nodes where the tables are located need to be found first. Therefore, the ultimate objects to be accessed by the processing operations are all nodes, and the nodes to be accessed by the data processing operations can be determined according to the processing operations and operands in the query logic representation.

[0084] The nodes to be accessed by the processing operations can be one or more. When the nodes to be accessed by the processing operations are multiple, these multiple nodes may come from one database system or may come from multiple database systems. That is, the method of this embodiment supports multiple data sources and allows multiple different types of data sources to be referenced in a single data processing request. This flexibility enables the system to store across multiple data sources, thus better meeting the requirements of complex queries.

[0085] After the management server parses the processing operations corresponding to the data processing requests and the nodes to be accessed by the processing operations, it generates a task to be processed according to the processing operations corresponding to the data processing requests. The task to be processed is used to describe the specific tasks that the agent service needs to process. The task to be processed is a task that the agent service can directly execute or process, and the task to be processed includes user information, processing operation information, etc. The scheduling nodes associated with the task to be processed are the nodes to be accessed by the processing operations, and the management server generates a task to be processed for each scheduling node.

[0086] Taking the SQL statement as an example, mysql>select server_id, server_ip, status from server-list. Through this SQL statement, the status (status) of all servers in the database system can be queried. Correspondingly, the task to be processed is used to query the status of the servers (i.e., the power-on / off status), and the scheduling nodes associated with the task to be processed are all servers. Among them, the tasks to be processed generated for each scheduling node are the same, all for querying the status of the server.

[0087] In one implementation manner, according to the query logic representation, the processing operation corresponding to the data processing request and the node to be accessed by the processing operation are determined. Specifically: according to the operators included in the query logic representation and the operators corresponding to each predefined operation type, the operation type corresponding to the data processing request is determined; according to the operation type corresponding to the data processing request and the query logic representation, the processing operation corresponding to the data processing request and the node to be accessed by the processing operation are determined.

[0088] In the embodiments of the present application, multiple operation types can be defined, and each operation type corresponds to one or more operators. Exemplarily, when the operation types include: data operation type and data query type, wherein, the data operation type is used to operate on the node or the data on the node, and the data query type is used to query the data of the node.

[0089] Optionally, the operators corresponding to the data operation type include: update, delete, count, etc., and the operator corresponding to the data query type includes select.

[0090] The operator included in the query logic representation is matched with the operator corresponding to each predefined operation type. When the operator included in the query logic representation is the same as the operator corresponding to the target operation type, it is determined that the operation type corresponding to the data processing request is the target operation type.

[0091] When the operation type corresponding to the data processing request is the data operation type, the management server determines the processing operation corresponding to the data processing request and the node to be accessed by the processing operation according to the query logic representation. When the operation type corresponding to the data processing request is the data query type, the query logic representation is optimized for query, and an execution plan corresponding to the query logic representation is obtained. According to the execution plan corresponding to the query logic representation, the processing operation corresponding to the data processing request and the node to be accessed by the processing operation are determined.

[0092] Through the method of this embodiment, the dynamic management of the database data can be realized. The dynamic management of the database system includes but is not limited to: querying the status of the server list of the database system, querying the running status of the server, querying the log, deleting the log, restarting the server, restarting the service, etc. Among them, the running status of the server includes the utilization rate of the Central Processing Unit (CPU), the utilization rate of the memory, and the Input Output (IO) rate, etc.

[0093] For example, when a data query request is used to query a list of servers, the corresponding processing operation for the data query request is a server status query operation, and the nodes to be accessed by the processing operation are all servers. The SQL statement corresponding to this data processing request can be mysql>select server_id, server_ip, status from server-list. Through this SQL statement, it can be queried that the status of all servers in the database system is 0. In this example, status = 0 is defined as the powered-on state.

[0094] Another example is that when a data query request is used to query the logs on a specific server, the corresponding processing operation for the data query request is to query the logs of the server, and the nodes to be accessed by the processing operation are the servers with the specified ID. The servers with the specified ID can be one or more. The SQL statement corresponding to this data processing request can be mysql>select server_id, level, msg from server_log where server_id = 1. Through this SQL statement, the logs of the server with server_id = 1 in the database system can be queried.

[0095] Another example is that when a data processing request is used to aggregate the number of logs on each server, the corresponding processing operation for the data query request is to aggregate and query the number of logs on each server (i.e., the number of log entries), and the nodes to be accessed by the processing operation are all servers. The SQL statement corresponding to this data processing request can be mysql>select server_id, count(*) from server_log group by server_id. Through this SQL statement, the number of logs on each server can be obtained.

[0096] Another example is that when a data query request is used to shut down a specific server, the corresponding processing operation for the data query request is to shut down the server, and the nodes to be accessed by the processing operation are the servers with the specified ID. The SQL statement corresponding to this data processing request can be mysql>update server_list, set status = 1 where server_id = 1. Through this SQL statement, the server with server_id = 1 is shut down. Among them, in this example, status = 1 is defined as the powered-off state.

[0097] It should be emphasized that the data sources of the power-on / off status data and log data of the above-mentioned servers are both dynamically generated or real-time generated. There is no data table in the database system for storing the power-on / off status data and log data. In the offline data processing method, the power-on / off status data and log data of the servers are stored in a fixed data table, and the management server can directly read the data from this fixed data table.

[0098] For example, for the power-on / off status data of the servers, the management server cannot directly read it from a certain data table. Instead, it needs to detect the power-on / off status of the servers in real time through the agent services of each server. For example, the agent services of each server judge the current power-on / off status of each server through heartbeat detection.

[0099] S102. Send the task to be processed to the proxy service corresponding to the scheduling node. The proxy service corresponding to the scheduling node is used to communicate with the scheduling node according to the task to be processed and obtain the execution result of the task to be processed.

[0100] The management server establishes connections with each node through the agent service of each node and manages each node through the agent service of each node.

[0101] The management server sends the corresponding task to be processed to the agent service corresponding to each scheduling node according to the generated task to be processed and the scheduling node associated with the task to be processed. The agent service executes the task to be processed to obtain the execution result.

[0102] In the embodiments of the present application, the online data to be processed by the task to be processed is both dynamically generated or real-time generated. Or, it is described that the data source of the task to be processed is dynamically generated or real-time generated. There is no data table in the database system for storing the online data. The online data is obtained in real time by the agent service of the scheduling node through a certain acquisition method. Different from offline data management, in the offline data processing method, the offline data to be managed by the management server is stored in a fixed data table, and the management server can directly read the offline data from this fixed data table. The offline data can be collected and stored in this fixed data table through some services.

[0103] S103. Receive the execution result returned by the proxy service corresponding to the scheduling node.

[0104] The agent services corresponding to each scheduling node return the execution result corresponding to the task to be processed to the management server.

[0105] When a data processing request is used to request the processing of online data in multiple databases, the scheduling nodes associated with the task to be processed include the nodes of the multiple databases, that is, the nodes to be accessed for the processing operation are the nodes of the multiple databases, or it can be described that the task to be processed is associated with the nodes of multiple databases. Among them, the scheduling nodes associated with the task to be processed can be all the nodes in the multiple databases, or may be some of the nodes in the multiple databases.

[0106] For example, when a data processing request is used to request the processing of online data in database A and database B, in actual situations, the scheduling nodes associated with the task to be processed may include all the nodes in database A and all the nodes in database B, or include all the nodes in database A and some of the nodes in database B, or include some of the nodes in database A and all the nodes in database B, or include some of the nodes in database A and some of the nodes in database B. The specific number of nodes in the databases included is related to the processing operation.

[0107] Among the multiple scheduling nodes associated with the task to be processed, some of the nodes may use a data query language that is partially different from the query language used by the management server. For example, among the multiple scheduling nodes associated with the task to be processed, at least one target scheduling node uses a second data query language, and the management server uses a first data query language. Then, when the management server sends the task to be processed to the proxy service corresponding to the scheduling node, it converts the task to be processed from the first data query language to the second data query language, and sends the converted task to be processed to the proxy service corresponding to the target scheduling node. The converted task to be processed uses the second data query language.

[0108] Correspondingly, when determining the processing result corresponding to the data processing request according to the execution result corresponding to the task to be processed, the execution result returned by the proxy service corresponding to the target scheduling node is converted from the second data query language to the first data query language. According to the converted execution result, the processing result corresponding to the data processing request is determined. The converted execution result uses the first data query language.

[0109] When the data to be processed by the data processing request belongs to multiple different types of databases, the method of the embodiment of the present application can provide a unified interface to access each database and use a unified data processing method for processing. The management server does not need to distinguish the storage differences of the underlying databases, simplifies the architecture of the entire system, and improves the flexibility of data processing.

[0110] S104. Determine the processing result corresponding to the data processing request according to the execution result corresponding to the task to be processed, and send the processing result to the client.

[0111] The management server receives the execution results corresponding to the tasks returned by the agent services corresponding to each scheduling node, comprehensively processes all the execution results, and obtains the processing result corresponding to the data processing request. For example, when the data query request is used to query the power-on / off status of all servers, the management server aggregates the power-on / off status of all servers and returns it to the client as the processing result. Among them, the formats of the power-on / off status data returned by the agent services of each server may be inconsistent, and the management server can also convert the formats of the power-on / off status data returned by the agent services of each server into a unified format.

[0112] For another example, when the data query request is used to update the status of a certain server, the management server returns the update result of the server to the client as the query request.

[0113] In this embodiment, the management server determines the to-be-processed task corresponding to the data processing request and the scheduling node associated with the to-be-processed task according to the data processing request sent by the client. The data processing request is used to request the processing of the online data of the database system; the to-be-processed task is sent to the proxy service corresponding to the scheduling node, and the proxy service obtains the execution result of the to-be-processed task and returns it. According to the execution result corresponding to the to-be-processed task, the processing result corresponding to the data processing request is determined, and the processing result is sent to the client. This method can process the online data of the database system in a unified manner and realize the management of the dynamic data of the database system.

[0114] Based on Embodiment 1, Embodiment 2 of the present application provides a method for processing online data of a database, which is used to describe the method of Embodiment 2 of the present application from the perspective of the agent services corresponding to each node of the database. Figure 5 This is the flowchart of the method for processing online data of the database provided in Embodiment 2 of the present application. As Figure 5 shown, the method provided in this embodiment includes the following steps.

[0115] S201. The proxy service receives the to-be-processed task sent by the management server.

[0116] The management server is used to manage at least one database. Each database includes at least one node, and each node is provided with an agent service. The agent service is used to control the corresponding node.

[0117] An agent service may include the following functions: (1) operating on node data; (2) querying node data; (3) user permission verification.

[0118] When the pending task determined by the management server according to the data processing request is to operate the data of one or more nodes, the management server sends the pending task to the agent service of each node. The pending task is used to operate the node data. The operation on the node data includes but is not limited to: changing the state of the node and changing the data of the node. Among them, changing the state of the node includes turning on, shutting down or restarting the node, and changing the data of the node includes restarting the node service, deleting or changing the data stored on the node (such as log data), etc.

[0119] Querying the data of a node refers to querying the data stored on the node, for example, querying the log data on the node, querying the user data on the node, etc.

[0120] User authority verification is used to determine whether the management user corresponding to the pending task has the authority to perform the processing operation corresponding to the pending task. The management user corresponding to the pending task refers to the user who initiates the data processing request. Different users have different operation permissions on the database.

[0121] The tasks to be processed include processing operation information and user information of management users. The processing operation information is used to describe the specific operation and operation object. Figure 6 As shown, Figure 6 The figure is a flowchart of node shutdown. When the management server determines to shut down a node of the database according to the data processing request, the management server sends a pending task to the agent service corresponding to the node. The pending task is an update task. The update task includes user information of the management user and update operation information. The user information is user:root as shown in the figure. The update operation information includes: field: states, old_val:0, new_val:1, wherein field: states indicates the object to be updated, old_val:0 indicates that the value of states before the update is 0, new_val:1 indicates that the value of states after the update is 1, wherein the value of states is 1, indicating that the node is shut down.

[0122] S202: The proxy service communicates with the node corresponding to the proxy service according to the task to be processed, and obtains the execution result of the task to be processed.

[0123] Optionally, when the agent service supports user authority verification, the agent service determines whether the management user corresponding to the task to be processed has the authority to perform the processing operation corresponding to the task to be processed; when the management user has the authority to perform the processing operation corresponding to the task to be processed, the agent service communicates with the node corresponding to the agent service according to the task to be processed, and obtains the execution result of the task to be processed.

[0124] Specifically, the agent service obtains the user information of the managing user and the processing operation corresponding to the task to be processed from the task to be processed; determines the operation permissions of the managing user according to the user information, and determines whether the managing user has the permission for the processing operation corresponding to the task to be processed according to the operation permissions of the managing user and the processing operation corresponding to the task to be processed.

[0125] The user information may be the user ID and user type. The operation permissions of the managing user are determined according to the user ID and user type. The managing user may have one or more operation permissions, and different users may have different operation permissions.

[0126] Taking Figure 6 the process shown as an example, before the agent service updates the states of the node, it determines whether the managing user corresponding to user:root has the permission to shut down the node. If the managing user corresponding to user:root does not have the permission to shut down the node, the task to be processed is rejected, and a rejection response is returned to the management server. Optionally, the rejection reason is also included in the rejection response. If the managing user corresponding to user:root has the permission to shut down the node, the shutdown operation is executed.

[0127] The agent service changes the state or data of the node corresponding to the proxy service according to the task to be processed, and obtains the state change result or data change result of the node corresponding to the proxy service; or, queries data on the node corresponding to the proxy service according to the task to be processed, and obtains the data processing result corresponding to the task to be processed.

[0128] In this embodiment, the agent service may be located on the node or on an independent server outside the node. When the agent service is located on the node, the communication between the agent service and the node can be understood as the communication between two modules or components on the node. When the agent service is located on an independent server outside the node, the communication between the agent service and the node is the communication between two devices.

[0129] Taking Figure 6 the shutdown process shown as an example, when the agent service determines that the managing user has the permission to shut down the node, it sends a shutdown instruction to the node to implement the shutdown of the node.

[0130] When the task to be processed is used to query the power-on / off state of the node, the agent service may send a heartbeat instruction to the query node to query whether the node can receive and send information normally. When the node can receive and send information normally, it is determined that the node is in the power-on state. When the node cannot receive and send information normally, it is determined that the node is in the power-off state.

[0131] S203. The proxy service sends the execution result to the management server.

[0132] After the proxy server obtains the execution result of the task to be processed, it feeds back the execution result to the management server. The management server processes the execution results returned by each agent to obtain the execution result of the data processing request, and returns the execution result to the user.

[0133] In this embodiment, by setting a proxy service for each node of the database, the management server processes the data processing request, splits the data processing request into one or more tasks to be processed, each task to be processed corresponds to a node, and the task to be processed is sent to the agent service of the corresponding node. The agent services of each node execute the tasks to be processed to obtain the online data of the database and return it to the management server, thereby realizing the management of the online data of the database system.

[0134] Based on Embodiment 1 and Embodiment 2, Embodiment 3 of the present application provides a method for processing online data of a database, and manages the online data of the database through the MySQL protocol. Figure 7 For the signaling flowchart of the method for processing online data of the database provided in Embodiment 3 of the present application, as Figure 7 shown, the method provided in this embodiment includes the following steps.

[0135] S301. The client sends an SQL request to the management server.

[0136] In this embodiment, the MySQL protocol is used for communication between the client and the management server. The user logs in to the management server through the MySQL protocol. In the MySQL database, the data processing request is an SQL request.

[0137] S302. The management server parses the SQL request into an abstract syntax tree, and determines the type of the processing operation corresponding to the SQL request according to the abstract syntax tree.

[0138] S303. The management server determines the processing operation corresponding to the SQL request and the node to be accessed by the processing operation according to the type of the processing operation corresponding to the SQL request and the abstract syntax tree.

[0139] S304. The management server generates a task to be processed according to the processing operation corresponding to the SQL request, and determines the scheduling node associated with the task to be processed according to the node to be accessed by the processing operation.

[0140] S305. The management server sends the task to be processed to the proxy services corresponding to each scheduling node.

[0141] It can be understood that Figure 7Only one scheduling node and the corresponding proxy service are shown, and in an actual scenario, there may be multiple scheduling nodes.

[0142] S306. The proxy service corresponding to each scheduling node communicates with the corresponding scheduling node according to the task to be processed, and obtains the execution result of the task to be processed.

[0143] S307. The proxy service corresponding to each scheduling node sends the execution result of the task to be processed to the management server.

[0144] S308. The management server determines the processing result of the SQL statement according to the execution results returned by each proxy service.

[0145] Figure 8 It is a schematic diagram of a functional module of the management server. As Figure 8 shown, the management server includes the following modules: MySQL protocol support, SQL parser, optimizer, distributed data source reading proxy, node task scheduler, and return data processing module. The manager server implements the foregoing steps through these modules.

[0146] The MySQL protocol support module is used to implement and expand the MySQL protocol as needed, and support users to log in to the system through the unified MySQL protocol.

[0147] The SQL parser is used to convert the SQL request into an abstract syntax tree using Calcite (that is, Calcite is Apache Calcite). Calcite is an open-source data management framework that provides a general-purpose and embeddable SQL processing framework, enabling developers to use the powerful expressive power of SQL for data query and processing, and at the same time being able to fully utilize the optimization functions provided by Calcite.

[0148] The SQL parser is also used to determine whether the type of processing operation corresponding to the SQL request is a data operation type or a data query type according to the abstract syntax tree. The SQL parser can determine whether the operation type corresponding to the SQL request is a data operation type or a data query type according to the operators included in the abstract syntax tree, the operators corresponding to the predefined data operation types, and the operators corresponding to the predefined data query types.

[0149] The optimizer is used to perform query optimization on the abstract syntax tree when the operation type corresponding to the SQL request is a data query type, and obtain the execution plan corresponding to the abstract syntax tree. The optimizer can be CBO or RBO, and developers can customize the optimizer according to needs.

[0150] The optimizer is used to find the optimal execution plan (or query plan). The optimizer can determine the optimal query plan through query rewriting, operation order adjustment, appropriate indexing, and cost prediction, etc.

[0151] The distributed data source reading agent is used to determine the processing operations corresponding to the data processing request and the nodes to be accessed by the processing operations according to the execution plan corresponding to the abstract syntax tree, generate specific tasks to be processed according to the processing operations corresponding to the data processing request, and determine the scheduling nodes associated with the tasks to be processed according to the nodes to be accessed by the processing operations, and send each task to be processed to the agent service corresponding to each scheduling node.

[0152] The scheduling node associated with the task can be a node of the Spark system, and the Spark system is an efficient distributed computing system.

[0153] The distributed data source reading agent is also used to receive the execution results corresponding to the tasks to be processed returned by the agent services of each scheduling node, and send the execution results corresponding to the tasks to be processed to the returned data processing module.

[0154] The node task scheduler is used to, when the operation type corresponding to the SQL request is a data operation type, determine the processing operations corresponding to the data processing request and the nodes to be accessed by the processing operations according to the abstract syntax tree, generate specific tasks to be processed according to the processing operations corresponding to the data processing request, and determine the scheduling nodes associated with the tasks to be processed according to the nodes to be accessed by the processing operations, and send each task to be processed to the agent service corresponding to each scheduling node.

[0155] The node task scheduler is also used to receive the execution results corresponding to the tasks to be processed returned by the agent services of each scheduling node, and send the execution results corresponding to the tasks to be processed to the returned data processing module.

[0156] The data processing module is used to receive the execution results corresponding to the processing tasks returned by the agent services of each scheduling node obtained from the distributed data source reading agent or the node scheduling tasker, determine the processing results corresponding to the SQL request according to all the obtained execution results, and send the processing results to the client.

[0157] S309. The management server sends the processing result of the SQL statement to the client.

[0158] S310. The client displays the processing result of the SQL statement.

[0159] To facilitate better implementation of the method for processing online data based on a database in the embodiments of the present application, the embodiments of the present application also provide an apparatus for processing online data of a database. Figure 9FIG. 0 is a schematic structural diagram of an apparatus for processing online data of a database provided in the fourth embodiment of the present application. The apparatus 100 is applied to a management server, and the management server is used to manage at least one database. Each database includes at least one node, and each node is provided with an agent service. The agent service is used to control the corresponding node, such as Figure 9 As shown, the apparatus 100 includes:

[0160] A determination module 11, configured to determine a to-be-processed task corresponding to the data processing request and a scheduling node associated with the to-be-processed task according to a data processing request sent by a client. The data processing request is used to request processing of online data of a database;

[0161] A distribution module 12, configured to send the to-be-processed task to an agent service corresponding to the scheduling node. The agent service corresponding to the scheduling node is used to communicate with the scheduling node according to the to-be-processed task and obtain an execution result of the to-be-processed task;

[0162] A receiving module 13, configured to receive the execution result returned by the agent service corresponding to the scheduling node;

[0163] A sending module 14, configured to determine a processing result corresponding to the data processing request according to the execution result corresponding to the to-be-processed task, and send the processing result to the client.

[0164] In one implementation, the determination module 11 is specifically configured to:

[0165] Parse the data processing request into a query logic representation for internal use;

[0166] According to the query logic representation, determine a processing operation corresponding to the data processing request and a node to be accessed by the processing operation;

[0167] Generate the to-be-processed task according to the processing operation corresponding to the data processing request, and determine the node to be accessed by the processing operation as the scheduling node associated with the to-be-processed task.

[0168] In one implementation, the determination module 11 is specifically configured to:

[0169] According to an operator included in the query logic representation and operators corresponding to each predefined operation type, determine an operation type corresponding to the data processing request;

[0170] According to the operation type corresponding to the data processing request and the query logic representation, determine a processing operation corresponding to the data processing request and a node to be accessed by the processing operation.

[0171] In one implementation, the determining module 11 is specifically configured to:

[0172] When the operation type corresponding to the data processing request is a data operation type, determine the processing operation corresponding to the data processing request and the node to be accessed by the processing operation according to the query logic representation;

[0173] When the operation type corresponding to the data processing request is a data query type, perform query optimization on the query logic representation to obtain an execution plan corresponding to the query logic representation;

[0174] Determine the processing operation corresponding to the data processing request and the node to be accessed by the processing operation according to the execution plan corresponding to the query logic representation.

[0175] In one implementation, the data processing request is used to request processing of online data of multiple databases, and the scheduling nodes associated with the task to be processed include the nodes of the multiple databases.

[0176] In one implementation, when at least one target scheduling node among the multiple scheduling nodes associated with the task to be processed uses a second data query language different from the first data query language used by the management server; the distribution module 12 is specifically configured to:

[0177] Convert the task to be processed from the first data query language to the second data query language;

[0178] Send the converted task to be processed to the proxy service corresponding to the target scheduling node;

[0179] The sending module 14 is specifically configured to:

[0180] Convert the execution result returned by the proxy service corresponding to the target scheduling node from the second data query language to the first data query language;

[0181] Determine the processing result corresponding to the data processing request according to the converted execution result.

[0182] In one implementation, the management server uses the data query language corresponding to MySQL.

[0183] It should be understood that the device embodiments and the method embodiments can correspond to each other, and similar descriptions can refer to the method embodiments. To avoid repetition, they are not elaborated here.

[0184] Figure 10Schematic diagram of the structure of an online data processing device for a database provided in the fifth embodiment of this application. The device 200 is applied to proxy services. The database includes at least one node, and each node is provided with a proxy service for controlling the corresponding node. For example, Figure 10 As shown, the device 200 includes:

[0185] A receiving module 21, configured to receive a to-be-processed task sent by a management server;

[0186] A processing module 22, configured to communicate with the node corresponding to the proxy service according to the to-be-processed task, and obtain an execution result of the to-be-processed task;

[0187] A sending module 23, configured to send the execution result to the management server.

[0188] In one implementation, the processing module 22 is specifically configured to:

[0189] Determine whether the management user corresponding to the to-be-processed task has permission for the processing operation corresponding to the to-be-processed task;

[0190] When the management user has permission for the processing operation corresponding to the to-be-processed task, communicate with the node corresponding to the proxy service according to the to-be-processed task, and obtain an execution result of the to-be-processed task.

[0191] In one implementation, the processing module 22 is specifically configured to:

[0192] Perform a status change or data change on the node corresponding to the proxy service according to the to-be-processed task, to obtain a status change result or data change result of the node corresponding to the proxy service;

[0193] Or, query data on the node corresponding to the proxy service according to the to-be-processed task, to obtain a data processing result corresponding to the to-be-processed task.

[0194] In one implementation, the processing module 22 is specifically configured to:

[0195] Obtain user information of the management user and the processing operation corresponding to the to-be-processed task from the to-be-processed task;

[0196] Determine the operation permission of the management user according to the user information;

[0197] Determine whether the management user has permission for the processing operation corresponding to the to-be-processed task according to the operation permission of the management user and the processing operation corresponding to the to-be-processed task.

[0198] It should be understood that the device embodiments and the method embodiments can correspond to each other, and similar descriptions can refer to the method embodiments. To avoid repetition, they will not be elaborated here.

[0199] In the above, the devices 100 and 200 of the embodiments of the present application have been described from the perspective of functional modules in combination with the accompanying drawings. It should be understood that the functional modules can be implemented in the form of hardware, or in the form of instructions in software, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in the embodiments of the present application can be completed through the integrated logic circuit in the hardware in the processor and / or instructions in software form. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed by the hardware decoding processor, or executed by a combination of the hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps in the above method embodiments.

[0200] The embodiments of the present application also provide a server. Figure 11 As a schematic structural diagram of the server provided by the embodiments of the present application, as Figure 11 shown, the server 300 may include: a communication interface 31, a memory 32, a processor 33, and a communication bus 34. The communication interface 31, the memory 32, and the processor 33 communicate with each other through the communication bus 34. The communication interface 31 is used for the computer device 1200 to perform data communication with external devices. The memory 32 can be used to store software programs and modules, and the processor 33 runs the software programs and modules stored in the memory 32, such as the software programs for the corresponding operations in the foregoing method embodiments.

[0201] Optionally, the processor 33 may call the software programs and modules stored in the memory 32 to perform the following operations:

[0202] According to the data processing request sent by the client, determine the to-be-processed task corresponding to the data processing request and the scheduling node associated with the to-be-processed task, where the data processing request is used to request to process the online data in the database;

[0203] Send the to-be-processed task to the proxy service corresponding to the scheduling node, where the proxy service corresponding to the scheduling node is used to communicate with the scheduling node according to the to-be-processed task and obtain the execution result of the to-be-processed task;

[0204] Receive the execution result returned by the proxy service corresponding to the scheduling node;

[0205] Determine the processing result corresponding to the data processing request according to the execution result corresponding to the task to be processed, and send the processing result to the client.

[0206] Alternatively, the processor 33 may call a software program and a module stored in the memory 32 to perform the following operations:

[0207] Receive a task to be processed sent by the management server;

[0208] Communicate with a node corresponding to the proxy service according to the task to be processed, and obtain the execution result of the task to be processed;

[0209] Send the execution result to the management server.

[0210] In some embodiments of the present application, the processor 33 may include but are not limited to:

[0211] General-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like.

[0212] In some embodiments of the present application, the memory 32 includes but are not limited to:

[0213] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be Read-Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), or flash memory. The volatile memory can be Random Access Memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double DataRate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synch link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0214] In some embodiments of the present application, the computer program can be divided into one or more modules, and the one or more modules are stored in the memory 32 and executed by the processor 33 to complete the method provided by the present application. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the server.

[0215] In some embodiments of the present application, the processor 33 can control the communication interface 31 to communicate with other devices. Specifically, it can send information or data to other devices, or receive information or data sent by other devices. The communication interface 31 can include a transmitter and a receiver. The communication interface 31 can further include an antenna, and the number of antennas can be one or more.

[0216] It can be understood that although Figure 11 not shown in the figure, the server 300 can further include a positioning module, a display, a controller, an audio circuit, a power supply, etc., which will not be elaborated here.

[0217] The present application also provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a computer, the computer can execute the methods in the above method embodiments. Or rather, the embodiments of the present application also provide a computer program product containing instructions. When the instructions are executed by a computer, the computer executes the methods in the above method embodiments.

[0218] The present application also provides a computer program product. The computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the electronic device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the electronic device executes the corresponding processes in the above method embodiments. For the sake of brevity, details are not described herein again.

[0219] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or modules can be in electrical, mechanical or other forms.

[0220] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, in each embodiment of the present application, the functional modules can be integrated into a processing module, or each module can exist physically alone, or two or more modules can be integrated into one module.

[0221] The above are only the specific implementation manners of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An online data processing method for a database, characterized in that, Applied to a management server, the management server is used to manage at least one database. Each database includes at least one node, and each node is provided with an agent service. The agent service is used to control the corresponding node. The method includes: According to a data processing request sent by a client, determine a to-be-processed task corresponding to the data processing request and a scheduling node associated with the to-be-processed task. The data processing request is used to request processing of online data in the database. The to-be-processed task is used to process the online data, and the online data processed by the to-be-processed task is generated in real time by the scheduling node associated with the to-be-processed task. There is no data table in the database for storing the online data to be processed by the to-be-processed task; Send the to-be-processed task to the agent service corresponding to the scheduling node. The agent service corresponding to the scheduling node is used to communicate with the scheduling node according to the to-be-processed task and obtain the execution result of the to-be-processed task; Receive the execution result returned by the agent service corresponding to the scheduling node; According to the execution result corresponding to the to-be-processed task, determine the processing result corresponding to the data processing request, and send the processing result to the client.

2. The method according to claim 1, wherein The step of determining a to-be-processed task corresponding to the data processing request and a scheduling node associated with the to-be-processed task according to the data processing request includes: Parse the data processing request into a query logic representation for internal use; According to the query logic representation, determine the processing operation corresponding to the data processing request and the node to be accessed by the processing operation; Generate the to-be-processed task according to the processing operation corresponding to the data processing request, and determine the node to be accessed by the processing operation as the scheduling node associated with the to-be-processed task.

3. The method according to claim 2, wherein The step of determining the processing operation corresponding to the data processing request and the node to be accessed by the processing operation according to the query logic representation includes: According to the operator included in the query logic representation and the operators corresponding to each predefined operation type, determine the operation type corresponding to the data processing request; According to the operation type corresponding to the data processing request and the query logic representation, determine the processing operation corresponding to the data processing request and the node to be accessed by the processing operation.

4. The method according to claim 3, wherein The step of determining the processing operation corresponding to the data processing request and the node to be accessed by the processing operation according to the operation type corresponding to the data processing request and the query logic representation includes: When the operation type corresponding to the data processing request is a data operation type, determine the processing operation corresponding to the data processing request and the node to be accessed by the processing operation according to the query logic representation; When the operation type corresponding to the data processing request is a data query type, perform query optimization on the query logic representation to obtain an execution plan corresponding to the query logic representation; According to the execution plan corresponding to the query logic representation, determine the processing operation corresponding to the data processing request and the node to be accessed by the processing operation.

5. The method according to any one of claims 1-4, characterized in that, The data processing request is used to request the processing of online data in multiple databases, and the scheduling nodes associated with the task to be processed include the nodes of the multiple databases.

6. The method according to claim 5, characterized in that, When at least one target scheduling node among the multiple scheduling nodes associated with the task to be processed uses a second data query language different from the first data query language used by the management server; The step of sending the task to be processed to the proxy service corresponding to the scheduling node includes: Converting the task to be processed from the first data query language to the second data query language; Sending the converted task to be processed to the proxy service corresponding to the target scheduling node; Determining the processing result corresponding to the data processing request according to the execution result corresponding to the task to be processed includes: Converting the execution result returned by the proxy service corresponding to the target scheduling node from the second data query language to the first data query language; Determining the processing result corresponding to the data processing request according to the converted execution result.

7. The method according to any one of claims 1-4, characterized in that, The management server uses the data query language corresponding to MySQL.

8. An online data processing method for a database, characterized in that, The database includes at least one node, and each node is provided with a proxy service for controlling the corresponding node. The method includes: The proxy service receives the task to be processed sent by the management server. The task to be processed is used to process online data, and the online data processed by the task to be processed is generated in real time by the node corresponding to the proxy service. There is no data table in the database for storing the online data to be processed by the task to be processed; The proxy service communicates with the node corresponding to the proxy service according to the task to be processed and obtains the execution result of the task to be processed; The proxy service sends the execution result to the management server.

9. The method according to claim 8, characterized in that The proxy service communicates with the node corresponding to the proxy service according to the task to be processed and obtains the execution result of the task to be processed, including: Determining whether the management user corresponding to the task to be processed has the permission for the processing operation corresponding to the task to be processed; When the management user has the permission for the processing operation corresponding to the task to be processed, communicating with the node corresponding to the proxy service according to the task to be processed and obtaining the execution result of the task to be processed.

10. The method according to claim 9, characterized in that The step of communicating with the node corresponding to the proxy service according to the task to be processed and obtaining the execution result of the task to be processed includes: Changing the state or data of the node corresponding to the proxy service according to the task to be processed to obtain the state change result or data change result of the node corresponding to the proxy service; Alternatively, querying data on the node corresponding to the proxy service according to the task to be processed to obtain the data processing result corresponding to the task to be processed.

11. The method according to claim 9, characterized in that Determining whether the management user corresponding to the task to be processed has the permission for the processing operation corresponding to the task to be processed includes: Obtaining the user information of the management user and the processing operation corresponding to the task to be processed from the task to be processed; Determining the operation permission of the management user according to the user information; Determine whether the management user has the permission for the processing operation corresponding to the task to be processed according to the operation permission of the management user and the processing operation corresponding to the task to be processed.

12. An online data processing device for a database, characterized in that, The device is applied to a management server, which is used to manage at least one database. Each database includes at least one node, and each node is provided with an agent service. The agent service is used to control the corresponding node. The device includes: A determination module, configured to determine the task to be processed corresponding to the data processing request and the scheduling node associated with the task to be processed according to the data processing request sent by the client. The data processing request is used to request processing of the online data in the database. The task to be processed is used to process the online data, and the online data processed by the task to be processed is generated in real time by the scheduling node associated with the task to be processed. There is no data table in the database for storing the online data to be processed by the task to be processed. A distribution module, configured to send the task to be processed to the agent service corresponding to the scheduling node. The agent service corresponding to the scheduling node is used to communicate with the scheduling node according to the task to be processed and obtain the execution result of the task to be processed. A receiving module, configured to receive the execution result returned by the agent service corresponding to the scheduling node. A sending module, configured to determine the processing result corresponding to the data processing request according to the execution result corresponding to the task to be processed, and send the processing result to the client.

13. An online data processing device for a database, characterized in that, The database includes at least one node, and each node is provided with an agent service. The agent service is used to control the corresponding node. The device is applied to the agent service. The device includes: A receiving module, configured to receive the task to be processed sent by the management server. The task to be processed is used to process the online data, and the online data processed by the task to be processed is generated in real time by the node corresponding to the agent service. There is no data table in the database for storing the online data to be processed by the task to be processed. A processing module, configured to communicate with the node corresponding to the agent service according to the task to be processed and obtain the execution result of the task to be processed. A sending module, configured to send the execution result to the management server.

14. A management server, characterized in that, Including: A processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 7.

15. A server, characterized in that, Including: A processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method according to any one of claims 8 to 11.

16. A computer-readable storage medium, characterized in that, For storing a computer program, the computer program causes a computer to execute the method according to any one of claims 1 to 7, or execute the method according to any one of claims 8 - 11.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 7, or executes the method according to any one of claims 8 - 11.

Images