Abnormality processing method and device of cloud database and computing device cluster
By using target memory to store SQL statements and their execution descriptions in the cloud database system, and by using anomaly analysis rules to automatically analyze anomalies, the performance loss caused by full logs and the time-consuming nature of manual analysis are solved, achieving efficient anomaly handling and rapid location.
Patent Information
- Application Number
- CN202310009589.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-01-04
- Publication Date
- 2025-11-07
- Estimated Expiration
- 2043-01-04
AI Technical Summary
In existing technologies, cloud databases are inefficient in handling anomalies because the use of full logs leads to severe performance degradation, and anomaly analysis relies on the experience and capabilities of operations and maintenance personnel, resulting in lengthy analysis times.
In cloud database systems, target memory is used to store SQL statements and their execution description information. Anomaly analysis information is generated using anomaly analysis rules to avoid disk write operations, reduce the impact on performance, and automatically analyze anomalies through shared memory and multi-dimensional execution description information.
It improves the efficiency of anomaly handling, reduces the impact on system performance, reduces the workload of operations and maintenance personnel, shortens anomaly analysis time, and enhances the availability and anomaly localization capabilities of cloud databases.
Smart Images

Figure CN115952153B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computers, and in particular, to an exception processing method and device for a cloud database and a computing device cluster. BACKGROUND
[0002] Compared with a traditional database, a cloud database has advantages such as low cost, easy deployment, and high reliability, and can provide more reliable database services and a more comfortable user experience. An exception that occurs during the running of the cloud database, for example, a program crash or an out of memory (OOM) reason, causes the process of the cloud database to be terminated, which directly causes the service of the cloud database to be unavailable. Therefore, exception processing for the cloud database is a key to improving the availability of the cloud database.
[0003] At present, an operation and maintenance personnel usually locates an exception root cause according to a general log of the cloud database, and thus gives an exception processing scheme. The general log is a log recording function provided by a relational database management system MySQL, which is used to store structured query language (SQL) statements received by the MySQL service to a hard disk. After an exception occurs in the database, the operation and maintenance personnel can view the general log, and find the cause of the exception by analyzing the stored SQL statements.
[0004] However, if the database system starts the general log, nearly 30% of the performance needs to be consumed to ensure that the log data is effectively written to the disk during the running of the database, which seriously affects the performance of the cloud database. Moreover, since the general log only stores SQL statements, the operation and maintenance personnel cannot directly determine which SQL statement causes the exception by the general log, and the accuracy of analyzing the exception cause depends on the experience and ability of the operation and maintenance personnel, which causes the exception processing to take a long time. Therefore, the efficiency of the exception processing for the cloud database by using the above technical scheme is very low. SUMMARY
[0005] Embodiments of the present application provide an exception processing method and device for a cloud database and a computing device cluster, which can improve the efficiency of the exception processing for the cloud database. The technical scheme is as follows:
[0006] In a first aspect, an exception processing method for a cloud database is provided, and the method is applied to a cloud database system, and the method comprises the following steps.
[0007] In the running process of the cloud database system, a structured query language (SQL) statement for the cloud database system and execution description information of the SQL statement are stored in a target memory of the cloud database system, the target memory being configured to provide a shared data read-write space for at least one process running in the cloud database system, and the execution description information being configured to describe an execution process of the SQL statement.
[0008] In response to detecting an exception occurring in the cloud database system, based on an exception analysis rule and execution description information of at least one SQL statement stored in the target memory, exception analysis information is generated, the exception analysis rule indicating a feature to which an execution process of a SQL statement causing the exception conforms, and the exception analysis information indicating a target SQL statement causing the exception in the at least one SQL statement.
[0009] The SQL statement and the execution description information stored in the target memory have no need to be written to a disk, and have little impact on the performance of the database system. In addition, the exception analysis information can be generated based on the preset exception analysis rule, thereby reducing the workload of relevant personnel, reducing the time consumption of exception processing, and improving the efficiency of exception analysis and problem positioning.
[0010] In a possible implementation, the target memory is configured to store a target number of SQL statements and execution description information of the target number of SQL statements; and the storing, in the running process of the cloud database system, of a SQL statement for the cloud database system and execution description information of the SQL statement in the target memory of the cloud database system comprises:
[0011] In response to receiving any SQL statement for the cloud database system, if the target number of SQL statements and the execution description information of the target number of SQL statements have been stored in the target memory, a first SQL statement with the earliest execution time and execution description information of the first SQL statement are deleted.
[0012] The SQL statement for the cloud database system and the execution description information of the SQL statement are stored in the target memory of the cloud database system.
[0013] Through the above process, it can be ensured that only the target number of SQL statements and the execution description information of the target number of SQL statements that are recently executed are retained in the target memory, thereby ensuring the tracing capability of the abnormal statement within a certain time length without affecting the performance of the system.
[0014] In a possible implementation, the storing, in the running process of the cloud database system, of a SQL statement for the cloud database system and execution description information of the SQL statement in the target memory of the cloud database system comprises:
[0015] For any one of the SQL statements, according to a first process executing the SQL statement, a sub-memory space corresponding to the SQL statement in the target memory is determined, the sub-memory space being used to store execution description information of the SQL statement executed by the first process;
[0016] In the sub-memory space, the SQL statement and the execution description information of the SQL statement are stored.
[0017] In some embodiments, the target memory is divided into a plurality of sub-memory spaces (slots), each of which is allocated to a corresponding process for data reading and writing.
[0018] By using the above technical solution to store the SQL statement and the execution description information thereof, the flexible storage characteristics of the shared memory are utilized, the influence of the disk writing process on the performance of the database system is avoided, and by dividing a plurality of sub-memory areas, the data consistency in the multi-process reading and writing process does not need to be ensured by locking, thereby greatly improving the data access efficiency.
[0019] In a possible implementation, the execution description information includes at least one of resource usage information, statement status information, and environment status information;
[0020] The resource usage information indicates the computing resources occupied in the execution process of the SQL statement; the statement status information indicates the execution status of the SQL statement; and the environment status information indicates the running status of the cloud database system when the SQL statement is executed.
[0021] The execution description information collected through the above process can describe the computing resources occupied in the execution process of the SQL statement from the dimensions of the running memory and the execution time, so as to understand the usage of the computing resources by the SQL statement; can describe the running status of the cloud database when the statement is executed, so as to understand the execution environment on the cloud; and can describe the execution status of the statement at multiple levels, so as to understand the execution of the statement from multiple granularities. Based on this, the execution process of the SQL statement is comprehensively described from multiple angles, which can provide a reference for the positioning of abnormal problems.
[0022] In a possible implementation, the abnormal analysis information is generated based on the exception analysis rule and the execution description information of at least one SQL statement stored in the target memory, and includes:
[0023] If the execution process described by the execution description information of a target SQL statement in the at least one SQL statement conforms to the target feature indicated by the exception analysis rule, the abnormal analysis information of the target SQL statement is generated, the abnormal analysis information of the target SQL statement including a problem type corresponding to the target feature, the problem type being used to indicate the reason for causing the exception of the target SQL statement.
[0024] Through the above process, multi-dimensional execution description information can be integrated, risks that can exist in each SQL statement can be analyzed, corresponding abnormal analysis information can be generated, the generation process of the abnormal analysis information is automated based on preset abnormal analysis rules, the workload of related personnel in the abnormal processing process is reduced through automatic analysis of the problem type, the time consumption of analyzing the abnormality is reduced, and the efficiency of problem analysis and positioning is greatly improved.
[0025] In a possible implementation, the abnormality includes a process active end and a process passive end.
[0026] The technical solution provided in the present application can cover a variety of abnormal scenarios, thereby performing abnormal processing in a variety of abnormal scenarios, and improving the comprehensiveness and practicality of the abnormal processing method.
[0027] In a possible implementation, before the abnormal analysis information is generated based on the abnormal analysis rule and the execution description information of the at least one SQL statement stored in the target memory, the method further includes:
[0028] storing the at least one SQL statement and the execution description information of the at least one SQL statement stored in the target memory into a target storage node, and releasing the target memory;
[0029] The abnormal analysis information is generated based on the abnormal analysis rule and the execution description information of the at least one SQL statement stored in the target memory, including:
[0030] The abnormal analysis information is generated based on the abnormal analysis rule and the at least one SQL statement and the execution description information of the at least one SQL statement obtained from the target storage node.
[0031] The target storage node is configured to provide a storage service. In some embodiments, the target storage node can be a storage server in a cloud database system, and can also be a storage node outside the cloud database system, and the embodiments of the present application do not limit this. In some embodiments, the target storage node can be a storage node outside the cloud database system for providing an OBS, and the cloud database system can conveniently use the OBS provided by the storage node by calling an application programming interface (API).
[0032] The technical scheme provided in the application can release the memory in time after an exception occurs, so as to ensure that the cloud database can recover from the exception as soon as possible and effectively improve the availability of cloud database services; and the related information for exception analysis is stored in a safe and stable manner, thereby providing reliable data support for subsequent exception analysis and processing.
[0033] In a second aspect, the application provides an exception processing apparatus of a cloud database, configured in a cloud database system, comprising at least one function module, used for executing the exception processing method of the cloud database provided in the foregoing first aspect or any possible implementation manner of the first aspect.
[0034] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used for executing instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the exception processing method of the cloud database provided in the foregoing first aspect or any possible implementation manner of the first aspect.
[0035] In a fourth aspect, a computer readable storage medium is provided, used for storing at least one piece of program code, when the at least one piece of program code is run by a computing device, so that the computing device executes the exception processing method of the cloud database provided in the foregoing first aspect or any possible implementation manner of the first aspect. The storage medium comprises but is not limited to a volatile memory such as a random access memory, a non-volatile memory such as a flash memory, a hard disk drive (HDD) and a solid state drive (SSD).
[0036] In a fifth aspect, a computer program product is provided, when the computer program product is run on a computing device, so that the computing device executes the exception processing method of the cloud database provided in the foregoing first aspect or any possible implementation manner of the first aspect. The computer program product can be a software installation package, and in the case where the function of the foregoing computing device needs to be implemented, the computer program product can be downloaded and executed on the computing device. BRIEF DESCRIPTION OF DRAWINGS
[0037] Figure 1 is a schematic diagram of an implementation environment of an exception processing method of a cloud database provided in an embodiment of the application;
[0038] Figure 2 is a flowchart of an exception processing method of a cloud database provided in an embodiment of the application;
[0039] Figure 3 is a schematic diagram of a cloud database exception processing method provided by an embodiment of the present application;
[0040] Figure 4 is a structural schematic diagram of a cloud database exception processing apparatus provided by an embodiment of the present application;
[0041] Figure 5 is a hardware structural schematic diagram of a computing device provided by an embodiment of the present application;
[0042] Figure 6 is a structural schematic diagram of a computing device cluster provided by an embodiment of the present application;
[0043] Figure 7 is a connection mode schematic diagram of a computing device cluster provided by an embodiment of the present application. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.
[0045] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the SQL statements, the execution user of the statement and the running state of the database involved in the present application are all obtained under sufficient authorization.
[0046] The key terms and abbreviations involved in the present application will be described below.
[0047] SQL statement: refers to a statement written in SQL language. SQL language is a database language used to implement operations such as adding, deleting, modifying and querying a database to realize functions such as accessing data, querying, updating and managing a database system. The types of SQL statements include data definition language (DDL), data query language (DQL), data manipulation language (DML) and data control language (DCL).
[0048] Database (DB): an ordered collection of structured information or data, generally stored in electronic form on a computer system. It is usually controlled by a database management system (DBMS). Illustratively, data, DBMS, and associated applications together are referred to as a database system, often simply referred to as a database. A database generally includes a client and a server. The client of the database provides an interface for data query services. The server of the database is used to execute the parsed query of the database and data communication processes, etc.
[0049] Cloud DB: a database deployed into a virtual computing environment, which can achieve advantages such as pay-as-you-go, on-demand scaling, high availability, and storage integration. The virtual computing environment can be a cloud platform. The cloud platform is a short form of cloud computing platform, which can provide computing, network, and storage capabilities based on hardware resources and software resources. Through the network "cloud", huge data computing is processed and analyzed in the remote end and then returned to the user, which has the characteristics of large scale, distribution, virtualization, high availability, scalability, on-demand service, and security. The cloud platform can achieve rapid delivery and release of configurable computing resources with small management cost or low interaction complexity of the user and the service provider.
[0050] OOM: Out Of Memory, which refers to the existence of unrecoverable memory or excessive use of memory in the system, eventually causing the memory used by the program to be greater than the maximum memory provided by the system.
[0051] Crash: refers to the failure of the database system to work, stop responding, or process interruption due to triggering a bug, failing to apply system resources, or memory leakage during the running of the program.
[0052] General log: a general log is a log recording function provided by a relational database management system MySQL, which is used to record the client connections established by the database system and the statements executed.
[0053] GDB (GNU debugger) tool: a portable debugger based on the open source operating system GNU, which is used to backtrack the causes of program abnormal end and change the program execution flow.
[0054] Coredump technology: a technology that writes the current process state into a coredump file on the hard disk by the operating system when the process abnormally ends. Users can use the GDB tool to analyze the coredump file to obtain the state of the process at the time of abnormality.
[0055] Object storage service (OBS): an object-based mass storage service, which provides users with mass, secure and highly reliable data storage capabilities, including creating, modifying and deleting buckets, uploading, downloading and deleting objects, etc. The basic components of OBS are buckets and objects. A bucket is a container for storing objects in OBS, and each bucket has its own storage class, access permission, region, etc. Users can locate a bucket through its access domain name on the Internet. An object is the basic unit of data storage in OBS, and an object is actually a collection of file data and its related attribute information.
[0056] The implementation environment related to the present application is introduced as follows.
[0057] Figure 1 is a schematic diagram of an implementation environment of an abnormality processing method of a cloud database provided by an embodiment of the present application. As shown in the figure, the implementation environment includes a terminal 101 and a server 102, and the terminal 101 is directly or indirectly connected to the server 102 through a wireless network or a wired network. Figure 1
[0058] The terminal 101 can be used as a client of the cloud database to provide an interface for using the database service. The terminal 101 can be at least one of a smart phone, a desktop computer, an augmented reality terminal, a tablet computer, an e-book reader and a laptop computer. The terminal 101 is installed and runs an application program supporting access to the cloud database. The application program can be a client of the cloud database or a browser, etc., which is not limited in the present application. For example, the application program is a client, and relevant personnel (such as an operation and maintenance personnel) can obtain abnormality analysis information of the database through the client, and perform corresponding operations based on the received abnormality analysis information to give an abnormality avoidance scheme or repair system problems.
[0059] The server 102 is a server of a cloud database system, and is configured to manage the cloud database, for example, to execute a parsing query request for the cloud database, to provide data communication, and the like. The server 102 is a physical server, or a server cluster or a distributed file system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms. In some embodiments, the server 102 is taken as an example of a cloud server, and the cloud server refers to a service based on hardware resources and software resources, and provides computing, network, and storage capabilities. Through a network "cloud", huge data computing is processed and analyzed in a remote end and then returned to a user, and has characteristics such as large-scale, distribution, virtualization, high availability, scalability, on-demand service, and security. The cloud server can realize rapid delivery and release of configurable computing resources with a small management cost or a low interaction complexity between the user and the service provider. Illustratively, the server 102 is configured to provide database services for a client of the cloud database running on the terminal 101.
[0060] In some embodiments, the server 102 and the terminal 101 can work cooperatively in the process of handling the exception of the cloud database. For example, the server 102 can store a structured query language (SQL) statement for the cloud database system and execution description information of the SQL statement into a target memory of the cloud database system during running of the cloud database system; in response to detecting that an exception occurs in the cloud database system, generate exception analysis information based on an exception analysis rule and the execution description information of at least one SQL statement stored in the target memory, and send the exception analysis information to the terminal 101; and the terminal 101 can display the received exception analysis information to provide a function of viewing and checking for a relevant person, to assist an operation and maintenance personnel to quickly give an exception avoidance scheme or repair a system problem. The process of generating the exception analysis information by the server 102 will be further described in subsequent embodiments, and will not be described herein.
[0061] It should be noted that the terminal 101 can refer to one of a plurality of terminals, or a set of a plurality of terminals; and the server 102 can refer to one of a plurality of servers, or a set of a plurality of servers. The number and type of each device in the implementation environment are not limited in the embodiments of the present application.
[0062] In some embodiments, the wireless or wired networks described above use standard communications technologies and / or protocols. Examples of networks include, but are not limited to, data center networks, storage area networks (SANs), local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), wireless networks, wired or wireless networks, private networks or virtual private networks (VPNs) of any combination thereof. In some implementations, technologies and / or formats including hyper text markup language (HTML), extensible markup language (XML) and the like are used to represent data exchanged over the networks. In addition, conventional encryption technologies such as secure sockets layer (SSL), transport layer security (TLS), virtual private networks (VPNs), internet protocol security (IPsec) and the like can be used to encrypt all or portions of links. In other embodiments, custom and / or proprietary data communications technologies can be used in place of or in combination with the data communications technologies described above.
[0063] The following describes an exception handling method of a cloud database related to the present application.
[0064] Figure 2 is a flowchart of an exception handling method of a cloud database provided by an embodiment of the present application. As shown in Figure 2 , the exception handling method of the cloud database is applied to Figure 1 the implementation environment shown in the figure, and schematically, the method is executed by a server 102, and the method includes the following steps 201 and 202.
[0065] 201. During the running of the cloud database system, the server stores structured query language (SQL) statements for the cloud database system and execution description information of the SQL statements into a target memory of the cloud database system.
[0066] The target memory is used to provide a shared data read-write space for at least one process running in the cloud database system. In some embodiments, a plurality of service processes of the cloud database run in the cloud database system, and the plurality of service processes can run based on mutually isolated memory spaces, thereby being able to provide database services for a plurality of clients (terminals) in parallel. For example, a service process 1 is used to establish a communication connection with a client 1, thereby receiving and executing an SQL statement sent by the client 1, and a service process 2 is used to establish a communication connection with a client 2, thereby receiving and executing an SQL statement sent by the client 2. Exemplarily, the service process can be a DBMS process, and a user can write an SQL statement through a client, thereby sending data to be stored or a database operation (such as a query, a deletion, or an update) to be performed in the form of an SQL statement to the DBMS process, and the DBMS process returns queried data to the client or deletes or updates data in the database by executing the received SQL statement.
[0067] In some embodiments, the target memory is a part of memory space divided based on a shared memory technology. Shared memory refers to a large-capacity memory that can be accessed by different processors in a multi-processor computing system. In some embodiments, when a service process running in a server is started, a corresponding virtual address is allocated to the service process, and the virtual address is used to map to a physical address. The server can map the virtual addresses of a plurality of service processes to the same physical address, which is also the physical address of the shared memory. Based on this, a plurality of processes running in the cloud database system can perform data read-write on the shared memory.
[0068] In some embodiments, the target memory is independent of the mutually isolated memory spaces occupied by the plurality of service processes. The plurality of service processes can store executed SQL statements and execution description information of the SQL statements in the target memory, so that in the case of an exception of any process, the data stored in the target memory can not be affected, thereby providing reliable data support for subsequent exception handling and analysis processes.
[0069] In some embodiments, the SQL statement can come from a client of the cloud database running in a plurality of terminals. The server receives the SQL statement, schedules a corresponding process to execute the SQL statement according to a database identifier carried by the SQL statement, thereby performing corresponding operations such as an increase, a deletion, a modification, or a query on a sub-database indicated by the database identifier. In some embodiments, the server is a server cluster composed of a plurality of physical servers, and different service processes can run in the same or different physical servers, which is not limited in the present application.
[0070] The execution description information is used to describe the execution process of the SQL statement. In some embodiments, the execution description information includes at least one of resource usage information, statement status information, and environment status information. In this example, the execution description information includes information of multiple dimensions, thereby comprehensively describing the execution process of the SQL statement from multiple perspectives. The multiple information included in the execution description information is described below.
[0071] In some embodiments, the resource usage information indicates the computing resources occupied in the execution process of the SQL statement. For example, the resource usage information includes at least one of thread memory application and SQL time slice information. The thread memory application refers to the running memory occupied by the thread executing the SQL statement, for example, 32 megabytes (M) or 1 gigabyte (G). In some embodiments, a process can have multiple threads, and each thread is used to execute corresponding program code. The SQL time slice information includes the number of CPU time slices consumed by the execution of the SQL statement to the current state, which can be used to count the time consumed in the execution process of the statement, thereby providing a reference for calculating the actual execution time of the SQL statement. The CPU time slice refers to the time length of the CPU allocated to each program for running the process, for example, 20 milliseconds (ms). In some embodiments, a process can have multiple threads, and each thread is used to execute corresponding program code. The CPU allocates a time period to the thread, and the CPU runs the program code under the thread, for example, the program code corresponding to the SQL statement executed by the thread, in the time period. Therefore, the time consumed in executing the SQL statement can be estimated by multiplying the number of CPU time slices and the time length corresponding to the time slice.
[0072] The execution description information collected through the above process describes the computing resources occupied in the execution process of the SQL statement from the dimensions of running memory and execution time length, thereby enabling to understand the use of computing resources by the SQL statement, and further providing a reference for locating abnormal problems.
[0073] In some embodiments, the statement status information indicates an execution status of the SQL statement. Illustratively, the statement status information includes at least one of a SQL execution status and a SQL execution site. The SQL execution status is the execution status of the SQL statement in the database process. According to different statement execution flows, the type of the execution status can include: waiting, optimizing, executing, rolling back, waiting for locks, success, and failed, etc., which are not limited by the present application. The SQL execution site is the site of the SQL statement in the executing state, which is equivalent to the execution progress of the executing state. The SQL execution site can include: tableScan, filesort, and join, etc. In some embodiments, the different execution sites described above can be executed in parallel, and therefore, the same SQL statement can be in multiple execution sites at the same time.
[0074] The execution description information collected through the above process describes the execution status of the statement at multiple levels, so that the execution of the statement can be understood from multiple granularities, and further provides a reference for the positioning of abnormal problems.
[0075] In some embodiments, the environment status information indicates a running status of the cloud database system when executing the SQL statement. Illustratively, the environment status information includes at least one of a total number of SQL executed by the cloud database and a consumed memory of the cloud database. The total number of SQL executed by the cloud database is the total number of SQL statements executed by the cloud database, and the consumed memory is the number of memory currently used by the cloud database system for running processes. The environment status information can indicate the running load of the cloud database system, so as to reflect the running status of the cloud database system. For example, when the consumed memory reaches a target threshold (e.g., 80%), it indicates that the running load of the cloud database system is too large, and the running status of the cloud database system can be affected.
[0076] The execution description information collected through the above process describes the running status of the cloud database when executing the statement, so that the execution environment on the cloud can be understood, and further provides a reference for the positioning of abnormal problems.
[0077] In some embodiments, the execution description information further comprises more information for describing the execution process of the SQL statement, such as information related to the attributes of the statement, e.g., the type of the SQL and the corresponding client or user of the SQL; or information related to the thread executing the statement, e.g., the thread executing the SQL statement and the stack information of the thread; or information related to the execution of the statement, e.g., the start / end time of executing the SQL statement and the number of opened tables, without limitation.
[0078] In some embodiments, the server can flexibly adjust the information composition of the collected execution description information according to the granularity requirement of the exception analysis and the load condition of the cloud database, without limitation.
[0079] In some embodiments, the target memory is used to store the target number of SQL statements and the execution description information of the target number of SQL statements, in which case, the step 201 comprises the following step 2011 and step 2012.
[0080] The step 2011, in response to receiving any SQL statement for the cloud database system, if the target memory has stored the target number of SQL statements and the execution description information of the target number of SQL statements, deleting the first SQL statement with the earliest execution time and the execution description information of the first SQL statement.
[0081] In some embodiments, the server sequentially stores the SQL statements and the execution description information thereof according to the time when the statement starts to execute, using the first in first out (FIFO) strategy. In this example, in response to receiving any SQL statement, if the target memory has stored the target number of SQL statements and the execution description information thereof, the earliest stored SQL statement and the execution description information thereof are deleted, so as to ensure that only the target number of recently executed SQL statements and the execution description information thereof are retained in the target memory, thereby ensuring the traceability of the abnormal statement within a certain time length without affecting the system performance.
[0082] The step 2012, storing the SQL statement for the cloud database system and the execution description information of the SQL statement into the target memory of the cloud database system.
[0083] In some embodiments, the target memory is divided into multiple slot memory spaces, each of which is allocated to a corresponding process for data reading and writing. In this example, the step 2012 comprises the following step 1 and step 2.
[0084] Step 1, for any one of the SQL statements, according to a first process executing the SQL statement, determine a sub-memory space corresponding to the SQL statement in the target memory, the sub-memory space being used to store execution description information of the SQL statement executed by the first process.
[0085] In some embodiments, the server can query a mapping relationship between a process and a sub-memory space according to a process identifier (PID) of the first process, so as to determine the sub-memory space corresponding to the first process. Illustratively, the mapping relationship can be indicated by a starting address and an offset of the target memory.
[0086] Step 2, store the SQL statement and the execution description information of the SQL statement in the sub-memory space.
[0087] In some embodiments, the server obtains the execution description information of the SQL statement by using a burying point (dotting) technology. Illustratively, the server triggers a collection process for the SQL statement in response to receiving the SQL statement. The server records the execution description information of the SQL statement according to a preset data format during execution of the SQL statement, and stores the execution description information into the target memory. In some embodiments, the data format indicates how to record different information included in the execution description information by using different fields, for example, field 1 is used to record resource usage information, the first 2 bytes of field 1 are used to record thread memory application, and the last 2 bytes of field 1 are used to store CPU time slice quantity; field 2 is used to record statement state information, the first 2 bytes of field 2 are used to record SQL execution state, and the last 2 bytes of field 2 are used to record SQL execution site.
[0088] By using the above technical solution to store the SQL statement and the execution description information thereof, the flexible storage characteristics of the shared memory are utilized, the influence of the disk writing process on the performance of the database system is avoided, and by dividing a plurality of sub-memory areas, the data consistency does not need to be ensured by using locking during the reading and writing processes of multiple processes, so that the data access efficiency is greatly improved.
[0089] 202, in response to detecting that an exception occurs in the cloud database system, generate exception analysis information based on an exception analysis rule and execution description information of at least one SQL statement stored in the target memory.
[0090] In some embodiments, the server actively detects whether an exception occurs in the cloud database system by using a polling manner. In some embodiments, the exception of the process includes active ending of the process and passive ending of the process. Illustratively, the active ending of the process includes the following case 1 and case 2, and the passive ending of the process includes the following case 3.
[0091] Case 1, process crash leads to process end.
[0092] In some embodiments, during the execution of the SQL statement by the process, the corresponding program code is read, and during the execution of the program code, if a code bug is triggered, process crash occurs, the program cannot continue to execute, and the process ends actively.
[0093] Case 2, process management mechanism leads to process end.
[0094] In some embodiments, during the execution of the SQL statement by the process, a code bug may not cause the program to be unable to continue to execute, but cause the statement to be unable to end normally or be interrupted, and fall into a dead loop. In this case, the process management mechanism in the cloud database system will spontaneously control the process to end when the process falls into a dead loop for a target length of time.
[0095] Case 3, operating system controls process end.
[0096] In some embodiments, during the execution of the SQL statement by the process, if too much memory (for example, exceeding a memory threshold) is used, OOM of the operating system is triggered, and the operating system actively ends the process by sending instructions and the like to release the memory occupied by the process in order to maintain the normal operation of the system.
[0097] In related technologies, core dump technology is used to record information related to exceptions in the form of a coredump file, but the core dump technology does not cover a comprehensive range of exception scenarios, for example, the operating system cannot generate a coredump file when ending the process due to OOM. The technical solution provided in the present application can cover a variety of exception scenarios, thereby performing exception handling in a variety of exception scenarios, and improving the comprehensiveness and practicality of the exception handling method.
[0098] In the embodiments of the present application, the exception analysis rule indicates the characteristics of the execution process of the SQL statement that leads to the exception, and the exception analysis information indicates the target SQL statement in the at least one SQL statement that leads to the exception.
[0099] In some embodiments, the exception analysis rule is pre-set by relevant personnel according to common exception scenarios and past engineering experience. The exception analysis rule can be continuously optimized and improved with the accumulation of experience of relevant personnel and business feedback of the cloud database system.
[0100] In some embodiments, the abnormality analysis rule is set according to a known problem type that can cause an abnormality. The problem type indicates a cause of the abnormality, for example, triggering a program code bug, an OOM occurring, or involving a high-risk function module, etc. Accordingly, the feature indicated by the abnormality analysis rule can describe an execution process of a statement when a program code bug occurs, for example, a SQL execution state or a SQL execution site of the statement when bug 1 occurs, etc.; can also describe an execution process of a statement when an OOM occurs, for example, a size of memory applied for by a thread when an OOM occurs; and can also describe a program code block, a data table identifier, etc. involved by a high-risk function module.
[0101] In some embodiments, if an execution process described by execution description information of a target SQL statement in the at least one SQL statement matches a target feature indicated by the abnormality analysis rule, abnormality analysis information of the target SQL statement is generated. The abnormality analysis information of the target SQL statement includes a problem type corresponding to the target feature, and the problem type is used to indicate a cause of the target SQL causing an abnormality. For example, if the SQL execution state and the SQL execution site included in the execution description information of the target SQL statement match the target feature corresponding to bug 1, it is indicated in the abnormality analysis information of the target SQL that the problem type corresponding to the SQL statement is bug 1; and if the thread memory application included in the execution description information of the target SQL statement matches the target feature corresponding to an OOM problem, it is indicated in the abnormality analysis information of the target SQL that the problem type corresponding to the SQL statement is an OOM occurring.
[0102] In some embodiments, the abnormality analysis information further includes risk point information. When determining the problem type, the server further indicates an execution step with a risk according to the SQL execution state, the SQL execution site, the program code block, etc. that cause the problem, so as to provide more detailed abnormality analysis reference. In some embodiments, the server can generate the abnormality analysis information within 5 seconds.
[0103] In some embodiments, the server can send the abnormality analysis information to a terminal of a relevant person in the form of an abnormality analysis report, so as to efficiently assist the relevant person in locating an abnormality root cause and timely providing an abnormality avoidance scheme in a clear and intuitive display form.
[0104] Through the above process, the risk possibly existing in each SQL statement can be analyzed based on multi-dimensional execution description information, and corresponding abnormality analysis information is generated. The generation process of the abnormality analysis information is automated based on the preset abnormality analysis rule. The problem type is automatically analyzed and the risk point is prompted, so as to reduce the workload of the relevant person in the abnormality processing process, reduce the time consumption of analyzing the abnormality, and greatly improve the efficiency of problem analysis and positioning.
[0105] In some embodiments, the server stores the SQL statement and the execution description information of the SQL statement in a target storage node in response to detecting the exception in the cloud database system. The server stores at least one SQL statement and the execution description information of the at least one SQL statement stored in the target memory into the target storage node in response to detecting the exception in the cloud database system, and releases the target memory. The target storage node is configured to provide a storage service. In some embodiments, the target storage node can be a storage server in the cloud database system, or a storage node outside the cloud database system, which is not limited in the embodiments of the present application. In some embodiments, the target storage node can be a storage node outside the cloud database system for providing OBS, and the cloud database system can conveniently use the OBS provided by the storage node by calling an API. In the case where the server stores the execution description information in the target storage node, the server performing the process of step 202 can include: obtaining the at least one SQL statement and the execution description information of the at least one SQL statement from the target storage node; and generating the exception analysis information based on the exception analysis rule and the at least one SQL statement and the execution description information of the at least one SQL statement obtained from the target storage node. The process of generating the exception analysis information is the same as the above process, which is not described herein.
[0106] In some embodiments, the server performs preliminary analysis on the log file of the cloud database system in response to detecting the exception, and obtains a log analysis result. When storing the SQL statement and the execution description information of the SQL statement in the target storage node, the server synchronously stores the log analysis result in the target storage node to enrich the data scale available for the exception analysis. In some embodiments, the server compresses the log analysis result and the SQL statement and the execution description information of the SQL statement into a file, and sends the file to the target storage node. In some embodiments, the process from detecting the exception to sending the file to the target storage node can be completed within 5 seconds, and then the server generates the exception analysis information based on the obtained file within 5 seconds. Based on this, from detecting the exception to generating the exception analysis information, only 10 seconds are required, and the time cost for exception processing is significantly shortened.
[0107] In the related art, the process of using core dump technology to dump the coredump file consumes tens of seconds to several minutes, thereby seriously delaying the process of restarting and switching the database from an exception and reducing the availability of the cloud database service. The technical solution provided in the present application does not have additional processing logic in the process of obtaining and storing the SQL statements and the execution description information, and does not affect the normal operation of the business. Storing the SQL statements and the execution description information in the target memory into the target storage node can release the memory in time after an exception occurs, so as to ensure that the cloud database can recover from the exception as soon as possible and effectively improve the availability of the cloud database service. The related information used for exception analysis is stored in a safe and stable storage mode, thereby providing reliable data support for the subsequent exception analysis and processing process.
[0108] In the related art, the full log is usually used to store the SQL statements executed by the database, so that in the case of an exception of the database, the full log stored in the hard disk can be provided to the operation and maintenance personnel. On the one hand, the mechanism of the full log requires that the executed SQL statements be written to the disk during operation, which will cause a performance loss of about 30% to the database system, thereby seriously affecting the service efficiency of the cloud database. In addition, when the full log is provided to the operation and maintenance personnel, a certain time is also required to read the hard disk. On the other hand, the full log only records the executed SQL statements, but does not record the execution state corresponding to the statements. Therefore, the operation and maintenance personnel cannot directly determine which statement caused the database process exception by checking the full log, and experienced personnel need to spend a lot of time to analyze the exception. The accuracy of analyzing the cause of the exception depends on the experience and ability of the operation and maintenance personnel, thereby causing a long time consumption of the exception processing. Therefore, the exception processing efficiency in the related art is not high.
[0109] The technical solution provided in the embodiments of the present application can effectively overcome the problems in the related art. On the one hand, the present application uses shared memory to store the executed SQL statements and the related execution description information, and the information does not need to be written to the disk, thereby having little effect on the performance of the database system. For the high-concurrency scenario of the cloud database, the performance overhead during the operation of the database can be greatly reduced, and the effect on the normal business is reduced, thereby making the cloud database more robust. In some embodiments, the performance overhead caused by using the technical solution provided in the present application to store the SQL statements and the execution description information can be controlled within 3%. On the other hand, the execution description information in multiple dimensions can be integrated to analyze the possible risks of each SQL statement, generate corresponding exception analysis information, and make the generation process of the exception analysis information automatic based on the preset exception analysis rule, thereby reducing the workload of the related personnel in the exception processing process, reducing the time consumption of analyzing the exception, and greatly improving the efficiency of problem analysis and positioning.
[0110] Based on the above Figure 1 Provided is an implementation environment and Figure 2 Provided is an embodiment, and the present application provides a schematic diagram of an exception handling method of a cloud database, referring to Figure 3 The exception handling method of the cloud database can be cooperatively implemented by a plurality of functional modules deployed on a server. The data acquisition module 301 is configured to perform the acquisition and storage process of the execution description information described in step 201; the exception detection module 302 is configured to perform the process of detecting an exception and packaging and sending relevant data to a target storage node described in step 202; the data analysis module 303 is configured to perform the process of generating exception analysis information described in step 202, which can be sent to an operation and maintenance personnel in the form of an exception analysis report to assist the operation and maintenance personnel in quickly giving an exception avoidance scheme based on the exception analysis report; the exception analysis information can be sent to a technical personnel in the form of an exception analysis report to assist the technical personnel in quickly analyzing and repairing system problems based on the exception analysis report. For details, refer to the foregoing, and no further description is given here.
[0111] Figure 4 is a structural schematic diagram of an exception handling device of a cloud database provided by an embodiment of the present application. The exception handling device of the cloud database is configured in a cloud database system and can implement the foregoing exception handling method of the cloud database through software, hardware, or a combination of both. As shown in Figure 4 The device comprises:
[0112] The storage module 401 is configured to, during the operation of the cloud database system, store the structured query language (SQL) statements for the cloud database system and the execution description information of the SQL statements in a target memory of the cloud database system, the target memory being configured to provide a shared data read-write space for at least one process running in the cloud database system, and the execution description information being configured to describe the execution process of the SQL statements;
[0113] The analysis module 402 is configured to, in response to detecting an exception occurring in the cloud database system, generate exception analysis information based on an exception analysis rule and the execution description information of at least one SQL statement stored in the target memory, the exception analysis rule indicating the characteristics that the execution process of the SQL statement causing the exception conforms to, and the exception analysis information indicating the target SQL statement causing the exception in the at least one SQL statement.
[0114] In a possible implementation manner, the target memory is configured to store a target number of SQL statements and execution description information of the target number of SQL statements; and the storage module 401 is configured to:
[0115] In response to receiving any SQL statement for the cloud database system, if the target memory has stored the target number of SQL statements and the execution description information of the target number of SQL statements, the first SQL statement with the earliest execution time and the execution description information of the first SQL statement are deleted.
[0116] The SQL statement for the cloud database system and the execution description information of the SQL statement are stored in the target memory of the cloud database system.
[0117] In a possible implementation, the storage module 401 is configured to:
[0118] For any SQL statement, according to a first process executing the SQL statement, a sub-memory space corresponding to the SQL statement in the target memory is determined, and the sub-memory space is used to store the execution description information of the SQL statement executed by the first process.
[0119] The SQL statement and the execution description information of the SQL statement are stored in the sub-memory space.
[0120] In a possible implementation, the execution description information includes at least one of resource usage information, statement status information and environment status information; the resource usage information indicates the computing resource occupied in the execution process of the SQL statement; the statement status information indicates the execution status of the SQL statement; and the environment status information indicates the running status of the cloud database system when the SQL statement is executed.
[0121] In a possible implementation, the exception includes active ending of a process and passive ending of a process.
[0122] In a possible implementation, the analysis module 402 is configured to:
[0123] If the execution process described by the execution description information of the target SQL statement in the at least one SQL statement meets the target feature indicated by the exception analysis rule, the exception analysis information of the target SQL is generated, and the exception analysis information of the target SQL includes a problem type corresponding to the target feature, and the problem type is used to indicate the reason for causing the exception of the target SQL.
[0124] In a possible implementation, the storage module 401 is further configured to:
[0125] The at least one SQL statement and the execution description information of the at least one SQL statement stored in the target memory are stored in a target storage node, and the target memory is released.
[0126] The analysis module 402 is configured to generate the abnormality analysis information based on the abnormality analysis rule and the at least one SQL statement and the execution description information of the at least one SQL statement obtained from the target storage node.
[0127] In the technical solution provided in the present application, the shared memory is used to store the executed SQL statement and the related execution description information, and the information does not need to be written to the disk, which has little impact on the performance of the database system. For the high-concurrency scenario of the cloud database, the performance overhead of the database running period can be greatly reduced, and the impact on the normal business is reduced, so that the cloud database is more robust and reliable. In some embodiments, the performance overhead caused by the use of the technical solution provided in the present application to store the SQL statement and the execution description information thereof can be controlled within 3%. On the other hand, the execution description information of multiple dimensions can be integrated to analyze the possible risks of each SQL statement, and the corresponding abnormality analysis information is generated. Based on the preset abnormality analysis rule, the generation process of the abnormality analysis information is automated, the workload of the related personnel in the abnormality processing process is reduced, the time consumption for analyzing the abnormality is reduced, and the efficiency of problem analysis and positioning is greatly improved.
[0128] In addition, the storage module 401 and the analysis module 402 can be implemented by software or by hardware. For example, the implementation of the storage module 401 is described below. Similarly, the implementation of the analysis module 402 and other modules can refer to the implementation of the storage module 401.
[0129] As an example of a software functional unit, the storage module 401 can include code running on a computing instance. The computing instance can include at least one of a physical host (computing device), a virtual machine, and a container. Further, the computing instance can be one or more. For example, the storage module 401 can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region (region) or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, and each AZ includes a data center or multiple data centers with similar geographical locations. Generally, one region can include multiple AZs.
[0130] Similarly, the plurality of hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Among them, usually one VPC is set in one region, and a communication gateway needs to be set in each VPC for cross-region communication between two VPCs in the same region and between VPCs in different regions, and the interconnection between VPCs is realized through the communication gateway.
[0131] As an example of a hardware functional unit, the storage module 401 can include at least one computing device. Alternatively, the storage module 401 can also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), etc. Among them, the above-mentioned PLD can be implemented by a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.
[0132] The plurality of computing devices included in the storage module 401 can be distributed in the same region or in different regions. The plurality of computing devices included in the storage module 401 can be distributed in the same AZ or in different AZs. Similarly, the plurality of computing devices included in the storage module 401 can be distributed in the same VPC or in multiple VPCs. Among them, the plurality of computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
[0133] It should be noted that in other embodiments, the storage module 401 can be used to perform any step of the cloud database exception handling method, that is, the steps implemented by the storage module 401 and the analysis module 402 can be specified as needed, and the entire function of the cloud database exception handling device can be realized by the storage module 401 and the analysis module 402 respectively implementing different steps of the cloud database exception handling method. In addition, the cloud database exception handling device provided in the above-mentioned embodiments and the cloud database exception handling method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0134] The present application also provides a computing device that can be configured as a server in the above-mentioned implementation environment. For referenceFigure 5 , Figure 5 is a hardware structure schematic diagram of a computing device provided by an embodiment of the present application. As shown in Figure 5 the computing device 500 includes a bus 502, a processor 504, a memory 506 and a communication interface 508. The processor 504, the memory 506 and the communication interface 508 communicate through the bus 502. It should be understood that the number of processors and memories in the computing device 500 is not limited by the present application.
[0135] The bus 502 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 only one line is used in the figure, but it does not mean that there is only one bus or only one type of bus. The bus 504 can include a path for transmitting information between various components (for example, the memory 506, the processor 504, the communication interface 508) of the computing device 500.
[0136] The processor 504 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.
[0137] The memory 506 can include a volatile memory (for example, a random access memory (RAM)). The processor 504 can also include a non-volatile memory (for example, a read-only memory (ROM), a flash memory, a mechanical hard disk drive (HDD) or a solid state drive (SSD)).
[0138] The executable program code is stored in the memory 506. Taking the computing device configured as the server as an example, the processor 504 executes the executable program code to respectively implement the functions of the aforementioned storage module 401 and analysis module 402, thereby implementing the functions of the server in the cloud database exception processing method. That is, the memory 506 stores instructions for executing the cloud database exception processing method.
[0139] The communication interface 503 uses a transceiving module such as but not limited to a network interface card and a transceiver to implement communication between the computing device 500 and other devices or a communication network.
[0140] The embodiments of the present application further provide a computing device cluster. The computing device cluster includes at least one computing device. Figure 6 is a structural schematic diagram of a computing device cluster provided by the embodiments of the present application. As shown in the figure, the computing device cluster includes at least one computing device 500. The memory 506 in one or more computing devices 500 in the computing device cluster can store the same instructions for executing the cloud database exception processing method. Figure 6
[0141] In some possible implementations, the memory 506 of one or more computing devices 500 in the computing device cluster can also respectively store partial instructions for executing the cloud database exception processing method. In other words, the combination of one or more computing devices 500 can collectively execute the instructions for the cloud database exception processing method.
[0142] It should be noted that the memories 506 in different computing devices 500 in the computing device cluster can store different instructions, respectively used to execute Figure 5 the partial functions of the cloud database exception processing apparatus as shown in the figure. That is, the instructions stored in the memories 506 in different computing devices 500 can implement the functions of one or more of the storage module 401 and analysis module 402.
[0143] In some embodiments, one or more computing devices in the computing device cluster can be connected through a network. The network can be a wide area network or a local area network, etc. Figure 7 is a connection mode schematic diagram of a computing device cluster provided by the embodiments of the present application. As shown in the figure, Figure 7 two computing devices 500 are connected through a network. Specifically, the communication interface in each computing device is connected to the network. In this type of possible implementation, Figure 7 the connection mode between the computing device clusters as shown in the figure takes into account the cloud database exception processing method provided by the present application. Figure 4 The illustrated cloud database anomaly processing apparatus involves different types of modules, and thus the memory of different computing devices stores instructions for executing the functions of the different modules. For example, the memory 506 in one computing device 500 stores instructions for executing the functions of the storing module 401. The memory 506 in another computing device 500 stores instructions for executing the functions of the analyzing module 402.
[0144] It should be understood that Figure 7 The functions of the computing device 500 illustrated in the foregoing embodiments can also be completed by multiple computing devices 500.
[0145] The terms "first", "second", and the like in the present application are used to distinguish between elements or items that have substantially the same function and are similar to each other, and it should be understood that there is no logical or chronological dependency between "first", "second", and "nth", and the number and execution order are not limited. It should also be understood that although the following description uses the terms first, second, and the like to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of various examples, a first process can be referred to as a second process, and similarly, a second process can be referred to as a first process. The first process and the second process can both be processes, and in some cases, can be separate and distinct processes.
[0146] In the present application, the term "at least one" means one or more, and the term "multiple" in the present application means two or more, for example, multiple processes means two or more processes.
[0147] The above description is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
[0148] In the above embodiments, all or part can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part can be implemented in the form of program structure information. The program structure information includes one or more program instructions. When the program instructions are loaded and executed on a computing device, all or part of the flow or function in the embodiments of the present application is generated.
[0149] A person of ordinary skill in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or by a program instructing related hardware, and the program can be stored in a computer readable storage medium, such as a read-only memory, a magnetic disk or an optical disk.
[0150] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those ordinarily skilled in the art should understand: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. An abnormality processing method of a cloud database, characterized by, The method is applied to a cloud database system, and comprises the following steps: In the running process of the cloud database system, structured query language (SQL) statements for the cloud database system and execution description information of the SQL statements are stored in a target memory of the cloud database system, the target memory is a part of memory space divided based on a shared memory technology and used for providing a shared data read-write space for at least one process running in the cloud database system, the target memory is independent of a memory space occupied by the at least one process, the execution description information is used for describing an execution process of a SQL statement, and the execution description information comprises at least one of resource usage information, statement state information and environment state information; the resource usage information indicates an operation resource occupied in the execution process of the SQL statement; the statement state information indicates an execution state of the SQL statement; and the environment state information indicates a running state of the cloud database system when the SQL statement is executed; In response to detecting an exception occurring in the cloud database system, exception analysis information is generated based on an exception analysis rule and execution description information of at least one SQL statement stored in the target memory, the exception analysis rule indicates a feature to which an execution process of a SQL statement causing the exception conforms, and the exception analysis information indicates a target SQL statement causing the exception in the at least one SQL statement; The execution description information of the SQL statement is acquired through a burying point technology; the resource usage information comprises at least one of thread application memory and SQL time slice information, the thread application memory is running memory occupied by a thread executing the SQL statement, and the SQL time slice information indicates time consumed in the execution process of the SQL statement; the statement state information comprises at least one of a SQL execution state and a SQL execution site, the SQL execution state is an execution state of the SQL statement in a database process, and the SQL execution site is a site at which the SQL statement is in an executing state; and the environment state information comprises at least one of a total number of SQLs executed by the cloud database and memory consumed by the cloud database, the total number of SQLs executed by the cloud database is a total number of SQL statements executed by the cloud database, and the consumed memory is a number of memory currently used by the cloud database system for running processes.
2. The method of claim 1, wherein, The target memory is used for storing a target number of SQL statements and execution description information of the target number of SQL statements; and the step of storing, in the running process of the cloud database system, the SQL statements for the cloud database system and the execution description information of the SQL statements in the target memory of the cloud database system comprises the following steps: In response to receiving any SQL statement for the cloud database system, if the target number of SQL statements and the execution description information of the target number of SQL statements have been stored in the target memory, a first SQL statement with the earliest execution time and execution description information of the first SQL statement are deleted. The SQL statement for the cloud database system and execution description information of the SQL statement are stored in a target memory of the cloud database system.
3. The method according to claim 1 or 2, characterized in that, The storing of the SQL statement for the cloud database system and execution description information of the SQL statement in the target memory of the cloud database system comprises: For any one of the SQL statements, a sub-memory space corresponding to the SQL statement in the target memory is determined according to a first process executing the SQL statement, and the sub-memory space is used to store execution description information of the SQL statement executed by the first process; The SQL statement and the execution description information of the SQL statement are stored in the sub-memory space.
4. The method according to claim 1 or 2, characterized in that, The generating of the abnormal analysis information based on the abnormal analysis rule and the execution description information of at least one of the SQL statements stored in the target memory comprises: If an execution process described by execution description information of a target SQL statement in the at least one of the SQL statements conforms to a target feature indicated by the abnormal analysis rule, abnormal analysis information of the target SQL statement is generated, and the abnormal analysis information of the target SQL statement comprises a problem type corresponding to the target feature, and the problem type is used to indicate a reason for causing an exception of the target SQL statement.
5. The method according to claim 1 or 2, characterized in that, Before the generating of the abnormal analysis information based on the abnormal analysis rule and the execution description information of at least one of the SQL statements stored in the target memory, the method further comprises: The at least one of the SQL statements and the execution description information of the at least one of the SQL statements stored in the target memory are stored in a target storage node to release the target memory; The generating of the abnormal analysis information based on the abnormal analysis rule and the execution description information of at least one of the SQL statements stored in the target memory comprises: The abnormal analysis information is generated based on the abnormal analysis rule and the at least one of the SQL statements and the execution description information of the at least one of the SQL statements obtained from the target storage node.
6. An abnormality processing apparatus of a cloud database, characterized by comprising: The device is configured in a cloud database system, and the device comprises: A storage module is configured to store, in a running process of the cloud database system, a structured query language (SQL) statement for the cloud database system and execution description information of the SQL statement in a target memory of the cloud database system, the target memory being a part of memory space divided based on a shared memory technology and used to provide a shared data read-write space for at least one process running in the cloud database system, the target memory being independent of a memory space occupied by the at least one process, and the execution description information being used to describe an execution process of the SQL statement and comprising at least one of resource usage information, statement state information and environment state information, the resource usage information indicating an operation resource occupied in the execution process of the SQL statement, the statement state information indicating an execution state of the SQL statement, and the environment state information indicating a running state of the cloud database system when the SQL statement is executed. The analysis module is configured to, in response to detecting that an exception occurs in the cloud database system, generate exception analysis information based on an exception analysis rule and execution description information of at least one SQL statement stored in the target memory, the exception analysis rule indicating a feature that a SQL statement execution process leading to the exception conforms to, and the exception analysis information indicating a target SQL statement in the at least one SQL statement that leads to the exception. The execution description information of the SQL statement is obtained through a point embedding technology; the resource usage information includes at least one of thread memory application and SQL time slice information, the thread memory application refers to running memory occupied by a thread executing the SQL statement, and the SQL time slice information indicates time consumed by the SQL statement execution process; the statement state information includes at least one of a SQL execution state and a SQL execution site, the SQL execution state refers to an execution state of the SQL statement in a database process, and the SQL execution site refers to a site of the SQL statement in an executing state; and the environment state information includes at least one of a total number of SQLs executed by the cloud database and memory consumed by the cloud database, the total number of SQLs executed by the cloud database refers to a total number of SQL statements executed by the cloud database, and the consumed memory refers to a number of memories currently used by the cloud database system for running processes.
7. The apparatus of claim 6, wherein, The target memory is configured to store a target number of SQL statements and execution description information of the target number of SQL statements; and the storage module is configured to: in response to receiving any SQL statement for the cloud database system, if the target number of SQL statements and the execution description information of the target number of SQL statements have been stored in the target memory, delete a first SQL statement with the earliest execution time and execution description information of the first SQL statement; store the SQL statement for the cloud database system and the execution description information of the SQL statement into the target memory of the cloud database system.
8. The apparatus of claim 6 or 7, wherein, The storage module is configured to: for any SQL statement, determine a sub-memory space corresponding to the SQL statement in the target memory according to a first process executing the SQL statement, the sub-memory space being configured to store execution description information of the SQL statement executed by the first process; and store the SQL statement and the execution description information of the SQL statement in the sub-memory space.
9. The apparatus of claim 6 or 7, wherein, The analysis module is configured to: if execution description information of a target SQL statement in the at least one SQL statement describes an execution process that conforms to a target feature indicated by the exception analysis rule, generate exception analysis information of the target SQL statement, the exception analysis information of the target SQL statement including a problem type corresponding to the target feature, and the problem type being configured to indicate a reason why the target SQL statement causes the exception.
10. The apparatus of claim 6 or 7, wherein, The storage module is further configured to: store at least one SQL statement stored in the target memory and execution description information of the at least one SQL statement into the target storage node, and release the target memory; The analysis module is configured to generate the abnormality analysis information based on an abnormality analysis rule and the at least one SQL statement and the execution description information of the at least one SQL statement obtained from the target storage node.
11. A cluster of computing devices, characterized in that, The at least one computing device includes a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the abnormality processing method of the cloud database according to any one of claims 1 to 5.
12. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store at least one program code, when the at least one program code is executed by a computing device, so that the computing device executes the abnormality processing method of the cloud database according to any one of claims 1 to 5.
13. A computer program product, characterised in that, When the computer program product is executed on a computing device, the computing device executes the abnormality processing method of the cloud database according to any one of claims 1 to 5.
Citation Information
Patent Citations
Database operation statement processing method and device, electronic equipment and medium
CN112181948A
SQL statement detection method and device, terminal equipment and storage medium
CN112948415A
Method and device for acquiring execution information
CN113297245A
Data processing method and device, electronic equipment and storage medium
CN114265900A