Database overload protection processing method and device, storage medium

Abstract

The present disclosure relates to a database overload protection processing method, device, and storage medium. The database overload protection processing method includes: collecting target database information in real time according to the overload monitoring index in the configuration file; judging whether the target database triggers the overload threshold according to the target database information; and in the case where the target database information triggers the overload threshold, overload control is performed on the target database according to the overload control rules in the configuration file. The present disclosure can improve the timeliness of database overload monitoring and improve the effectiveness of database overload control.

Description

Technical Field

[0001] The present disclosure relates to the field of communications, and in particular to a database overload protection processing method and device, and a storage medium. Background Art

[0002] A database is a system that uses a certain data structure to organize, store, and manage data. It plays a very important role in information systems. In system operation and maintenance, database failures often cause global failures of the information system. How to ensure stable and efficient operation of the database is the primary issue faced by every DBA (Database Administrator).

[0003] Among database failures, a large part of the reasons are caused by database overload, which means that the consumption of database resources exceeds a certain limit, resulting in reduced database availability. The main manifestations of database overload include the inability of the client to connect to the database and slow database response. The factors that cause database overload are high server CPU (central processing unit) load, high database I / O (input / output) pressure, insufficient database server memory, database connection number exceeds the limit, insufficient network bandwidth, application bugs, and low efficiency of SQL (Structured Query Language). If the database overload is not monitored in time and effective control measures are not taken, it will cause further shortage of database resources and even database downtime. Summary of the invention

[0004] In view of at least one of the above technical problems, the present disclosure provides a database overload protection processing method and device, and a storage medium, which can improve the timeliness of database overload monitoring.

[0005] According to one aspect of the present disclosure, a database overload protection processing method is provided, comprising:

[0006] According to the overload monitoring indicators in the configuration file, the target database information is collected in real time;

[0007] According to the target database information, determining whether the target database information triggers an overload threshold;

[0008] When the target database information triggers the overload threshold, the target database is overloaded according to the overload control rules in the configuration file.

[0009] In some embodiments of the present disclosure, the database overload protection processing method further includes:

[0010] A configuration file is pre-set, wherein the configuration file includes overload monitoring indicators and overload control rules for different database types.

[0011] In some embodiments of the present disclosure, when the target database information triggers the overload threshold, overload control of the target database according to the overload control rule in the configuration file includes:

[0012] Collecting slow structured query language information of a target database whose resource consumption is greater than a predetermined threshold, and determining an overloaded process number according to the slow structured query language information;

[0013] A cleanup action is performed on the overloaded process with the overloaded process number.

[0014] In some embodiments of the present disclosure, when the target database information triggers the overload threshold, overload control of the target database according to the overload control rule in the configuration file further includes:

[0015] Record the overload cleanup log and notify the relevant maintenance personnel of the overload cleanup log.

[0016] In some embodiments of the present disclosure, for a MYSQL database, the slow structured query language information in the target database that consumes resources greater than a predetermined threshold is collected, including:

[0017] Use MYSQL abnormal connection log to locate slow structured query language information.

[0018] In some embodiments of the present disclosure, for a MYSQL database, the overload control rule includes:

[0019] When the MYSQL abnormal connection log shows the first number of too many connections errors or the second number of unavailable connections errors, the slow structured query language information is located to determine the overload process number and perform overload cleanup.

[0020] In some embodiments of the present disclosure, the performing a cleanup action on the overloaded process with the overloaded process number includes:

[0021] Filter query statements to filter processes corresponding to specific abnormal events. The first step is to clean up local processes on a small scale.

[0022] If the database overload situation does not improve, take further steps to expand the scope of the process cleanup.

[0023] In some embodiments of the present disclosure, for an ORACLE database, collecting slow structured query language information of a target database in which resource consumption is greater than a predetermined threshold, and determining an overload process number according to the slow structured query language information includes:

[0024] Categorize abnormal events that cause overload in ORACLE database;

[0025] Determine whether resource utilization has reached a specified threshold, wherein the resource utilization includes CPU, memory, and swap area resource utilization;

[0026] When the resource usage reaches a specified threshold, the overload process ID is determined.

[0027] According to another aspect of the present disclosure, a database overload protection processing device is provided, comprising:

[0028] An overload monitoring module is used to collect target database information in real time according to the overload monitoring indicators in the configuration file; and to determine whether the target database information triggers an overload threshold according to the target database information;

[0029] The overload control and protection module is used to perform overload control on the target database according to the overload control rules in the configuration file when the target database information triggers the overload threshold.

[0030] In some embodiments of the present disclosure, the database overload protection processing is used to execute operations to implement the database overload protection processing method described in any of the above embodiments.

[0031] According to another aspect of the present disclosure, a database overload protection processing device is provided, comprising:

[0032] A memory for storing instructions;

[0033] The processor is used to execute the instruction so that the device performs the operation of implementing the database overload protection processing method as described in any of the above embodiments.

[0034] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the database overload protection processing method as described in any of the above embodiments is implemented.

[0035] The present disclosure can improve the timeliness of database overload monitoring and improve the effectiveness of database overload control. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 Schematic diagram of some embodiments of the database overload protection processing method disclosed in the present invention.

[0038] Figure 2 Schematic diagram of other embodiments of the database overload protection processing method disclosed in the present invention.

[0039] Figure 3 Schematic diagram of some further embodiments of the database overload protection processing method disclosed in the present invention.

[0040] Figure 4 Schematic diagram of some further embodiments of the database overload protection processing method disclosed in the present invention.

[0041] Figure 5 Schematic diagram of some embodiments of the database overload protection processing device disclosed in the present invention.

[0042] Figure 6 Schematic diagrams of other embodiments of the database overload protection processing device disclosed in the present invention. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0044] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0045] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0046] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0047] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0048] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0049] The inventors have found through research that, in response to the database overload problem, the relevant technology adopts the following three processing methods:

[0050] First, service degradation: During peak hours, through business control or human management intervention, only some key businesses are limited to be available, and other marginal non-key businesses are suspended or handled in a staggered manner to control the database load and avoid overload.

[0051] Second, business flow control: Set a resource high-water mark limit at the information system middleware layer. For example, set a high-water mark limit on the number of requests on the middleware server. If the number of requests within a certain period of time exceeds a certain set threshold, the request will be intercepted and the system busy page will be output to ensure that the abnormal pressure on the front end is not transmitted to the database and the database load is controlled.

[0052] Third, database monitoring and manual processing: Introduce a monitoring system on the database side, periodically sample and monitor the database, evaluate the comprehensive performance indicators of the database, and establish a monitoring alarm mechanism. After the database administrator receives the alarm, manual intervention is required to make judgments and handle the problem.

[0053] The first technology mentioned above uses service degradation. First, it is necessary to distinguish between business peaks and valleys, and distinguish between key businesses and non-key businesses, in order to achieve overload protection. However, if key businesses will cause database overload during business peaks, this method cannot be used to achieve database overload protection.

[0054] The second technology mentioned above uses the middleware current limiting method, which generally limits the middleware load flow by setting the number of middleware requests. However, programs such as database interconnection access or data storage processes do not go through the middleware. The overload caused by this part of the load cannot be protected by this current limiting method.

[0055] The third technology mentioned above is to use a database monitoring system. The accuracy of monitoring and alarming depends on the rationality of indicators and the timeliness of data collection. After receiving the alarm, the DBA needs a certain amount of response time. After logging into the system to view various information, it will analyze and judge, and make emergency treatment. However, when the database triggers an overload alarm, the two factors of increased load and tight resources often interact with each other, exacerbating the database overload in a short period of time. The time left for the DBA to manually intervene is very short, making it difficult to effectively deal with the database overload situation.

[0056] In summary, the above three technologies have certain effects in solving the problem of database overload protection through various different methods, but they all have defects and deficiencies to varying degrees. Overload monitoring coverage is not comprehensive, there is a lack of effective overload protection control, and overload protection cannot be provided in a timely manner. For example, if the application SQL statement is inefficient, a large number of slow SQL statements appear, and concurrent SQL statements continue to pour in, if overload protection is not taken in a timely manner, the number of database connections will reach the maximum value in a short period of time or the database's I / O resources and CPU resources will be exhausted, eventually causing the entire database to hang or crash. Local errors are rapidly amplified by business peak pressure, and the impact is quickly expanded to a global database system problem. Since the processing capacity of a single node in a distributed database is relatively weak, the above phenomenon is particularly evident in a distributed database.

[0057] The inventors have found through research that related technologies for information system overload control are mainly implemented at the middleware layer by means of current limiting or service degradation, and overload anomalies at the database end are mainly monitored, and overload control at the database end is not timely.

[0058] In view of various deficiencies in database overload protection of mainstream database monitoring systems in related technologies, the above-mentioned embodiments of the present disclosure propose a database overload protection processing method and device, and the present disclosure is described below through specific embodiments.

[0059] Figure 1 Schematic diagram of some embodiments of the database overload protection processing method disclosed in the present invention. Preferably, this embodiment can be executed by the database overload protection processing device disclosed in the present invention. The method includes the following steps:

[0060] Step 11: Collect target database information in real time according to the overload monitoring indicators in the configuration file.

[0061] Step 12: Determine whether the target database information triggers an overload threshold based on the target database information.

[0062] Step 13: When the target database information triggers the overload threshold, overload control is performed on the target database according to the overload control rules in the configuration file.

[0063] In some embodiments of the present disclosure, the step of overload controlling the target database according to the overload control rules in the configuration file may include: the overload control rules determine the priority of the overload protection intervention steps according to the degree of impact that the overload intervention may cause to the information system, and the steps with smaller impacts are done first. If there is no effect, the steps with larger impacts are implemented step by step.

[0064] Based on the database overload protection processing method provided by the above embodiment of the present disclosure, an overload protection scheme on the database side is adopted, which is transparent and non-intrusive to the application program. A database overload monitoring method and a database overload protection control method are proposed, which can improve the timeliness of database overload monitoring and improve the effectiveness of database overload control.

[0065] Figure 2 Schematic diagram of other embodiments of the database overload protection processing method disclosed in the present invention. Preferably, this embodiment can be executed by the database overload protection processing device disclosed in the present invention. The method includes the following steps:

[0066] Step 21 : pre-set a configuration file, wherein the configuration file may include overload monitoring indicators and overload control rules for different database types.

[0067] In some embodiments of the present disclosure, step 21 may include: configuring a configuration file to save data and host collection indicators, and these indicators may be supplemented with indicator values ​​based on the experience of the system administrator or new features of the new version. The configuration file includes two parts: collection monitoring rules and overload control rules. In the collection monitoring rule part, it is possible to set whether the indicator needs information collection and subsequent overload control. The overload control rule part sets different thresholds for each indicator, and starts the overload protection process when the collected data reaches the predetermined threshold.

[0068] In some embodiments of the present disclosure, step 21 may include: setting a configuration file, configuring various objects and related indicators for data overload monitoring, and determining whether to collect the indicator information and whether to perform the overload control action corresponding to the indicator by setting the value of the monitoring indicator.

[0069] Step 22, collecting host information, wherein the host information includes the host's database type, CPU usage, memory usage, and swap area usage, which are used for comprehensive evaluation with other indicators in subsequent steps.

[0070] Step 23: Collect and analyze slow SQL statements.

[0071] In some embodiments of the present disclosure, step 23 may include: collecting slow structured query language information in the target database that consumes resources greater than a predetermined threshold.

[0072] In some embodiments of the present disclosure, step 23 may include: using MYSQL abnormal connection log to locate slow structured query language information.

[0073] In some embodiments of the present disclosure, step 23 may include: collecting the slow SQL that consumes the most resources in the database, and the threshold value of the slow SQL is set in the configuration file.

[0074] In some embodiments of the present disclosure, different databases have different analysis methods for slow SQL.

[0075] In some embodiments of the present disclosure, for MYSQL (a relational database management system), step 23 may include: using a slow log to locate a slow SQL.

[0076] In some other embodiments of the present disclosure, for ORACLE (another relational database management system), step 23 may include: associating active session history v$active_session_history with v$sql, filtering connections with session type as foreground (session_type='FOREGROUND'), taking a session set of the last two minutes, accumulating the number of sessions (number of sessions) with session type as session_state='ON CPU' on the CPU, obtaining a total database request time db_time_total, sorting the total database request time db_time_total and taking out the top 50 sessions (TOP 50 sessions), dividing the total database request time db_time_total by the number of different sessions to obtain the execution time single_sql_times of a single SQL, and determining the slow SQL according to single_sql_times.

[0077] Step 24: determine the overloaded process number according to the slow structured query language information.

[0078] In some embodiments of the present disclosure, step 24 may include: in order not to affect data security and transaction validity, extract the query statement select type SQL from the slow SQL extracted in the previous step, associate it with v$process (for ORACLE database) or process list processlist (for MYSQL database), and determine the overloaded process number.

[0079] Step 25: Perform a cleanup operation on the overloaded process with the overloaded process number.

[0080] In some embodiments of the present disclosure, step 25 may include: passing the overloaded process ID to a kill command of the host to perform a cleanup action.

[0081] In some embodiments of the present disclosure, step 25 may include: filtering query statements, filtering processes corresponding to specific abnormal events, and first cleaning up local processes on a small scale; if the database overload situation does not improve, taking further measures to expand the scope of process cleaning.

[0082] Step 26, record the overload clearing log, and notify the relevant maintenance personnel of the overload clearing log via SMS or email.

[0083] In some embodiments of the present disclosure, different methods are used for overload monitoring, analysis and overload clearing of abnormal connections and abnormal events of ORACLE and MYSQL.

[0084] The following describes in detail the overload monitoring and analysis and overload clearing method for abnormal connections and abnormal events of ORACLE and MYSQL through specific embodiments.

[0085] Figure 3 Schematic diagram of some other embodiments of the database overload protection processing method disclosed in the present invention. Preferably, this embodiment can be executed by the database overload protection processing device disclosed in the present invention. For MYSQL database, the overload monitoring analysis and overload clearing method disclosed in the present invention (for example Figure 2 The database overload protection processing method of step 22 to step 25 of the embodiment may include the following steps:

[0086] Step 31, when the MYSQL abnormal connection log shows a first number of too many connections errors or a second number of unavailable connections errors, locate the slow structured query language information, and then determine the overloaded process number.

[0087] In some embodiments of the present disclosure, step 31 may include: for abnormal connections of MYSQL, the reasons causing the connection feedback to be 2013 error (failed to connect) or 1040 error (too many connections) are different. The adopted overload control rule is to call the overload cleanup script to perform cleanup work when a 1040 error occurs once or 10 2013 errors occur 10 times.

[0088] Step 32, filter the query statements, filter the processes corresponding to specific abnormal events, and clean up the local processes on a small scale in the first step.

[0089] In some embodiments of the present disclosure, step 32 may include: firstly, taking out the processes with info like 'select %', command != 'sleep' and time>= 30 seconds through processlist, and calling the cleanup script to clean up the overloaded processes.

[0090] Step 33: If the database overload situation does not improve, take further measures to expand the scope of process cleanup.

[0091] In some embodiments of the present disclosure, step 33 may include: if the process is still overloaded, taking further overload cleanup measures, taking out the first 300 processes in processlist that meet the conditions of time>=30 seconds and command='sleep', and then calling a cleanup script to clean up the overloaded processes.

[0092] Figure 4 Schematic diagram of some other embodiments of the database overload protection processing method disclosed in the present invention. Preferably, this embodiment can be executed by the database overload protection processing device disclosed in the present invention. For ORACLE database, the overload monitoring analysis and overload clearing method disclosed in the present invention (for example Figure 2 The database overload protection processing method of step 22 to step 25 of the embodiment may include the following steps:

[0093] Step 41, classifying the abnormal events causing overload of the ORACLE database.

[0094] In some embodiments of the present disclosure, step 41 may include: for ORACLE abnormal events, mainly including latch class, enq lock, memory lock, RAC hot block and other related abnormal events, by associating v$session with v$sqlarea, taking out the ORACLE abnormal waiting events that meet the configuration file, each waiting event has a corresponding abnormal number threshold, and then taking out the query statement select type SQL, and associating v$process to obtain the process number.

[0095] Step 42, determining whether the resource utilization rate reaches a specified threshold, wherein the resource utilization rate includes the CPU, memory, and swap area resource utilization rates.

[0096] Step 43: When the resource usage reaches a specified threshold, determine the overload process number.

[0097] In some embodiments of the present disclosure, step 43 may include: associating Figure 2 In the embodiment, the host CPU, memory, and swap area resource usage collected in step 22, if the resource usage reaches the threshold specified by the configuration file, the cleanup script is called to perform the cleanup of the overloaded process. Step 43 cleans up the specific process corresponding to the specific abnormal event.

[0098] Based on the database overload protection processing method provided by the above embodiment of the present disclosure, a database overload monitoring method is provided.

[0099] The database overload monitoring method of the above-mentioned embodiment of the present invention sets a configuration file, configures various objects and related indicators of data overload monitoring, and determines whether to collect the indicator information and whether to perform the overload control action corresponding to the indicator by setting the value of the monitoring indicator.

[0100] The database overload monitoring method of the above embodiment of the present disclosure, for an ORACLE database, calculates slow SQL in real time and locates the process that consumes resources by associating v$active_session_history with v$sql.

[0101] The database overload monitoring method of the above embodiment of the present disclosure adopts different monitoring methods of overload abnormal connection and abnormal events for different databases. The MYSQL database detects abnormal connection by analyzing the mechanism of abnormal connection generation, monitoring the log error number 2013 and the error number 1040. The abnormal events that cause overload in the ORACLE database are classified into latch, enq lock, memory lock, RAC hot block, etc., and the utilization rate of CPU, memory, and swap area resources is comprehensively analyzed to obtain the overload process.

[0102] Based on the database overload protection processing method provided by the above embodiment of the present disclosure, a database overload protection control method is provided.

[0103] The database overload protection control method of the above embodiment of the present disclosure flexibly configures the control rules of overload protection through a configuration file. Different control events correspond to different trigger thresholds, and the overload protection intervention of a certain event can be shielded through the configuration file.

[0104] The database overload protection control method of the above embodiment of the present disclosure is implemented in stages in a progressive manner to minimize the impact of overload control cleaning. The overload control rule used by the MYSQL abnormal connection is to call the overload cleaning script to perform cleaning work when a 1040 error or 10 2013 errors occur. First, clean up the processes corresponding to select and slow SQL. If there is no effect, take further overload cleaning measures to clean up the processes corresponding to the first 300 statements whose execution time exceeds 30 seconds.

[0105] The above embodiment of the present disclosure adopts an overload protection solution on the database side, which is transparent and non-invasive to the application. The above embodiment of the present disclosure proposes a database overload monitoring method and a database overload protection control method and device, which can improve the timeliness of database overload monitoring and improve the effectiveness of database overload control.

[0106] The resource indicators and monitoring objects of the database overload monitoring in the above-mentioned embodiment of the present disclosure are configurable through configuration files and are compatible with different mainstream database types.

[0107] The above-mentioned embodiment of the present disclosure can flexibly configure database overload control rules in the configuration file, and different control rules are configured for each monitoring indicator. The overload protection control rules fully consider data security and minimize the impact of overload cleaning on database availability. When locating the overload process, first filter the query statements and filter the processes corresponding to specific abnormal events. The first step is to clean up local processes on a small scale. If the database overload situation does not improve, take further measures to expand the cleaning scope.

[0108] The above-mentioned embodiments of the present disclosure can be widely used in database operation and maintenance of IT systems, high availability design of IT system databases, etc.

[0109] Figure 5 Schematic diagram of some embodiments of the database overload protection processing device disclosed in the present invention. The database overload protection processing device disclosed in the present invention may include an overload monitoring module 51 and an overload control and protection module 52, wherein:

[0110] The overload monitoring module 51 is used to collect target database information in real time according to the overload monitoring index in the configuration file; and determine whether the target database information triggers an overload threshold according to the target database information.

[0111] The overload control and protection module 52 is used to perform overload control on the target database according to the overload control rules in the configuration file when the target database information triggers the overload threshold.

[0112] In some embodiments of the present disclosure, the database overload protection processing device of the present disclosure may also be used to pre-set a configuration file, wherein the configuration file includes overload monitoring indicators and overload control rules for different database types.

[0113] In some embodiments of the present disclosure, the overload monitoring module 51 can be used to collect slow structured query language information in the target database that consumes resources greater than a predetermined threshold, and determine the overloaded process number based on the slow structured query language information; thereafter, the overload control and protection module 52 can be used to perform a cleanup action on the overloaded process with the overloaded process number.

[0114] In some embodiments of the present disclosure, in some embodiments of the present disclosure, the database overload protection processing device of the present disclosure can also be used to record an overload clearing log and notify the overload clearing log to relevant maintenance personnel.

[0115] In some embodiments of the present disclosure, for a MYSQL database, the overload monitoring module 51 may be used to locate slow structured query language information using a MYSQL abnormal connection log.

[0116] In some embodiments of the present disclosure, for a MYSQL database, the overload control rules may include: when a first number of too many connection errors or a second number of unavailable connection errors appear in the MYSQL abnormal connection log, locating slow structured query language information, and then determining the overload process number to perform overload clearing.

[0117] In some embodiments of the present disclosure, the overload control and protection module 52 can be used to filter query statements and processes corresponding to specific abnormal events. The first step is to clean up local processes on a small scale. If the database overload situation does not improve, further measures are taken to expand the scope of process cleaning.

[0118] In some embodiments of the present disclosure, a database overload protection processing device can be used for an ORACLE database to classify abnormal events that cause overload in the ORACLE database; determine whether the resource utilization rate reaches a specified threshold, and the resource utilization rate includes the CPU, memory, and swap area resource utilization rate; when the resource utilization rate reaches the specified threshold, determine the overload process number.

[0119] In some embodiments of the present disclosure, the database overload protection process is used to implement any of the above embodiments (for example Figure 1-Figure 4 The operation of the database overload protection processing method described in any embodiment).

[0120] Based on the database overload protection processing device provided by the above embodiment of the present disclosure, an overload protection agent is added to the database side. The agent includes two parts, one is an overload monitoring module, and the other is an overload control and protection module. In order to be compatible with different types of databases, the above embodiment of the present disclosure sets a configuration file, and the configuration file includes overload monitoring indicators and overload control strategies for different mainstream database types. The overload monitoring module collects information of the target database in real time according to the monitoring indicators in the configuration file, and uses the overload model to analyze whether it is overloaded. If the overload threshold is triggered, the scheduler performs overload control according to the overload control rules in the configuration file. The overload control rules determine the priority of the overload protection intervention steps according to the degree of impact that the overload intervention may cause to the information system. The steps with a smaller impact are done first. If there is no effect, the steps with a larger impact are gradually implemented. The purpose is to minimize the impact of the overload protection intervention on the database and improve the overall availability of the system.

[0121] Figure 6 Schematic diagrams of other embodiments of the database overload protection processing device disclosed in the present invention. The database overload protection processing device disclosed in the present invention may include a memory 61 and a processor 62, wherein:

[0122] The memory 61 is used for storing instructions.

[0123] The processor 62 is used to execute the instructions so that the device performs any of the above embodiments (for example Figure 1-Figure 4 The operation of the database overload protection processing method described in any embodiment).

[0124] The above-mentioned embodiments of the present disclosure can effectively monitor and timely control the local resource overload on the database side, so as not to expand to the global database system, thereby avoiding the overload of the entire database and ensuring the overall availability of the database system. It mainly includes the following two points:

[0125] The above-mentioned embodiment of the present disclosure proposes a monitoring device for database overload, which can improve the timeliness and effectiveness of database overload perception monitoring.

[0126] The above-mentioned embodiment of the present disclosure proposes a control device for database overload protection, which can improve the timeliness of database overload protection control.

[0127] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, any of the above embodiments (for example Figure 1-Figure 4 The database overload protection processing method described in any embodiment).

[0128] Based on the computer-readable storage medium provided in the above embodiments of the present disclosure, an overload protection solution on the database side is adopted, which is transparent and non-intrusive to the application program, can improve the timeliness of database overload monitoring, and can improve the effectiveness of database overload control.

[0129] The database overload protection processing device described above can be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component or any appropriate combination thereof for performing the functions described in this application.

[0130] So far, the present disclosure has been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.

[0131] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0132] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A database overload protection processing method, characterized in that: include: According to the overload monitoring indicators in the configuration file, the target database information is collected in real time; According to the target database information, determining whether the target database information triggers an overload threshold; When the target database information triggers the overload threshold, the target database is overloaded according to the overload control rules in the configuration file. For the MYSQL database, the overload control rules include: when the MYSQL abnormal connection log shows a first number of too many connections errors or a second number of unavailable connections errors, the slow structured query language information is located, and then the overload process number is determined to perform overload cleaning; The overload control of the target database according to the overload control rules in the configuration file includes: determining the priority of multiple overload protection intervention steps according to the degree of impact that the overload intervention may cause to the information system, and first executing the overload protection intervention step with a small impact; if the execution of the overload protection intervention step with a small impact has no effect, gradually executing the overload protection intervention step with a large impact; Wherein, when the target database information triggers the overload threshold, overload control of the target database according to the overload control rule in the configuration file includes: Collecting slow structured query language information of a target database whose resource consumption is greater than a predetermined threshold, and determining an overloaded process number according to the slow structured query language information; Performing a cleanup action on the overloaded process with the overloaded process number; For an ORACLE database, the collecting of slow structured query language information of a target database in which the amount of resource consumption is greater than a predetermined threshold, and determining the overload process number according to the slow structured query language information include: Categorize abnormal events that cause overload in ORACLE database; Determine whether resource utilization has reached a specified threshold, wherein the resource utilization includes CPU, memory, and swap area resource utilization; When the resource usage reaches a specified threshold, the overload process number is determined; Comprehensively analyze abnormal event classification and resource utilization to determine overloaded processes.

2. The database overload protection processing method according to claim 1, characterized in that: Also includes: A configuration file is pre-set, wherein the configuration file includes overload monitoring indicators and overload control rules for different database types.

3. The database overload protection processing method according to claim 1 or 2, characterized in that: In the case where the target database information triggers the overload threshold, performing overload control on the target database according to the overload control rule in the configuration file further includes: Record the overload cleanup log and notify the relevant maintenance personnel of the overload cleanup log.

4. The database overload protection processing method according to claim 1 or 2, characterized in that: For a MYSQL database, the slow structured query language information in the target database that consumes resources greater than a predetermined threshold includes: Use MYSQL abnormal connection log to locate slow structured query language information.

5. The database overload protection processing method according to claim 1 or 2, characterized in that: The performing of the cleanup action on the overloaded process with the overloaded process number includes: Filter query statements to filter processes corresponding to specific abnormal events. The first step is to clean up local processes on a small scale. If the database overload situation does not improve, take further steps to expand the scope of the process cleanup.

6. A database overload protection processing device, characterized in that: include: An overload monitoring module is used to collect target database information in real time according to the overload monitoring indicators in the configuration file; According to the target database information, determining whether the target database information triggers an overload threshold; An overload control and protection module is used to perform overload control on the target database according to the overload control rules in the configuration file when the target database information triggers the overload threshold; The database overload protection process is used to execute operations to implement the database overload protection process method according to any one of claims 1 to 5.

7. A database overload protection processing device, characterized in that: include: A memory for storing instructions; The processor is used to execute the instruction so that the device performs the operation of implementing the database overload protection processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the database overload protection processing method according to any one of claims 1 to 5 is implemented.

Images