Database intelligent monitoring method and system based on historical experience and storage medium
An intelligent monitoring and database technology, applied in the field of big data processing, can solve problems such as low efficiency of database alarm processing, and achieve the effect of improving positioning and processing efficiency, improving processing efficiency, and reducing the impact of human factors
Pending Publication Date: 2021-04-20
CHINA PING AN LIFE INSURANCE CO LTD
0 Cites 0 Cited by
AI-Extracted Technical Summary
Problems solved by technology
[0008] The invention provides a database intelligent monitoring method, system and computer-readable storage medi...
Method used
In a specific embodiment, also have a kind of dynamic reading value database monitoring based on historical experience, utilize python time series model and Prophet time series prediction storehouse to set dynamic monitoring threshold according to time trend change, thereby set up intelligent data monitoring. That is to say, in the step of judging whether the information data conforms to the preset alarm rule, if the preset alarm rule is a specific value, intelligent data monitoring can be realized.
In general, the database intelligent monitoring method and device based on historical experience of the present invention filter the alarm by setting the shielding time period, provide historical processing data for the historical alarm problem by setting the historical record table, and set the preprocessing The scheme table automatically handles common alarm problems; the combination of the three improves database positioning and processing speed; in addition, through the timely update of the history record table and preprocessing scheme table, as the usage time of the database intelligent monitoring system increases, The continuous enrichment of historical experience and the continuous increase of pre-set processing schemes make it more efficient, thus achieving the effect of improving the intelligence of database monitoring; by adopting fixed processing methods to deal with common alarm problems, the efficiency of alarm problem processing is greatly improved , and reduce the influence of human factors. When an alarm problem occurs, by extracting the processing data in the history record table, the historical processing data is sent together with the alarm information and displayed visually, or directly processed through the preprocessing scheme in the preprocessing scheme table, thereby reducing the processing time. Requirements for human experience in alarm handling.
In summary, the database intelligent monitoring method based on historical experience of the present invention filters the alarm by setting the shielding time period, provides historic...
Abstract
The invention relates to a big data processing technology, and discloses an intelligent database monitoring method based on historical experience, and the method comprises the steps: receiving a database task; receiving a database task; querying an SQL (Structured Query Language) corresponding to the database task in a preset SQL query database, querying information data corresponding to the SQL in the database according to the SQL, and processing the information data conforming to a preset alarm rule; giving an alarm and generating alarm information; traversing a preset historical record table, and determining historical processing data matched with the information data in the historical record table; and sending the historical processing data and the alarm information together. The invention further relates to a blockchain technology, data is stored in the blockchain, and when shielding is set, the alarm positioning speed and the alarm processing speed of the database are improved by setting the historical record table and the preprocessing scheme table.
Application Domain
Hardware monitoring
Technology Topic
BlockchainTable (database) +9
Image
Examples
- Experimental program(1)
Example Embodiment
[0043]It should be understood that the specific embodiments described herein are merely intended to illustrate the invention and are not intended to limit the invention.
[0044]The present invention provides a database intelligent monitoring method based on historical experience. Based on the exchange hidden dangers in the bus system, the historical record table and the pretreatment scheme are set in advance, and the alarm status is to be queried. Or is it a condition for filing in the pre-processing schedule, and then processed according to historical processing record or processing in accordance with the presence scheme in the pre-processing scheme, improve the alarm processing efficiency.
[0045]The present invention provides a database intelligent monitoring method based on historical experience.figure 1 A process of better embodiments preferred embodiments of a database intelligent monitoring method based on historical experience is shown. Referfigure 1 As shown, the method includes step S110-Step S140:
[0046]S110, receive database tasks.
[0047]The monitoring of the database is waiting for the event monitoring, and the event monitoring template will be developed according to the actual needs.
[0048]Before the receiving database task, it is also included including the pretreatment step of setting a shielding time period according to actual needs.
[0049]After receiving the database task, the task filtering step includes: determining whether the database task is in a pre-set shield time period, and if, the database task is started.
[0050]In the specific implementation process, it is determined whether or not the current database monitor timing task is in a preset shield time period. If so, the monitor timing task is ended; if, then the monitor timing task is started; specifically, By comparing the currently monitored shielding time field and the current time, if the current time is within the set shielded time field, no monitoring operation is performed.
[0051]In a specific embodiment, the monitoring items for timing tasks include: monitoring titles, database types, databases, monitoring item coding, associated system TAG, monitoring instructions, and alarm time. Taking the Oracle Alarm as an example, monitor the title is [Major] [Oracle] [LUCS0] Top SQL monitoring; CPU consumption>10; the alarm time is September 30, 2020 09:58:55.
[0052]It should be noted that the scenes that need to be set to provide a shielding time period, shield some alarms caused by certain changes, an irresistible alarm or a warning of the alarm to avoid persistent alarms.
[0053]S120, query the SQL corresponding to the database task in the preset SQL query database, according to the information data corresponding to the SQL in the database, and the information data that meets the predetermined alarm rule A alarm, generate alarm information.
[0054]Specifically, according to the database SQL corresponding to the timing task query, the corresponding database SQL is queried, and according to the queryed SQL, the information data corresponding to the database is queried, and the information data queried is in line with the preset alarm rule. If it is, then the current database monitoring timing task; if it is, the alarm is triggered.
[0055]Among them, pre-set alarm rules is to query SQL based on monitoring settings, queries content, will be considered alarm, and the query is not subject to the alarm.
[0056]S130, traverse the pre-set history table, determine historical processing data that matches the information data in the history table; if, end traversal history tables, perform normal alarms.
[0057]Among them, the records and monitoring templates of history tables are binding, that is, the monitoring template corresponds to the history information. The main fields of the historical record include: monitor template ID, match field, matching value, and record connection.
[0058]The history information is as follows: Match field: event; matching value: GC Buffer busy acquire; Record connection: You can have a document link for you to summarize the summary of similar issues, or you can be an article link of online related issues. In the process of waiting for the event monitoring, if an ID in the monitoring template is monitored, the ID in the monitor template corresponds to the history information; the corresponding record link in the history information will be monitored when monitoring alarms.
[0059]Specific embodiments include: traversing history tables, determining if the query is matched with a history table; if there is, the corresponding historical processing record in the history table is told, and will The history processing record is sent with the alarm information; if there is no existence, the alarm information is directly sent.
[0060]It should be noted that the matching rule for matching, for example, a historical experience of monitoring is {"Field 1": "Value 1", "Value 2"}, then see this alarm Whether the value of the corresponding field meets the rules, the quantity of the fields in the match rules is unfained, and the match is included in the alarm, which is convenient for handlers.
[0061]It should be noted that in the specific execution process, it is also included in time to store various alarms to be stored in time to increase the reserve of the history record. Among them, in time, timely storage of various alarms includes two cases, one, which is a more typical alarm history according to actual needs; second, the supplemental pretreatment scheme is entered into manually.
[0062]In a specific embodiment, after step S120, the pretreatment scheme in which the pre-arranged pretreatment scheme is traveled, and the pretreatment scheme data that matches the information data of the trigger alarm in the pre-processing scheme is determined; The pretreatment scheme data will process the database task to run the pretreatment scheme data, and send the alarm processing to the process. If, end traverse the pre-processing schedule is ended, and the normal alarm program is executed.
[0063]Specifically, the parallel traversation of the pre-processing scheme is used to traverse the history record, and it is judged whether or not the queried data matches the pretreatment scheme; where parallel traversal is implemented by asynchronous task. It includes, including the actual needs to set a historical record table and a pretreatment scheme step with the pretreatment scheme.
[0064]It should be noted that the preprocessing scheme is implemented through the interface. The specific scheme information in the pretreatment schedule is bound to a single ID of the monitoring template. The preprocessing scheme includes three parts: entry rule field, entry calling method, and entry into the pass parameters. Where the rule field is a matching alarm content, the JSON format; the pass parameter field includes the parameter field and method field; the parameter field is the transfer parameter data corresponding to the call method; the method field is the name of the specific method of the call. The calling method is a pre-set specific processing method, a call method corresponds to a method function to provide a call to call; such as Kill SQL, analyze SQL, and reporting an event to DBA. Further, the rule field and the passage field are embodied in the form of a table, and a alarm event corresponds to a rule field, and matches its method fields and parameter fields according to the rule field, and calls the method function corresponding to the method field.
[0065]A specific method of calling the pretreatment scheme process includes acquiring a rule field that matches the information data of the trigger alert; match the parametric field corresponding to the rule field according to the rule field; call the passage The method function corresponding to the field handles the database task.
[0066]Taking the monitor (TOPSQL) alarm as an example: assuming SQL 9BU62WYWG41TU monitoring alarm has a pre-processing scheme record; its rule field: {"SQL_ID": "9bu62wywg41tu"}; parameter field: "sql_id"; method field "kill_sql". During the specific implementation process, if the SQL9BU62WYWG41TU alarm is found, the parameter field and method fields are associated by the rule field, and then the method is called to process the alarm problem. Specifically, the specific alarm rule matches the SQL_ID field in the alarm content to the 9bu62wywg41tu; the parameter field is called the parameter of the SQL_ID field; the method field is called a pre-method, KILL fell to the SQL. In summary, if you find that SQL 9BU62wYWG41TU has alarm problem, as long as the TOP SQL alarm appears in the above SQL, automatically follow the call method in the pre-processing scheme record.
[0067]S140, transmitting the history processing data and the alarm information.
[0068]Specifically, the specific implementation of the alarm is an email alarm, in a specific application scenario, can also use a telephone alert.
[0069]When the traverstation of the history table is set, history processing data is generally attached to the alarm email to handle people's reference, accelerate the solution, and make historical experience to emit current problems.
[0070]Taking Oracle Alarms as an example, the alarm details include the following: SNAP_ID, database node, time range, SQL_ID, execution user, average CPU consumption, SQL average execution time (s), execution number, plan_hash_value, buffer_gets_delta, rows_processed_delta, sql_text .
[0071]In a specific embodiment, there is also a dynamic value-visual database monitoring based on historical experience, using the Python Time Series Model and the Prophet Time Series Prediction Library to set dynamic monitoring thresholds based on time trend changes, thereby establishing intelligent data monitoring. That is, in the step of determining whether the information data conforms to a preset alarm rule, the preset alarm rule is a specific value, intelligent data monitoring can be achieved.
[0072]Users can create intelligent monitoring, such monitoring can be used to monitor single data, such as underwriting failure, monitoring, monitoring, monitoring according to historical data trends, such as 15 o'clock, unfailed single quantity E 1000 single, according to historical trends , 15 points failure single prediction upper limit is 900, then the alarm, 19 points Monitor the failure single quantity is 1000 sheets, but according to historical trends, 19 points failure single prediction upper limit is 1100 orders, so no alarm .
[0073]It should be noted that the monitoring dynamic threshold can be configured, can select the predicted upper limit or predicted lower limit, or simultaneously select the predicted upper limit (not in the upper and lower limit ranges); intelligent monitoring can also add additional rules, additional rules Levels than predicting, such as 19-point underwriting failure is 1000 single, the upper limit of the forecast is 1100 orders, it is not alarm, but the additional rules are configured, more than 1000 alarms are also alarm.
[0074]In summary, the present invention based on historical database intelligent monitoring method is filtered by setting the shielding time period. By setting a history alarm problem with historical alarm issues, the common alarm problem is performed by setting the pre-processing plan. Automatic processing; three combinations, improve database positioning and processing speed; in addition, through the timely update of historical record tables and pre-processing schemes, the history experience is constantly rich with the growth of the database intelligent monitoring system, and The preset processing scheme is increasing, and more efficient, thereby achieving the effect of enhancing the monitoring of intelligence.
[0075]figure 2 A schematic diagram of a logical structure of a database intelligent monitoring system based on historical experience of the present invention;figure 2 Distance
[0076]To achieve the above object, the present invention provides a database intelligent monitoring system 200 based on historical experience, the task receiving unit 210, a trigger alarm unit 230, a history processing data matching unit 240, and alarm display unit 260;
[0077]The task receiving unit 210 is configured to receive a database task;
[0078]The trigger alarm unit 230 is configured to query the SQL corresponding to the database task in the preset SQL query database, inquiry the information data corresponding to the SQL in accordance with the SQL query database, and in accordance with pre-set The information data of the alarm rules; the alarm is performed, and the alarm information is generated;
[0079]The history processing data matching unit 240 is used to traverse the pre-set history table, determine a history processing data that matches the information data in the history table.
[0080]The alarm display unit 260 is configured to transmit the history processing data from the alarm information.
[0081]Further, preferably, the pretreatment scheme matching unit 250 is further included for traversing the pre-arranged pretreatment scheme, and determines the pre-processing scheme data that matches the information data of the trigger alert in the pre-processing scheme; Pre-processing scheme data handles the database task.
[0082]In a specific embodiment, the task screening unit 220 is also included for determining whether the database task received by the task receiving unit is in a pre-set shield time period, and if, the database task is started.
[0083]In a specific embodiment, there is also a dynamic value-visual database monitoring based on historical experience, using the Python Time Series Model and the Prophet Time Series Prediction Library to set dynamic monitoring thresholds based on time trend changes, thereby establishing intelligent data monitoring. That is, in the step of determining whether the information data conforms to a preset alarm rule, the preset alarm rule is a specific value, intelligent data monitoring can be achieved.
[0084]Users can create intelligent monitoring, such monitoring can be used to monitor single data, such as underwriting failure, monitoring, monitoring, monitoring according to historical data trends, such as 15 o'clock, unfailed single quantity E 1000 single, according to historical trends , 15 points failure single prediction upper limit is 900, then the alarm, 19 points Monitor the failure single quantity is 1000 sheets, but according to historical trends, 19 points failure single prediction upper limit is 1100 orders, so no alarm .
[0085]It should be noted that the monitoring dynamic threshold can be configured, can select the predicted upper limit or predicted lower limit, or simultaneously select the predicted upper limit (not in the upper and lower limit ranges); intelligent monitoring can also add additional rules, additional rules Levels than predicting, such as 19-point underwriting failure is 1000 single, the upper limit of the forecast is 1100 orders, it is not alarm, but the additional rules are configured, more than 1000 alarms are also alarm.
[0086]In summary, the present invention based on historical experienced database intelligent monitoring systems, by setting the hood, boost the database alarm positioning speed and the alarm processing speed; by using a fixed processing method Common alarm issues have greatly improved the processing efficiency of the alarm, and reduced the impact of human factors. The alarm problem is generated, by extracting processing data in the history table, sending history processing data, as a alarm message, or performing intuitive display, or directly processing through the preprocessing scheme, thereby reducing processing Human alarm processing experience requirements.
[0087]The present invention provides a database intelligent monitoring method based on historical experience, and is applied to an electronic device 3.
[0088]image 3A application environment of a preferred embodiment of a database intelligent monitoring method based on historical experience in accordance with the present invention is shown.
[0089]Referimage 3As shown in the present embodiment, the electronic device 3 can be a server, smartphone, a tablet, a portable computer, a desktop computer, and the like a terminal device with an operational function.
[0090]The electronic device 3 includes a processor 32, a memory 31, a communication bus 33, and a network interface 35.
[0091]Memory 31 includes at least one type of readable storage medium. The at least one type of readable storage medium can be a nonvolatile storage medium such as flash memory, hard disk, multimedia card, card type memory 31, and the like. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device 3, such as a hard disk of the electronic device 3. In other embodiments, the readable storage medium may also be an outer memory 31 of the electronic device 3, such as a sleeve hard disk equipped on the electronic device 3, a smart media card, SMC. Secure Digital, SD) card, flash card (Flash Card), etc.
[0092]In the present embodiment, the readable storage medium of the memory 31 is typically used to store historical experience-related database intelligent monitor 30, which is mounted to the electronic device 3. The memory 31 can also be used to temporarily store data that has been output or will output.
[0093]Processor 32 may be a central processor (CPU), a microprocessor, or other data processing chip, a program code or processing data stored in the memory 31, such as performing historical experience. Database Intelligent Monitor 30, etc.
[0094]Communication bus 33 is used to implement connection communication between these components.
[0095]Network interface 34 can optionally include a standard wired interface, a wireless interface, such as a Wi-Fi interface, is typically used to establish a communication connection between the electronic device 3 and other electronic devices.
[0096]image 3Only electronic devices 3 having components 31-34, but it should be understood that all displayed components are not required to replace more or fewer components.
[0097]Alternatively, the electronic device 3 may further include a user interface that may include an input unit such as a keyboard, a voice input device, such as a microphone, and the like, a voice-output device such as an audio, a headset, etc. The optional user interface can also include a standard wired interface, a wireless interface.
[0098]Alternatively, the electronic device 3 may further include a display, and a display may also be referred to as a display screen or a display unit. In some embodiments, it can be an LED display, a liquid crystal display, a touch liquid crystal display, and an organic light-emitting diode, an OLED. The display is used to display information processed in the electronic device 3 and to display visualized user interfaces.
[0099]Alternatively, the electronic device 3 can also include a radio frequency, an RF circuit, a sensor, an audio circuit, and the like, and will not be described herein again.
[0100]inimage 3In the device embodiment shown, the memory 31 as a computer storage medium may include an operating system, as well as a historical database intelligent monitor 30; processor 32 performs historical experience-based database intelligent monitoring stored in memory 31. When the program 30 is implemented: Receive the database task; query the SQL corresponding to the database task in the preset SQL query database, inquiry the information data corresponding to the SQL in accordance with the SQL inquiry database, and in line with pre-set The information data of the alarm rule; alarm, generate alarm information; traverse the previously set history table, determine historical processing data that matches the information data in the history table; handling the history processing data Send with the alarm information.
[0101]In other embodiments, the database intelligent monitoring program 30 based on historical experience can also be split into one or more modules, one or more modules are stored in memory 31, and is performed by processor 32 to complete the present invention. The module referred to in the present invention refers to a series of computer program blocks capable of completing a particular function. The database intelligent monitoring program 30 based on historical experience can be divided into task receiving unit 210, trigger alarm unit 230, history processing data matching unit 240, and alarm display unit 260.
[0102]Further, the present invention also provides a computer readable storage medium, which mainly includes a storage data area and a storage program area, wherein the storage data area can store data created according to the use of the block link node, the storage program area can be stored. System, at least one function required, the computer readable storage medium includes historical experience-based database intelligent monitoring programs, the history-based database intelligent monitor is implemented as historical experience when executed by the processor The operation of the database intelligent monitoring method.
[0103]The specific embodiment of the computer readable storage medium of the present invention is substantially identical to the above-described historical database intelligent monitoring method, system, and electronic device, and will not be described herein.
[0104]In general, the present invention is based on historical experience of the database intelligent monitoring method and apparatus, by setting the shield time period, the alarm is filtered, and the historical alarm is provided for historical alarm, and the historical alarm is provided by setting the pre-processing scheme. Common alarm issues automatic processing; three combinations, enhance database positioning and processing speed; in addition, through historical record tables and pre-processing schedules, the timely update of the pre-processing scheme will increase with the use time of the database intelligent monitoring system, historical experience Continuously rich, and more efficient and more efficient, thereby achieving the effect of improving database monitoring intelligence; by using a fixed processing method to handle common alarm problems, it greatly improves the processing efficiency of the alarm problem, and decreases The impact of human factors. The alarm problem is generated, by extracting processing data in the history table, sending history processing data, as a alarm message, or performing intuitive display, or directly processing through the preprocessing scheme, thereby reducing processing Human alarm processing experience requirements.
[0105]The zipper chain referred to in the present invention is a new application model for computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm. Blockchain, essentially a detrimental database, is a string of data blocks generated by the cryptographic method, containing a batch of network transactions in each data block for verifying it The validity of information (anti-counterfeiting) and generate the next block. The block chain can include block chain bottom platform, platform product service layer, and application service layer.
[0106]It should be noted that the term "comprising", "comprising" or any other variable is intended to cover non-exclusive contained, so that the process, apparatus, article, or method including a series of elements, not only those elements, but Also included in other elements that are not explicitly listed, or include elements inherent to such processes, devices, items, or methods. In the absence of more limitations, the elements defined by the statement "include a ...", and there is no other same element in the process, apparatus, article, or method including the element.
[0107]The above-described embodiments of the present invention described only for describing, and does not represent the advantages and disadvantages of the embodiment. Through the description of the above embodiments, those skilled in the art will clearly understand that the above-described embodiment can be implemented by means of software plus necessary general hardware platforms. Of course, you can pass hardware, but in many cases, the former is better. Embodiment. Based on this understanding, the technical solution of the present invention essentially or contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium as described above (such as ROM / RAM). , A disk, an optical disk, including a number of procedures to enable a terminal device (can be a mobile phone, computer, server, server, or network device, etc.) to perform various embodiments of the present invention.
[0108]The above is only the preferred embodiments of the present invention, and is not limited to the patent scope of the present invention, and an equivalent structure or equivalent process transform is used to use the specification of the present invention and the accompanying drawings, or use directly or indirectly in other related technical fields. The same is included within the patent protection range of the present invention.
PUM
Description & Claims & Application Information
We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.
Similar technology patents
Drilling device for wheel insert
Owner:CITIC DICASTAL
Neural network-based grinding machining working condition detection method
Owner:NANJING UNIV +1
Shaft gear composite processing machine tool
Owner:CHONGQING MACHINE TOOL GROUP
Device and method for magnetically grinding inner surfaces of large-diameter long and straight pipes
Owner:UNIV OF SCI & TECH LIAONING
Classification and recommendation of technical efficacy words
- Improve processing efficiency
- Reduce the influence of human factors
Business risk assessment method and device, and risk control system
Owner:BEIJING QIYI CENTURY SCI & TECH CO LTD
Method for automatically acquiring new words from Chinese webpages
Owner:TSINGHUA UNIV
Full-automatic cloth paving and cutting integrated machine
Owner:CHONGQING UNIV OF TECH
Method for Managing Local IP Access Connection
Owner:ZTE CORP
Method and device for processing concurrent data and electronic accounting system
Owner:ADVANCED NEW TECH CO LTD
Coke automatic sample collecting and making method
Owner:TIANJIN TIANTIE METALLURGICAL GRP
Method and device for grading and coloring thematic map
Owner:BEIJING NORMAL UNIVERSITY
Surrounding rock engineering geology fine classification method in hydropower project
Owner:POWERCHINA CHENGDU ENG
Automatic welding equipment
Owner:GUANGDONG ZHENGYE TECH