Method, device, storage medium and equipment for cross-system deployment of log monitoring rules

Abstract

This invention discloses a method, apparatus, storage medium, and device for cross-system deployment of log monitoring rules, relating to the field of log monitoring technology, and solving the technical problem of insufficient cross-system coverage of current log monitoring rules. The method includes: based on the technology stack set of each system and the abnormal log keywords in the system logs of each system, statistically determining the associated systems, associated technology stack sets, and the frequency of each associated system for each abnormal log keyword; calculating the cross-system influence coefficient of each abnormal log keyword based on the associated systems and associated technology stack sets; then, comparing it with the corresponding frequency of each associated system, determining the abnormal log keywords that meet the conditions for generating monitoring rules as monitoring keywords from among multiple abnormal log keywords; automatically generating monitoring rules based on the monitoring keywords and a preset monitoring template; and determining the target systems for batch deployment of monitoring rules among all systems according to the preset deployment rules.

Description

Technical Field

[0001] This invention relates to the field of log monitoring technology, and in particular to a method, apparatus, storage medium and device for cross-system deployment of log monitoring rules. Background Technology

[0002] Application system exception logs are an indispensable resource in monitoring configuration activities, playing an irreplaceable role in problem discovery and fault localization. In the current production environment, monitoring rules are mainly added manually based on experience. However, the monitoring rules of each system are independent of each other, and there is no way to deploy monitoring rules across systems. As a result, for exception log keywords that are common to multiple systems, monitoring rules are only configured on systems where the exception log keywords have already appeared, resulting in blind spots in monitoring coverage. When an exception occurs in a blind spot system, the lack of corresponding monitoring rules prevents the triggering of alarms in real time, affecting business availability. Summary of the Invention

[0003] In view of this, the present invention provides a method, apparatus, storage medium and device for cross-system deployment of log monitoring rules, which can solve the technical problem of insufficient cross-system coverage of current log monitoring rules.

[0004] According to one aspect of the present invention, a method for cross-system deployment of log monitoring rules is provided, the method comprising: Based on the technology stack set of each system and the abnormal log keywords in the system logs of each system, the associated systems, associated technology stack sets, and the frequency of single associated systems appearing in each associated system for each abnormal log keyword are statistically obtained. Based on the associated systems and the associated technology stack sets, the cross-system influence coefficient of each abnormal log keyword is calculated. Here, one system corresponds to one technology stack set, and the technology stack sets corresponding to different systems are the same or different. The types of abnormal log keywords corresponding to one system are greater than or equal to 0. Based on the cross-system impact coefficient of each of the aforementioned abnormal log keywords and its corresponding single-related system frequency, among the various abnormal log keywords, the abnormal log keywords that meet the monitoring rule generation conditions are determined as monitoring keywords. Monitoring rules are automatically generated based on the monitoring keywords and preset monitoring templates. Based on the preset deployment rules, target systems for batch deployment of the monitoring rules are determined in all the systems. The monitoring rules are then deployed in the target systems to monitor the logs of the target systems according to the monitoring rules.

[0005] Preferably, the step of statistically analyzing the associated systems, associated technology stack sets, and the frequency of single associated systems appearing in each associated system based on the technology stack set of each system and the abnormal log keywords in the system logs of each system includes: Obtain the configuration information of each system, extract the technical components corresponding to the preset technical component items from the configuration information, and use the technical components as elements to form the technology stack set of the system; Collect system logs for each system, identify abnormal logs from all system logs using a preset abnormal log rule matching library, extract abnormal log keywords based on the abnormal logs, take the systems in which the abnormal log keywords appear as associated systems, and count the frequency of each abnormal log keyword appearing in each associated system in a single associated system. The set of technology stacks corresponding to the associated systems is taken as the associated technology stack set.

[0006] Preferably, the step of calculating the cross-system impact coefficient of each of the abnormal log keywords based on the associated system and the associated technology stack set includes: The number of systems associated with the aforementioned abnormal log keywords is taken as the number of affected systems; The technology stack concentration is obtained by dividing the maximum number of the same associated technology stack set of the abnormal log keywords by the total number of all associated systems. The cross-system impact coefficient of each abnormal log keyword is obtained by multiplying the number of affected systems for each abnormal log keyword by the concentration of its corresponding technology stack.

[0007] Preferably, the step of determining, from among a variety of abnormal log keywords, the abnormal log keywords that meet the monitoring rule generation conditions as monitoring keywords based on the cross-system impact coefficient of each of the abnormal log keywords and its corresponding single-related system frequency includes: Obtain monitoring rule generation conditions, wherein the monitoring rule generation conditions include a preset influence coefficient threshold and a preset rule generation frequency threshold; For each of the aforementioned abnormal log keywords, if the cross-system impact coefficient is greater than or equal to the preset impact coefficient threshold, and the single-associated system frequency of the abnormal log keyword is greater than or equal to the preset rule generation frequency threshold, then it is determined to be an abnormal log keyword that meets the monitoring condition generation rule, and the abnormal log keyword that meets the monitoring condition generation rule is used as a monitoring keyword.

[0008] Preferably, the step of automatically generating monitoring rules based on the monitoring keywords and preset monitoring templates includes: Obtain a preset monitoring template, wherein the preset monitoring template includes: an alarm title field to be filled in and a preset alarm trigger frequency threshold; The monitoring keyword is filled into the alarm title field to be filled to obtain the alarm title, and the alarm title is the monitoring keyword alarm; Automatically generate monitoring rules that include the alarm title and the preset alarm trigger frequency threshold.

[0009] Preferably, determining the target systems for batch deployment of the monitoring rules in all the systems according to preset deployment rules includes: The preset deployment rules include: if the type of the associated technology stack set corresponding to the monitoring keyword is one, then the associated technology stack set corresponding to the monitoring keyword is taken as the target associated technology stack set; The preset deployment rules also include: if the types of associated technology stack sets corresponding to the monitoring keyword are greater than one, then obtain the number of each type of associated technology stack set corresponding to the monitoring keyword, and take the largest number of associated technology stack sets as the target associated technology stack set; Systems corresponding to the same technology stack set as the target are used as the target systems for batch deployment of the monitoring rules.

[0010] Preferably, the step of monitoring the target system logs according to the monitoring rules includes: Obtain the target system logs; If the monitoring keyword is found in the target system log, it is determined whether the frequency of occurrence of the monitoring keyword is greater than or equal to the preset alarm triggering frequency threshold. If so, an alarm is triggered and an alarm is pushed.

[0011] According to another aspect of the present invention, a cross-system deployment apparatus for log monitoring rules is provided, the apparatus comprising: The statistics module is used to obtain the associated systems, associated technology stacks, and single associated system frequency of each abnormal log keyword based on the technology stack set of each system and the abnormal log keywords in the system logs of each system. Based on the associated systems and the associated technology stacks, the cross-system influence coefficient of each abnormal log keyword is calculated. Here, one system corresponds to one technology stack set, and the technology stack sets corresponding to different systems may be the same or different. The types of abnormal log keywords corresponding to one system are greater than or equal to 0. The determination module is used to determine, from among the various abnormal log keywords, the abnormal log keywords that meet the conditions for generating monitoring rules as monitoring keywords, based on the cross-system impact coefficient of each abnormal log keyword and its corresponding single-related system frequency. The deployment module is used to automatically generate monitoring rules based on the monitoring keywords and preset monitoring templates, determine the target systems for batch deployment of the monitoring rules in all the systems according to the preset deployment rules, and deploy the monitoring rules in the target systems to monitor the logs of the target systems according to the monitoring rules.

[0012] According to another aspect of the present invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the cross-system deployment method for the above-described log monitoring rules.

[0013] According to another aspect of the present invention, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the cross-system deployment method for the above-described log monitoring rules.

[0014] By employing the above technical solutions, this invention provides a method, apparatus, storage medium, and device for cross-system deployment of log monitoring rules. Through this invention, all abnormal log keywords from all systems are aggregated, and the correspondence between abnormal log keywords, associated systems, associated technology stack sets, and the frequency of single associated systems is statistically analyzed. Based on this, the cross-system influence coefficient and single associated system frequency of each abnormal log keyword are calculated. Thus, among multiple abnormal log keywords, those that meet the conditions for generating monitoring rules are determined as monitoring keywords. This solves the technical problem of existing technologies struggling to identify common monitoring keywords from massive amounts of abnormal log keywords. Monitoring rules are automatically generated from monitoring keywords and preset monitoring templates, rather than manually, improving the efficiency of monitoring rule generation. Since systems with the same technology stack set have the same underlying technical architecture and a consistent probability of experiencing the same type of anomaly, the same set of monitoring rules can be reused. This solves the technical problem of existing technologies struggling to determine the target systems for batch deployment of monitoring rules. The entire process is automated, requiring no manual intervention, and efficiently solves the current technical problem of insufficient cross-system coverage of log monitoring rules.

[0015] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of this application. In the drawings: Figure 1The diagram illustrates a flowchart of a cross-system deployment method for log monitoring rules provided by an embodiment of the present invention. Figure 2 This paper illustrates a flowchart of another method for cross-system deployment of log monitoring rules provided by an embodiment of the present invention. Figure 3 This diagram illustrates the structure of a cross-system deployment device for log monitoring rules provided in an embodiment of the present invention. Figure 4 This diagram illustrates the structure of another cross-system deployment device for log monitoring rules provided in an embodiment of the present invention. Detailed Implementation

[0017] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0018] This embodiment provides a method for deploying log monitoring rules across systems, such as... Figure 1 As shown, the method includes: 101. Based on the technology stack set of each system and the abnormal log keywords in the system logs of each system, the associated systems, associated technology stack sets, and the frequency of single associated systems appearing in each associated system are statistically obtained for each abnormal log keyword. Based on the associated systems and the associated technology stack sets, the cross-system influence coefficient of each abnormal log keyword is calculated. Here, one system corresponds to one technology stack set, and the technology stack sets corresponding to different systems are the same or different. The types of abnormal log keywords corresponding to one system are greater than or equal to 0.

[0019] In this embodiment, the purpose is to summarize all abnormal log keywords from all systems and to statistically determine the correspondence between abnormal log keywords, associated systems, associated technology stack sets, and the frequency of single associated systems.

[0020] 102. Based on the cross-system impact coefficient of each of the aforementioned abnormal log keywords and its corresponding single-related system frequency, among the various abnormal log keywords, determine the abnormal log keywords that meet the monitoring rule generation conditions as monitoring keywords.

[0021] In this embodiment, the cross-system impact coefficient and the frequency of a single associated system are used to determine whether it is necessary to trigger the generation of monitoring rules. If the cross-system impact coefficient of a certain abnormal log keyword is high, it means that the abnormal log keyword affects many associated systems, and these associated systems belong to the same set of associated technology stacks. Therefore, it is necessary to configure monitoring rules in batches. If the frequency of a certain abnormal log keyword in a single associated system is high, it means that the severity of the abnormal log keyword in a certain associated system is higher. Therefore, it is necessary to configure monitoring rules.

[0022] 103. Automatically generate monitoring rules based on the monitoring keywords and preset monitoring templates. Based on the preset deployment rules, determine the target systems for batch deployment of the monitoring rules in all the systems, and deploy the monitoring rules in the target systems to monitor the logs of the target systems according to the monitoring rules.

[0023] In this embodiment, since systems with the same technology stack have the same underlying technical architecture and the probability of the same type of anomaly occurring is consistent, the same set of monitoring rules can be reused, thereby enabling batch deployment of monitoring rules.

[0024] This invention provides a method, apparatus, storage medium, and device for cross-system deployment of log monitoring rules. Through this invention's technical solution, all abnormal log keywords from all systems are aggregated, and the correspondence between abnormal log keywords, associated systems, associated technology stack sets, and the frequency of single associated systems is statistically determined. Based on this, the cross-system influence coefficient and single associated system frequency of each abnormal log keyword are calculated. Thus, among multiple abnormal log keywords, those that meet the conditions for generating monitoring rules are identified as monitoring keywords. This solves the technical problem of existing technologies struggling to identify common monitoring keywords from massive amounts of abnormal log keywords. Monitoring rules are automatically generated from monitoring keywords and preset monitoring templates, rather than manually, improving the efficiency of monitoring rule generation. Since systems with the same technology stack set have the same underlying technical architecture and a consistent probability of experiencing the same type of anomaly, the same set of monitoring rules can be reused. This solves the technical problem of existing technologies struggling to determine the target systems for batch deployment of monitoring rules. The entire process is automated, requiring no manual intervention, and efficiently solves the current technical problem of insufficient cross-system coverage of log monitoring rules.

[0025] Furthermore, as a refinement and extension of the specific implementation methods of the above embodiments, and to fully illustrate the specific implementation process in this embodiment, another method for cross-system deployment of log monitoring rules is provided, such as... Figure 2 As shown, the method includes: 201. Based on the technology stack set of each system and the abnormal log keywords in the system logs of each system, the associated systems, associated technology stack sets, and the frequency of single associated systems appearing in each associated system are statistically obtained for each abnormal log keyword. Based on the associated systems and the associated technology stack sets, the cross-system influence coefficient of each abnormal log keyword is calculated. Here, one system corresponds to one technology stack set, and the technology stack sets corresponding to different systems are the same or different. The types of abnormal log keywords corresponding to one system are greater than or equal to 0.

[0026] In this embodiment, the step of calculating the associated systems, associated technology stack sets, and single-associated system frequency for each type of abnormal log keyword based on the technology stack set of each system and the abnormal log keywords in the system logs of each system includes: obtaining the configuration information of each system; extracting the technology components corresponding to preset technology component items from the configuration information; using the technology components as elements to form the technology stack set of the system; collecting the system logs of each system; using a preset abnormal log rule matching library to identify abnormal logs from the system logs of all systems; extracting abnormal log keywords based on the abnormal logs; using the systems in which the abnormal log keywords appear as associated systems; calculating the single-associated system frequency for each type of abnormal log keyword in each associated system; and using the technology stack set corresponding to the associated system as the associated technology stack set.

[0027] In the scenario of a banking business system cluster, the systems include: mobile banking, wealth management platform, core system, and personal online banking. Configuration information for each system is obtained from the configuration management database. Preset technical component items corresponding to these items are extracted from the configuration information. These preset technical component items include: development language, database, middleware, and runtime environment. Table 1 shows the correspondence between systems and technology stack sets. For example, the preset technical component item "development language" extracted from the mobile banking configuration information corresponds to the technical component "Java"; the preset technical component item "database" extracted from the mobile banking configuration information corresponds to the technical component "MySQL"; the preset technical component item "middleware" extracted from the mobile banking configuration information corresponds to the technical component "Redis"; and the preset technical component item "runtime environment" extracted from the mobile banking configuration information corresponds to the technical component "WAS".

[0028] Table 1:

[0029] It should be noted that the technology stacks of different systems may be the same or different. In the example in Table 1, the technology stacks of different systems are different.

[0030] System logs from each system are collected and aggregated in a log center. A pre-defined exception log rule library is constructed based on fuzzy matching and regular expressions. Each pre-defined exception log rule in the library is matched against the system logs of all systems to identify exception logs that match at least one pre-defined exception log rule. Then, exception log keywords are extracted from these logs. Pre-defined exception log rules include, for example, Timeout and ConnectionException.

[0031] For each type of abnormal log keyword, if the abnormal log keyword appears in the system log of a certain system, then that system is the associated system of the abnormal log keyword. After obtaining the associated system of each abnormal log keyword, the frequency of each abnormal log keyword in the system log of a certain associated system is counted, which is used as the single associated system frequency of the abnormal log keyword. For example, the correspondence table of associated systems, abnormal log keywords, and single associated system frequencies shown in Table 2 is obtained.

[0032] Table 2:

[0033] The set of technology stacks corresponding to the associated systems is called the associated technology stack set. Thus, the correspondence between systems and technology stack sets, and the correspondence between associated systems, exception log keywords, and the frequency of single associated systems, can be summarized to obtain the correspondence between exception log keywords, associated systems, the frequency of single associated systems, and associated technology stack sets. For example, the correspondence table shown in Table 3 is obtained.

[0034] Table 3:

[0035] In addition, the preset exception log rule library is updated promptly. If new preset exception log rules exist, they are added to the library. The method for obtaining new preset exception log rules is as follows: ERROR-level logs from all systems are cleaned by removing irrelevant characters (such as timestamps and special symbols), then segmented into words or phrases, combined with technical terms (such as ConnectionRefused), and finally, high-frequency words with a frequency greater than a preset frequency threshold are extracted and compared with existing preset exception log rules. High-frequency words that differ from existing preset exception log rules are designated as new preset exception log rules to be confirmed and added to the confirmation queue. After confirmation, they are synchronized to the preset exception log rule library. Dynamically updating the preset exception log rule library ensures dynamic deployment of monitoring rules and strong adaptability.

[0036] In this embodiment, the step of calculating the cross-system impact coefficient of each abnormal log keyword based on the associated system and the associated technology stack set includes: taking the number of associated systems of the abnormal log keyword as the number of affected systems; calculating the maximum number of the same associated technology stack set of the abnormal log keyword divided by the total number of all associated systems to obtain the technology stack concentration; and calculating the product of the number of affected systems and the corresponding technology stack concentration of each abnormal log keyword to obtain the cross-system impact coefficient of each abnormal log keyword.

[0037] For each type of abnormal log keyword, the cross-system impact coefficient is the product of the number of affected systems and the technology stack concentration. The number of affected systems refers to the number of related systems, which is used to represent the scope covered by the abnormal log keyword. The more systems affected by a certain abnormal log keyword, the more common the problem is and the more unified monitoring is needed. The technology stack concentration refers to the maximum number of the same related technology stack set divided by the total number of all related systems, which is used to represent the related technology stack set where the abnormal log keyword is most concentrated.

[0038] For example, if the number of associated systems for a certain abnormal log keyword is 10, it means that the number of systems affected by this abnormal log keyword is 10. For instance, if the number of associated systems for a certain abnormal keyword is 10, and among them, 2 associated systems belong to associated technology stack set 1, 1 associated system belongs to associated technology stack set 2, 5 associated systems belong to associated technology stack set 3, and 2 associated systems belong to associated technology stack set 4, the maximum number of the same associated technology stack set for this abnormal log keyword is 5, and the number of associated systems is 10, then the technology stack concentration of this abnormal keyword is 0.5. Associated technology stack set 1, associated technology stack set 2, associated technology stack set 3, associated technology stack set 4, and associated technology stack set 5 are all identifiers of associated technology stack sets.

[0039] The cross-system impact coefficient of each abnormal log keyword is obtained by multiplying the number of affected systems for each abnormal log keyword with the concentration of its corresponding technology stack. Specifically, by multiplying the two, we can see how many related systems a certain abnormal log keyword affects, and also see the set of related technology stacks to which these related systems are concentrated.

[0040] In addition, when the technology stack of any system changes, it is necessary to redetermine the target system for the deployment of monitoring keywords, update the deployment, and ensure that the monitoring rules are dynamically deployed and highly adaptable.

[0041] 202. Based on the cross-system impact coefficient of each of the aforementioned abnormal log keywords and its corresponding single-related system frequency, among the various abnormal log keywords, determine the abnormal log keywords that meet the monitoring rule generation conditions as monitoring keywords.

[0042] In this embodiment, determining the abnormal log keywords that meet the monitoring rule generation conditions as monitoring keywords from among a variety of abnormal log keywords based on the cross-system influence coefficient of each abnormal log keyword and its corresponding single-associated system frequency includes: obtaining monitoring rule generation conditions, wherein the monitoring rule generation conditions include a preset influence coefficient threshold and a preset rule generation frequency threshold; for each abnormal log keyword, if the cross-system influence coefficient is greater than or equal to the preset influence coefficient threshold, and the single-associated system frequency of the abnormal log keyword is greater than or equal to the preset rule generation frequency threshold, then it is determined to be an abnormal log keyword that meets the monitoring condition generation rules, and the abnormal log keyword that meets the monitoring condition generation rules is used as a monitoring keyword.

[0043] 203. Automatically generate monitoring rules based on the monitoring keywords and preset monitoring templates.

[0044] In this embodiment, the step of automatically generating monitoring rules based on the monitoring keywords and the preset monitoring template includes: obtaining the preset monitoring template, wherein the preset monitoring template includes: an alarm title field to be filled in and a preset alarm triggering frequency threshold; filling the monitoring keywords into the alarm title field to be filled in to obtain an alarm title, wherein the alarm title is the monitoring keyword alarm; and automatically generating monitoring rules including the alarm title and the preset alarm triggering frequency threshold.

[0045] After determining which abnormal log keywords are monitoring keywords in step 203 of the embodiment, monitoring rules can be automatically generated based on a preset monitoring template without manual intervention. The preset alarm trigger frequency threshold refers to the condition that an alarm is triggered if the frequency of the monitoring keyword in the target system's log is greater than or equal to the preset alarm trigger frequency threshold. One monitoring rule is generated for each monitoring keyword.

[0046] It should be noted that the preset monitoring template also includes alarm frequency, etc., which are not limited here. Alarm frequency refers to the frequency of alarms being triggered once every preset time period.

[0047] 204. According to the preset deployment rules, determine the target systems for batch deployment of the monitoring rules in all the systems, deploy the monitoring rules in the target systems to obtain the target system logs, and if the monitoring keyword is found in the target system logs, determine whether the frequency of occurrence of the monitoring keyword is greater than or equal to the preset alarm triggering frequency threshold. If so, trigger an alarm and push the alarm.

[0048] In this embodiment, determining the target systems for batch deployment of the monitoring rules in all systems according to preset deployment rules includes: the preset deployment rules include: if the associated technology stack set corresponding to the monitoring keyword has only one type, then the associated technology stack set corresponding to the monitoring keyword is taken as the target associated technology stack set; the preset deployment rules also include: if the associated technology stack set corresponding to the monitoring keyword has more than one type, then the quantity of each type of associated technology stack set corresponding to the monitoring keyword is obtained, and the associated technology stack set with the largest quantity is taken as the target associated technology stack set; the systems corresponding to the technology stack sets that are the same as the target associated technology stack set are taken as the target systems for batch deployment of the monitoring rules.

[0049] For example, if the associated technology stack set corresponding to monitoring keyword 1 is of one type, namely associated technology stack set 1, then the target associated technology stack set for monitoring keyword 1 is associated technology stack set 1. The system whose technology stack set is also associated technology stack set 1 is taken as the target system, and monitoring rule 1 corresponding to monitoring keyword 1 is deployed in each target system.

[0050] For example, if the number of associated systems corresponding to monitoring keyword 2 is 4, and there are two types of associated technology stack sets, namely associated technology stack set 2 (3 items) and associated technology stack set 3 (1 item), the largest number of associated technology stack sets is associated technology stack set 2. Therefore, the target associated technology stack set for monitoring keyword 2 is associated technology stack set 2. The system whose technology stack set is also associated technology stack set 2 is taken as the target system, and monitoring rule 2 corresponding to monitoring keyword 2 is deployed in each target system.

[0051] This invention provides a method, apparatus, storage medium, and device for cross-system deployment of log monitoring rules. Through this invention's technical solution, all abnormal log keywords from all systems are aggregated, and the correspondence between abnormal log keywords, associated systems, associated technology stack sets, and the frequency of single associated systems is statistically determined. Based on this, the cross-system influence coefficient and single associated system frequency of each abnormal log keyword are calculated. Thus, among multiple abnormal log keywords, those that meet the conditions for generating monitoring rules are identified as monitoring keywords. This solves the technical problem of existing technologies struggling to identify common monitoring keywords from massive amounts of abnormal log keywords. Monitoring rules are automatically generated from monitoring keywords and preset monitoring templates, rather than manually, improving the efficiency of monitoring rule generation. Since systems with the same technology stack set have the same underlying technical architecture and a consistent probability of experiencing the same type of anomaly, the same set of monitoring rules can be reused. This solves the technical problem of existing technologies struggling to determine the target systems for batch deployment of monitoring rules. The entire process is automated, requiring no manual intervention, and efficiently solves the current technical problem of insufficient cross-system coverage of log monitoring rules.

[0052] Furthermore, as Figure 1 and Figure 2 The specific implementation of the method shown in this invention provides a cross-system deployment device for log monitoring rules, such as... Figure 3 As shown, the device includes: a statistics module 31, a determination module 32, and a deployment module 33; The statistics module 31 is used to obtain the associated systems, associated technology stacks, and single associated system frequency of each abnormal log keyword based on the technology stack set of each system and the abnormal log keywords in the system logs of each system. Based on the associated systems and the associated technology stacks, the cross-system influence coefficient of each abnormal log keyword is calculated. Here, one system corresponds to one technology stack set, and the technology stack sets corresponding to different systems are the same or different. The types of abnormal log keywords corresponding to one system are greater than or equal to 0. The determination module 32 is used to determine, from among the various abnormal log keywords, the abnormal log keywords that meet the conditions for generating monitoring rules as monitoring keywords, based on the cross-system impact coefficient of each abnormal log keyword and its corresponding single-related system frequency. The deployment module 33 is used to automatically generate monitoring rules based on the monitoring keywords and preset monitoring templates, determine the target systems for batch deployment of the monitoring rules in all the systems according to the preset deployment rules, and deploy the monitoring rules in the target systems to monitor the logs of the target systems according to the monitoring rules.

[0053] Accordingly, in order to statistically determine the associated systems, associated technology stack sets, and the frequency of each associated system for each type of abnormal log keyword based on the technology stack set of each system and the abnormal log keywords in the system logs of each system, the statistics module 31 is specifically used to obtain the configuration information of each system, extract the technology components corresponding to the preset technology component items from the configuration information, and use the technology components as elements to form the technology stack set of the system; collect the system logs of each system, use the preset abnormal log rule matching library to identify abnormal logs from the system logs of all systems, extract abnormal log keywords based on the abnormal logs, take the systems in which the abnormal log keywords appear as associated systems, and statistically determine the frequency of each type of abnormal log keyword in each associated system; and take the technology stack set corresponding to the associated system as the associated technology stack set.

[0054] Accordingly, in order to calculate the cross-system impact coefficient of each abnormal log keyword based on the associated systems and the associated technology stack set, the statistics module 31 is specifically used to take the number of associated systems of the abnormal log keyword as the number of affected systems; calculate the maximum number of the same associated technology stack set of the abnormal log keyword divided by the number of all associated systems to obtain the technology stack concentration; and calculate the product of the number of affected systems of each abnormal log keyword and its corresponding technology stack concentration to obtain the cross-system impact coefficient of each abnormal log keyword.

[0055] Accordingly, in order to determine the abnormal log keywords that meet the monitoring rule generation conditions as monitoring keywords among a variety of abnormal log keywords based on the cross-system impact coefficient of each abnormal log keyword and its corresponding single-associated system frequency, the determining module 32 is specifically used to obtain the monitoring rule generation conditions, wherein the monitoring rule generation conditions include a preset impact coefficient threshold and a preset rule generation frequency threshold; for each abnormal log keyword, if the cross-system impact coefficient is greater than or equal to the preset impact coefficient threshold, and the single-associated system frequency of the abnormal log keyword is greater than or equal to the preset rule generation frequency threshold, then it is determined to be an abnormal log keyword that meets the monitoring condition generation rules, and the abnormal log keyword that meets the monitoring condition generation rules is used as the monitoring keyword.

[0056] Accordingly, in order to automatically generate monitoring rules based on the monitoring keywords and the preset monitoring template, the deployment module 33 is specifically used to obtain the preset monitoring template, wherein the preset monitoring template includes: an alarm title field to be filled in and a preset alarm triggering frequency threshold; fill the monitoring keywords into the alarm title field to be filled in to obtain an alarm title, the alarm title being the monitoring keyword alarm; and automatically generate monitoring rules including the alarm title and the preset alarm triggering frequency threshold.

[0057] Accordingly, in order to determine the target system for batch deployment of the monitoring rules in all the systems according to the preset deployment rules, the deployment module 33 specifically includes a first determining unit 331 and a second determining unit 332. The first determining unit 331 is specifically used for the preset deployment rule including: if the type of the associated technology stack set corresponding to the monitoring keyword is one, then the associated technology stack set corresponding to the monitoring keyword is taken as the target associated technology stack set; The second determining unit 332 is specifically used for the preset deployment rule, which further includes: if the types of the associated technology stack set corresponding to the monitoring keyword are greater than one, then obtaining the number of each type of associated technology stack set corresponding to the monitoring keyword, and taking the associated technology stack set with the largest number as the target associated technology stack set; taking the system corresponding to the technology stack set that is the same as the target associated technology stack set as the target system for batch deployment of the monitoring rule.

[0058] In order to monitor the target system logs according to the monitoring rules, the deployment module 33 is specifically used to obtain the target system logs; if the monitoring keyword is identified in the target system logs, it is determined whether the frequency of occurrence of the monitoring keyword is greater than or equal to the preset alarm triggering frequency threshold; if so, an alarm is triggered and an alarm is pushed.

[0059] It should be noted that other corresponding descriptions of the functional units involved in the cross-system deployment device for log monitoring rules provided in this embodiment can be found in [reference needed]. Figures 1 to 2 The corresponding description will not be repeated here.

[0060] Based on the above, Figures 1 to 2 Accordingly, this embodiment also provides a storage medium, which may be volatile or non-volatile, storing a computer program that, when executed by a processor, implements the above-described method. Figures 1 to 2 The method for deploying log monitoring rules across systems is shown.

[0061] Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present invention.

[0062] Based on the above, Figures 1 to 2 The method shown and Figure 3 , Figure 4 To achieve the above objectives, the present application also provides a computer device, specifically a personal computer, server, network device, etc., as shown in the illustrated embodiment. This computer device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figure 1 and Figure 2 The method for deploying log monitoring rules across systems is shown.

[0063] Optionally, the computer device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0064] Those skilled in the art will understand that the computer device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0065] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned computer device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the non-volatile storage medium, as well as communication with other hardware and software in the information processing entity device.

[0066] Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware.

[0067] This invention provides a method, apparatus, storage medium, and device for cross-system deployment of log monitoring rules. Through this invention's technical solution, all abnormal log keywords from all systems are aggregated, and the correspondence between abnormal log keywords, associated systems, associated technology stack sets, and the frequency of single associated systems is statistically determined. Based on this, the cross-system influence coefficient and single associated system frequency of each abnormal log keyword are calculated. Thus, among multiple abnormal log keywords, those that meet the conditions for generating monitoring rules are identified as monitoring keywords. This solves the technical problem of existing technologies struggling to identify common monitoring keywords from massive amounts of abnormal log keywords. Monitoring rules are automatically generated from monitoring keywords and preset monitoring templates, rather than manually, improving the efficiency of monitoring rule generation. Since systems with the same technology stack set have the same underlying technical architecture and a consistent probability of experiencing the same type of anomaly, the same set of monitoring rules can be reused. This solves the technical problem of existing technologies struggling to determine the target systems for batch deployment of monitoring rules. The entire process is automated, requiring no manual intervention, and efficiently solves the current technical problem of insufficient cross-system coverage of log monitoring rules.

[0068] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or they can be located in one or more apparatuses different from this embodiment, with corresponding changes. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0069] The serial numbers used above are for descriptive purposes only and do not represent the superiority or inferiority of the implementation scenarios. The above disclosures are merely a few specific implementation scenarios of the present invention; however, the present invention is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A method for cross-system deployment of log monitoring rules, characterized in that, The method includes: Based on the technology stack set of each system and the abnormal log keywords in the system logs of each system, the associated systems, associated technology stack sets, and the frequency of single associated systems appearing in each associated system for each abnormal log keyword are statistically obtained. Based on the associated systems and the associated technology stack sets, the cross-system influence coefficient of each abnormal log keyword is calculated. Here, one system corresponds to one technology stack set, and the technology stack sets corresponding to different systems are the same or different. The types of abnormal log keywords corresponding to one system are greater than or equal to 0. Based on the cross-system impact coefficient of each of the aforementioned abnormal log keywords and its corresponding single-related system frequency, among the various abnormal log keywords, the abnormal log keywords that meet the monitoring rule generation conditions are determined as monitoring keywords. Monitoring rules are automatically generated based on the monitoring keywords and preset monitoring templates. Based on the preset deployment rules, target systems for batch deployment of the monitoring rules are determined in all the systems. The monitoring rules are then deployed in the target systems to monitor the logs of the target systems according to the monitoring rules.

2. The method according to claim 1, characterized in that, The step of statistically analyzing the associated systems, associated technology stacks, and the frequency of single associated systems appearing in each associated system based on the technology stack set of each system and the abnormal log keywords in the system logs of each system includes: Obtain the configuration information of each system, extract the technical components corresponding to the preset technical component items from the configuration information, and use the technical components as elements to form the technology stack set of the system; Collect system logs for each system, identify abnormal logs from all system logs using a preset abnormal log rule matching library, extract abnormal log keywords based on the abnormal logs, take the systems in which the abnormal log keywords appear as associated systems, and count the frequency of each abnormal log keyword appearing in each associated system in a single associated system. The set of technology stacks corresponding to the associated systems is taken as the associated technology stack set.

3. The method according to claim 1, characterized in that, The step of calculating the cross-system impact coefficient for each of the abnormal log keywords based on the associated systems and the associated technology stack set includes: The number of systems associated with the aforementioned abnormal log keywords is taken as the number of affected systems; The technology stack concentration is obtained by dividing the maximum number of the same associated technology stack set of the abnormal log keywords by the total number of all associated systems. The cross-system impact coefficient of each abnormal log keyword is obtained by multiplying the number of affected systems for each abnormal log keyword by the concentration of its corresponding technology stack.

4. The method according to claim 1, characterized in that, The step of determining, from among a variety of abnormal log keywords, the abnormal log keywords that meet the monitoring rule generation conditions as monitoring keywords, based on the cross-system impact coefficient of each of the abnormal log keywords and its corresponding single-related system frequency, includes: Obtain monitoring rule generation conditions, wherein the monitoring rule generation conditions include a preset influence coefficient threshold and a preset rule generation frequency threshold; For each of the aforementioned abnormal log keywords, if the cross-system impact coefficient is greater than or equal to the preset impact coefficient threshold, and the single-associated system frequency of the abnormal log keyword is greater than or equal to the preset rule generation frequency threshold, then it is determined to be an abnormal log keyword that meets the monitoring condition generation rule, and the abnormal log keyword that meets the monitoring condition generation rule is used as a monitoring keyword.

5. The method according to claim 1, characterized in that, The automatic generation of monitoring rules based on the monitoring keywords and preset monitoring templates includes: Obtain a preset monitoring template, wherein the preset monitoring template includes: an alarm title field to be filled in and a preset alarm trigger frequency threshold; The monitoring keyword is filled into the alarm title field to be filled to obtain the alarm title, and the alarm title is the monitoring keyword alarm; Automatically generate monitoring rules that include the alarm title and the preset alarm trigger frequency threshold.

6. The method according to claim 1, characterized in that, The step of determining the target systems for batch deployment of the monitoring rules in all the systems according to preset deployment rules includes: The preset deployment rules include: if the type of the associated technology stack set corresponding to the monitoring keyword is one, then the associated technology stack set corresponding to the monitoring keyword is taken as the target associated technology stack set; The preset deployment rules also include: if the types of associated technology stack sets corresponding to the monitoring keyword are greater than one, then obtain the number of each type of associated technology stack set corresponding to the monitoring keyword, and take the largest number of associated technology stack sets as the target associated technology stack set; Systems corresponding to the same technology stack set as the target are used as the target systems for batch deployment of the monitoring rules.

7. The method according to claim 5, characterized in that, The monitoring of the target system logs according to the monitoring rules includes: Obtain the target system logs; If the monitoring keyword is found in the target system log, it is determined whether the frequency of occurrence of the monitoring keyword is greater than or equal to the preset alarm triggering frequency threshold. If so, an alarm is triggered and an alarm is pushed.

8. A cross-system deployment device for log monitoring rules, characterized in that, The device includes: The statistics module is used to obtain the associated systems, associated technology stacks, and single associated system frequency of each abnormal log keyword based on the technology stack set of each system and the abnormal log keywords in the system logs of each system. Based on the associated systems and the associated technology stacks, the cross-system influence coefficient of each abnormal log keyword is calculated. Here, one system corresponds to one technology stack set, and the technology stack sets corresponding to different systems may be the same or different. The types of abnormal log keywords corresponding to one system are greater than or equal to 0. The determination module is used to determine, from among the various abnormal log keywords, the abnormal log keywords that meet the conditions for generating monitoring rules as monitoring keywords, based on the cross-system impact coefficient of each abnormal log keyword and its corresponding single-related system frequency. The deployment module is used to automatically generate monitoring rules based on the monitoring keywords and preset monitoring templates, determine the target systems for batch deployment of the monitoring rules in all the systems according to the preset deployment rules, and deploy the monitoring rules in the target systems to monitor the logs of the target systems according to the monitoring rules.

9. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the cross-system deployment method of the log monitoring rules as described in any one of claims 1 to 7.

10. A computer device comprising a memory, a processor, and a computer program stored on a storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the cross-system deployment method for log monitoring rules as described in any one of claims 1 to 7.

Images