Log analyzing and filtering method

Abstract

The invention relates to the technical field of log filtering, in particular to a log analyzing and filtering method. Comprising the steps of constructing a log filtering model according to historical attack events; obtaining a log data packet of the security equipment according to a preset monitoring time node, and generating a plurality of sub-events for an analysis result of the log data packet based on a preset log analysis model; and generating an attack risk value according to the filtering result of the log filtering model on all the sub-events, judging whether to generate an early warning instruction according to the attack risk value, generating a filtering evaluation value according to the filtering result, and judging whether to generate a correction instruction according to the filtering evaluation value. A plurality of feature codes are generated based on comprehensive analysis of historical attack events, a plurality of filtering sub-models are constructed according to random combination of the feature codes, and multi-dimensional recognition filtering is performed on all logs in the security equipment, so that the recognition filtering efficiency and precision of the logs are improved, and the analysis precision of the security equipment detection capability is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of log filtering, in particular to a log analysis and filtering method. BACKGROUND

[0002] The detection capability verification process of the current security device needs to be manually designed by security experts Attack cases, high cost, long cycle, and difficult to cover new threats. It is impossible to verify the detection continuity of the device in daily operation in real time. Based on the fixed attack feature library (such as Snort rule set), it lacks the verification ability of customized attacks (such as APT organization special tool chain). The rule update lags behind, and it is impossible to quickly respond to 0day attacks.

[0003] The general log platform only provides log aggregation and retrieval, and lacks special evaluation ability for security detection efficiency. The automatic mapping mechanism of attack feature code and device log has not been established, and manual correlation analysis is required, which is inefficient. SUMMARY

[0004] The purpose of the present application is: to solve the above technical problems, the present application provides a log analysis and filtering method, which aims to improve the analysis accuracy of the detection capability of the security device.

[0005] In some embodiments of the present application, based on the comprehensive analysis of historical attack events, a plurality of feature codes are generated, and a plurality of filtering sub-models are constructed according to the random combination of each feature code, and all logs in the security device are identified and filtered in multiple dimensions, improving the identification and filtering efficiency and accuracy of the logs, thereby improving the analysis accuracy of the detection capability of the security device.

[0006] In some embodiments of the present application, the performance of each filtering sub-model is periodically evaluated, and the filtering sub-models with low filtering efficiency are timely eliminated, thereby improving the filtering efficiency of the security device logs and reducing the overall misidentification rate.

[0007] In some embodiments of the present application, a log analysis and filtering method is provided, which includes: Constructing a log filtering model according to historical attack events; Obtaining log data packets of a security device according to a preset monitoring time node, generating a plurality of sub-events based on the analysis result of the log data packets by a preset log analysis model; Generating an attack risk value according to the filtering result of all sub-events by the log filtering model, and determining whether to generate a warning instruction according to the attack risk value; Generating a filtering evaluation value according to the filtering result, and determining whether to generate a correction instruction according to the filtering evaluation value.

[0008] In some embodiments of the present application, the construction of the log filtering model comprises: generating a plurality of feature codes according to historical attack events; establishing a feature code sequence P, P=(p1, p2…p i …p m1 ), wherein p i is the i-th feature code; n is the number of feature codes; constructing a plurality of filter sub-models according to the feature code sequence P; establishing a filter sub-model sequence W, W=(w1, w2…w i …w m2 ), wherein w i is the i-th filter sub-model; m2 is the number of filter sub-models; constructing a log filter model according to all filter sub-models.

[0009] In some embodiments of the present application, when generating a filter result, the following steps are included: establishing a sub-event sequence A of the current monitoring time node, A=(a1, a2…a i …a n ), wherein a i is the i-th sub-event of the current monitoring time node; n is the number of sub-events; setting a i as a target sub-event according to the sub-event sequence A; generating an abnormal evaluation value b of the target sub-event according to the log filter model; generating abnormal evaluation values of each sub-event in turn; establishing an abnormal evaluation value sequence B, B=(b1, b2…b i …b n ), wherein b i is the i-th sub-event abnormal evaluation value; n is the number of sub-events; setting an abnormal evaluation value threshold B1; if b i >B1, setting the i-th event as a first-level event; establishing a first-level event sequence A1, A1=(a 11 ,a 12 …a 1i …a 1n1 ), wherein a 1i is the i-th first-level event of the current monitoring time node; n1 is the number of first-level events; generating a filter result of the current monitoring time node according to all first-level events.

[0010] In some embodiments of the present application, when generating an abnormal evaluation value b of a target sub-event, the following steps are included: setting w ia target sub-event is generated according to the target sub-model; a fitting evaluation value c of the target sub-event is generated according to the target sub-model; c= β i *v i ; wherein θ1 is the number of characteristic codes in the target sub-model; β i is the influence factor of the i th characteristic code in the target sub-model; v i is the matching value of the target sub-event and the i th characteristic code in the target sub-model; a fitting evaluation value c of the target sub-event is generated according to the target sub-model; a fitting evaluation value series C, C=(c1, c2…c i …c m2 ), of the target sub-event is established, wherein c i is the fitting evaluation value of the target sub-event and the i th filter sub-model; an abnormal evaluation value b of the target sub-event is generated according to the fitting evaluation value series C.

[0011] In some embodiments of the present application, when the abnormal evaluation value b of the target sub-event is generated, the following is further included: b=e1*Q1*(r*c')+e2*Q2*[ η i *s i ] wherein e1 is a preset first weight coefficient; e2 is a preset second weight coefficient; Q1 is a preset first fixed coefficient; Q2 is a preset second fixed coefficient; c' is the average value of all parameters in the fitting evaluation value series C; r is a compensation coefficient; θ2 is the number of abnormal evaluation indexes; η i is the influence factor of the i th abnormal evaluation index; s i is the reference value of the i th abnormal evaluation index generated based on the target sub-event.

[0012] In some embodiments of the present application, when the compensation coefficient r is set, the following is included: a compensation evaluation value d is generated according to the fitting evaluation value series C; d=e3*Q3*[ (c i -c') 2 ]+e4*Q4*[ Y(i)*(c i -Δc)]; wherein e3 is a preset third weight coefficient; e4 is a preset fourth weight coefficient; Q3 is a preset third fixed coefficient; Q4 is a preset fourth fixed coefficient; c' is the average value of all parameters in the fitting evaluation value series C; Δc is a preset fitting evaluation value threshold; Y(i) is a selection coefficient; if (ci - Δc) > 0, Y(i) = 1; if (c i - Δc) < 0, Y(i) = 0; A compensation coefficient r is set according to the compensation evaluation value d.

[0013] In some embodiments of the present application, when determining whether to generate a warning instruction according to the attack risk value, the following steps are included: generating an attack risk value f of the current monitoring time node according to the primary event sequence A1; f = e1 * Q1 λ i * (c i - c ), wherein n1 is the number of primary events; j i is the record evaluation value of the i-th primary event; λ i is the influence factor of the i-th primary event; a preset attack risk evaluation value threshold F1 is set; if f > F1, a primary warning instruction is generated at the current monitoring time node.

[0014] In some embodiments of the present application, when determining whether to generate a correction instruction according to the filtering evaluation value, the following steps are included: setting w i as a to-be-evaluated submodel according to the filtering submodel sequence W in turn; generating a filtering evaluation value h of the to-be-evaluated submodel according to the primary event sequence A1; generating the filtering evaluation value of each filtering submodel in turn; establishing a filtering evaluation value sequence H, H = (h1, h2…h i …h m2 ), wherein m2 is the number of filtering submodels; h i is the filtering evaluation value of the i-th filtering submodel at the current monitoring time node; a preset filtering evaluation value threshold H1 is set; if h < H1, a primary correction instruction of the i-th filtering submodel is generated at the current monitoring time node.

[0015] In some embodiments of the present application, when generating the filtering evaluation value h, the following steps are included: h = e5 * Q5 λ i * (c 1i - c 1i ) + e6 * Q6 λ i * c 1i ]; wherein e5 is a preset fifth fixed coefficient; e6 is a preset sixth fixed coefficient; Q5 is a preset fifth fixed coefficient; n1 is the number of primary events; λi is the impact factor of the i-th primary event; c 1i the sub-model to be evaluated generates the fitting evaluation value of the i-th primary event; c' 1i is the average fitting value of the i-th primary event.

[0016] In some embodiments of the present application, when judging whether to generate a correction instruction according to the filtering evaluation value, the following is further included: generating an updated evaluation value k according to the filtering evaluation value sequence H; k= µ i *g i ; wherein θ3 is the number of updated evaluation indexes; µ i is the i-th updated evaluation index; g i is the reference value of the i-th updated evaluation index generated based on the filtering evaluation value sequence H; a preset updated evaluation value threshold K1; if k>K1, a secondary correction instruction is generated at the current monitoring time node.

[0017] Compared with the prior art, the log analysis and filtering method according to an embodiment of the present application has the beneficial effects that: Based on the comprehensive analysis of historical attack events, multiple characteristic codes are generated, and multiple filtering sub-models are constructed according to random combinations of the characteristic codes, so that all logs in the security device are identified and filtered in multiple dimensions, the identification and filtering efficiency and accuracy of the logs are improved, and thus the analysis accuracy of the detection capability of the security device is improved.

[0018] By periodically evaluating the performance of each filtering sub-model, filtering sub-models with low filtering efficiency are timely eliminated, the filtering efficiency of the logs of the security device is improved, and the overall misidentification rate is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of a log analysis and filtering method according to a preferred embodiment of the present application. DETAILED DESCRIPTION

[0020] The specific embodiments of the present application will be further described in detail below in combination with the drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.

[0021] In the description of the present application, it needs to be understood that the terms "center", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0022] The terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0023] In the description of the present application, it needs to be explained that, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connection" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integrally connected; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through intermediate medium, or the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0024] As Figure 1 shown, a log analysis and filtering method of the preferred embodiment of the present application comprises: S101: constructing a log filtering model according to historical attack events; S102: obtaining log data packets of security devices according to preset monitoring time nodes, and generating a plurality of sub-events based on the analysis result of the log data packets according to a preset log analysis model; S103: generating an attack risk value according to the filtering result of all sub-events according to the log filtering model, and judging whether to generate a warning instruction according to the attack risk value; S104: generating a filtering evaluation value according to the filtering result, and judging whether to generate a correction instruction according to the filtering evaluation value.

[0025] Specifically, when constructing the log filtering model, it comprises: generating a plurality of characteristic codes according to historical attack events; establishing a characteristic code sequence P, P=(p1, p2…pn), wherein pi is the i-th characteristic code; n is the number of characteristic codes; i …p m1 ), wherein pi is the i-th characteristic code; n is the number of characteristic codes; i ​A plurality of filter sub-models are constructed according to the feature code sequence P; A sequence of filter sub-models W is established, W=(w1, w2…w i …w m2 ), wherein w i is the i th filter sub-model; m2 is the number of filter sub-models; A log filter model is constructed according to all the filter sub-models.

[0026] Specifically, the log analysis model includes a plurality of analysis methods (including but not limited to regular expression analysis, separator analysis, key-value pair analysis method, etc.), which can be analyzed according to the original log content in the security device, and the original log content is divided into a plurality of sub-events according to the analysis result.

[0027] Specifically, a single sub-event represents a continuous log content.

[0028] Specifically, by analyzing the historical log data of the security device, a plurality of attack cases such as ransomware and APT organizations are obtained, and a plurality of feature codes associated with attack risks are generated by extracting feature parameters of each attack case.

[0029] Specifically, the feature code includes but is not limited to abnormal time features (abnormal timestamp, high-frequency repeated events, etc.), abnormal source features (source IP address, etc.), abnormal target features (sensitive resource access), same IP access to multiple user accounts, and internal system port traversal in a short time. Parameters such as.

[0030] Specifically, a plurality of filter sub-models are constructed by randomly combining each feature code, and the combination process includes the number of feature codes in the filter sub-model and the verification order of the feature codes contained.

[0031] Specifically, the number of feature codes and the verification order in each filter sub-model are not completely the same.

[0032] It can be understood that in the above embodiment, a plurality of feature codes are generated based on the comprehensive analysis of historical attack events, and a plurality of filter sub-models are constructed according to the random combination of each feature code, which improves the identification and filtering efficiency and accuracy of the log.

[0033] In the preferred embodiment of the present application, when generating a filter result, it includes: A sequence of sub-events A is established for the current monitoring time node, A=(a1, a2…a i …a n ), wherein a i is the i th sub-event of the current monitoring time node; n is the number of sub-events; According to the sub-event sequence A, a i is set in sequence The abnormal evaluation value b of the target sub-event is generated according to the log filtering model; The abnormal evaluation value of each sub-event is generated in sequence The abnormal evaluation value sequence B is established, B=(b1, b2…b i …b n ), wherein b i is the abnormal evaluation value of the i-th sub-event; n is the number of sub-events; The abnormal evaluation value threshold B1 is predicted; If b i >B1, the i-th event is set as a first-level event; The first-level event sequence A1 is established, A1=(a 11 ,a 12 …a 1i …a 1n1 ), wherein a 1i is the i-th first-level event of the current monitoring time node; n1 is the number of first-level events; The filtering result of the current monitoring time node is generated according to all first-level events.

[0034] Specifically, the abnormal evaluation value threshold can be set according to historical parameters. When the abnormal evaluation value of the target sub-event is greater than the preset abnormal evaluation value threshold, it indicates that the log content of the current target sub-event has a potential attack risk.

[0035] Specifically, the greater the abnormal evaluation value, the greater the correlation between the target sub-event and the attack risk.

[0036] Specifically, when the abnormal evaluation value b of the target sub-event is generated, it includes: According to the filtering sub-model sequence W, w i is set in sequence The fitting evaluation value c of the target sub-event is generated according to the target sub-model; c= β i *v i ; Wherein θ1 is the number of feature codes in the target sub-model; β i is the influence factor of the i-th feature code in the target sub-model; v i is the matching value of the target sub-event and the i-th feature code in the target sub-model; The fitting evaluation value of the target sub-event and each filtering sub-model is generated in sequence The fitting evaluation value sequence C is established, C=(c1, c2…c i …c m2 ), wherein ci a fitting evaluation value of the target sub-event and the ith filtering sub-model; generating an anomaly evaluation value b of the target sub-event according to the fitting evaluation value sequence C.

[0037] Specifically, according to the significance of each feature code in the historical attack case, the corresponding influence factor is set, and the greater the significance, the greater the corresponding influence factor.

[0038] Specifically, the greater the fitting evaluation value, the greater the possibility of the existence of the attack risk corresponding to the target sub-model in the target sub-event.

[0039] Specifically, when generating the anomaly evaluation value b of the target sub-event, it further includes: b = e1*Q1*(r*c') + e2*Q2*[ η i *s i ] Wherein, e1 is a preset first weight coefficient; e2 is a preset second weight coefficient; Q1 is a preset first fixed coefficient; Q2 is a preset second fixed coefficient; c' is the average value of all parameters in the fitting evaluation value sequence C; r is a compensation coefficient; θ2 is the number of anomaly evaluation indexes; η i is the influence factor of the ith anomaly evaluation index; s i is the reference value of the ith anomaly evaluation index generated based on the target sub-event.

[0040] Specifically, the anomaly evaluation indexes include but are not limited to log timestamp discontinuity, log file process abnormal termination, abnormal empty line or deletion mark, log file checksum mismatch, and other abnormal parameters of log metadata. Through the quantitative processing of each anomaly evaluation index, the comprehensive judgment of the target sub-event is realized.

[0041] Specifically, according to the analysis result of the log analysis model, the reference value of each anomaly evaluation index in the target sub-event is generated, and the greater the reference value of each anomaly evaluation index, the greater the potential attack risk of the corresponding log content of the current target sub-event.

[0042] Specifically, the greater the correlation between each anomaly evaluation index and the attack risk, the greater the value of the corresponding influence factor.

[0043] Specifically, the preset first fixed coefficient and the second fixed coefficient are used to normalize all parameters in the model, so that each parameter in the model is in the same value range.

[0044] It can be understood that, in the above embodiment, the multi-dimensional identification filtering of all logs in the security device is performed by constructing multiple filtering sub-models, the identification filtering efficiency and accuracy of the logs are improved, and thus the analysis accuracy of the security device detection capability is improved.

[0045] In the preferred embodiment of the present application, when setting the compensation coefficient r, the following steps are included: generate a compensation evaluation value d according to the fitting evaluation value sequence C; d = e3*Q3*[ (c i -c') 2 ]+e4*Q4*[ Y(i)*( c i -Δc ); Wherein, e3 is a preset third weight coefficient; e4 is a preset fourth weight coefficient; Q3 is a preset third fixed coefficient; Q4 is a preset fourth fixed coefficient; c' is the average value of all parameters in the fitting evaluation value sequence C; Δc is a preset fitting evaluation value threshold; Y(i) is a selection coefficient; if (c i -Δc) > 0, Y(i) = 1; if (c i -Δc) < 0, Y(i) = 0; set the compensation coefficient r according to the compensation evaluation value d.

[0046] Specifically, the larger the compensation evaluation value d is, the greater the fluctuation of the judgment result of each filtering sub-model based on the current target sub-event, and the greater the possibility of misjudgment.

[0047] Specifically, the larger the compensation evaluation value is, the larger the corresponding compensation coefficient is, the value range of the compensation coefficient is always greater than one, and the mapping relationship between the compensation evaluation value and the compensation coefficient can be set according to historical parameters. By dynamically adjusting the compensation coefficient, the saliency of the risk features in the target sub-event is improved, and the filtering efficiency of the log content is improved.

[0048] Specifically, the third fixed coefficient and the fourth fixed coefficient are used to normalize all parameters in the model, so that each parameter in the model is in the same value range.

[0049] It can be understood that, in the above embodiment, based on the judgment result of a single sub-event by all filtering sub-models, the value of the compensation coefficient is dynamically adjusted, so as to improve the saliency of the risk features in the sub-event, avoid misjudgment, and improve the filtering efficiency of the log content.

[0050] In the preferred embodiment of the present application, when determining whether to generate a warning instruction according to the attack risk value, the following steps are included: generate an attack risk value f of the current monitoring time node according to the primary event sequence A1; f= λ i *j i ; wherein, n1 is the number of primary events; j i is the record evaluation value of the i-th primary event; λ i is the influence factor of the i-th primary event; a preset attack risk evaluation value threshold F1; if f>F1, a primary warning instruction is generated at the current monitoring time node.

[0051] Specifically, the influence factor of each primary event is set according to the risk degree of the attack content recorded by the primary event, and the greater the risk degree, the greater the corresponding influence factor.

[0052] Specifically, the record evaluation value of the target primary event refers to the completeness and identification accuracy of the record content of the current primary event by the security device, and the greater the record evaluation value, the lower the completeness and identification accuracy of the corresponding record content.

[0053] Specifically, the attack risk value threshold can be set according to historical parameters.

[0054] Specifically, the primary warning instruction refers to that the detection efficiency of the current security device for the attack behavior is low, and the security device needs to be updated in time.

[0055] In the preferred embodiment of the present application, when judging whether to generate a correction instruction according to the filtering evaluation value, it includes: According to the filtering sub-model sequence W, w i is set as a to-be-evaluated sub-model; According to the primary event sequence A1, a filtering evaluation value h of the to-be-evaluated sub-model is generated; The filtering evaluation values of each filtering sub-model are generated in sequence; A filtering evaluation value sequence H, H=(h1, h2…h i …h m2 ) is established, wherein m2 is the number of filtering sub-models; h i is the filtering evaluation value of the i-th filtering sub-model at the current monitoring time node; A preset filtering evaluation value threshold H1 is set; If h<H1, a primary correction instruction of the i-th filtering sub-model is generated at the current monitoring time node.

[0056] Specifically, when generating the filtering evaluation value h, it includes: h=e5*Q5* λ i *(c 1i -c' 1i] + e6 * Q6 λ i *c 1i ]; wherein e5 is a preset fifth fixed coefficient; e6 is a preset sixth fixed coefficient; Q5 is a preset fifth fixed coefficient; n1 is a number of primary events; λ i is an influence factor of the i-th primary event; c 1i is a fitting evaluation value of the i-th primary event generated by the to-be-evaluated sub-model; c' 1i is an average fitting value of the i-th primary event.

[0057] Specifically, the filtering evaluation value threshold can be set according to historical parameters.

[0058] Specifically, the higher the filtering evaluation value is, the better the filtering efficiency of the to-be-evaluated sub-model is.

[0059] Specifically, the average fitting value is a fitting evaluation value of a target primary event generated by all filtering sub-models.

[0060] Specifically, the preset fifth fixed coefficient and the preset sixth fixed coefficient are used to normalize all parameters in the model, so that all parameters in the model are in the same value range.

[0061] Specifically, the primary correction instruction refers to eliminating the current filtering sub-model.

[0062] It can be understood that in the above embodiments, the performance of each filtering sub-model is periodically evaluated, and the filtering sub-model with low filtering efficiency is timely eliminated, thereby improving the filtering efficiency of the safety device log and reducing the overall misrecognition rate.

[0063] In the preferred embodiments of the present application, when determining whether to generate a correction instruction according to the filtering evaluation value, the following steps are further included: generating an updated evaluation value k according to the filtering evaluation value sequence H; k= µ i *g i ; wherein θ3 is a number of updated evaluation indexes; µ i is the i-th updated evaluation index; g i is a reference value of the i-th updated evaluation index generated based on the filtering evaluation value sequence H; a preset updated evaluation value threshold K1; if k > K1, a secondary correction instruction is generated at the current monitoring time node.

[0064] Specifically, the updated evaluation value threshold can be set according to historical parameters.

[0065] Specifically, the update evaluation index includes but is not limited to the average value of the data in the filtered evaluation value sequence, the difference between the maximum value and the minimum value, the variance and the like. The greater the update evaluation value is, the greater the difference between the judgment results of the sub-events by the respective filtering sub-models is, and the greater the possibility of misjudgment is. The greater the difference between the current attack event and the attack event in the historical data is.

[0066] Specifically, the secondary correction instruction refers to that when there are multiple newly added attack events, the newly added attack events need to be analyzed, so as to update the feature code parameters.

[0067] According to the first concept of the present application, multiple feature codes are generated based on the comprehensive analysis of the historical attack events, and multiple filtering sub-models are constructed according to the random combination of the respective feature codes, so as to perform multi-dimensional recognition filtering on all logs in the security device, improve the recognition filtering efficiency and accuracy of the logs, and thus improve the analysis accuracy of the detection capability of the security device.

[0068] According to the second concept of the present application, the performance of each filtering sub-model is periodically evaluated, and the filtering sub-models with low filtering efficiency are timely eliminated, so as to improve the filtering efficiency of the logs of the security device and reduce the overall misrecognition rate.

[0069] The above is only the preferred embodiment of the present application. It should be pointed out that for those skilled in the art, without departing from the technical principles of the present application, a number of improvements and replacements can be made, which should also be considered as the protection scope of the present application.

Claims

1. A log parsing and filtering method, characterized by, The application relates to a log filtering model construction method and device, a log filtering method and device, and a log filtering system. The log filtering model is constructed according to historical attack events; Log data packets of security devices are acquired according to preset monitoring time nodes, and a plurality of sub-events are generated based on the analysis results of the log data packets according to a preset log analysis model; An attack risk value is generated according to the filtering results of all the sub-events according to the log filtering model, and a warning instruction is generated according to the attack risk value; A filtering evaluation value is generated according to the filtering results, and a correction instruction is generated according to the filtering evaluation value.

2. The log parsing and filtering method of claim 1, wherein, When the log filtering model is constructed, the method comprises the following steps: A plurality of characteristic codes are generated according to historical attack events; A characteristic code sequence P, P=(p1, p2…p i …p m1 ), is established, wherein p i is the i-th characteristic code; n is the number of characteristic codes; A plurality of filtering sub-models are constructed according to the characteristic code sequence P; Establish the filter sub-model sequence W, W=(w1,w2…w i …w m2 ), where w i Let m1 be the i-th filter sub-model; m2 is the number of filter sub-models. The log filtering model is constructed according to all the filtering sub-models.

3. The log parsing and filtering method of claim 2, wherein, When the filtering results are generated, the method comprises the following steps: A, A=(a1, a2…a i …a n ), wherein a i is the i-th sub-event of the current monitoring time node; n is the number of sub-events According to the sub-event sequence A, a is set in turn i Target sub-event; An abnormal evaluation value b of a target sub-event is generated according to the log filtering model; Abnormal evaluation values of all the sub-events are generated in sequence; Establish an anomaly evaluation value sequence B, B=(b1, b2…b i …b n ), where b i is the anomaly evaluation value of the i-th sub-event; n is the number of sub-events; An abnormal evaluation value threshold B1 is set; If b i > B1, set the i-th event as a first-level event; A1, A1=(a 11 ,a 12 …a 1i …a 1n1 ), wherein a 1i is the i-th primary event of the current monitoring time node; n1 is the number of primary events; A filtering result of the current monitoring time node is generated according to all the primary events.

4. The log parsing and filtering method of claim 3, wherein, When the abnormal evaluation value b of the target sub-event is generated, the method comprises the following steps: According to the filter sub-model sequence W, in turn, set w i Target sub-model; A fitting evaluation value c of the target sub-event is generated according to the target sub-model; c= β i *v i ; wherein θ1 is the number of feature codes in the target sub-model; β i is the impact factor of the i-th feature code in the target sub-model; v i is the matching value of the target sub-event and the i-th feature code in the target sub-model. Fitting evaluation values of the target sub-event and all the filtering sub-models are generated in sequence; A fitting evaluation value sequence C, C=(c1, c2…c i …c m2 ), is established, wherein c i is a fitting evaluation value of the target sub-event and the i-th filtering sub-model; The abnormal evaluation value b of the target sub-event is generated according to the fitting evaluation value sequence C.

5. The log parsing and filtering method of claim 4, wherein, When the abnormal evaluation value b of the target sub-event is generated, the method further comprises the following steps: b = e1*Q1*(r*c') + e2*Q2*[ η i *s i ] Wherein, e1 is a preset first weight coefficient; e2 is a preset second weight coefficient; Q1 is a preset first fixed coefficient; Q2 is a preset second fixed coefficient; c' is the average value of all parameters in the fitting evaluation value sequence C; r is a compensation coefficient; θ2 is the number of abnormal evaluation indexes; η i is the influence factor of the i-th abnormal evaluation index; s i is the reference value of the i-th abnormal evaluation index generated based on the target sub-event.

6. The log parsing and filtering method of claim 5, wherein, When the compensation coefficient r is set, the method comprises the following steps: A compensation evaluation value d is generated according to the fitting evaluation value sequence C; d = e3 * Q3 * [ Y(i) * (c + Δc) - Y(i) * (c - Δc) ] / 2 (c i - c') 2 + e4 * Q4 * [ Y(i) * (c + Δc) + Y(i) * (c - Δc) ] / 2 Y(i) * (c i - Δc) ] wherein e3 is a preset third weight coefficient; e4 is a preset fourth weight coefficient; Q3 is a preset third fixed coefficient; Q4 is a preset fourth fixed coefficient; c' is an average value of all parameters in the fitting evaluation value sequence C; Δc is a preset fitting evaluation value threshold; Y(i) is a selection coefficient; if (c i - Δc) > 0, Y(i) = 1; if (c i - Δc) < 0, Y(i) = 0. The compensation coefficient r is set according to the compensation evaluation value d.

7. The log parsing and filtering method of claim 6, wherein, When it is judged whether a warning instruction is generated according to the attack risk value, the method comprises the following steps: An attack risk value f of the current monitoring time node is generated according to the primary event sequence A1; f= λ i *j i ; wherein n1 is the number of primary events; j i is the record evaluation value of the i-th primary event; λ i is the influence factor of the i-th primary event; A preset attack risk evaluation value threshold F1 is set; If f>F1, a primary warning instruction of the current monitoring time node is generated.

8. The log parsing and filtering method of claim 7, wherein, When it is judged whether a correction instruction is generated according to the filtering evaluation value, the method comprises the following steps: According to the filter sub-model sequence W, in turn, set w i To be evaluated sub-model; A filtering evaluation value h of a sub-model to be evaluated is generated according to the primary event sequence A1; Filtering evaluation values of all the filtering sub-models are generated in sequence; Establish a filter evaluation value sequence H, H=(h1, h2…h i …h m2 ), where m2 is the number of filter sub-models; h i This represents the filtering evaluation value of the i-th filtering sub-model at the current monitoring time point. A preset filtering evaluation value threshold H1 is set; If h<H1, a primary correction instruction of the i-th filtering sub-model of the current monitoring time node is generated.

9. The log parsing and filtering method of claim 8, wherein, When the filtering evaluation value h is generated, the method comprises the following steps: h = e5*Q5 λ i * (c 1i - c 1i ) + e6*Q6 λ i * c 1i ; Wherein, e5 is a preset fifth fixed coefficient; e6 is a preset sixth fixed coefficient; Q5 is a preset fifth fixed coefficient; n1 is the number of first-level events; λ i is an influence factor of the i-th first-level event; c 1i The to-be-evaluated sub-model generates a fit evaluation value of the i-th first-level event; c' 1i is an average fit value of the i-th first-level event.

10. The log parsing and filtering method of claim 7, wherein, When it is judged whether a correction instruction is generated according to the filtering evaluation value, the method further comprises the following steps: An update evaluation value k is generated according to the filtering evaluation value sequence H; k = 1 μ i g i ; where θ3 is the number of updated evaluation indexes; µ i is the i-th updated evaluation index; g i is the reference value of the i-th updated evaluation index generated based on the filtered evaluation value series H; A preset update evaluation value threshold K1 is set; If k>K1, a secondary correction instruction of the current monitoring time node is generated.

Images