Syslog log blacklist generation method and device based on compression technology
Through the Syslog log blacklist generation method based on compression technology, log features are extracted and compressed vectors are generated, which solves the problem of low Syslog log parsing efficiency in the existing technology, and realizes efficient log matching and analysis.
Patent Information
- Application Number
- CN202411668605.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-21
- Publication Date
- 2025-03-07
AI Technical Summary
When processing Syslog logs that cannot be parsed, the prior art needs to traverse a large number of log parsing templates, resulting in inefficient parsing, which may cause log accumulation and server memory overflow.
The Syslog log blacklist generation method based on compression technology is adopted. By establishing a log blacklist processing mechanism, log features are extracted and compressed vectors are generated, and the log blacklist is updated, thereby improving matching efficiency and avoiding unresolved logs occupying a large amount of computing resources.
The compressed vector generated by the compression algorithm improves the blacklist matching efficiency of Syslog logs, greatly avoids the use of unresolved logs on system resources, and improves the log parsing efficiency.
Smart Images
Figure CN119577757A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of industrial control safety technology, and in particular to a Syslog log blacklist generation method and device based on compression technology. Background Art
[0002] Syslog logs are the most common log format in computer and network systems, and generally follow the RFC 3164 and RFC5424 standards. Syslog logs are used to record key processes in the operation of business systems or network devices for auditing or analyzing problems. For log auditing and analysis systems, parsing multi-source heterogeneous Syslog log data is an important task.
[0003] The usual solution is to write a log parsing template (usually a regular expression) for each device of different models and different versions of software or system logs, and provide parsing capabilities for logs in that format. Because the log collection device cannot determine the format of the received logs when it is deployed, a traversal matching solution is generally adopted, that is, traversing the log parsing template (regular expression), finding a template that can be matched normally, and then parsing according to the template.
[0004] Disadvantages of current technology: If you encounter a log that cannot be parsed, you need to traverse all the log parsing templates. The number of templates may reach hundreds or even thousands. It takes a long time to traverse them all, which will greatly affect the efficiency of log parsing. In a short period of time, it may cause log accumulation or even server memory overflow.
[0005] Based on this, the present invention proposes a Syslog blacklist generation method and device based on compression technology. Summary of the invention
[0006] The present invention provides a Syslog log blacklist generation method based on compression technology, comprising:
[0007] Step S1, establish a log blacklist processing mechanism;
[0008] Step S2: after receiving the Syslog log, perform blacklist matching on the Syslog log according to the log blacklist processing mechanism. If the match is successful, perform blacklist processing on the Syslog log. If the match is not successful, execute step S3.
[0009] Step S3: determine whether the Syslog log can be parsed. If yes, perform normal processing on the Syslog log; otherwise, execute step S4;
[0010] Step S4, extracting features from the Syslog log, generating a compression vector according to the extracted features according to the compression algorithm, re-updating the log blacklist processing mechanism according to the generated compression vector, and returning to execute step S2;
[0011] Generate compression vectors according to the extracted features and the compression algorithm, specifically including: converting fixed pattern strings in the log or variable-length log content into single characters according to the compression algorithm, thereby compressing the original log into a short string, and then comparing the compressed strings, that is, ignoring the variable content of the log and focusing on comparing the log format.
[0012] As described above, a Syslog log blacklist generation method based on compression technology, wherein the log blacklist includes a log source blacklist and a log format blacklist; the log source blacklist records the log source IP addresses that have security risks, and the log format blacklist records the compression vectors generated by extracting log features from Syslog logs that cannot be parsed.
[0013] The Syslog log blacklist generation method based on compression technology as described above, wherein the Syslog log is matched with the blacklist according to the log blacklist processing mechanism, specifically including:
[0014] Step S21: first determine whether the source address of the Syslog log matches the log source blacklist. If yes, perform blacklist processing. If not, execute step S22.
[0015] Step S22: determine whether the Syslog log matches the log format blacklist. If yes, perform blacklist processing. If no match is successful, execute step S3.
[0016] The Syslog blacklist generation method based on compression technology as described above, wherein determining whether the Syslog log matches the log format blacklist specifically includes the following sub-steps:
[0017] Step S221, taking the first compression vector from the compression vectors in the log format blacklist;
[0018] Step S222, traverse each character, if it is a character in the compression dictionary, use the corresponding regular expression to match, otherwise directly match the character, if the character match is successful, it means that the log belongs to the blacklist; if the character match fails, execute step S223;
[0019] Step S223, sequentially obtain the next compression vector, and then traverse until all blacklist traversals are completed.
[0020] As described above, a Syslog log blacklist generation method based on compression technology is described, wherein a compression vector is generated according to the extracted features according to the compression algorithm, specifically: a simplified fixed dictionary is used, and there is no need to dynamically build a dictionary, and the inherent string pattern in the log is encoded and mapped to a single character; the variable-length content in the log is encoded and mapped to a single character; special characters in the log are retained without the need for mapping changes.
[0021] The Syslog blacklist generation method based on compression technology as described above, wherein a compression vector is generated according to the extracted features in accordance with a compression algorithm, specifically comprising the following sub-steps:
[0022] Define the compression dictionary table based on the characteristics of Syslog logs;
[0023] Traverse the Syslog log and perform string replacement and compression according to the dictionary table;
[0024] Blacklist compression vectors.
[0025] The Syslog blacklist generation method based on compression technology is described above, wherein the compression dictionary is sorted according to the inclusion relationship between standard pattern strings, longer and more complex compression vectors are matched first, and longer and more complex string patterns are recognized first.
[0026] As described above, in the Syslog blacklist generation method based on compression technology, during the Syslog blacklist processing, if the number of Syslog format blacklists of a certain log source exceeds a set threshold limit, the log source is added to the log source blacklist.
[0027] The present invention also provides a Syslog log blacklist generation device based on compression technology, comprising:
[0028] Log blacklist mechanism establishment module, used to establish a log blacklist processing mechanism;
[0029] The blacklist matching module is used to perform blacklist matching on the Syslog log according to the log blacklist processing mechanism after receiving the Syslog log. If the match is successful, the blacklist processing is performed on the Syslog log. If the match is not successful, the parsing module is triggered;
[0030] The parsing module is used to parse the Syslog log and determine whether the Syslog log can be parsed. If yes, the Syslog log is processed normally, otherwise the blacklist update module is triggered;
[0031] The blacklist update module is used to extract features from the Syslog log, generate compression vectors according to the compression algorithm based on the extracted features, and re-update the log blacklist processing mechanism based on the generated compression vectors; generate compression vectors according to the compression algorithm based on the extracted features, specifically including: converting the fixed pattern string in the log, or the variable length log content, into a single character according to the compression algorithm, thereby compressing the original log into a short string, and then comparing the compressed strings, that is, ignoring the variable content of the log and focusing on comparing the log format.
[0032] The present invention also provides a computer storage medium, comprising: at least one memory and at least one processor;
[0033] The memory is used to store one or more program instructions;
[0034] The processor is used to run one or more program instructions to execute any one of the above-mentioned Syslog log blacklist generation methods based on compression technology.
[0035] The beneficial effects achieved by the present invention are as follows: for logs that cannot be parsed normally, the present invention uses an improved compression algorithm, adopts automated means to extract the features of the logs, generates a compression vector corresponding to the logs, and then uses a regular expression to check whether the logs belong to the blacklist. The matching efficiency is very high, which greatly avoids the problem that logs that cannot be parsed normally occupy a large amount of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0037] Figure 1 This is a flow chart of a Syslog log blacklist generation method based on compression technology provided in Example 1 of the present application;
[0038] Figure 2 This is a schematic diagram of a Syslog log blacklist generation device based on compression technology provided in Example 2 of the present application. DETAILED DESCRIPTION
[0039] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0040] Embodiment 1
[0041] like Figure 1 As shown, the first embodiment of the present invention provides a Syslog log blacklist generation method based on compression technology, including:
[0042] Step 110: Establish a log blacklist processing mechanism;
[0043] The log blacklist includes the log source blacklist and the log format blacklist. The log source blacklist records the log source IP addresses that pose security risks, and the log format blacklist records the compression vectors generated by extracting log features from Syslog logs that cannot be parsed. When blacklisting different Syslog logs in the future, the log source blacklist and log format blacklist will be updated in real time. After receiving the Syslog log, blacklist matching is performed first. If the match is successful, these logs will be discarded or specially processed to avoid affecting the parsing of normal logs.
[0044] Step 120: After receiving the Syslog log, perform blacklist matching on the Syslog log according to the log blacklist processing mechanism. If the match is successful, perform blacklist processing on the Syslog log. If the match is not successful, execute step 130.
[0045] Syslog logs are matched against blacklists based on the log blacklist processing mechanism, including:
[0046] Step 121: first determine whether the source address of the Syslog log matches the log source blacklist. If yes, perform blacklist processing. If not, execute step 122.
[0047] Specifically, since the IP address can be obtained directly, the IP address is matched first to reduce the number of matches and improve the matching efficiency. After receiving the Syslog log, first determine whether the IP address from which the Syslog log comes is in the log source blacklist. If so, the Syslog log has a security risk and can be discarded or otherwise processed in a special way; if the IP address from which the received Syslog log comes is not in the log source blacklist, the subsequent log format blacklist mechanism is processed.
[0048] Step 122: determine whether the Syslog log matches the log format blacklist. If yes, perform blacklist processing. If no match is successful, execute step 130.
[0049] Specifically, features are extracted from the Syslog log, a compression vector is generated according to the extracted features, and a blacklist match is performed on the compression vector of the Syslog log according to the log format blacklist. If the match is successful, blacklist processing is performed; if the match is not successful, step 130 is executed.
[0050] Among them, the blacklist format check is performed, which specifically includes the following sub-steps:
[0051] Step 1221: Take the first compression vector from the compression vectors in the log format blacklist;
[0052] Step 1222, traverse each character, if it is a character in the compression dictionary, use the corresponding regular expression to match, otherwise directly match the character, if the character match is successful, it means that the log belongs to the blacklist; if the character match fails, execute step 1223;
[0053] Step 1223, sequentially obtain the next compression vector, and then traverse until all blacklist traversals are completed.
[0054] Step 130: determine whether the Syslog log can be parsed. If yes, perform normal processing on the Syslog log; otherwise, execute step 140;
[0055] Step 140 , extract features from the Syslog log, generate compression vectors according to the extracted features and the compression algorithm, re-update the log blacklist processing mechanism according to the generated compression vector, and return to step 120 .
[0056] Among them, the compression vector is generated according to the extracted features, specifically: referring to the LZW compression algorithm (LZW compression algorithm is a lossless compression algorithm based on a dictionary, which encodes the repeated patterns in the string into a shorter binary string by dynamically building a dictionary to achieve compression), the fixed pattern string in the log, or the variable length log content, is converted into a single character according to the compression algorithm, thereby compressing the original log into a short string. Then, the compressed strings are compared, that is, the variable content of the log is ignored, and the log format is compared.
[0057] In the embodiment of the present application, since the string pattern in the Syslog log is relatively fixed, the purpose of the present application is to identify log features, ignore log content, and do not require lossless compression, so the LZW compression algorithm is modified: a simplified fixed dictionary is used, and there is no need to dynamically build a dictionary. The inherent string pattern in the log is encoded and mapped to a single character; the variable-length content in the log (excluding special characters) is encoded and mapped to a single character; special characters (non-word characters) in the log are retained, and no mapping changes are required.
[0058] Specifically, generating a compressed vector according to the extracted features using a compression algorithm includes the following sub-steps:
[0059] Step 141: Define a compression dictionary table according to the characteristics of the Syslog log;
[0060] The compression dictionary table mainly includes the following contents:
[0061] ①Some standard character strings usually appear in the logs, including but not limited to: IP address, date and time, MAC address, SNMPOID, etc.
[0062] ② The variable-length text content that appears in the log is generally a word string.
[0063] Since word strings will not appear in the compressed vector, English letters can be used instead of standard pattern strings. A typical compression dictionary is shown in the following table:
[0064] Serial number String Pattern String Example Compressed Characters 1 OID 1.3.6.1.2.1.1.4 O 2 IPV4 192.168.10.244 I 3 MAC 00-23-24-E8-F9-8D M 4 DATETIME 2006-03-1220:12:23 D 5 COMMON_WORD PPLive-888 W
[0065] Because these special string patterns have a containment relationship, the long string pattern should be matched first and the short string pattern should be ignored. If a part of the string in the log has already matched a longer string pattern, there is no need to consider the short string pattern. In this way, the extracted log pattern is more effective and accurate.
[0066] Therefore, the first thing to do is to sort the compression dictionary according to the inclusion relationship, give priority to matching longer and more complex compression vectors, and give priority to identifying longer and more complex string patterns. It should be noted that the regular expression sorting principle involved in this application may have different known string patterns in different industries, and it is preferred that they should be compared and sorted according to this principle. For example: the string pattern of SNMPOID contains the string pattern of IP V4, so the compression dictionary of SNMPOID should be ranked before the string pattern of IP V4.
[0067] Step 142, traverse the Syslog log, and perform string replacement compression according to the dictionary table;
[0068] For Syslog log strings that cannot be parsed normally, traverse the compression dictionary table, search for string patterns stored in the dictionary one by one, and replace them with characters in the dictionary table. When searching for string patterns, you can use regular expressions for matching. When the corresponding string pattern is found, it is replaced with compressed characters and the corresponding end position is recorded.
[0069] Start a new traversal from the last end position until the end of the log.
[0070] For example, a log file is:
[0071] <1> 2006-03-12 20:12:23 kemel IDS 0PPLive online streaming 192.168.10.244138192.168.10.255138
[0072] After compression and replacement, the generated compressed vector string is:
[0073] <w>DWWWWWIWIW
[0074] It can also be converted to ASCII encoding, represented as a binary vector:
[0075] {60,87,62,32,68,32,87,32,87,32,87,32,87,32,87,32,87,32,73,32,87,32,73,32,87}
[0076] Step 143: Add the compression vector to the blacklist.
[0077] During the Syslog blacklist processing, if the number of Syslog format blacklists for a log source exceeds the set threshold limit, the log source will be added to the log source blacklist. This can avoid the problem of too many log format blacklists.
[0078] The present invention aims at the logs that cannot be parsed normally, uses an improved compression algorithm, adopts an automated means to extract the features of the logs, generates a compression vector corresponding to the logs, and then uses a regular expression to check whether the logs belong to the blacklist. The matching efficiency is very high, and the problem that the logs that cannot be parsed normally occupy a large amount of computing resources is greatly avoided.
[0079] Because of the use of compression algorithm, the generated blacklist content is much simpler than the complete regular expression, and has obvious advantages in readability and storage efficiency.
[0080] For example, the following log file:
[0081] <1> 2006-03-12 20:12:23 kemel IDS 0PPLive online streaming 192.168.10.244138192.168.10.255138
[0082] If a regular expression is used for validation, the regular expression is:
[0083] <[\w\u2014-\u9FFF\uFF01-\uFF1F]+>
[0084] ([1-9]{1}[0-9]{3})[\-\.\\ / ]? ([0-1]?[0-9]{1})[\-\.\\ / ]? ([0-3]?[0-9]{1})[T\s]+[0-2]? ([0-9]{1}:[0-9]{1,2}:[0-9]{1,2}) [\w\u2014-\u9FFF\uFF01-\uFF1F]+[\w\u2014-\u9FFF\uFF01-\uFF1F]+ [\w\u2014-\u9FFF\uFF01-\uFF1F]+[\w\u2014-\u9FFF\uFF01-\uFF1F]+ [\w\u2014-\u9FFF\uFF01-\uFF1F]+((25[0-5]|2[0-4][0-9]|
[01] ?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|
[01] ?[0-9][0-9]?)
[0085] [\w\u2014-\u9FFF\uFF01-\uFF1F]+
[0086] ((25[0-5]|2[0-4][0-9]|
[01] ?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|
[01] ?[0-9][0-9]?)
[0087] [\w\u2014-\u9FFF\uFF01-\uFF1F]+
[0088] The total length of the regular expression is 541, while the total length of the compressed vector is 23, a difference of 23 times.
[0089] Embodiment 2
[0090] like Figure 2 As shown, Embodiment 2 of the present invention provides a Syslog log blacklist generation device based on compression technology, including:
[0091] A log blacklist mechanism establishment module 21 is used to establish a log blacklist processing mechanism;
[0092] The blacklist matching module 22 is used to perform blacklist matching on the Syslog log according to the log blacklist processing mechanism after receiving the Syslog log. If the match is successful, the blacklist processing is performed. If the match is not successful, the parsing module 23 is triggered;
[0093] The parsing module 23 is used to parse the Syslog log and determine whether the Syslog log can be parsed. If yes, the Syslog log is processed normally, otherwise the blacklist update module 24 is triggered;
[0094] The blacklist update module 24 is used to extract features from the Syslog log, generate compression vectors according to the extracted features according to the compression algorithm, and re-update the log blacklist processing mechanism based on the generated compression vectors; generating compression vectors according to the compression algorithm according to the extracted features, specifically including: converting the fixed pattern string in the log, or the variable length log content, into a single character according to the compression algorithm, thereby compressing the original log into a short string, and then comparing the compressed strings, that is, ignoring the variable content of the log and focusing on comparing the log format.
[0095] Corresponding to the above embodiment, an embodiment of the present invention provides a computer storage medium, including: at least one memory and at least one processor;
[0096] The memory is used to store one or more program instructions;
[0097] The processor is used to run one or more program instructions to execute a Syslog log blacklist generation method based on compression technology.
[0098] Corresponding to the above embodiment, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer storage medium contains one or more program instructions, and the one or more program instructions are used by a processor to execute a Syslog log blacklist generation method based on compression technology.
[0099] The embodiment disclosed in the present invention provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed on a computer, the computer executes the above-mentioned Syslog log blacklist generation method based on compression technology.
[0100] In the embodiment of the present invention, the processor may be an integrated circuit security chip having the signal processing capability. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0101] The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.
[0102] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.
[0103] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.
[0104] The volatile memory may be a random access memory (RAM) which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM).
[0105] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.
[0106] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the present invention can be implemented using a combination of hardware and software. When software is used, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.
[0107] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.< / w>
Claims
1. A Syslog blacklist generation method based on compression technology, characterized in that: include: Step S1, establish a log blacklist processing mechanism; Step S2: after receiving the Syslog log, perform blacklist matching on the Syslog log according to the log blacklist processing mechanism. If the match is successful, perform blacklist processing on the Syslog log. If the match is not successful, execute step S3. Step S3: determine whether the Syslog log can be parsed. If yes, perform normal processing on the Syslog log; otherwise, execute step S4; Step S4, extracting features from the Syslog log, generating a compression vector according to the extracted features according to the compression algorithm, re-updating the log blacklist processing mechanism according to the generated compression vector, and returning to execute step S2; Generate compression vectors according to the extracted features and the compression algorithm, specifically including: converting fixed pattern strings in the log or variable-length log content into single characters according to the compression algorithm, thereby compressing the original log into a short string, and then comparing the compressed strings, that is, ignoring the variable content of the log and focusing on comparing the log format.
2. A Syslog log blacklist generation method based on compression technology as described in claim 1, characterized in that: The log blacklist includes the log source blacklist and the log format blacklist; the log source blacklist records the log source IP addresses that pose security risks, and the log format blacklist records the compression vectors generated by extracting log features from Syslog logs that cannot be parsed.
3. A Syslog log blacklist generation method based on compression technology as described in claim 2, characterized in that: Syslog logs are matched against blacklists based on the log blacklist processing mechanism, including: Step S21: first determine whether the source address of the Syslog log matches the log source blacklist. If yes, perform blacklist processing. If not, execute step S22. Step S22: determine whether the Syslog log matches the log format blacklist. If yes, perform blacklist processing. If no match is successful, execute step S3.
4. A Syslog log blacklist generation method based on compression technology as described in claim 3, characterized in that: Determine whether the Syslog log matches the log format blacklist, including the following sub-steps: Step S221, taking the first compression vector from the compression vectors in the log format blacklist; Step S222, traverse each character, if it is a character in the compression dictionary, use the corresponding regular expression to match, otherwise directly match the character, if the character match is successful, it means that the log belongs to the blacklist; if the character match fails, execute step S223; Step S223, sequentially obtain the next compression vector, and then traverse until all blacklist traversals are completed.
5. The Syslog blacklist generation method based on compression technology as claimed in claim 1, characterized in that: A compression vector is generated according to the extracted features and the compression algorithm. Specifically, a simplified fixed dictionary is used, and there is no need to dynamically build a dictionary. The inherent string patterns in the log are encoded and mapped to single characters. The variable-length content in the log is encoded and mapped to single characters. Special characters in the log are retained without the need for mapping changes.
6. A Syslog blacklist generation method based on compression technology as claimed in claim 5, characterized in that: Generate a compressed vector according to the extracted features using a compression algorithm, specifically including the following sub-steps: Define the compression dictionary table based on the characteristics of Syslog logs; Traverse the Syslog log and perform string replacement and compression according to the dictionary table; Blacklist compression vectors.
7. A Syslog blacklist generation method based on compression technology as claimed in claim 6, characterized in that: The compression dictionary is sorted according to the inclusion relationship between the standard pattern strings, and longer and more complex compression vectors are matched first, and longer and more complex string patterns are recognized first.
8. The Syslog blacklist generation method based on compression technology as claimed in claim 6, characterized in that: During the Syslog blacklist processing, if the number of Syslog format blacklists for a log source exceeds the set threshold, the log source is added to the log source blacklist.
9. A Syslog blacklist generation device based on compression technology, characterized in that: include: Log blacklist mechanism establishment module, used to establish a log blacklist processing mechanism; The blacklist matching module is used to perform blacklist matching on the Syslog log according to the log blacklist processing mechanism after receiving the Syslog log. If the match is successful, the blacklist processing is performed on the Syslog log. If the match is not successful, the parsing module is triggered; The parsing module is used to parse the Syslog log and determine whether the Syslog log can be parsed. If yes, the Syslog log is processed normally, otherwise the blacklist update module is triggered; The blacklist update module is used to extract features from Syslog logs, generate compression vectors according to the extracted features and the compression algorithm, and re-update the log blacklist processing mechanism according to the generated compression vectors; Generate compression vectors according to the extracted features and the compression algorithm, specifically including: converting fixed pattern strings in the log or variable-length log content into single characters according to the compression algorithm, thereby compressing the original log into a short string, and then comparing the compressed strings, that is, ignoring the variable content of the log and focusing on comparing the log format.
10. A computer storage medium, characterized in that: include: at least one memory and at least one processor; The memory is used to store one or more program instructions; A processor is used to run one or more program instructions to execute the Syslog log blacklist generation method based on compression technology as described in any one of claims 1 to 8.