A log information processing method and device and medium

Abstract

The application discloses a kind of log information processing method, device and medium, applicable to server technical field.According to the mapping of log information according to hash rule obtains hash fingerprint;According to the relationship between hash fingerprint and alarm information, the state level of server is determined;When the state level of server is lower than preset level, then determine that server is abnormal state and output current alarm information to facilitate user to view.According to the mapping of log information according to hash rule, hash fingerprint is obtained, and then the state level of server is determined according to the relationship between hash fingerprint and alarm information, and when the state level is lower than preset level, alarm information is output, the state level of server is obtained by analyzing log information, reduces the troubleshooting time of staff, facilitates fault location, when problem occurs, alarm information is prompted, facilitate staff to view, avoid existing according to log information scattered data and relevant experience troubleshooting, reduce the loss caused by equipment failure.

Description

Technical Field

[0001] This invention relates to the field of server technology, and in particular to a method, apparatus and medium for processing log information. Background Technology

[0002] In today's era of big data, the importance of data is becoming increasingly apparent. Organizations are establishing their own data centers and server rooms, or renting cloud servers from cloud service providers, to store critical data. Therefore, monitoring servers within data centers or server rooms is of paramount importance.

[0003] Equipment logs are an essential part of data center or data monitoring. Log information includes not only various warning messages but also operational data such as user logins and hard drive replacements. Essentially, logs record the entire lifecycle of a server from its initial setup. Currently, log information is simply collected and rarely used. It's only when a server malfunctions that staff review the logs. However, at this stage, the collected information is limited, allowing troubleshooting based on fragmented data and prior experience. If staff lack experience or the logs are incomplete, troubleshooting becomes time-consuming, making it difficult to accurately pinpoint the cause of the problem and impacting normal server operation.

[0004] Therefore, improving the speed of troubleshooting is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, and medium for processing log information, thereby improving the speed of troubleshooting.

[0006] To address the aforementioned technical problems, this invention provides a method for processing log information, comprising:

[0007] Collect log information and alarm information generated during server operation, where the alarm information is reference information provided for server production;

[0008] The log information is mapped according to the hash rules to obtain the hash fingerprint;

[0009] The server's status level is determined based on the relationship between hash fingerprints and alarm information;

[0010] When the server's status level is lower than the preset level, the server is determined to be in an abnormal state and the current alarm information is output for the user to view.

[0011] Preferably, the log information is mapped to obtain a hash fingerprint according to a hash rule, including:

[0012] Retrieve the current information from the log messages, which include multiple current messages.

[0013] The filtered information is obtained by filtering the current information based on the core content fields;

[0014] The filtered information is segmented into words to obtain keyword information, and weights are assigned based on word frequency to obtain corresponding weight information;

[0015] The hash string information is obtained by mapping the keyword information using a hash function.

[0016] The corresponding keyword weight information is obtained based on the hash string information and the weight information.

[0017] The hash fingerprint is obtained by merging the weight information of each keyword.

[0018] Preferably, the server status level is determined based on the relationship between the hash fingerprint and alarm information, including:

[0019] Reference fingerprint information is obtained by mapping the reference information of alarm information according to the hash rule. The reference fingerprint information consists of multiple pieces of information.

[0020] The hash fingerprint corresponding to the current information is matched with the similarity information of each reference fingerprint to obtain the corresponding similarity information;

[0021] When the similarity information is greater than the similarity threshold, the current information will be matched with a similar warning message;

[0022] The similarity score of the current information is obtained based on the various similarity information.

[0023] The server's status level is determined based on the similarity score of the current information.

[0024] Preferably, the server status level is obtained based on the similarity score of the current information, including:

[0025] Obtain similar warning messages for the current information and their corresponding warning importance coefficients;

[0026] The server's status level is obtained by weighting and averaging similar warning messages, warning importance coefficients, and similarity scores.

[0027] Preferably, the hash fingerprint corresponding to the current information is matched with the reference fingerprint information to obtain the corresponding similarity information, including:

[0028] The hash fingerprint is matched with each reference fingerprint using cosine similarity to obtain the corresponding similarity information.

[0029] Preferably, the alarm importance coefficient is determined through the following steps:

[0030] The current number of warnings, the total number of alarms, and the number of log entries containing the current warning information are retrieved.

[0031] The alarm importance coefficient is determined based on the relationship between the number of warnings, the total number of alarms, and the number of log entries.

[0032] Preferably, when the server is in an abnormal state, the current alarm information is output, including:

[0033] The current alarm information will be displayed via SMS or email.

[0034] To address the aforementioned technical problems, the present invention also provides a log information processing apparatus, comprising:

[0035] The data acquisition module is used to collect log information and alarm information generated during server operation, where the alarm information is reference information provided for server production.

[0036] The mapping module is used to map log information according to hash rules to obtain hash fingerprints;

[0037] The determination module is used to determine the server's status level based on the relationship between hash fingerprints and alarm information;

[0038] The output module is used to determine that the server is in an abnormal state and output the current alarm information for users to view when the server's status level is lower than the preset level.

[0039] To address the aforementioned technical problems, the present invention also provides a log information processing apparatus, comprising:

[0040] Memory, used to store computer programs;

[0041] A processor is used to implement the steps of the log information processing method described above when executing a computer program.

[0042] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the log information processing method described above.

[0043] This invention provides a method for processing log information, including collecting log information and alarm information generated during server operation, wherein the alarm information serves as reference information provided by the server; mapping the log information to obtain a hash fingerprint according to a hash rule; determining the server's status level based on the relationship between the hash fingerprint and the alarm information; and determining the server's abnormal state and outputting the current alarm information when the server's status level is lower than a preset level. This method maps log information to obtain a hash fingerprint according to a hash rule, then determines the server's status level based on the relationship between the hash fingerprint and the alarm information, and outputs alarm information when the status level is lower than a preset level. By analyzing the log information to obtain the server's status level, this method reduces the time required for staff to troubleshoot and facilitates fault location. When a problem occurs, alarm information is provided for easy staff review, avoiding the need for existing troubleshooting methods that rely on fragmented log data and related experience, thus reducing losses caused by equipment failures.

[0044] In addition, the present invention also provides a log information processing apparatus, which has the same beneficial effects as the log information processing method described above. Attached Figure Description

[0045] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating a log information processing method provided in an embodiment of the present invention;

[0047] Figure 2 A structural diagram of a log information processing device provided in an embodiment of the present invention;

[0048] Figure 3 This is a structural diagram of another log information processing device provided in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0050] The core of this invention is to provide a method, apparatus, and medium for processing log information, thereby improving the speed of troubleshooting.

[0051] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] It should be noted that the log information processing method provided by this invention is used to collect server logs so as to perform unified log aggregation and analysis on servers in the data center, promptly reflect abnormal system states, and issue alarm information to staff. Figure 1 A flowchart of a log information processing method provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes:

[0053] S11: Collect log information and alarm information generated during server operation, where alarm information is reference information provided for server production.

[0054] Specifically, log information generated during server runtime can be collected at preset intervals or proactively retrieved by users for periodic checks, without specific limitations. Log information records program runtime data; different technical frameworks implement logging differently. A logging interface is defined, which can be adapted when the project's business logic uses a different logging framework. Log information serves two purposes: troubleshooting and data source. In the big data field, data sources include relational databases, web crawlers, and log information.

[0055] The alarm information is a reference information provided by the server manufacturer based on common fault alarms that occur during server testing before the server leaves the factory. This information is intended to help users troubleshoot and repair the problem during use.

[0056] S12: Map the log information according to the hash rules to obtain the hash fingerprint.

[0057] In hash rule mapping, finding target information in log information requires powerful retrieval. This invention does not impose specific limitations and supports structured retrieval, full-text retrieval, multi-field retrieval, approximate matching, partial matching, etc. After finding the target information, it is analyzed. During the analysis, a scheduled task is maintained to periodically call the retrieval interface, retrieve all log information collected in the current time period, group the log information, and analyze each grouped log individually.

[0058] The grouped log information is mapped according to hash rules. The hash rule transforms an input of arbitrary length into a fixed-length output, which is the hash value. This transformation process is a compression mapping; the space of hash values ​​is usually much smaller than the space of inputs. Different inputs may hash to the same output, so it is impossible to uniquely determine the input value from the hash value. Commonly used hash functions include the direct modulo method, the multiplication and integer method, and the mid-square method. This invention does not impose specific limitations, as long as they can be mapped to obtain a hash fingerprint.

[0059] S13: Determine the server's status level based on the relationship between hash fingerprints and alarm information.

[0060] The hash fingerprint obtained in S12 above, wherein the alarm information can be obtained by obtaining reference fingerprint information through the above embodiment, and the hash fingerprint and reference fingerprint information are compared and matched to determine the status level of the server.

[0061] Specifically, the collected log information consists of multiple entries. Each entry is processed individually and compared with the reference fingerprint information in the alarm information list file to determine the level of the current entry. Then, the levels of the multiple entries are summarized to obtain the server's status level.

[0062] S14: When the server's status level is lower than the preset level, the server is determined to be in an abnormal state and the current alarm information is output for the user to view.

[0063] When the server's status level is lower than the defined threshold, which is the preset level, the server's current status is determined to be abnormal, and an alarm needs to be pushed and the current alarm information is output. It can be understood that the current alarm information can be the same as the alarm information in step S11, or other alarm information can be added.

[0064] Alarm information can be output via a web page. Considering that users will not spend long periods of time in the server room or near their computers, alarm information can be sent to users via SMS or email so that they can check and maintain it in a timely manner.

[0065] This invention provides a log information processing method, including collecting log information and alarm information generated during server operation, wherein the alarm information serves as reference information provided by the server; mapping the log information according to hash rules to obtain hash fingerprints; determining the server's status level based on the relationship between the hash fingerprints and alarm information; and determining the server's abnormal state and outputting current alarm information when the server's status level is lower than a preset level. Mapping the log information to obtain hash fingerprints according to hash rules, and then determining the server's status level based on the relationship between the hash fingerprints and alarm information, outputting alarm information when the status level is lower than a preset level, reduces the time required for staff to troubleshoot and facilitates fault location. Alarm information is provided when problems occur, making it convenient for staff to check and avoiding the need for existing troubleshooting methods based on fragmented log data and related experience, thus reducing losses caused by equipment failures.

[0066] Based on the above embodiments, step S12, which maps log information according to hash rules to obtain hash fingerprints, includes:

[0067] Retrieve the current information from the log messages, which include multiple current messages.

[0068] The filtered information is obtained by filtering the current information based on the core content fields;

[0069] The filtered information is segmented into words to obtain keyword information, and weights are assigned based on word frequency to obtain corresponding weight information;

[0070] The hash string information is obtained by mapping the keyword information using a hash function.

[0071] The corresponding keyword weight information is obtained based on the hash string information and the weight information.

[0072] The hash fingerprint is obtained by merging the weight information of each keyword.

[0073] Specifically, the current information of multiple log entries is obtained, and the core content field is used to filter this information. The core content field is categorized during log entry creation, based on log name, log time, and core content field. The filtering in this embodiment is not the conventional frequency or noise filtering; instead, it filters out special characters and stop words. Keywords can also be extracted to achieve the same filtering effect. To capture subsequent keyword sets, filtering is performed here based on special characters and stop words. For example, if the core content field is "fan module malfunction, location: fan 0\n", filtering yields "fan module malfunction, location: fan 0". Filtering special characters and stop words avoids interference with text similarity calculations. Stop words are words that appear frequently but have no inherent meaning, such as adverbs, prepositions, and modal particles.

[0074] The filtered information is segmented into words to obtain keyword information. At this point, the keyword information is a keyword set, and the corresponding weight information is obtained by assigning weights according to word frequency. Combining with the example above, it is {"fan", "module", "generate", "fault", "location", "0"}, where the weight of fan is W1=2, and the weight of the other keywords is W2=1.

[0075] The keyword information is mapped using a hash function to generate an n-bit binary string, i.e., the hash string. For example, the hash string for the keyword "fan" is H1 = 10011101, and the hash string for the keyword "module" is H2 = 11001010, and so on. It should be noted that the hash function here is not fixed and can be chosen according to actual needs, as long as the mapping distribution of different keywords is sufficiently even and the mapping of the same keyword is the same.

[0076] The corresponding keyword weight information is obtained by combining the hash string information and the weight information, using the following formula:

[0077] V wi =W i *H i

[0078] Among them, W i H represents the weight of the current keyword. i This is the hash value of the current keyword.

[0079] After weighted calculation, the weight vector of each keyword is obtained, such as the weight vector V for the keyword "fan". w2 = (1, 1, -1, -1, 1, -1, 1, -1, -1), the weight vector V of the keyword "module" w2= (1, 1, -1, -1, 1, -1, 1, -1, 1, -1). Then, the weight vectors of all keywords are combined using the following formula:

[0080]

[0081] Here, n is the number of keywords in the current text. For the merged weight vector, positions greater than 0 are set to 1, and positions less than 0 are set to 0, thus obtaining the hash fingerprint of the current information. It should be noted that the weight vector merging is parallel. For example, if the hash string information of each keyword is 8 bits, and there are a total of 6 keywords, then the merged weight vector will be 8 bits.

[0082] This invention provides a method for mapping log information to obtain hash fingerprints based on hash rules. First, filtering is performed to obtain filtered information, then word segmentation is used to obtain keyword information, which is then mapped according to a hash function. The filtering process avoids interference with text similarity calculations, and the different log information is organized to obtain fixed-length hash fingerprints, facilitating subsequent data similarity comparisons.

[0083] Based on the above embodiments, step S13, determining the server's status level according to the relationship between the hash fingerprint and alarm information, includes:

[0084] Reference fingerprint information is obtained by mapping the reference information of alarm information according to the hash rule. The reference fingerprint information consists of multiple pieces of information.

[0085] The hash fingerprint corresponding to the current information is matched with the similarity information of each reference fingerprint to obtain the corresponding similarity information;

[0086] When the similarity information is greater than the similarity threshold, the current information will be matched with a similar warning message;

[0087] The similarity score of the current information is obtained based on the various similarity information.

[0088] The server's status level is determined based on the similarity score of the current information.

[0089] Specifically, in the above embodiments, the hash fingerprint of the current information needs to be matched one by one with the reference information of the alarm information. The reference information of the alarm information needs to have the same data length as the hash fingerprint, so the same hash rule mapping is required to obtain the reference fingerprint information.

[0090] The similarity information is obtained by matching the hash fingerprint corresponding to the current information with the similarity information of each reference fingerprint. That is, if the log information contains multiple pieces of information, the current information needs to be matched with the alarm information first, and then the next current information is matched to obtain the similarity information of the corresponding log information. A similarity score is obtained based on the similarity information corresponding to the current information, with one similarity score for each piece of information. All log information is then processed to obtain the corresponding similarity information. For example, if there are 5 logs per minute, the corresponding similarity score set is {A, B, C, D, E}. If the server has multiple log information, all log information is retrieved to obtain the server's similarity list. The server's status level is then determined based on the server's similarity list.

[0091] If the similarity information exceeds the similarity threshold, the current information will be matched with a similarity warning message to determine the server's loading level. Similarity matching can be performed using cosine similarity, Euclidean metric, or Pearson correlation coefficient, etc. This invention does not impose specific limitations, as long as the corresponding similarity information can be obtained.

[0092] This embodiment provides a method for determining the server's status level based on the relationship between hash fingerprints and alarm information. It involves matching the hash fingerprints obtained from multiple log entries with the reference fingerprints of each alarm entry to obtain a similarity score. The server's status level is then determined based on this similarity score. This refined information facilitates a more accurate determination of the server's status level.

[0093] Based on the above embodiments, the server status level is obtained according to the similarity score of the current information, specifically including:

[0094] Obtain similar warning messages for the current information and their corresponding warning importance coefficients;

[0095] The server's status level is obtained by weighting and averaging similar warning messages, warning importance coefficients, and similarity scores.

[0096] Specifically, the similarity score list is represented by the following formula:

[0097] Scores={Sc1,Sc2,...,Sc n}

[0098] Among them, Sc i This is the similarity score of the current log.

[0099] Based on the acquired similar warning information, the corresponding similarity alarm level is obtained, and the basic weight is derived based on the similarity alarm level. It is expressed by the following formula:

[0100]

[0101] Here, level refers to the alarm level recorded in the log list, which is a total of 10 levels from 1 to 10 for similar alarm levels.

[0102] The server's status level is determined by a weighted average algorithm based on the base weight obtained from the similar warning information, the warning importance coefficient, and the similarity score. The formula for the weighted average is as follows:

[0103]

[0104] Where Level represents the state level, and Sc i The similarity score for the current log. The base weight is δ, which is the alarm importance coefficient.

[0105] The present invention provides a method for determining the server status level based on the similarity score of current information, which refines the determination form of the status level, increases the accuracy of the server status level, and facilitates the accurate output of subsequent alarm information.

[0106] As a preferred embodiment, the hash fingerprint corresponding to the current information is matched with each reference fingerprint information to obtain the corresponding similarity information, including:

[0107] The hash fingerprint is matched with each reference fingerprint using cosine similarity to obtain the corresponding similarity information.

[0108] Specifically, the similarity information is obtained through cosine similarity, and the two fingerprint information (hash fingerprint and reference fingerprint information) are calculated using the following formula:

[0109]

[0110] Where n is the length of the fingerprint information, A i For the i-th bit of the hash fingerprint, B i For the i-th reference point of fingerprint information, the smaller the cosine value of two fingerprints, that is, the smaller the included angle θ, the higher the similarity between the two fingerprints.

[0111] In conjunction with the above embodiments, the similarity threshold is set to 0.8. That is, when cosθ > 0.8, the current information is matched with similar warning information. If it is lower than the threshold, the information is considered irrelevant and can be stored or discarded without specific limitation.

[0112] This invention provides a method for obtaining corresponding similarity information by performing cosine similarity matching between hash fingerprints and each reference fingerprint. Compared with other similarity methods, cosine similarity measures the consistency of the value directions between dimensions, emphasizing the differences between dimensions.

[0113] In the above embodiments, the alarm importance coefficient is determined through the following steps:

[0114] The current number of warnings, the total number of alarms, and the number of log entries containing the current warning information are retrieved.

[0115] The alarm importance coefficient is determined based on the relationship between the number of warnings, the total number of alarms, and the number of log entries.

[0116] Specifically, the alarm importance coefficient is used to measure whether a particular alarm is a common alarm. If it is rare but appears multiple times in the logs collected this time, it may indicate a serious fault on the server. For a given alarm, the total number of similar warnings matching the current alarm in this collection is the alarm count α. The total number of alarms β in the alarm information, and the number of log entries containing the current alarm's warning information in the server's logs collected this time (i.e., the number of log entries containing the current alarm's warning information), are used to determine the alarm importance coefficient δ based on the relationship between these three data points, obtained through the following formula:

[0117]

[0118] This embodiment provides a method for determining the alarm importance coefficient based on the relationship between the number of warnings, the total number of alarms, and the number of log entries. This facilitates the determination of the server's status level based on the alarm importance coefficient and other coefficients.

[0119] Based on the above embodiments, when the server is in an abnormal state, the current alarm information is output, specifically including:

[0120] The current alarm information will be displayed via SMS or email.

[0121] Alarm messages can be output via a web page. Considering users won't spend extended periods in the server room or near their computers, alarm messages can also be sent to users via SMS or email for timely monitoring and maintenance. A combination of these three methods can also be used; this invention is not specifically limited, but can be configured according to actual needs, as long as it promptly alerts users.

[0122] The present invention provides a method for outputting current alarm information via SMS or email, which improves the user experience and promptly reminds users to check and maintain the system.

[0123] The foregoing has described in detail various embodiments of the log information processing method. Based on this, the present invention also discloses a log information processing apparatus corresponding to the above-described method. Figure 2 This is a structural diagram of a log information processing device provided in an embodiment of the present invention. Figure 2 As shown, the log information processing device includes:

[0124] The acquisition module 11 is used to collect log information and alarm information generated during server operation, wherein the alarm information is reference information provided for server production.

[0125] Mapping module 12 is used to map log information according to hash rules to obtain hash fingerprints;

[0126] Module 13 is used to determine the server's status level based on the relationship between the hash fingerprint and alarm information;

[0127] Output module 14 is used to determine that the server is in an abnormal state and output the current alarm information for the user to view when the server's status level is lower than the preset level.

[0128] Since the embodiments of the device part correspond to the embodiments described above, please refer to the embodiments described in the method part for the embodiments of the device part, and will not be repeated here.

[0129] This invention provides a log information processing device, comprising collecting log information and alarm information generated during server operation, wherein the alarm information serves as reference information provided by the server; mapping the log information to obtain a hash fingerprint according to a hash rule; determining the server's status level based on the relationship between the hash fingerprint and the alarm information; and determining the server's abnormal state and outputting the current alarm information when the server's status level is lower than a preset level. Mapping the log information to obtain a hash fingerprint according to a hash rule, and then determining the server's status level based on the relationship between the hash fingerprint and the alarm information, outputting alarm information when the status level is lower than a preset level, reduces the time required for staff to troubleshoot faults, facilitates fault location, and provides alarm information prompts when problems occur, making it convenient for staff to check and avoiding the need for existing troubleshooting methods based on fragmented log data and related experience, thus reducing losses caused by equipment failures.

[0130] Figure 3 A structural diagram of another log information processing device provided in an embodiment of the present invention is shown below. Figure 3 As shown, the device includes:

[0131] Memory 21 is used to store computer programs;

[0132] Processor 22 is used to implement the steps of a log information processing method when executing a computer program.

[0133] The log information processing device provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0134] The processor 22 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 22 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 22 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 22 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 22 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0135] The memory 21 may include one or more computer-readable storage media, which may be non-transitory. The memory 21 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 21 is used to store at least the following computer program 211, which, after being loaded and executed by the processor 22, is capable of implementing the relevant steps of the log information processing method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, etc., and the storage method may be temporary storage or permanent storage. The operating system 212 may include Windows, Unix, Linux, etc. The data 213 may include, but is not limited to, the data involved in the log information processing method, etc.

[0136] In some embodiments, the log information processing device may further include a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27.

[0137] Those skilled in the field can understand, Figure 3 The structure shown does not constitute a limitation on the log information processing device and may include more or fewer components than shown.

[0138] The processor 22 implements the log information processing method provided in any of the above embodiments by calling instructions stored in the memory 21.

[0139] This invention provides a log information processing device, comprising collecting log information and alarm information generated during server operation, wherein the alarm information serves as reference information provided by the server; mapping the log information to obtain a hash fingerprint according to a hash rule; determining the server's status level based on the relationship between the hash fingerprint and the alarm information; and determining the server's abnormal state and outputting the current alarm information when the server's status level is lower than a preset level. Mapping the log information to obtain a hash fingerprint according to a hash rule, and then determining the server's status level based on the relationship between the hash fingerprint and the alarm information, outputting alarm information when the status level is lower than a preset level, reduces the time required for staff to troubleshoot faults, facilitates fault location, and provides alarm information prompts when problems occur, making it convenient for staff to check and avoiding the need for existing troubleshooting methods based on fragmented log data and related experience, thus reducing losses caused by equipment failures.

[0140] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by processor 22, implements the steps of the log information processing method described above.

[0141] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] For an introduction to the computer-readable storage medium provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above-described log information processing method.

[0143] The above provides a detailed description of a log information processing method, apparatus, and medium provided by the present invention. The various embodiments are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0144] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

Claims

1. A method of processing log information, characterized by, include: Collect log information and alarm information generated during server operation, wherein the alarm information is reference information provided for server production; The log information is mapped according to the hash rules to obtain a hash fingerprint; Specifically, the process includes: obtaining the current information of the log information, wherein the log information includes multiple pieces of the current information; filtering the current information according to the core content field to obtain filtered information; segmenting the filtered information into words to obtain keyword information, and assigning weights according to word frequency to obtain corresponding weight information; mapping the keyword information according to a hash function to obtain hash string information; obtaining corresponding keyword weight information based on the hash string information and the weight information; and merging the keyword weight information to obtain the hash fingerprint. The server's status level is determined based on the relationship between the hash fingerprint and the alarm information; wherein, when there are multiple log messages, the level of each individual log message is determined, and the server's status level is obtained by summarizing them. When the status level of the server is lower than the preset level, the server is determined to be in an abnormal state and the current alarm information is output for the user to view; Correspondingly, determining the server's status level based on the relationship between the hash fingerprint and the alarm information includes: The reference information of the alarm information is mapped according to the hash rule to obtain reference fingerprint information, wherein the reference fingerprint information consists of multiple pieces of information; The hash fingerprint corresponding to the current information is matched with each of the reference fingerprint information to obtain the corresponding similarity information; When the similarity information is greater than the similarity threshold, the current information will be matched with a similarity warning message; The similarity score of the current information is obtained based on each of the aforementioned similarity information. The server's status level is obtained based on the similarity score of the current information; Correspondingly, obtaining the server's status level based on the similarity score of the current information includes: Obtain similar warning information and corresponding alarm importance coefficients for the current information; The server's status level is obtained by weighting and averaging the similarity warning information, the alarm importance coefficient, and the similarity score. Correspondingly, the alarm importance coefficient is determined through the following steps: The number of warnings in the current information, the total number of alarms in the alarm information, and the number of log entries in the log information that contain the warning information of the current information are obtained; The importance coefficient of an alarm is determined based on the relationship between the number of warnings, the total number of alarms, and the number of log entries, specifically including: ； wherein, is the number of alarms for the current information, is the total number of alarms, is the number of log entries.

2. The method of claim 1, wherein, The step of performing similarity matching between the hash fingerprint corresponding to the current information and each of the reference fingerprint information to obtain the corresponding similarity information includes: The hash fingerprint is matched with each of the reference fingerprint information by cosine similarity to obtain the corresponding similarity information.

3. The method of claim 1, wherein, When the server is in an abnormal state, the current alarm information is output, including: The current alarm information will be output via SMS or email.

4. A processing apparatus of log information, characterized by, include: The collection module is used for collecting log information generated by a server runtime and alarm information, wherein the alarm information is reference information provided by the server production; The mapping module is used for mapping the log information according to a hash rule to obtain a hash fingerprint; Specifically, the current information of the log information is obtained, wherein the log information includes a plurality of current information; the current information is filtered according to a core content field to obtain filtered information; the filtered information is segmented to obtain keyword information, and a weight is assigned according to a word frequency to obtain corresponding weight information; the keyword information is mapped according to a hash function to obtain hash string information; the hash string information and the weight information are used to obtain corresponding keyword weight information; and the keyword weight information is combined to obtain the hash fingerprint; The determination module is used for determining a state level of the server according to a relationship between the hash fingerprint and the alarm information; wherein when the log information is a plurality of information, the level of a single log information is determined respectively, and the state level of the server is obtained by summarizing; The output module is used for determining that the server is in an abnormal state and outputting current alarm information for a user to view when the state level of the server is lower than a preset level. Correspondingly, the determination of the state level of the server according to the relationship between the hash fingerprint and the alarm information includes: The reference information of the alarm information is mapped according to the hash rule to obtain reference fingerprint information, wherein the reference fingerprint information is a plurality of information; The hash fingerprint corresponding to the current information is matched with each reference fingerprint information to obtain corresponding similarity information; When the similarity information is greater than a similarity threshold, the current information is matched with similar alarm information; The similarity score of the current information is obtained according to the similarity information; The state level of the server is obtained according to the similarity score of the current information. Correspondingly, the determination of the state level of the server according to the similarity score of the current information includes: The similar alarm information of the current information and the corresponding alarm importance coefficient are obtained; The similar alarm information, the alarm importance coefficient and the similarity score are processed by weighted average to obtain the state level of the server. Correspondingly, the determination of the alarm importance coefficient is through the following steps: The warning times of the current information, the total number of alarms of the alarm information and the number of log information containing the warning information of the current information are obtained; The alarm importance coefficient is determined according to the relationship among the warning times, the total number of alarms and the number of log information, and specifically includes: ； wherein, is the number of alarms for the current information, is the total number of alarms, is the number of log entries.

5. A processing apparatus of log information, characterized by, The memory is used for storing a computer program; The processor is used for executing the computer program to realize the steps of the log information processing method in any one of claims 1 to 3. The computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to realize the steps of the log information processing method in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, ​

Images