Information processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an information processing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring to-be-processed log information and source equipment to which the to-be-processed log information belongs; determining the type classification to which the source equipment belongs; then candidate analysis rules corresponding to the type classification are screened from a target analysis rule set, and the target analysis rule set is an analysis rule set corresponding to the log type to which the log information belongs; and finally, determining a target analysis rule from the candidate analysis rules to analyze the log information to obtain an analysis result. According to the scheme, the parsing rules are customized for different model classifications and different log types, so that the device logs under the same model classification can reuse the same parsing rule, the universality of the parsing rules is improved, and meanwhile, the overall parsing efficiency when different logs are parsed is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an information processing method, device, electronic device, computer storage medium, and computer program product. Background Art

[0002] The BMC (Baseboard Management Controller) is a critical module in the server and plays a vital role in server hardware management and control. It can perform operations such as firmware upgrades and device checks on the machine even when the machine is not powered on. The server baseboard controller records a variety of key logs and other information about the server hardware, including SDR (Sensor Data Record), SEL (Sensor Event Log), and SNMP Trap (Simple Network Management Protocol Trap). The communication protocols, log formats, and log content formats provided by the baseboard management controller vary significantly depending on the server vendor and model.

[0003] The log parsing solutions in related technologies only support parsing the output logs of servers from specific vendors or server manufacturers, and are not universal for other logs, resulting in low processing performance when parsing different logs and poor overall parsing efficiency. Summary of the Invention

[0004] The embodiments of the present application provide an information processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve the overall parsing efficiency when parsing different logs.

[0005] An information processing method, comprising:

[0006] Obtain the log information to be processed and the source device to which it belongs;

[0007] Determine the model category to which the source device belongs;

[0008] Filter candidate parsing rules corresponding to the model classification from a target parsing rule set, wherein the target parsing rule set is a parsing rule set corresponding to the log type to which the log information belongs;

[0009] A target parsing rule is determined from the candidate parsing rules to parse the log information and obtain a parsing result.

[0010] Accordingly, an embodiment of the present application further provides an information processing device, including:

[0011] An acquisition unit, used to obtain the log information to be processed and the source device to which it belongs;

[0012] A determination unit, configured to determine the model category to which the source device belongs;

[0013] a screening unit, configured to screen candidate parsing rules corresponding to the model classification from a target parsing rule set, wherein the target parsing rule set is a parsing rule set corresponding to the log type to which the log information belongs;

[0014] The parsing unit is configured to determine a target parsing rule from the candidate parsing rules to parse the log information and obtain a parsing result.

[0015] Optionally, in some implementations, the parsing unit is specifically configured to:

[0016] Normalizing the log information to obtain normalized log information;

[0017] Determining a corresponding classification label according to the content of the normalized log information;

[0018] Based on the classification label, a target parsing rule is determined from the candidate parsing rules to parse the log information.

[0019] Optionally, in some implementations, when determining a target parsing rule from the candidate parsing rules based on the classification label to parse the log information, the parsing unit is further configured to:

[0020] Prioritizing the candidate parsing rules based on the classification labels to obtain a first priority ranking result;

[0021] According to the first priority sorting result, the corresponding candidate parsing rules are sequentially obtained to parse the log information until the log information is successfully parsed, and the candidate parsing rule that successfully parses the log information is determined as the target parsing rule.

[0022] Optionally, in some implementations, when parsing of the log information based on all the candidate parsing rules fails, the parsing unit is further configured to:

[0023] Acquire other parsing rules except the candidate rules from the target parsing rule set;

[0024] Prioritizing the other parsing rules based on the classification labels to obtain a second priority ranking result;

[0025] According to the second priority sorting result, the corresponding other parsing rules are sequentially obtained to parse the log information until the log information is successfully parsed.

[0026] Optionally, in some embodiments, the parsing unit is further configured to:

[0027] After the log information is successfully parsed based on the other parsing rules, the candidate parsing rules corresponding to the model classification in the target parsing rule set are updated based on the other parsing rules that successfully parsed the log information.

[0028] Optionally, in some implementations, when normalizing the log information to obtain normalized log information, the parsing unit is further configured to:

[0029] extracting key information from the log information based on a preset expression;

[0030] Normalized log information is constructed based on the key information.

[0031] Optionally, in some implementations, the log information is forwarded by the source device through an intermediate device; and the apparatus further includes:

[0032] a generating unit, configured to determine a target parsing rule from the candidate parsing rules, parse the log information, and generate structured data of the log information based on the parsing result after obtaining the parsing result;

[0033] A sending unit is used to forward the structured data to the source device through the intermediate device.

[0034] An embodiment of the present application further provides an electronic device, comprising a processor and a memory, wherein the memory stores an application program, and the processor is configured to run the application program in the memory to execute the steps of any of the above-described information processing methods.

[0035] An embodiment of the present application further provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for loading by a processor to execute the steps in any of the above-mentioned information processing methods.

[0036] An embodiment of the present application further provides a computer program product, comprising a computer program or instructions, which implement the steps of any of the above-mentioned information processing methods when executed by a processor.

[0037] In this application, we first obtain the log information to be processed and the source device to which it belongs; then determine the model category to which the source device belongs; then filter the candidate parsing rules corresponding to the model category from the target parsing rule set, wherein the target parsing rule set is the parsing rule set corresponding to the log type to which the log information belongs; finally, determine the target parsing rule from the candidate parsing rules to parse the log information and obtain the parsing result. In this solution, by customizing parsing rules for different model categories and different log types, the device logs under the same model category can reuse the same parsing rules, which improves the versatility of the parsing rules and improves the overall parsing efficiency when parsing different logs. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 This is a schematic diagram of a scenario of the information processing method provided in an embodiment of the present application;

[0040] Figure 2 This is a flow chart of the information processing method provided by an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the architecture of the unified access module provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the architecture of the log parsing system provided in an embodiment of the present application;

[0043] Figure 5 A flow chart of log structured processing provided in an embodiment of the present application;

[0044] Figure 6 A schematic diagram of the process of parsing rule matching provided in an embodiment of the present application;

[0045] Figure 7 Another flow chart of log structured processing provided in an embodiment of the present application;

[0046] Figure 8 is a schematic structural diagram of an information processing device provided in an embodiment of the present application;

[0047] Figure 9 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0049] Related log parsing solutions, such as those that analyze log information using key-value pairs and store information in a tree structure, can only parse output logs from servers of specific vendors or manufacturers. These solutions only address SNMP Trap logs and lack a universal solution for other types of server logs. Furthermore, the tree structure is suitable for SNMP Trap logs but not for other types of logs, potentially leading to high storage costs and low processing performance.

[0050] The embodiments of the present application provide an information processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve information processing efficiency. The information processing apparatus can be integrated into an electronic device, which can be a server, a terminal, or other device.

[0051] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services, and big data and artificial intelligence platforms. The terminal can be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a car terminal, an aircraft, etc., but is not limited to these. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and this application does not limit this.

[0052] For example, see Figure 1 Taking the information processing device integrated into an electronic device as an example, the electronic device first obtains the log information to be processed and the source device to which it belongs; then determines the model category to which the source device belongs; then filters the candidate parsing rules corresponding to the model category from the target parsing rule set, where the target parsing rule set is the parsing rule set corresponding to the log type to which the log information belongs; finally, determines the target parsing rule from the candidate parsing rules to parse the log information and outputs the parsing result.

[0053] It can be understood that in the specific implementation of this application, related data such as attribute data, attribute sets and attribute subsets are involved. When the following embodiments of this application are applied to specific products or technologies, permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0054] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0055] This embodiment will be described from the perspective of an information processing device. The information processing device can be specifically integrated into an electronic device, which can be a server or a terminal. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), or other smart devices that can perform data processing.

[0056] An embodiment of the present application provides an information processing method, including: obtaining log information to be processed and the source device to which it belongs; determining the model category to which the source device belongs; screening candidate parsing rules corresponding to the model category from a target parsing rule set, wherein the target parsing rule set is a parsing rule set corresponding to the log type to which the log information belongs; determining a target parsing rule from the candidate parsing rules to parse the log information to obtain a parsing result.

[0057] like Figure 2 As shown, the specific process of the information processing method is as follows:

[0058] 101. Obtain the log information to be processed and the source device to which it belongs.

[0059] In the embodiments of the present application, the log information to be processed can be the original log of a server or the original log of a terminal; the source device represents the device from which the log information originates. For example, if the log information is the original log of a server, the source device is the server; if the log information is the original log of a terminal, the source device is the terminal.

[0060] In order to facilitate the unified management of log information, this application proposes a unified access solution for different log information from different source devices. Specifically, multiple control servers can be set up to collect log information in a distributed manner. Each control server collects log information from the devices under its control, processes and aggregates the collected log information, and transmits it to the log analysis service, thereby avoiding the huge network overhead caused by all source devices reporting logs separately. That is, in this embodiment, log information can be obtained by the source device through forwarding by the control server.

[0061] During the specific implementation process, when the management and control server processes and aggregates log information, it can attach collection information to each original log as a mark of the original log, such as the device identification of the source device and the collection time of the log information, to facilitate subsequent queries or statistics; in addition, multiple log data can be combined into one or more requests. When the log data packet reaches a certain size or reaches a certain time, it is transmitted to the message queue of the operation database access layer (such as Kafka message queue) for processing.

[0062] Specifically, when obtaining the log information to be processed and its source device, this embodiment can extract data from the message queue of the operational database access layer, including the original log and the corresponding collection information. Therefore, the source device to which the log information belongs can be obtained based on the extracted collection information.

[0063] 102. Determine the model category to which the source device belongs.

[0064] Specifically, machine model classification can be based on the specific hardware models and specifications of the equipment for different application scenarios. For example, taking a server as an example, when it is necessary to apply to multiple scenarios such as server high temperature abnormality alarms, server hardware abnormality detection and event query, and server downtime diagnosis, the server fault or abnormality detection can be achieved through the log information of the server baseboard management controller (i.e., BMC log). For different server vendors, the communication protocols, log formats, and log content formats provided by the baseboard management controller vary greatly. Therefore, when classifying models in this solution, the classification can be based on the similarity between the models and specifications of the baseboard management controllers in each server.

[0065] In actual applications, models from the same server vendor generally use the same or similar baseboard management controllers and can be classified into the same model category. However, models from different server vendors may also use the same or similar baseboard management controllers. Therefore, these server vendor models that use the same or similar baseboard management controllers can also be classified into the same model category.

[0066] It should be noted that, when classifying machine models in the embodiments of the present application, the classification can be done in a single-level classification or in a multi-level classification based on the complexity of the actual machine models in the field and in combination with the actual application scenario requirements. A multi-level classification can include multiple machine model categories, each of which includes multiple machine model subcategories at the next level, each of which can also include a further subcategories at the next level, and so on.

[0067] 103. Filter candidate parsing rules corresponding to the machine model classification from a target parsing rule set, wherein the target parsing rule set is a parsing rule set corresponding to the log type to which the log information belongs.

[0068] Specifically, log information can be classified in advance to obtain multiple different log types. For each log type, a corresponding parsing rule set is pre-configured. The configured parsing rule set includes a variety of parsing rules that can analyze log information under this log type, and there are more or less differences between each parsing rule. In this embodiment, for the acquired log information, the model classification of the source device is first determined, and then the log type to which the log information belongs is determined, and the target parsing rule set corresponding to the log type is obtained. Finally, based on the determined model classification, one or more matching parsing rules are obtained from the target parsing rule set as candidate parsing rules to implement the initial screening of the parsing rules corresponding to the model classification.

[0069] It should be noted that, in this embodiment, a mapping relationship between the model classification and the parsing rules needs to be pre-built. When executing rule screening, the matching candidate parsing rules are screened out from the target parsing rule set through the mapping relationship.

[0070] In the embodiment of the present application, the log type is the classification of log information in the relevant application scenario, and the log type can be divided based on the actual scenario involved. For example, taking the BMC log that records the server hardware management information as an example, it can record the CPU temperature, fan speed, power status, etc. The BMC log is generally stored in the server's BIOS chip or a dedicated BMC chip, and the administrator can obtain the BMC log information through the IPMI (Intelligent Platform Management Interface) protocol, Refish protocol or other management tools. In the specific implementation process, the BMC log can be divided into multiple log types such as SDR logs, SEL logs, SNMP Trap logs, etc. based on different protocol divisions, and an independent parsing matching table can be configured for each log type such as SDR logs, SEL logs, SNMP Trap logs, etc., to record the mapping relationship between the model classification and the parsing rule under each log type.

[0071] Among them, the SDR log is mainly used for active and regular information collection, collecting the probe data of each component on the motherboard, and then judging the basic information and health status information of each component, and using it in combination with the actual server operation needs. The SEL log mainly records and reports key events after the probes of each component on the motherboard collect data. SNMP belongs to the application layer protocol in the TCP / IP five-layer protocol and is a protocol used for network management. SNMP Trap is part of SNMP. When a specific event occurs in the observed segment (such as performance problems, network device interface interruption, etc.), the agent will send an alarm event to the management station.

[0072] In server management, BMC logs are a crucial information source, helping to address server hardware failures. For example, in server overheating alarm scenarios, server SDR logs can be parsed to obtain information such as CPU temperature, GPU temperature, and air inlet and outlet temperatures for each server model. These temperature values ​​can then be passed to downstream observation services, which can then configure alarms, enabling real-time awareness of server overheating anomalies.

[0073] For example, in the server hardware anomaly detection scenario, this application can determine what abnormal problems the server has reported by collecting and parsing the SNMP Trap logs reported by each model. The abnormal event content can be passed to the downstream hardware anomaly detection service. If the service determines that the abnormal event may affect the server, it can issue an alarm to enable timely handling of the abnormal problem.

[0074] 104. Determine a target parsing rule from the candidate parsing rules to parse the log information and obtain a parsing result.

[0075] Specifically, the best target parsing rule can be matched from the selected candidate parsing rules to parse the log information, thereby obtaining a parsing result. In this embodiment, in order to facilitate unified processing, the log information can be normalized. Then, in order to improve the performance of subsequent matching rules, classification marking can be performed based on the normalized log. That is, in one embodiment, determining the target parsing rule from the candidate parsing rules to parse the log information can include the following process:

[0076] Normalize the log information to obtain normalized log information;

[0077] Determine the corresponding classification label based on the content of the normalized log information;

[0078] Based on the classification labels, the target parsing rules are determined from the candidate parsing rules to parse the log information.

[0079] Specifically, when performing normalization processing, key information in the log information can be disassembled by pre-set rules. That is, in one embodiment, normalization processing is performed on the log information to obtain normalized log information, which can include the following process:

[0080] Extract key information from log information based on preset expressions;

[0081] Construct normalized log information based on key information.

[0082] Specifically, you can extract key information from the raw log by setting a preset expression. The preset expression specifies the key information to be extracted. For example, the preset expression can be a regular expression. For the raw log "Inlet_Temp0|08h|ok|3.0|22degrees C", you can construct a regular expression based on actual needs to extract the sensor ID information "Inlet_Temp0" in the first column, the standard status information "ok" in the third column, and the sensor reading status information "22degrees C" in the fifth column. The extracted information is the normalized log.

[0083] Specifically, when determining the corresponding classification label based on the content of the normalized log information, taking the normalized log extracted above as an example, a string matching judgment can be performed on the sensor ID information in the log. If it contains the word "Temp", the label classification of the log information corresponding to the normalized log can be determined as the temperature category and classified and marked.

[0084] In one embodiment, when determining a target parsing rule from candidate parsing rules based on the classification label to parse log information, the following process may be included:

[0085] Prioritize the candidate parsing rules based on the classification labels to obtain a first priority ranking result;

[0086] According to the first priority sorting result, corresponding candidate parsing rules are sequentially obtained to parse the log information until the log information is successfully parsed, and the candidate parsing rule that successfully parses the log information is determined as the target parsing rule.

[0087] Specifically, in this embodiment, the classification labels of each normalized log information can be matched with candidate rules, a higher priority can be set for candidate rules with a high degree of matching, and a lower priority can be set for candidate rules with a low degree of matching. The candidate parsing rules can also be sorted in descending order of priority to obtain a first priority sorting result. Then, according to the first priority sorting result, the highest priority candidate parsing rule is first obtained to parse the log information. If the parsing is successful, the matching of subsequent candidate parsing rules is stopped, and the highest priority candidate parsing rule is determined as the target parsing rule. If the parsing fails, the next priority candidate parsing rule is obtained to parse the log information, and so on, until the log information is successfully parsed.

[0088] In the embodiment of the present application, the parsing rules can be implemented based on a variety of different methods that can achieve matching effects, such as regular parsing, Grok expressions, string matching and other methods. For example, taking regular parsing as an example, if the original log information is "Inlet_Temp0", it represents the air inlet temperature sensor No. 0; the regular expression of an air inlet temperature matching rule is "Inlet_Temp(\d+)", which means matching the "Inlet_Temp" string + an integer of at least 1 digit, and capturing this integer value. If the match is successful, the next step of information matching will be carried out, such as matching the standard status information and the sensor reading value. It is determined whether the rule requires reading a continuous value (reading) or a discrete value (status). If it is a continuous value, the reading value regular expression is used to match the reading value information. If it is a discrete value, the reading value regular expression is used to match the standard status information. If the match fails at the beginning or the match fails in the middle, other parsing rules are obtained for parsing.

[0089] The rule determines whether to read a continuous value (reading) or a discrete value (status). If the value is continuous, the reading value regular expression is used to match the reading value information. If the value is discrete, the reading value regular expression is used to match the standard status information. As shown in the figure, the "(\d+)\s*degrees C" rule represents an integer of at least one digit, any number of spaces, and the string "degrees C," and captures this integer. This matches the reading "22degrees C," and the regular expression captures the sensor value "22," successfully parsing a log message.

[0090] In specific implementation, if all the candidate parsing rules screened out fail to successfully parse the log information, other parsing rules in the target parsing rule can be obtained to parse the log information. That is, in one embodiment, when parsing the log information based on all candidate parsing rules fails, the following process can also be included:

[0091] Obtain other parsing rules except candidate rules from the target parsing rule set;

[0092] Prioritize other parsing rules based on the classification label to obtain a second priority sorting result; according to the second priority sorting result, obtain corresponding other parsing rules in sequence to parse the log information until the log information is successfully parsed.

[0093] Specifically, after determining that none of the candidate parsing rules can successfully parse the log information, the classification label of the normalized log information can be used to match the other parsing rules in the target parsing rule set except the candidate rules. A higher priority can be set for other rules with a high degree of match, and a lower priority can be set for other rules with a low degree of match. The other parsing rules can be sorted in descending order of priority to obtain a second priority sorting result. Then, according to the second priority sorting result, the other parsing rules with the highest priority are first obtained to parse the log information. If the parsing is successful, the matching of subsequent other parsing rules is stopped, and the other parsing rules with the highest priority are determined as the target parsing rules. If the parsing fails, other parsing rules with the next priority level are continued to be obtained to parse the log information, and so on, until the log information is successfully parsed.

[0094] In one embodiment, after successfully parsing the log information based on other parsing rules, the candidate parsing rules corresponding to the model classification in the target parsing rule set can also be updated based on the other parsing rules that successfully parsed the log information, so as to establish a mapping relationship between the other parsing rules for parsing the log information in the source device and the model classification to which it belongs, and save the newly established mapping relationship to the parsing matching table corresponding to the log type to which the log information belongs, thereby improving the reliability of the parsing matching table and improving the parsing efficiency of subsequent log information.

[0095] In one embodiment, if the log information parsing ultimately fails, the log data will be transferred to the operation database for analysis of the failure cause.

[0096] In one embodiment, if the log information is forwarded by the source device through an intermediate device (such as a distributed management and control server), after determining the target parsing rule from the candidate parsing rules to parse the log information and obtaining the parsing result, the following process may also be included:

[0097] Generate structured data of log information based on the parsing results;

[0098] Forward structured data to the source device through the intermediate device.

[0099] Specifically, logs that are successfully parsed according to the parsing rules are structured logs. The information will be aggregated and forwarded to the message queue. The structured data will be forwarded to the source device through the intermediate device (such as the corresponding management and control server) for in-depth processing in scenarios such as log anomaly detection and hardware timing observation of downstream devices.

[0100] As can be seen from the above, the information processing method provided by the embodiment of the present application obtains the log information to be processed and the source device to which it belongs; then determines the model classification to which the source device belongs; then filters the candidate parsing rules corresponding to the model classification from the target parsing rule set, wherein the target parsing rule set is the parsing rule set corresponding to the log type to which the log information belongs; finally, determines the target parsing rule from the candidate parsing rules to parse the log information and obtain the parsing result. In this solution, by customizing parsing rules for different model classifications and different log types, the device logs under the same model classification can reuse the same parsing rules, which improves the versatility of the parsing rules and improves the overall parsing efficiency when parsing different logs.

[0101] In another embodiment of the present application, a log parsing method is provided. In this embodiment, the log parsing method can be applied to the log parsing of a server baseboard management controller. The log parsing method provided in this embodiment will be described in detail using the BMC log information to be parsed as an example.

[0102] First, due to different protocols, each server may generate BMC logs in various forms. For example, SDR logs and SEL logs require IPMI tools to actively execute command collection to return, while SNMP Trap logs require SNMP services and can only be obtained by the monitored server actively issuing alarm information. Therefore, this solution proposes a unified access solution for the original logs of each server. For details, refer to Figure 3 , Figure 3 This is a schematic diagram of the architecture of the unified access module provided in the embodiment of this application. Figure 3 As shown, by configuring a control server to centrally collect BMC log information from multiple servers, the information is then consolidated and forwarded to the message queue of the server operations database for processing. In addition to collecting BMC logs from servers under its control, the control server can also process and aggregate BMC log information. Specifically, each raw BMC log is accompanied by the corresponding SN (server serial number) information and collection time. Multiple log data items are then combined into one or more requests and sent to the Kafka message queue at the access layer. This significantly reduces the network overhead caused by separate reporting.

[0103] refer to Figure 4 , Figure 4 This is a schematic diagram of the architecture of the log parsing system provided in the embodiment of this application. Figure 4 As shown, this solution technically consists of a log parsing module whose input is BMC hardware logs of different types from different servers. First, the BMC logs are packaged and integrated through unified data access and transmitted to the log parsing service. Then, through log data extraction, key information carried in the logs (such as collection time and server serial number) is parsed, and the original BMC logs are obtained. Different log types are then fed into different data structuring plug-ins. Finally, the logs are parsed based on the corresponding rules.

[0104] Continue to refer Figure 4 The function of the log data extraction module is to extract data from the Kafka message queue of the server operation database access layer, extract data from different queue topics, and transfer it to different data structuring plug-ins in the back to process different types of logs separately, making it easier to decouple the log structuring module in the form of a plug-in.

[0105] Continue to refer Figure 4 ,The function of the log structured plug-in is to parse the original hardware log information and ,extract key information after the upstream log data extraction ,module flows to different log structured plug-ins according to different log types.

[0106] refer to Figure 5 , Figure 5 The following is a flow chart of log structure processing provided by the embodiment of this application. Taking SDR log as an example, the specific implementation logic of log structure is explained. Figure 5 As shown, first obtain the SDR original log data extracted from the log extraction module and the accompanying server serial number information, which can be specifically shown as follows:

[0107] Server SN: 12345

[0108] Collection time: 2024 / 01 / 01 01:01:01

[0109] Write time: 2024 / 01 / 01 01:02:02

[0110] Inlet_Temp0|08h|ok|3.0|22degrees C;

[0111] Outlet_Temp|05h|ok|3.0|36degrees C;

[0112] CPU0_Temp|17h|ok|3.3|57degrees C;

[0113] CPU1_Temp|18h|ok|3.3|50degrees C;

[0114] …

[0115] Then, based on the server serial number, the server's model is queried from the model table stored in the CMDB (Configuration Management Database) or Redis configuration file. After obtaining the model, the SDR standard parsing matching table is used to obtain the parsing rules that meet the requirements for the model, and the parsing rules to be matched are selected.

[0116] On the other hand, for the original SDR logs, in order to facilitate unified processing, log normalization will be performed. Specifically, for protocol formats such as IPMI / Redfish, the key information of the log data is disassembled. For example, if the original log is "Inlet_Temp0|08h|ok|3.0|22degrees C", a regular expression can be used to extract the sensor ID information "Inlet_Temp0" in the first column, the standard status information "ok" in the third column, and the sensor reading value status information "22degrees C" in the fifth column. The extracted information is called a normalized log. In addition, in order to improve the performance of subsequent matching rules, labeling will be performed on the basis of the normalized log, that is, coarse classification. Specifically, a string matching judgment is performed on the sensor ID information. If it contains the word "Temp", it will be classified as a temperature class. Taking the temperature class as an example, the normalized log data obtained can be shown as follows:

[0117] Temperature class+Inlet_Temp0+ok+22degrees C;

[0118] Temperature+Outlet_Temp+ok+36degrees C;

[0119] Temperature+CPU0_Temp+ok+57degrees C;

[0120] Temperature+CPU1_Temp+ok+50degrees C;

[0121] …

[0122] During subsequent matching, normalized temperature logs will be prioritized against the temperature matching rules, which reduces the number of matching rules and improves matching efficiency.

[0123] After the above preprocessing, a series of matching rules and a section of normalized log data are obtained. Then, the step of actually matching the structured data is carried out, which is called sub-classification. Figure 6 ,, Figure 6 The flowchart of the parsing rule matching provided in the embodiment of the present application is shown in FIG. Figure 6 As shown, the coarse classification labeling of the normalized log will be judged first, and the rules to be matched will be arranged according to the coarse classification. The rules arranged in order of priority within the coarse classification will be matched first, and then the rules arranged in order of priority outside the coarse classification will be matched. The matching method is to match the sensor ID information through regular expressions. For example, for the information "Inlet_Temp0", there is an inlet temperature matching rule with a regular expression of "Inlet_Temp(\d+)" which is a successful match. If the match is successful, the next step will be to match the standard status information and the sensor reading value. Determine whether the rule requires reading continuous values ​​or discrete values. If it is a continuous value (reading), the reading value regular expression is used to match the reading value information; if it is a discrete value (status), the reading value regular expression is used to match the standard status information. Specifically, as Figure 6 As shown, the "(\d+)\s*degrees C" rule can be used to match the reading "22 degrees C". Finally, the sensor value "22" is captured by the regular expression, and a log message is successfully parsed. The format of the structured log data obtained can be specifically shown as follows:

[0124] Server SN+Model+Model Classification

[0125] Collection type + collection definition + collection definition ID

[0126] Inlet_Temp0+22;

[0127] Outlet_Temp+36;

[0128] Power_status+0;

[0129] …

[0130] Specifically, if a sensor reading fails to match, it's likely because the reading is in text format, not a specific number. For example, the air inlet temperature reading might be "No Reading" or simply contain no text. In this case, a text-to-digital conversion step is required to convert the text into a numerical value to facilitate downstream sensor observation. Therefore, we've established an SDR text-to-digital dictionary to map text that can't be directly converted to a numerical value. For example, "No Reading" is converted to the value -6. This allows downstream users to determine if a no-reading anomaly has occurred simply by observing the sensor reading for -6.

[0131] In the log parsing plug-in, successfully parsed logs are structured logs, which are aggregated and forwarded to the message queue for in-depth processing in downstream scenarios such as log anomaly detection and hardware timing observation. If the sub-classification step fails, the data is transferred to the server operation database for analysis of the failure cause.

[0132] In one embodiment, for some types of hardware logs, if the model classification is more complex, in order to quickly reuse the matching rules of different models, the concept of model categories is introduced, and different models are classified using the model category mapping table. In this way, a rule can be shared by models of the same model category, which improves the configuration flexibility. The following takes the log structure plug-in of SNMP Trap log as an example to briefly describe the differences with the SDR log parsing process. Figure 7 , Figure 7 Another flowchart of log structured processing provided in an embodiment of the present application.

[0133] like Figure 7 As shown, in this embodiment, in the log parsing step, for the obtained server hardware log, first, based on the model category mapping table stored in the MySQL database, the model or pre-defined model category of the machine is determined; then, according to the protocol and model or model category corresponding to the log, combined with the SNMP Trap standard parsing matching table stored in the MySQL database, the log parsing rules to be matched are filtered out from the corresponding parsing rule set.

[0134] After rule matching is complete, the system attempts to parse log entries using these rules one by one, sorted by rule priority. Specifically, it matches the Trap name and Trap 9 fields, which include information such as the sensor and its description. After successful parsing, it uses text digitization to convert portions of the log text into observable values. This ultimately yields structured log data for use in scenarios such as log data observation.

[0135] Compared to SDR logs, the SNMP Trap log plug-in introduces the aforementioned model category functionality. The server SN first queries the database for model information and then determines the model category to which the model belongs in the model category mapping table. When filtering matching rules, it selects based on the model category or a combination of model and model category, providing greater flexibility and avoiding duplicate configuration for multiple models.

[0136] It should be noted that the technical solutions provided in the embodiments of this application use SDR logs and SNMP Trap logs as examples to describe the process for parsing server baseboard management controller logs. The parsing rules are not limited to regular expression parsing; methods such as Grok expressions and string matching can also achieve the same matching effect. Log formats are not limited to SDR logs; server in-band logs can also use a similar process to detect abnormal log text.

[0137] The log parsing method provided by the embodiment of the present application has improvements in system architecture, adaptation and maintenance, and log support. At the system architecture level, the log matching speed is improved. Specifically, the system deployment of this solution can support concurrent processing of billions of BMC log information per day, with significant performance optimization. For example, in an environment with an 8-core CPU and 16GB of memory, the node processing capacity can reach 50 million logs per day, while conventional technology can only process one million per day, greatly improving the processing speed.

[0138] At the adaptation and maintenance level, the matching logic is clearer. Specifically, the table configuration method is used to connect to the WEB page to provide a user configuration interface, which improves configuration flexibility.

[0139] In terms of log support, it offers greater compatibility. For example, SDR logs can simultaneously support protocols such as IPMI and Redfish, normalizing logs and providing support for emerging technology stacks. To add different logs, simply add a new log structured plug-in to quickly reuse parsing capabilities and support different log data observation scenarios.

[0140] In order to better implement the above method, an embodiment of the present application also provides an information processing device, which can be integrated into an electronic device, such as a server or terminal, and the terminal may include a tablet computer, a laptop computer and / or a personal computer, and other smart terminal devices.

[0141] For example, Figure 8 As shown, the information processing device may include: an acquisition unit 301, a determination unit 302, a screening unit 303, and a parsing unit 304, as follows:

[0142] The acquisition unit 301 is used to obtain the log information to be processed and the source device to which it belongs;

[0143] A determination unit 302 is configured to determine the model category to which the source device belongs;

[0144] A screening unit 303 is configured to screen candidate parsing rules corresponding to the model classification from a target parsing rule set, wherein the target parsing rule set is a parsing rule set corresponding to the log type to which the log information belongs;

[0145] The parsing unit 304 is configured to determine a target parsing rule from the candidate parsing rules and parse the log information to obtain a parsing result.

[0146] Optionally, in some implementations, the parsing unit 304 is specifically configured to:

[0147] Normalizing the log information to obtain normalized log information;

[0148] Determine a corresponding classification label according to the content of the normalized log information;

[0149] Based on the classification label, a target parsing rule is determined from the candidate parsing rules to parse the log information.

[0150] Optionally, in some implementations, when determining a target parsing rule from the candidate parsing rules based on the classification label to parse the log information, the parsing unit 304 is further configured to:

[0151] Prioritize the candidate parsing rules based on the classification label to obtain a first priority ranking result;

[0152] According to the first priority sorting result, the corresponding candidate parsing rules are sequentially obtained to parse the log information until the log information is successfully parsed, and the candidate parsing rule that successfully parses the log information is determined as the target parsing rule.

[0153] Optionally, in some implementations, when parsing of the log information based on all the candidate parsing rules fails, the parsing unit 304 is further configured to:

[0154] Obtaining other parsing rules except the candidate rule from the target parsing rule set;

[0155] Prioritize the other parsing rules based on the classification label to obtain a second priority ranking result;

[0156] According to the second priority sorting result, the corresponding other parsing rules are sequentially obtained to parse the log information until the log information is successfully parsed.

[0157] Optionally, in some implementations, the parsing unit 304 is further configured to:

[0158] After the log information is successfully parsed based on the other parsing rule, the candidate parsing rule corresponding to the model classification in the target parsing rule set is updated based on the other parsing rule that successfully parsed the log information.

[0159] Optionally, in some implementations, when normalizing the log information to obtain normalized log information, the parsing unit 304 is further configured to:

[0160] Extract key information from the log information based on a preset expression;

[0161] Normalized log information is constructed based on the key information.

[0162] Optionally, in some implementations, the log information is forwarded by the source device through an intermediate device; the apparatus further includes:

[0163] a generating unit, configured to determine a target parsing rule from the candidate parsing rules, parse the log information, and generate structured data of the log information based on the parsing result after obtaining the parsing result;

[0164] A sending unit is used to forward the structured data to the source device through the intermediate device.

[0165] As can be seen from the above, the information processing device provided by the embodiment of the present application obtains the log information to be processed and the source device to which it belongs; then determines the model classification to which the source device belongs; then filters the candidate parsing rules corresponding to the model classification from the target parsing rule set, wherein the target parsing rule set is the parsing rule set corresponding to the log type to which the log information belongs; finally, the target parsing rule is determined from the candidate parsing rules to parse the log information and obtain the parsing result. In this solution, by customizing parsing rules for different model classifications and different log types, the device logs under the same model classification can reuse the same parsing rules, which improves the versatility of the parsing rules and improves the overall parsing efficiency when parsing different logs.

[0166] The present application also provides an electronic device, such as Figure 9 , which shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0167] The electronic device may include one or more processing core processors 401, one or more computer-readable storage media memories 402, a power supply 403, an input unit 404 and other components. Those skilled in the art will understand that Figure 9 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0168] Processor 401 is the control center of the electronic device. It connects all parts of the electronic device using various interfaces and circuits. It performs various functions of the electronic device and processes data by running or executing software programs and / or modules stored in memory 402 and accessing data stored in memory 402. Optionally, processor 401 may include one or more processing cores. Preferably, processor 401 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 401.

[0169] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and information processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0170] The electronic device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 403 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0171] The electronic device may further include an input unit 404, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0172] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402 to implement various functions as follows:

[0173] Obtain the log information to be processed and the source device to which it belongs; determine the model category to which the source device belongs; filter candidate parsing rules corresponding to the model category from the target parsing rule set, where the target parsing rule set is the parsing rule set corresponding to the log type to which the log information belongs; determine the target parsing rule from the candidate parsing rules to parse the log information and obtain the parsing result.

[0174] In one embodiment, when determining a target parsing rule from the candidate parsing rules to parse the log information, the processor 401 is specifically configured to:

[0175] The log information is normalized to obtain normalized log information; a corresponding classification label is determined according to the content of the normalized log information; and based on the classification label, a target parsing rule is determined from the candidate parsing rules to parse the log information.

[0176] In one embodiment, when determining a target parsing rule from the candidate parsing rules based on the classification label to parse the log information, the processor 401 is further configured to:

[0177] The candidate parsing rules are prioritized based on the classification label to obtain a first priority sorting result; according to the first priority sorting result, the corresponding candidate parsing rules are obtained in turn to parse the log information until the log information is successfully parsed, and the candidate parsing rule that successfully parses the log information is determined as the target parsing rule.

[0178] In one embodiment, when parsing of the log information based on all the candidate parsing rules fails, the processor 401 is further configured to:

[0179] Obtain other parsing rules except the candidate rule from the target parsing rule set; prioritize the other parsing rules based on the classification label to obtain a second priority sorting result; according to the second priority sorting result, obtain the corresponding other parsing rules in turn to parse the log information until the log information is successfully parsed.

[0180] In one embodiment, after successfully parsing the log information based on the other parsing rule, the processor 401 is further configured to update the candidate parsing rule corresponding to the model classification in the target parsing rule set based on the other parsing rule that successfully parsed the log information.

[0181] In one embodiment, when normalizing the log information to obtain normalized log information, the processor 401 is further configured to:

[0182] Extract key information from the log information based on a preset expression; and construct normalized log information based on the key information.

[0183] In one embodiment, the log information is forwarded by the source device through an intermediate device; after determining a target parsing rule from the candidate parsing rules to parse the log information and obtain a parsing result, the processor 401 is further configured to:

[0184] The structured data of the log information is generated based on the parsing result; and the structured data is forwarded to the source device through the intermediate device.

[0185] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0186] From the above, it can be seen that in the embodiment of the present application, the electronic device obtains the log information to be processed and the source device to which it belongs; then determines the model classification to which the source device belongs; then filters the candidate parsing rules corresponding to the model classification from the target parsing rule set, wherein the target parsing rule set is the parsing rule set corresponding to the log type to which the log information belongs; finally, the target parsing rule is determined from the candidate parsing rules to parse the log information and obtain the parsing result. In this solution, by customizing parsing rules for different model classifications and different log types, the device logs under the same model classification can reuse the same parsing rules, which improves the versatility of the parsing rules and improves the overall parsing efficiency when parsing different logs.

[0187] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0188] To this end, an embodiment of the present application provides a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any of the information processing methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0189] Obtain the log information to be processed and the source device to which it belongs; determine the model category to which the source device belongs; filter candidate parsing rules corresponding to the model category from the target parsing rule set, where the target parsing rule set is the parsing rule set corresponding to the log type to which the log information belongs; determine the target parsing rule from the candidate parsing rules to parse the log information and obtain the parsing result.

[0190] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0191] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0192] Since the instructions stored in the computer-readable storage medium can execute the steps in any information processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any information processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0193] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the aforementioned information processing aspects.

[0194] The above is a detailed introduction to an information processing method, device, electronic device, computer-readable storage medium and computer program product provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An information processing method, characterized in that: include: Obtain the log information to be processed and the source device to which it belongs; Determine the model category to which the source device belongs; Filter candidate parsing rules corresponding to the model classification from a target parsing rule set, wherein the target parsing rule set is a parsing rule set corresponding to the log type to which the log information belongs; A target parsing rule is determined from the candidate parsing rules to parse the log information and obtain a parsing result.

2. The information processing method according to claim 1, wherein: Determining a target parsing rule from the candidate parsing rules to parse the log information includes: Normalizing the log information to obtain normalized log information; Determining a corresponding classification label according to the content of the normalized log information; Based on the classification label, a target parsing rule is determined from the candidate parsing rules to parse the log information.

3. The information processing method according to claim 2, wherein: The step of determining a target parsing rule from the candidate parsing rules based on the classification label to parse the log information includes: Prioritizing the candidate parsing rules based on the classification labels to obtain a first priority ranking result; According to the first priority sorting result, the corresponding candidate parsing rules are sequentially obtained to parse the log information until the log information is successfully parsed, and the candidate parsing rule that successfully parses the log information is determined as the target parsing rule.

4. The information processing method according to claim 3, wherein: Also includes: When parsing of the log information based on all the candidate parsing rules fails, obtaining other parsing rules except the candidate rules from the target parsing rule set; Prioritizing the other parsing rules based on the classification labels to obtain a second priority ranking result; According to the second priority sorting result, the corresponding other parsing rules are sequentially obtained to parse the log information until the log information is successfully parsed.

5. The information processing method according to claim 4, characterized in that After the log information is successfully parsed based on the other parsing rules, the method further includes: Based on the other parsing rules that successfully parse the log information, the candidate parsing rules corresponding to the model classification in the target parsing rule set are updated.

6. The information processing method according to claim 2, wherein: The normalizing the log information to obtain normalized log information includes: extracting key information from the log information based on a preset expression; Normalized log information is constructed based on the key information.

7. The information processing method according to any one of claims 1 to 6, characterized in that: The log information is forwarded by the source device through the intermediate device; After determining a target parsing rule from the candidate parsing rules to parse the log information and obtain a parsing result, the method further includes: Generating structured data of the log information based on the analysis result; The structured data is forwarded to the source device through the intermediate device.

8. An information processing device, characterized in that include: An acquisition unit, used to obtain the log information to be processed and the source device to which it belongs; A determination unit, configured to determine the model category to which the source device belongs; a screening unit, configured to screen candidate parsing rules corresponding to the model classification from a target parsing rule set, wherein the target parsing rule set is a parsing rule set corresponding to the log type to which the log information belongs; The parsing unit is configured to determine a target parsing rule from the candidate parsing rules to parse the log information and obtain a parsing result.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores an application program, and the processor is configured to run the application program in the memory to execute the steps of the information processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps of the information processing method according to any one of claims 1 to 7.

Images