Log processing method and device, equipment, medium and product

Abstract

The invention relates to the technical field of computers, and discloses a log processing method and device, equipment, a medium and a product. The method comprises the steps of obtaining initial transaction logs according to field extraction rule configuration, and performing log merging according to timestamps corresponding to the initial transaction logs to obtain merged transaction logs; according to the field extraction rule configuration, obtaining a regular analysis rule, a content replacement rule and a character string conversion rule, and splitting the merged transaction log according to the regular analysis rule to obtain a log body; performing specified content replacement on the log body according to a content replacement rule to obtain a hexadecimal log message, and performing code conversion on the hexadecimal log message according to a character string conversion rule to obtain a log message in an extensible markup language format; and carrying out key element extraction on the log message in the extensible markup language format to obtain a key element extraction result. According to the scheme of the embodiment, the log processing efficiency can be improved, and the log readability can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a log processing method, device, equipment, medium and product. Background Art

[0002] Humans' inherent reading habits make it difficult to understand the hexadecimal messages in the logs, resulting in extremely poor reading results.

[0003] At present, the existing log processing methods usually use pre-compiled scripts or online escape tools provided on the Internet to parse hexadecimal log messages. However, pre-compiled scripts are inefficient for batch log parsing, and when encountering different styles of hexadecimal logs, the scripts need to be significantly modified for adaptation; while the use of online escape tools is even less efficient and is only suitable for single log processing needs, and cannot be applied to large-scale log analysis and monitoring. Summary of the invention

[0004] The present invention provides a log processing method, device, equipment, medium and product, which can improve log processing efficiency and log readability.

[0005] According to one aspect of the present invention, a log processing method is provided, comprising:

[0006] According to the field extraction rule configuration, the initial transaction log of the enterprise service bus system is obtained, and the logs are merged according to the timestamps corresponding to the initial transaction logs to obtain the merged transaction log;

[0007] According to the field extraction rule configuration, a regular parsing rule, a content replacement rule and a string conversion rule are obtained, and the merged transaction log is split according to the regular parsing rule to obtain a log body;

[0008] Performing designated content replacement on the log body according to the content replacement rule to obtain a hexadecimal log message, and performing encoding conversion on the hexadecimal log message according to the string conversion rule to obtain an extensible markup language format log message;

[0009] Key elements are extracted from the log message in the extensible markup language format to obtain a key element extraction result.

[0010] According to another aspect of the present invention, there is provided a log processing device, comprising:

[0011] The log merging module is used to obtain the initial transaction log of the enterprise service bus system according to the field extraction rule configuration, and merge the logs according to the timestamps corresponding to the initial transaction logs to obtain the merged transaction log;

[0012] A log splitting module, configured to obtain regular parsing rules, content replacement rules and string conversion rules according to the field extraction rule configuration, and to split the merged transaction log according to the regular parsing rules to obtain a log body;

[0013] A content replacement module, used to replace the specified content of the log body according to the content replacement rule, obtain a hexadecimal log message, and perform encoding conversion on the hexadecimal log message according to the string conversion rule to obtain an extensible markup language format log message;

[0014] The element extraction module is used to extract key elements from the log message in the extensible markup language format and obtain key element extraction results.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the log processing method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the log processing method described in any embodiment of the present invention when executed.

[0020] According to another aspect of the present invention, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the log processing method according to any embodiment of the present invention is implemented.

[0021] The technical solution of the embodiment of the present invention obtains the initial transaction log of the enterprise service bus system according to the field extraction rule configuration, and merges the logs according to the timestamps corresponding to the initial transaction logs to obtain the merged transaction log; obtains the regular parsing rule, the content replacement rule and the string conversion rule according to the field extraction rule configuration, and splits the merged transaction log according to the regular parsing rule to obtain the log body; replaces the specified content of the log body according to the content replacement rule to obtain a hexadecimal log message, and encodes the hexadecimal log message according to the string conversion rule to obtain an extensible markup language format log message; extracts key elements from the extensible markup language format log message to obtain the key element extraction result; merges the transaction logs, and performs log splitting, content replacement and encoding conversion according to the pre-deployed field extraction rule configuration to obtain an extensible markup language format log message, thereby improving log processing efficiency and log readability.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flow chart of a log processing method provided according to Embodiment 1 of the present invention;

[0025] Figure 2 This is a schematic diagram of a rule-based new page provided according to Embodiment 1 of the present invention;

[0026] Figure 3 is a structural schematic diagram of a log processing device provided according to Embodiment 2 of the present invention;

[0027] Figure 4 It is a structural schematic diagram of an electronic device for implementing the log processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment 1

[0031] Figure 1 A flow chart of a log processing method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of processing transaction logs of an enterprise service bus (ESB) system. The method can be executed by a log processing device, which can be implemented in the form of hardware and / or software. Typically, the log processing device can be configured in an electronic device, such as a computer device or a server. Figure 1 As shown, the method includes:

[0032] S110. According to the field extraction rule configuration, the initial transaction log of the enterprise service bus system is obtained, and the logs are merged according to the timestamps corresponding to the initial transaction logs to obtain a merged transaction log.

[0033] The field extraction rule configuration may be a pre-set field extraction rule, which may be composed of multiple processing rules. In a specific example, the user may select a new field extraction rule in the settings page, and perform specific rule configuration in the rule creation page, such as filling in the rule name, log file path, application name, host ID, label, sample log information, etc., selecting the field extraction group, and adding parsing rules, etc., to generate a field extraction rule configuration.

[0034] Specifically, log collection can be performed according to the data source information in the field extraction rule configuration to obtain the initial transaction log of the ESB system. For example, the initial transaction log can be Figure 2 The log information in the rule creation page shown is that the first two lines are log headers (including time, log level, etc.), the subsequent lines are log bodies (including the main message body), and the middle part of the log body is the actual log message in hexadecimal. Furthermore, the initial transaction logs with the same timestamp can be merged into one, thereby obtaining a merged transaction log.

[0035] Optionally, according to the field extraction rule configuration, the initial transaction log of the enterprise service bus system is obtained, which may include:

[0036] According to the field extraction rule configuration, the log file path, application name and host identifier are obtained, and according to the log file path, the application name and the host identifier, the initial transaction log of the enterprise service bus system is obtained.

[0037] In this embodiment, the data source information in the field extraction rule configuration may include a log file path (corresponding to the source field), an application name (corresponding to the appname field) and a host identifier (corresponding to the hostname field). Specifically, based on the log file path, the application name and the host identifier, the initial transaction log of the ESB system corresponding to the application name may be automatically read from the log file path of the server host corresponding to the host identifier.

[0038] Optionally, performing log merging according to the timestamps corresponding to the initial transaction logs to obtain the merged transaction log may include:

[0039] Converting the timestamp corresponding to each of the initial transaction logs into a wildcard;

[0040] According to the wildcard, a matching test is performed on the initial transaction logs within a preset range, and the successfully matched initial transaction logs are merged to obtain a merged transaction log.

[0041] The timestamp is located at the beginning of each initial transaction log, such as Figure 2 The initial transaction log shown is timestamped at 2016-12-31 22:06:05,084.

[0042] In this embodiment, the timestamp of each initial transaction log can be converted into a wildcard, which can be a special statement used for fuzzy search of files and can be composed of asterisks and question marks. Then, the wildcard corresponding to each initial transaction log can be fuzzily matched with the wildcards corresponding to other initial transaction logs within a preset range (e.g., 1000 etc.). For multiple initial transaction logs that are successfully matched, their contents can be merged to obtain a whole merged transaction log.

[0043] S120. Obtain regular parsing rules, content replacement rules, and string conversion rules according to the field extraction rule configuration, and split the merged transaction log according to the regular parsing rules to obtain a log body.

[0044] Specifically, the field extraction rule configuration can be parsed to obtain regular parsing rules, content replacement rules, and string conversion rules. Among them, the regular parsing rules are used to split the merged transaction log into a log header and a log body; the content replacement rules are used to replace the log content with specified content; and the string conversion rules are used to convert the data format of the log content. Then, the regular parsing rules can be used to process each merged transaction log to parse and obtain the log body.

[0045] Optionally, splitting the merged transaction log according to the regular parsing rule to obtain the log body may include:

[0046] According to the regular parsing rule, a regular expression is obtained, and based on the regular expression, the merged transaction log is split to obtain a log body.

[0047] In this embodiment, a corresponding regular expression may be pre-set for the regular parsing rule. For example, the regular expression may be (? <timestamp>.*?,\d{3})\s+\[(?<thread_no>[\s\S]+)\]\s+\[(? <class> .*？)\]\s+\[(？ <level>\S+)\].*?-.*?Code---\s+(? <msg>[\s\S]+). Specifically, a regular expression can be used to parse the merged transaction log to obtain the log body. Figure 2 For example, the split result of the merged transaction log can be shown on the right side of the figure, where the msg field is the split log body.

[0048] S130. Replace the specified content of the log body according to the content replacement rule to obtain a hexadecimal log message, and perform encoding conversion on the hexadecimal log message according to the string conversion rule to obtain an extensible markup language format log message.

[0049] In this embodiment, the left and right data in the merged transaction log can be deleted according to the content replacement rule, so that only the hexadecimal log message in the middle part is obtained. Figure 2 For example, the first line of data in the log body is "0010(0016)3C 3F 78 6D 6C 20 76 65 72 7369 6F 6E 3D 22 31<?xml version="1", the left and right data are "0010(0016)" and "<?xml version="1" respectively, and the right data is the data format conversion text of the hexadecimal log message in the middle part, that is, ASCII information. By adopting the content replacement rule, the left and right data can be deleted, and only the middle part, that is, "3C 3F 78 6D 6C20 76 65 72 73 69 6F6E 3D 22 31", can be retained.

[0050] It should be noted that although the data on the right side of the log body is an Extensible Markup Language (XML) format text converted from the hexadecimal log message encoding, it is difficult to splice and it is difficult to determine the line break position due to its random line breaks. This embodiment performs message processing based on the hexadecimal log message in the middle part, which contains line break identification characters, namely "0D 0A", and the line break position can be easily determined, thereby reducing the difficulty of log processing and improving log processing efficiency.

[0051] Specifically, a string conversion rule may be used to convert the hexadecimal log message into a code, for example, into a GBK code, an ASCII code, etc., to obtain an XML format log message.

[0052] Optionally, performing specified content replacement on the log body according to the content replacement rule to obtain a hexadecimal log message may include:

[0053] According to the content replacement rule, obtaining a first content replacement sub-rule and a second content replacement sub-rule;

[0054] According to the first content replacement sub-rule and the second content replacement sub-rule, the left data and the right data of the log body are respectively searched and replaced with blank data to obtain a hexadecimal log message in the middle part of the log body.

[0055] The first content replacement sub-rule is used to replace the data on the left side of the log body; the second content replacement sub-rule is used to replace the data on the right side of the log body.

[0056] Specifically, the regular expression corresponding to the first content replacement sub-rule may be \S{4}\(\S{4}\)\s+, based on which the left data, for example, 0010(0016), 0020(0032), etc., may be replaced with blank data. Then, the regular expression corresponding to the second content replacement sub-rule may be \s{3}.*? \n\s, based on which the right data, for example, <? xml version="1、.0" encoding="UT, etc., may be replaced with blank data. At this point, after the data is replaced, only the hexadecimal log message in the middle remains in the log body.

[0057] Optionally, replacing the left data and the right data with blank data to obtain a hexadecimal log message in the middle part of the log body may include:

[0058] The left-side data and the right-side data are replaced with blank data to obtain an initial hexadecimal log message of the log body;

[0059] According to the content replacement rule, a content deletion sub-rule is obtained, and according to the content deletion sub-rule, the first space and the last space of each line in the initial hexadecimal log message are deleted to obtain the hexadecimal log message in the middle part of the log body.

[0060] In this embodiment, the first space and the last space in the initial hexadecimal log message after data replacement can also be deleted. Specifically, the content replacement rule can be parsed to obtain a content deletion sub-rule, and a regular expression corresponding to the content deletion sub-rule can be obtained, for example, it can be \s(? <hex>[\s\S]+)\s; Then, the regular expression can be used to perform regular parsing on the initial hexadecimal log message to delete the first space and the last space of each line, thereby obtaining the final hexadecimal log message.

[0061] In this embodiment, the beginning of the line of the log body data can be checked, and data like "0010 (0016)" can be deleted line by line, and the hexadecimal log message in the middle can be retained; further, each line can be deleted according to the mark, starting from the beginning of the line, and 37 bits are taken backward, and then the content exceeding 37 bits at the end of the same line is deleted / replaced. Alternatively, three consecutive space marks (\s) can be matched from the beginning of each line, and the matched data can be retained to obtain the hexadecimal log message.

[0062] S140: extract key elements from the log message in the extensible markup language format to obtain a key element extraction result.

[0063] Specifically, key fields can be extracted from XML format log messages to obtain field values ​​corresponding to the key fields as key element extraction results. For example, the key fields may be BranchID, T1rNo, etc. Furthermore, after obtaining the key element extraction results, the key element extraction results may be visualized.

[0064] The technical solution of the embodiment of the present invention obtains the initial transaction log of the enterprise service bus system according to the field extraction rule configuration, and merges the logs according to the timestamps corresponding to the initial transaction logs to obtain the merged transaction log; obtains the regular parsing rule, the content replacement rule and the string conversion rule according to the field extraction rule configuration, and splits the merged transaction log according to the regular parsing rule to obtain the log body; replaces the specified content of the log body according to the content replacement rule to obtain a hexadecimal log message, and encodes the hexadecimal log message according to the string conversion rule to obtain an extensible markup language format log message; extracts key elements from the extensible markup language format log message to obtain the key element extraction result; merges the transaction logs, and performs log splitting, content replacement and encoding conversion according to the pre-deployed field extraction rule configuration to obtain an extensible markup language format log message, thereby improving log processing efficiency and log readability.

[0065] The solution of this embodiment is suitable for solving the situation where a hexadecimal message log and other contents are mixed in a log, can solve the problem that traditional Linux commands or scripts cannot process mixed logs, and can improve the compatibility with complex logs.

[0066] Embodiment 2

[0067] Figure 3 This is a schematic diagram of the structure of a log processing device provided in Embodiment 2 of the present invention. Figure 3 As shown, the device includes: a log merging module 210, a log splitting module 220, a content replacement module 230 and an element extraction module 240; wherein,

[0068] The log merging module 210 is used to obtain the initial transaction log of the enterprise service bus system according to the field extraction rule configuration, and merge the logs according to the timestamps corresponding to the initial transaction logs to obtain the merged transaction log;

[0069] The log splitting module 220 is used to obtain regular parsing rules, content replacement rules and string conversion rules according to the field extraction rule configuration, and split the merged transaction log according to the regular parsing rules to obtain a log body;

[0070] The content replacement module 230 is used to replace the specified content of the log body according to the content replacement rule, obtain a hexadecimal log message, and perform encoding conversion on the hexadecimal log message according to the string conversion rule to obtain an extensible markup language format log message;

[0071] The element extraction module 240 is used to extract key elements from the log message in the extensible markup language format and obtain key element extraction results.

[0072] The technical solution of the embodiment of the present invention obtains the initial transaction log of the enterprise service bus system according to the field extraction rule configuration, and merges the logs according to the timestamps corresponding to the initial transaction logs to obtain the merged transaction log; obtains the regular parsing rule, the content replacement rule and the string conversion rule according to the field extraction rule configuration, and splits the merged transaction log according to the regular parsing rule to obtain the log body; replaces the specified content of the log body according to the content replacement rule to obtain a hexadecimal log message, and encodes the hexadecimal log message according to the string conversion rule to obtain an extensible markup language format log message; extracts key elements from the extensible markup language format log message to obtain the key element extraction result; merges the transaction logs, and performs log splitting, content replacement and encoding conversion according to the pre-deployed field extraction rule configuration to obtain an extensible markup language format log message, thereby improving log processing efficiency and log readability.

[0073] Optionally, the log merging module 210 is specifically configured to convert the timestamps corresponding to the initial transaction logs into wildcards;

[0074] According to the wildcard, a matching test is performed on the initial transaction logs within a preset range, and the successfully matched initial transaction logs are merged to obtain a merged transaction log.

[0075] Optionally, the log splitting module 220 is specifically configured to obtain a regular expression according to the regular parsing rule, and split the merged transaction log based on the regular expression to obtain a log body.

[0076] Optionally, the content replacement module 230 is specifically configured to obtain a first content replacement sub-rule and a second content replacement sub-rule according to the content replacement rule;

[0077] According to the first content replacement sub-rule and the second content replacement sub-rule, the left data and the right data of the log body are respectively searched and replaced with blank data to obtain a hexadecimal log message in the middle part of the log body.

[0078] Optionally, a content replacement module 230 is specifically configured to replace the left data and the right data with blank data to obtain an initial hexadecimal log message of the log body;

[0079] According to the content replacement rule, a content deletion sub-rule is obtained, and according to the content deletion sub-rule, the first space and the last space of each line in the initial hexadecimal log message are deleted to obtain the hexadecimal log message in the middle part of the log body.

[0080] Optionally, the log merging module 210 is specifically configured to obtain the log file path, application name and host identifier according to the field extraction rule configuration, and obtain the initial transaction log of the enterprise service bus system according to the log file path, the application name and the host identifier.

[0081] The log processing device provided in the embodiment of the present invention can execute the log processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0082] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0083] Embodiment 3

[0084] Figure 4 A schematic diagram of the structure of an electronic device 30 that can be used to implement an embodiment of the present invention is shown. The electronic device 30 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 30 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0085] like Figure 4 As shown, the electronic device 30 includes at least one processor 31, and a memory connected to the at least one processor 31, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory 32 or the computer program loaded from the storage unit 38 to the random access memory 33. In the RAM 33, various programs and data required for the operation of the electronic device 30 can also be stored. The processor 31, the ROM 32 and the RAM 33 are connected to each other through a bus 34. The input / output (I / O) interface 35 is also connected to the bus 34.

[0086] A number of components in the electronic device 30 are connected to the I / O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0087] The processor 31 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit, a graphics processing unit, various special artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any appropriate processors, controllers, microcontrollers, etc. The processor 31 executes the various methods and processes described above, such as the log processing method.

[0088] In some embodiments, the log processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the log processing method described above may be performed. Alternatively, in other embodiments, the processor 31 may be configured to execute the log processing method in any other appropriate manner (e.g., by means of firmware).

[0089] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays, application specific integrated circuits, application specific standard products, systems on a chip, load programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0090] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0091] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0092] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device 30 having: a display device (e.g., a cathode ray tube or a liquid crystal display) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device 30. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0093] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area networks, wide area networks, blockchain networks, and the Internet.

[0094] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other. The server may be a cloud server.

[0095] This embodiment may also include a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the log processing method provided by any embodiment of the present invention.

[0096] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0097] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.< / hex> < / msg> < / level> < / class> < / timestamp>

Claims

1. A log processing method, characterized in that: include: According to the field extraction rule configuration, the initial transaction log of the enterprise service bus system is obtained, and the logs are merged according to the timestamps corresponding to the initial transaction logs to obtain the merged transaction log; According to the field extraction rule configuration, a regular parsing rule, a content replacement rule and a string conversion rule are obtained, and the merged transaction log is split according to the regular parsing rule to obtain a log body; Performing designated content replacement on the log body according to the content replacement rule to obtain a hexadecimal log message, and performing encoding conversion on the hexadecimal log message according to the string conversion rule to obtain an extensible markup language format log message; Key elements are extracted from the log message in the extensible markup language format to obtain a key element extraction result.

2. The method according to claim 1, characterized in that Merging the logs according to the timestamps corresponding to the initial transaction logs to obtain a merged transaction log includes: Converting the timestamp corresponding to each of the initial transaction logs into a wildcard; According to the wildcard, a matching test is performed on the initial transaction logs within a preset range, and the successfully matched initial transaction logs are merged to obtain a merged transaction log.

3. The method according to claim 1, characterized in that The combined transaction log is split according to the regular parsing rule to obtain a log body, including: According to the regular parsing rule, a regular expression is obtained, and based on the regular expression, the merged transaction log is split to obtain a log body.

4. The method according to claim 1, characterized in that: Replacing the specified content of the log body according to the content replacement rule to obtain a hexadecimal log message includes: According to the content replacement rule, obtaining a first content replacement sub-rule and a second content replacement sub-rule; According to the first content replacement sub-rule and the second content replacement sub-rule, the left data and the right data of the log body are respectively searched and replaced with blank data to obtain a hexadecimal log message in the middle part of the log body.

5. The method according to claim 4, characterized in that The left data and the right data are replaced with blank data to obtain a hexadecimal log message in the middle part of the log body, including: The left-side data and the right-side data are replaced with blank data to obtain an initial hexadecimal log message of the log body; According to the content replacement rule, a content deletion sub-rule is obtained, and according to the content deletion sub-rule, the first space and the last space of each line in the initial hexadecimal log message are deleted to obtain the hexadecimal log message in the middle part of the log body.

6. The method according to claim 1, characterized in that According to the field extraction rule configuration, obtain the initial transaction log of the enterprise service bus system, including: According to the field extraction rule configuration, the log file path, application name and host identifier are obtained, and according to the log file path, the application name and the host identifier, the initial transaction log of the enterprise service bus system is obtained.

7. A log processing device, characterized in that: include: The log merging module is used to obtain the initial transaction log of the enterprise service bus system according to the field extraction rule configuration, and merge the logs according to the timestamps corresponding to the initial transaction logs to obtain the merged transaction log; A log splitting module, configured to obtain regular parsing rules, content replacement rules and string conversion rules according to the field extraction rule configuration, and to split the merged transaction log according to the regular parsing rules to obtain a log body; A content replacement module, used to replace the specified content of the log body according to the content replacement rule, obtain a hexadecimal log message, and perform encoding conversion on the hexadecimal log message according to the string conversion rule to obtain an extensible markup language format log message; The element extraction module is used to extract key elements from the log message in the extensible markup language format and obtain key element extraction results.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the log processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the log processing method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the log processing method according to any one of claims 1 to 6 when executed by a processor.

Images