Message extraction method, device, equipment, medium and program product of log file

Abstract

The present application relates to the field of big data technology and can be used in the financial field. It provides a method, device, computer equipment, storage medium and program product for extracting messages from log files. The present application can quickly and accurately extract messages from log files, reducing labor costs and implementation difficulties. The method includes: equalizing the start input timestamp and the end input timestamp extracted from the log file to form a timestamp array containing equalized timestamps, then determining the timestamp interval corresponding to the message to be searched in the timestamp array, then extracting the log text of each corresponding line from the log file according to the timestamp interval, and then segmenting the log text of each corresponding line in sequence according to preset keywords. If a preset part of the log text segment is segmented from the log text of the current corresponding line, it is determined whether the log text segment has the preset message node feature. If so, the log text segment is used as the message extraction result of the log file.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for extracting messages from a log file. Background Art

[0002] With the development of big data technology, more and more technicians choose to extract messages from log files to obtain script data for interface automation testing. Regularly running such script data helps technicians monitor the quality of interface development in real time, so as to detect potential problems that may cause program failures as early as possible and reduce the cost of repairing program failures.

[0003] Currently, the method of extracting messages from log files can be carried out with the help of text matching technology. However, text matching technology requires constructing keyword phrases into an ordered keyword tree. It is often difficult for keyword phrases in log files to have an association relationship to construct an ordered keyword tree. Therefore, this method is difficult to extract messages from log files quickly and accurately. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium and computer program product for extracting messages from a log file in order to address the above technical issues.

[0005] In a first aspect, the present application provides a method for extracting messages from a log file. The method comprises:

[0006] Equalize the start input timestamp and the end input timestamp extracted from the log file to form a timestamp array containing the equalized timestamps;

[0007] Determine the timestamp interval corresponding to the message to be searched in the timestamp array;

[0008] Extracting log text of each corresponding line from the log file according to the timestamp interval;

[0009] Sequentially segmenting the log texts in the corresponding lines according to the preset keywords;

[0010] If a preset portion of the log text segment is segmented from the log text of the current corresponding line, then determining whether the log text segment has a preset message node feature;

[0011] If so, the log text segment is used as the message extraction result of the log file.

[0012] In a second aspect, the present application further provides a message extraction device for a log file. The device comprises:

[0013] The data processing module performs equalization on the start input timestamp and the end input timestamp extracted from the log file to form a timestamp array containing the equalized timestamps;

[0014] An interval determination module determines a timestamp interval corresponding to a message to be searched in the timestamp array;

[0015] A text extraction module extracts log text of each corresponding line from the log file according to the timestamp interval;

[0016] A text segmentation module, which segments the log text of each corresponding line in sequence according to preset keywords;

[0017] A feature judgment module, if a preset portion of the log text segment is segmented from the log text of the current corresponding line, then judges whether the log text segment has a preset message node feature;

[0018] If yes, the result acquisition module uses the log text segment as the message extraction result of the log file.

[0019] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0020] The start input timestamp and the end input timestamp extracted from the log file are equally divided to form a timestamp array containing the equally divided timestamps; the timestamp interval corresponding to the message to be searched in the timestamp array is determined; the log text of each corresponding line is extracted from the log file according to the timestamp interval; the log text of each corresponding line is segmented in sequence according to preset keywords; if a preset part of the log text segment is segmented from the log text of the current corresponding line, it is determined whether the log text segment has a preset message node feature; if so, the log text segment is used as the message extraction result of the log file.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0022] The start input timestamp and the end input timestamp extracted from the log file are equally divided to form a timestamp array containing the equally divided timestamps; the timestamp interval corresponding to the message to be searched in the timestamp array is determined; the log text of each corresponding line is extracted from the log file according to the timestamp interval; the log text of each corresponding line is segmented in sequence according to preset keywords; if a preset part of the log text segment is segmented from the log text of the current corresponding line, it is determined whether the log text segment has a preset message node feature; if so, the log text segment is used as the message extraction result of the log file.

[0023] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0024] The start input timestamp and the end input timestamp extracted from the log file are equally divided to form a timestamp array containing the equally divided timestamps; the timestamp interval corresponding to the message to be searched in the timestamp array is determined; the log text of each corresponding line is extracted from the log file according to the timestamp interval; the log text of each corresponding line is segmented in sequence according to preset keywords; if a preset part of the log text segment is segmented from the log text of the current corresponding line, it is determined whether the log text segment has a preset message node feature; if so, the log text segment is used as the message extraction result of the log file.

[0025] The above-mentioned message extraction method, apparatus, computer equipment, storage medium and computer program product for log files will equally divide the start input timestamp and end input timestamp extracted from the log file to form a timestamp array containing the equally divided timestamps, and then determine the timestamp interval corresponding to the message to be searched in the timestamp array, and then extract the log text of each corresponding line from the log file according to the timestamp interval, and then segment the log text of each corresponding line in sequence according to the preset keywords. If a preset part of the log text segment is segmented from the log text of the current corresponding line, it is determined whether the log text segment has the preset message node feature. If so, the log text segment is used as the message extraction result of the log file. This solution can realize log message extraction based on the format characteristics of the log file without the need to construct an ordered keyword tree based on keyword phrases. It can not only accurately and quickly extract the messages of the log file, but also reduce the labor cost and implementation difficulty of extracting the messages of the log file. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Schematic diagram of a flow chart of a method for extracting messages from a log file in one embodiment;

[0027] Figure 2A flowchart illustrating a method for forming a timestamp array containing equally spaced timestamps in one embodiment;

[0028] Figure 3 A flowchart of a method for checking whether a predetermined number of characters in each corresponding line of log text is a valid timestamp in one embodiment;

[0029] Figure 4 A structural block diagram of a message extraction device for a log file in one embodiment;

[0030] Figure 5 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0032] The message extraction method for log files provided in the embodiments of the present application can be applied to computer devices such as servers and terminals. The server can be implemented as a standalone server or a server cluster consisting of multiple servers, and the terminal can be, but is not limited to, various personal computers, laptops, smartphones, and tablet computers.

[0033] In one embodiment, Figure 1 As shown, a method for extracting messages from a log file is provided, comprising the following steps:

[0034] In step S101 , the start input timestamp and the end input timestamp extracted from the log file are equalized to form a timestamp array containing the equalized timestamps.

[0035] In this step, the log file may be a system log file generated when the computer system is running; the start input timestamp refers to the time record generated when the system log content starts to be input, and the content recorded at the beginning of each line of system log text is the time record generated when the system log text starts to be input; the end input timestamp refers to the time record generated when the system log content is finished being input, and the content recorded at the end of each line of system log text is the time record generated when the system log text is finished being input.

[0036] Step S102: Determine the timestamp interval corresponding to the message to be searched in the timestamp array.

[0037] In this step, the message to be searched refers to the message that needs to be searched and extracted in the log file, and the content of the message to be searched can be any content contained in the log file; the timestamp interval refers to the time range between two equally divided timestamps in the timestamp array; determining the timestamp interval corresponding to the message to be searched in the timestamp array refers to the time range between the two equally divided timestamps in the timestamp array to which the start input timestamp corresponding to the message that needs to be searched and extracted in the log file belongs; the method for determining the timestamp interval corresponding to the message to be searched in the timestamp array can be binary positioning.

[0038] Step S103: extracting log text of each corresponding line from the log file according to the timestamp interval.

[0039] In this step, under the premise of a known timestamp interval, the log text of each corresponding line extracted from the log file refers to extracting the log text of each corresponding line whose start input timestamp recorded at the beginning of the log file falls within the timestamp interval.

[0040] For example, if the timestamp interval corresponding to the message to be searched obtained in the above step S102 in the timestamp array is [m, m+1], then the log text of each corresponding line whose starting input timestamp is between [m] and [m+1] is extracted from the log file; if there are 20 lines of log text whose starting input timestamp is between [m] and [m+1], then these 20 lines of log text are extracted from the log file for executing subsequent steps S104 to S106.

[0041] Step S104 : segmenting the log text in each corresponding line in sequence according to the preset keywords.

[0042] In this step, the preset keyword refers to the keyword used to split the log text of each corresponding line into multiple segments. The number of times the preset keyword appears in the log text of each corresponding line is used to determine the number of times the log text of each corresponding line is split. The method of using the preset keyword to split the log text of each corresponding line in sequence can be the split method in the JAVA language.

[0043] For example, it is known that the preset keyword is Req, and the preset keyword appears three times in the following log text line.

[0044] 20xx-0x-0x 1x:3x:4x.2xx[WebContainer:0]DEBUG-logType=->debugmessage=request enter hanlderchain=XXXXXXX? 20xx-0x-0x 1x:3x:4x.2xx[WebContainer:0]DEBUG-logType=->debug message=request paramis:Req[flowActionName=XXXX00270]Req[action=soct_pool_interface.flowc]Req[xmlString=<? xml version="1.0" encoding="UTF-8"? > <gcmc> <public> <holderacctid> 0200000609027330386< / holderacctid> <checkflag> 0< / checkflag> <zonecd> 0200< / zonecd> <netcd> 0998< / netcd> <oper> 1110< / oper> <operdt> 20xx0x2x< / operdt> <bak1>< / bak1> <bak2>< / bak2> <bak3>< / bak3> < / public> < / gcmc> ]

[0045] According to the specification for printing request messages into log messages, the request XML message needs to be split into four segments based on the preset keyword Req and printed in the log file. The content of each segmented log text line is the timestamp, flowActionName, action, and xmlString in the printing order. Based on this, when the string line is split (split) based on the keyword Req, the fourth segment is the request message in XML format. Therefore, when splitting the above log text line based on the preset keyword Req, a total of three splits are required. After the splitting is completed, the above log text line will be divided into the following four segments:

[0046] 20xx-0x-0x 1x:3x:4x.2xx[WebContainer:0]DEBUG-logType=->debugmessage=request enter hanlderchain=XXXXXXX? 20xx-0x-0x 1x:3x:4x.2xx[WebContainer:0]DEBUG-logType=->debug message=request param is:

[0047] Req (preset keyword)

[0048] [flowActionName=XXXX00270]

[0049] Req (preset keyword)

[0050] [action=soct_pool_interface.flowc]

[0051] Req (preset keyword)

[0052] [xmlString=<? xml version="1.0" encoding="UTF-8"? > <gcmc> <public> <holderacctid> 0200000609027330386< / holderacctid> <checkflag> 0< / checkflag> <zonecd> 0200< / zonecd> <netcd> 0998< / netcd> <oper> 1110< / oper> <operdt> 20xx0x2x< / operdt> <bak1>< / bak1> <bak2>< / bak2> <bak3>< / bak3> < / public> < / gcmc> ]

[0053] Step S105 : If a preset portion of the log text segment is segmented from the log text of the current corresponding line, it is determined whether the log text segment has a preset message node feature.

[0054] If the preset portion of the log text segment is not segmented from the log text of the current corresponding line, the process returns to step S104, i.e., the log text of the next line of the current corresponding line is segmented according to the preset keywords; if the preset portion of the log text segment is segmented from the log text of the current corresponding line, the log text segment is determined to have the preset message node feature. The preset portion of the log text segment refers to a log text segment that may contain a message that needs to be searched and extracted from the log file; the position of the preset portion of the log text segment in the log text of the corresponding line can be determined based on the message content that needs to be extracted from the log file.

[0055] For example, if the message that needs to be extracted from the log file is an XML message, then the preset part of the log text segment refers to the log text segment that contains the XML message in the log text of each corresponding line after segmentation, and the position of the preset part of the log text segment in the log text of the corresponding line refers to the position of the log text segment that contains the XML message in the log text of each corresponding line after segmentation.

[0056] Step S106: If yes, the log text segment is used as the message extraction result of the log file.

[0057] If not, the process returns to step S104, where the log text of the next line of the current corresponding line is segmented according to the preset keywords. If so, the log text segment is determined to have the preset message node characteristics, and the log text segment can be used as the message extraction result of the log file. The message extraction result of the log file refers to the log text segment extracted from the log file that contains the message to be extracted.

[0058] For example, if it is known that the message of the log file to be extracted is an XML message, the message extraction result can be parsed using a regular expression to automatically generate a case script in Excel format. Such case script data can be used by technicians to manually initiate interface call tests using existing testing tools, or uploaded to the automated testing system code library based on TestNG+Jenkins for daily scheduled operation, so as to help technicians monitor the quality of interface research and development in real time through test results. Among them, if the log file used to extract the message comes from the production environment, the Excel script data automatically generated based on it will be more real and effective, and R&D personnel can directly use such Excel script data to perform interface testing in the test environment without further adjustment.

[0059] The message extraction method of the log file mentioned above is to equally divide the start input timestamp and the end input timestamp extracted from the log file into a timestamp array containing the equally divided timestamps, and then determine the timestamp interval corresponding to the message to be searched in the timestamp array, and then extract the log text of each corresponding line from the log file according to the timestamp interval, and then segment the log text of each corresponding line in sequence according to the preset keywords. If a preset part of the log text segment is segmented from the log text of the current corresponding line, it is determined whether the log text segment has the preset message node feature. If so, the log text segment is used as the message extraction result of the log file. This solution can realize log message extraction based on the format characteristics of the log file without the need to construct an ordered keyword tree based on keyword phrases. It can not only accurately and quickly extract the messages of the log file, but also reduce the labor cost and implementation difficulty of extracting the messages of the log file.

[0060] Regarding the method of forming a timestamp array containing equalized timestamps, in one embodiment, as Figure 2 As shown, the above step S101 specifically includes:

[0061] In step S201 , the start input timestamp and the end input timestamp extracted from the log file are converted into a preset time representation format.

[0062] In this step, the start input timestamp refers to the start input timestamp recorded at the beginning of the first line of the log file; the end input timestamp refers to the end input timestamp recorded at the end of the last line of the log file; the preset time representation format can be any time representation format converted from the timestamp format, such as millisecond format.

[0063] Step S202: Obtain the difference between the end input timestamp and the start input timestamp in a preset time representation format.

[0064] In this step, the difference value refers to the difference value obtained by subtracting the start input timestamp from the end input timestamp in the preset time representation.

[0065] Step S203 : obtaining equalization timestamps corresponding to the preset equalization number according to the ratio of the difference and the preset equalization number.

[0066] In this step, the preset equal number refers to the number of equal-divided time stamps that need to be obtained based on the above-mentioned start input time stamp and end input time. For example, if the preset equal number is 10, 10 equal-divided time stamps need to be obtained based on the above-mentioned start input time stamp and end input time.

[0067] Furthermore, in one embodiment, step S203 specifically includes:

[0068] According to the ratio of the above difference to the preset equal number, and the start input timestamp and the end input timestamp in the preset time form, the equalization timestamp corresponding to the preset equal number and in the preset time form is obtained; the equalization timestamp corresponding to the preset equal number and in the preset time form is converted into a timestamp format to obtain the equalization timestamp corresponding to the preset equal number.

[0069] Step S204 , sequentially dividing the timestamps corresponding to the above number and the preset equal number into a timestamp array.

[0070] In this step, all the equally divided timestamps constituting the timestamp array may be arranged in the order of the time records they represent.

[0071] For example, given that the start input timestamp extracted from the log file is 2021-01-02 19:38:09.016 and the end timestamp is 2021-01-02 21:35:00.013, the preset time representation is in milliseconds, and the preset equal fraction is 10, the above steps S201 to S204 can be:

[0072] In step S201 , the start input timestamp 2021-01-02 19:38:09.016 and the end timestamp 2021-01-02 21:35:00.013 extracted from the log file are converted into millisecond format to obtain the start input timestamp 1649238360 and the end timestamp 1649238411 in millisecond format.

[0073] Step S202: Subtract the start input timestamp 1649238360 from the end input timestamp 1649238411 in millisecond format to obtain the difference 51.

[0074] Step S203, based on the ratio 5.1 of the above difference 51 and the preset equal number 10, the start input timestamp 1649238360 and the end timestamp 1649238411 in millisecond format, obtain 10 equal-division timestamps in millisecond format corresponding to the preset equal number, and convert the 10 equal-division timestamps in millisecond format into a timestamp format to obtain 10 equal-division timestamps corresponding to the preset equal number.

[0075] Step S204 , arranging the time records represented by the 10 equally divided time stamps in chronological order to form a time stamp array.

[0076] This embodiment obtains a timestamp array containing equally divided timestamps arranged in chronological order by dividing the start input timestamp and the end input timestamp extracted from the log file into equal parts, laying the foundation for subsequently determining the timestamp interval corresponding to the message to be searched in the timestamp array, and narrowing the range of log text required to extract messages from the log file in a manner that is highly consistent with the format characteristics of the log file, thereby effectively reducing the time cost spent on message extraction from the log file.

[0077] Regarding the method of checking whether the characters of the preset number of digits in the log text of each corresponding line are valid timestamps, in one embodiment, as shown in FIG. Figure 3 As shown, before step S104, the above step also includes:

[0078] In step S301 , for each corresponding line of log text, it is determined whether the first predetermined digits of characters of the corresponding line of log text conform to a valid timestamp format.

[0079] In this step, the length of the first preset digit is determined based on the character length of the valid timestamp. For example, if the character length of the valid timestamp is 23 digits, then when determining whether the characters of the first preset digit of the log text in the corresponding line conform to the valid timestamp format, it is necessary to read the first 23 characters of the log text in the corresponding line to make a judgment.

[0080] Step S302: If not, delete the log text of the corresponding line.

[0081] If so, the first predetermined digits of characters in the log text of the corresponding line are determined to conform to the valid timestamp format, the log text of the corresponding line is retained, and the first predetermined digits of characters in the log text of the next line of the current corresponding line are determined to conform to the valid timestamp format; if not, the first predetermined digits of characters in the log text of the corresponding line are determined to not conform to the valid timestamp format, the log text of the corresponding line is deleted, and the first predetermined digits of characters in the log text of the next line of the current corresponding line are determined to conform to the valid timestamp format. The retained log text will be used to execute step S104.

[0082] In combination with the above embodiment and taking the extraction of XML messages from a log file as an example, the process of obtaining the log text of each corresponding line from the log file and checking whether the characters of the preset digits in the log text of each corresponding line are valid timestamps is generally described:

[0083] First, it is known that the timestamp interval corresponding to the XML message to be extracted, found using binary search, is [m,m+1] (m is a positive integer less than n, and n is the number of equally spaced timestamps contained in the above timestamp array). According to the log file specification, the XML message is located at the end of each line of log text and can be displayed on the same line as the other content of the corresponding line of log text. The format of the XML message extracted from a line of log text is as follows:

[0084] <gcmc> <public> <holderacctid>< / holderacctid> <checkflag>< / checkflag> <zonecd>< / zonecd> <netcd>< / netcd> <oper>< / oper> <operdt>< / operdt> <bak1>< / bak1> <bak2>< / bak2> <bak3>< / bak3> < / public> < / gcmc>

[0085] Next, create a file named tmp.txt in TXT format. Copy the log text from each line with a start input timestamp between [m] and [m+1] to tmp.txt according to the original log file format. For example, if there are 20 lines of log text between [m] and [m+1], and each line begins with the start input timestamp, then tmp.txt will also need to contain 20 lines of log text, each starting with a timestamp. If the number of characters in a log line exceeds the system-specified maximum number of columns for TXT text (1024), the remaining text in that line will be automatically wrapped to the next line.

[0086] Next, the first 23 characters of each line of log text in tmp.txt are read line by line to determine whether it is in a valid timestamp format. If not, the line is deleted and the next line is entered. The line value of tmp.txt remains unchanged, and the operation of reading and determining whether it is in a valid timestamp format is continued line by line until the first 23 characters of a certain line of log text are determined to be in a valid timestamp format or the log text end position. If so, a carriage return is performed to retain the log text of the corresponding line. The line value of tmp.txt is accumulated by 1, and the operation of reading and determining whether the first 23 characters of a certain line of log text are in a valid timestamp format is continued line by line until the log text end position is determined. When the cursor reaches the log text end position, it indicates that the verification of the preset digits of the log text in each corresponding line is complete, that is, the structured processing for the current log text has ended. The total number of lines of tmp.txt obtained at this time is j, and the cursor returns to the leftmost end of the first line of tmp.txt, and step S104 is ready to be executed.

[0087] After the above processing, a line of log text is retained as follows:

[0088] 20xx-0x-0x 1x:3x:4x.2xx[WebContainer:0]DEBUG-logType=->debugmessage=request enter hanlderchain=XXXXXXX? 20xx-0x-0x 1x:3x:4x.2xx[WebContainer:0]DEBUG-logType=->debug message=request paramis:Req[flowActionName=XXXX00270]Req[action=soct_pool_interface.flowc]Req[xmlString=<? xml version="1.0" encoding="UTF-8"? > <gcmc> <public> <holderacctid> 0200000609027330386< / holderacctid> <checkflag> 0< / checkflag> <zonecd> 0200< / zonecd> <netcd> 0998< / netcd> <oper> 1110< / oper> <operdt> 20xx0x2x< / operdt> <bak1>< / bak1> <bak2>< / bak2> <bak3>< / bak3> < / public> < / gcmc> ]

[0089] This embodiment checks whether the characters of the preset digits in the log text of each corresponding line are valid timestamps. Before searching the log text according to the preset message node characteristics, the log text that is not necessary to be searched is removed, thereby effectively improving the efficiency of message extraction from the log file in a low-cost manner.

[0090] Regarding the method of searching for log text with preset message node characteristics in the preset portion of log text segments, in one embodiment, the above step S105 specifically includes:

[0091] A preset part of the log text segment is split out from the log text of the current corresponding line, and then a substring search method is used, with the preset part of the log text segment as the main string and the preset message node feature as the pattern string, to search whether the log text segment has the preset message node feature.

[0092] In this embodiment, the search for the preset message node features of the log text segments segmented from the log text of each corresponding line can be performed line by line, and the corresponding line currently searched is the current corresponding line. Wherein, when the preset message node features are found in the preset portion of the log text segment segmented from the log text of the current corresponding line, the log text segment can be used as the message extraction result of the log file, and the search for the next line will not continue; when the preset message node features are not found in the preset portion of the log text segment segmented from the log text of the current corresponding line, the search will continue in the preset portion of the log text segment segmented from the log text of the next corresponding line, until a log text segment with the preset message node features is found or all the log texts of the corresponding lines have been searched. Wherein, the preset message node features may include at least one of the name of the message node and the corresponding value of the message node.

[0093] Taking the corresponding value of the XML message node as the preset message node feature of the message to be searched as an example, the preset part of the log text segment shown below is extracted from the log file and used as the main string for searching for the preset message node feature:

[0094] [xmlString=<? xml version="1.0" encoding="UTF-8"? > <gcmc> <public> <holderacctid> 0200000609027330386< / holderacctid> <checkflag> 0< / checkflag> <zonecd> 0200< / zonecd> <netcd> 0998< / netcd> <oper> 1110< / oper> <operdt> 20220320< / operdt> <bak1>< / bak1> <bak2>< / bak2> <bak3>< / bak3> < / public> < / gcmc> ]

[0095] Given the pattern string "0200000609027330386," comparing the pattern string with the main string reveals that the main string contains the pattern string. Therefore, the text segment is determined to have the message node characteristics of the message being searched, and the log text segment can be used as the message extraction result from the log file. That is, if the pattern string matches the main string, the target message is obtained. If the match fails, the search continues in the log text line following the current line, until either line j (i.e., the total number of lines in tmp.txt) is reached or the target message is obtained.

[0096] This embodiment efficiently locates the log text with preset message node characteristics by searching for the log text with preset message node characteristics in the preset part of the log text segment, which not only reduces the difficulty of obtaining the message extraction results of the log file, but also effectively improves the accuracy of the message extraction results of the log file.

[0097] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0098] Based on the same inventive concept, the present application also provides a device for extracting messages from log files involved in the above-mentioned method. The solution provided by this device is similar to the solution described in the above-mentioned method. Therefore, the specific limitations of the embodiments of the device for extracting messages from one or more log files provided below can be found in the above-mentioned limitations of the method for extracting messages from log files, and will not be repeated here.

[0099] In one embodiment, Figure 4 As shown, a message extraction device for a log file is provided, and the device 400 includes:

[0100] The data processing module 401 performs equalization on the start input timestamp and the end input timestamp extracted from the log file to form a timestamp array including the equalized timestamps;

[0101] The interval determination module 402 determines the timestamp interval corresponding to the message to be searched in the timestamp array;

[0102] The text extraction module 403 extracts the log text of each corresponding line from the log file according to the timestamp interval;

[0103] The text segmentation module 404 segments the log text in each corresponding line in sequence according to preset keywords;

[0104] The feature determination module 405 determines whether a preset portion of the log text segment is segmented from the log text of the current corresponding line and whether the log text segment has a preset message node feature;

[0105] Result acquisition module 406: If yes, the log text segment is used as the message extraction result of the log file.

[0106] In one embodiment, the data processing module 401 is used to convert the start input timestamp and the end input timestamp extracted from the log file into a preset time representation; obtain the difference between the end input timestamp and the start input timestamp of the preset time representation; obtain the number of equalized timestamps corresponding to the preset equal number based on the ratio of the difference to the preset equal number; and sequentially form the timestamp array with the number of equalized timestamps corresponding to the preset equal number.

[0107] In one embodiment, the data processing module 401 is used to obtain the equalization timestamps corresponding to the preset equalization number and in the preset time expression form based on the ratio of the difference to the preset equalization number, and the start input timestamp and the end input timestamp of the preset time expression form; and convert the equalization timestamps corresponding to the preset equalization number and in the preset time expression form into a timestamp format to obtain the equalization timestamps corresponding to the preset equalization number.

[0108] In one embodiment, the text segmentation module 404 is further used to determine whether the first preset digits of characters of each corresponding line of log text conform to a valid timestamp format before segmenting the corresponding lines of log text in sequence according to preset keywords.

[0109] In one embodiment, the feature judgment module 405 is used to use a substring search method to search for the preset message node feature in the log text segment if a preset part of the log text segment is segmented out from the log text of the current corresponding line, with the preset part of the log text segment as the main string and the preset message node feature as the pattern string.

[0110] In one embodiment, the preset message node feature includes at least one of a name of the message node and a corresponding value of the message node.

[0111] Each module in the above-mentioned log file message extraction device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above modules can be embedded in or independent of the processor of the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0112] It should be noted that the message extraction method and device for log files of the present application can be used for message extraction from log files in the financial field, and can also be used in any field other than the financial field. The application field of the message extraction method and device for log files of the present application is not limited.

[0113] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data such as log files. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a message extraction method for a log file is implemented.

[0114] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0115] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0116] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0117] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0118] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0119] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0120] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for extracting messages from a log file, characterized in that: The method comprises: Convert the start input timestamp and the end input timestamp extracted from the log file into a preset time representation; Obtaining the difference between the end input timestamp and the start input timestamp of the preset time representation; According to the ratio of the difference value to the preset equal number, obtaining equalization timestamps corresponding to the preset equal number; Sequentially compose a timestamp array by dividing the timestamps corresponding to the number and the preset equal number; Determine the timestamp interval corresponding to the message to be searched in the timestamp array; Extracting log text of each corresponding line from the log file according to the timestamp interval; Sequentially segmenting the log texts in the corresponding lines according to the preset keywords; If a preset portion of the log text segment is segmented from the log text of the current corresponding line, a substring search method is used, with the preset portion of the log text segment as a main string and a preset message node feature as a pattern string, to search the log text segment for the preset message node feature; wherein the preset message node feature includes at least one of a message node name and a corresponding value of the message node; If so, the log text segment is used as the message extraction result of the log file.

2. The method according to claim 1, characterized in that The step of obtaining, based on a ratio of the difference to a preset equal number, equalization time stamps corresponding to a number of equal numbers, includes: Obtaining, according to the ratio of the difference to the preset equal number, the start input timestamp and the end input timestamp of the preset time representation, a number of equalized time stamps corresponding to the preset equal number and in the preset time representation; The equalization timestamps corresponding to the number and the preset equalization number and in the preset time representation format are converted into a timestamp format to obtain the equalization timestamps corresponding to the number and the preset equalization number.

3. The method according to claim 1, characterized in that Before sequentially segmenting the log texts of the corresponding lines according to the preset keywords, the method further includes: For each corresponding line of log text, determine whether the first predetermined number of characters of the corresponding line of log text conforms to a valid timestamp format; If not, delete the log text of the corresponding line.

4. The method according to claim 1, wherein The preset message node feature includes at least one of a name of the message node and a corresponding value of the message node.

5. A message extraction device for log files, characterized in that: The device comprises: The data processing module performs equalization on the start input timestamp and the end input timestamp extracted from the log file to form a timestamp array containing the equalized timestamps; An interval determination module determines a timestamp interval corresponding to a message to be searched in the timestamp array; A text extraction module extracts log text of each corresponding line from the log file according to the timestamp interval; A text segmentation module, which segments the log text of each corresponding line in sequence according to preset keywords; The feature judgment module, if a preset portion of the log text segment is segmented from the log text of the current corresponding line, then uses a substring search method, with the preset portion of the log text segment as a main string and a preset message node feature as a pattern string, to search the log text segment for the preset message node feature; wherein the preset message node feature includes at least one of a message node name and a corresponding value of the message node; If yes, the result acquisition module uses the log text segment as the message extraction result of the log file.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images