Log analysis method, device, equipment and storage medium

Abstract

The present application is about a log analysis method, device, equipment and storage medium, and specifically relates to the field of electronic digital data processing technology. The method includes: obtaining a symbol file, an executable file and a log file; parsing the symbol file to obtain a function address mapping table; parsing the executable file to obtain a string address mapping table; parsing the log file to obtain a log content table; based on the log content table, the string address mapping table and the function address mapping table, constructing a log content function table containing the function information referenced by each thread at different times to characterize the behavior of the application. The log content function table obtained by the above scheme can directly characterize the execution status of each thread of the application at different times, without the need for manual step-by-step searching in the log and source code, thereby improving the efficiency of log analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a log analysis method, device, equipment and storage medium. Background Art

[0002] Network devices, systems, and service programs all generate an event record called a log when they are operating; each line of the log records the date, time, user, action, and other related operation descriptions.

[0003] When an exception occurs in a program, the logs generated when the application is running can be analyzed to determine the cause of the exception. Usually, when analyzing the logs of an application, it is necessary to manually find the time point when the program exception occurs, find the relevant logs and threads in the log, extract the text of each line of the log, search in the source code, locate the source code that prints this log sentence, record it, and restore the event sequence of the code running before the exception occurs line by line, so as to find the cause of the application exception.

[0004] However, in the above solution, manual search is required in the logs and source codes step by step to obtain the event sequence of the code running, which has low efficiency in log analysis. Summary of the invention

[0005] The present application provides a log analysis method, apparatus, device and storage medium, which improve the efficiency of log analysis. The technical solution is as follows.

[0006] In one aspect, a log analysis method is provided, the method comprising:

[0007] Get symbol files, executable files, and log files;

[0008] Parsing the symbol file to obtain a function address mapping table; the function address mapping table is used to indicate a mapping relationship between a function name and a function address;

[0009] Parsing the executable file to obtain a string address mapping table; the string address mapping table is used to indicate a mapping relationship between string data and function addresses;

[0010] Parsing the log file to obtain a log content table; the log content table is used to indicate character string data corresponding to each thread and time;

[0011] A log content function table including function information referenced by each thread at different times is constructed according to the log content table, the string address mapping table and the function address mapping table to characterize the behavior of the application program.

[0012] In another aspect, a log analysis device is provided, the device comprising:

[0013] File acquisition module, used to obtain symbol files, executable files and log files;

[0014] A symbol file parsing module, used to parse the symbol file to obtain a function address mapping table; the function address mapping table is used to indicate the mapping relationship between function names and function addresses;

[0015] An executable file parsing module, used to parse the executable file to obtain a string address mapping table; the string address mapping table is used to indicate a mapping relationship between string data and function addresses;

[0016] A log file parsing module, used to parse the log file to obtain a log content table; the log content table is used to indicate the character string data corresponding to each thread and time;

[0017] The mapping table construction module is used to construct a log content function table containing function information referenced by each thread at different times according to the log content table, the string address mapping table and the function address mapping table, so as to characterize the behavior of the application.

[0018] In a possible implementation, the mapping table construction module includes:

[0019] A function address query unit, used for querying the target function address corresponding to the target string data in the log content table in the string address mapping table;

[0020] A function name query unit, used to query the target function name corresponding to the target function address in the function address mapping table;

[0021] The mapping table acquisition unit is used to insert the mapping relationship between the target function name and the target string data into the log content table to obtain the log content function table.

[0022] In a possible implementation, the function address query unit is further configured to:

[0023] Perform word segmentation processing on the target character string data to obtain a target word segmentation set of the target character string data;

[0024] The function address corresponding to the character string data with the highest similarity to the target word segmentation set in the character string address mapping table is determined as the target function address.

[0025] In a possible implementation manner, the device further includes:

[0026] The timeline display module is used to display the timelines corresponding to each thread in the log analysis interface, and to display the function names of the functions referenced by each thread on the timelines corresponding to each thread in chronological order.

[0027] In a possible implementation, the timeline display module is further used to:

[0028] Displaying a function control on the time axis corresponding to each of the threads; and displaying the function name of the referenced function superimposed on the function control;

[0029] The device also includes:

[0030] An operation information display module is used to display the operation information corresponding to the function control on the log analysis interface in response to receiving a specified operation on the function control; the operation information at least includes the local variables corresponding to the function name of the referenced function.

[0031] The device also includes:

[0032] The abnormal time point acquisition module is used to query the abnormal snapshot information table and obtain the abnormal time point;

[0033] The abnormal time point display module is used to display the abnormal time point on the log analysis interface.

[0034] In a possible implementation manner, the device further includes:

[0035] The abnormal snapshot information table generation module is used to read the memory process snapshot, obtain the process name, process ID, process version number and abnormal occurrence time stored in the memory process snapshot, and generate the abnormal snapshot information table corresponding to the memory process snapshot.

[0036] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the above-mentioned log analysis method.

[0037] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned log analysis method.

[0038] In another aspect, a computer program product is provided. A computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned log analysis method.

[0039] The technical solution provided by this application may have the following beneficial effects:

[0040] When it is necessary to analyze the log, the computer device can read the symbol file set, executable file and log file corresponding to the application, and parse each file to obtain the mapping relationship between function name and function address, the mapping relationship between string and function address, and string data related to time. Therefore, the computer device can establish the function name referenced by each string in the log file according to the above mapping relationship, so as to construct a log content function table indicating the function information referenced by each thread of the application at different times, thereby directly representing the execution status of each thread of the application at different times, without the need for manual step-by-step searching in the log and source code, thereby improving the efficiency of log analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 The diagram is a structural diagram of a log analysis system according to an exemplary embodiment.

[0043] Figure 2 The figure is a flow chart of a log analysis method according to an exemplary embodiment.

[0044] Figure 3 A schematic diagram of a process of converting a symbol file to a mapping table according to an embodiment of the present application is shown.

[0045] Figure 4 A schematic diagram of a process of converting an executable file string to a mapping table involved in an embodiment of the present application is shown.

[0046] Figure 5 A schematic diagram of a log file conversion mapping table process involved in an embodiment of the present application is shown.

[0047] Figure 6The figure is a flow chart of a log analysis method according to an exemplary embodiment.

[0048] Figure 7 A log analysis interface diagram according to an embodiment of the present application is shown.

[0049] Figure 8 A schematic diagram of a memory snapshot file conversion mapping representation involved in an embodiment of the present application is shown.

[0050] Fig. 9 A schematic diagram of a memory snapshot file conversion mapping representation involved in an embodiment of the present application is shown.

[0051] Fig.10 A schematic diagram of the software module structure of a log analysis system involved in an embodiment of the present application is shown.

[0052] Fig.11 The figure is a structural block diagram of a log analysis device according to an exemplary embodiment.

[0053] Fig.12 It is a schematic diagram of a computer device provided according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solution of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0055] It should be understood that the "indication" mentioned in the embodiments of the present application can be a direct indication, an indirect indication, or an indication of an association relationship. For example, A indicates B, which can mean that A directly indicates B, for example, B can be obtained through A; it can also mean that A indirectly indicates B, for example, A indicates C, and B can be obtained through C; it can also mean that there is an association relationship between A and B.

[0056] In the description of the embodiments of the present application, the term "corresponding" may indicate a direct or indirect correspondence between two items, or an association relationship between the two items, or a relationship between indication and being indicated, configuration and being configured, and the like.

[0057] In an embodiment of the present application, "predefinition" can be achieved by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in a device (for example, including a terminal device and a network device). The present application does not limit its specific implementation method.

[0058] Before describing the various embodiments shown in the present application, several concepts involved in the present application are first introduced.

[0059] 1) Log file (Log)

[0060] Log files are recording files or collections of files used to record system operation events, which can be divided into event logs and message logs. They play an important role in processing historical data, tracing diagnostic problems, and understanding system activities. In computers, log files are files that record events that occur in the operation of an operating system or other software or messages between different users of a communication software. Logging is the act of keeping a log. In the simplest case, messages are written to a single log file. Many operating systems, software frameworks, and programs include logging systems.

[0061] 2) Symbol files

[0062] Symbol Files is a data information file. It contains debugging information of application binary files, such as EXE (executable file), DLL (Dynamic Link Library), etc., which is specially used for debugging. The executable file generated in the end does not need this symbol file when running, but all variable information in the program is recorded in this file. Therefore, this file is very important when debugging an application. This file is used when debugging a program with Visual C++ and WinDbg. In Windows system, symbol files have the extension .pdb. For example: each Windows operating system has a GDI32.dll file. When the compiler compiles the DLL, it will generate a GDI32.pdb file. Once you have this PDB file, you can use it to debug and trace into the GDI32.dll. This file is closely related to the compiled version of the binary file. For example, if the output function of the DLL is modified and the DLL is compiled again, the original PDB (Palm DataBase) file will be outdated. The old PDB file can no longer be used for debugging, and the latest PDB file version must be used.

[0063] 3) Executable file

[0064] An executable file refers to a file that can be loaded and executed by the operating system. In different operating system environments, the presentation of executable programs is different. Under the Windows operating system, executable programs can be .exe files, .sys files, .com files and other types of files. Win32 executable files are called PE files. The basic structure of PE files is very different from DOS executable files. It divides different parts of the program into various sections, one of which can be a section for placing various resources, such as menus, dialog boxes, bitmaps, cursors, icons, and sounds. Although the resource part can be understood as similar to the "overlay" part in the DOS executable file, since the resource is a standard component of the Win32 executable file and a very important component, its format is fixed. Therefore, compared with the development process of DOS software, the development of Win32 software has an additional step of creating a resource file.

[0065] Figure 1 1 is a schematic diagram showing a log analysis system according to an exemplary embodiment. The log analysis system includes a server 110 and a terminal 120 .

[0066] When an application in the terminal 120 is running, a log file corresponding to the application is generated to record the running status of the application.

[0067] Optionally, after the terminal 120 generates a log file corresponding to the application, the log file is uploaded to the server 110 and stored in a data storage module in the server 110 so that the server 110 can subsequently analyze the log file.

[0068] Optionally, the terminal 120 can also upload the executable file and symbol file corresponding to the application to the server 110, and save them in the data storage module in the server 110, so that the server 110 can subsequently analyze the log file through the contents of the executable file, symbol file and log file to determine the historical working status of the application.

[0069] Optionally, when the terminal 120 detects an application exception, the terminal 120 converts the data stored in the memory at this time into a memory snapshot and dumps it in a memory snapshot file. The terminal 120 can upload the memory snapshot file to the server 110 so that the server 110 analyzes the log file according to the memory snapshot file.

[0070] Optionally, the log files, symbol files, executable files and memory snapshot files corresponding to the above-mentioned applications can also be stored in the terminal 120. In this case, the terminal 120 can also analyze the contents of the log files based on the above-mentioned files to determine the historical working status of the application.

[0071] Optionally, the above-mentioned server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms and other technical computing services.

[0072] Optionally, the system may further include a management device, which is used to manage the system (such as managing the connection status between each module and the server, etc.), and the management device is connected to the server via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0073] Optionally, the above-mentioned wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any other network, including but not limited to any combination of local area network, metropolitan area network, wide area network, mobile, limited or wireless network, private network or virtual private network. In some embodiments, the technology and / or format including hypertext markup language, extensible markup language, etc. are used to represent the data exchanged through the network. In addition, conventional encryption technologies such as secure socket layer, transport layer security, virtual private network, Internet protocol security, etc. can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technology can also be used to replace or supplement the above-mentioned data communication technology.

[0074] Figure 2 is a flow chart of a log analysis method according to an exemplary embodiment. The method is executed by a server or a terminal, and the server or the terminal may be as follows Figure 1 The server or terminal in the log analysis system shown. Figure 2 As shown, the log analysis method may include the following steps:

[0075] Step 201, obtaining symbol files, executable files and log files.

[0076] In the embodiment of the present application, the log file to be analyzed is the operation record generated by the application during operation. Taking the Windows platform as an example, the log text is a document recorded in the application code during operation, and the content is line text, such as gxxx_log.1.txt. The main function is to record the actions or exceptions on the timeline of the program running process, so that the operation and maintenance personnel can trace the behavior of the program during operation by analyzing the log.

[0077] In a possible implementation of the embodiment of the present application, the log file contains at least a module name, a version number, a time, a process ID, a thread ID, and a string text.

[0078] The module name in the log file is used to indicate which module generated the log text, the version number is the version number of the module, the time is the generation time, the process ID is used to indicate the process ID that generated the log, the thread ID is used to indicate the thread ID that generated the log, and the string text is used to indicate the actions actually performed by the application in the log text.

[0079] In the embodiment of the present application, since the log content table is expanded, the string text in the log content table is used in the log content table to query the mapping table generated by other files until the function corresponding to the log text is queried.

[0080] In the embodiment of the present application, taking the Windows platform as an example, the executable file can be a binary executable file, usually a PE format file (PE, Portable Executable) format, which is the standard file format of portable executable files (such as exe, dll, vxd, sys and vdm, etc.) in the Microsoft Win32 environment. It is an executable program module unit generated after code compilation. The main function is to load into the content at runtime to provide specific functional services. There are two common types: exe file or dll file.

[0081] In a possible implementation of the embodiment of the present application, the executable file includes a process name, a module name, an address, a length, a string text, and a reference address.

[0082] In the executable file, the module name is the name of the executable file (that is, the name of the execution module), the process name indicates the process corresponding to the executable file when the application is executed; the address is the address of the executable part in the executable file, the length is the length of the executable part, the string text is the execution resource in the executable file, and the reference address is the reference address of the function that processes the execution resource corresponding to the string text.

[0083] In the embodiment of the present application, taking the Windows platform as an example, the symbol file generally refers to a .pdb file, which is mainly used to map the name symbol in the source code with the machine code of the address at binary runtime during debugging, so that the runtime binary code can be translated by symbols to improve the readability of the debugging code. For example, the symbol file corresponding to the above calc.exe is calc.pdb. The symbol file corresponding to the above capisp.dll is capisp.pdb.

[0084] In a possible implementation of the embodiment of the present application, the symbol file includes a module name, a version number, a function name, a function address, and a function length. The module name is the name of the symbol file, which usually corresponds to the module name of the binary executable file, and the version number is the version number of the symbol file; and the symbol file also includes the function name, function address, and function length of each function called during the execution of the executable file.

[0085] Step 202, parse the symbol file to obtain a function address mapping table.

[0086] The function address mapping table is used to indicate the mapping relationship between function names and function addresses, and the function address mapping table is stored in a computer device as relational two-dimensional data. When the computer device obtains a symbol file, the module name, version number, function name, function address, and function length in the symbol file can be extracted to obtain the "module name, version number, function name, function address, and function length" in each line or each text interval in the symbol file, and a function address mapping table is generated according to the mapping relationship.

[0087] For example, Figure 3 A schematic diagram of a process of converting a symbol file to a mapping table involved in an embodiment of the present application is shown. When a symbol file parser (i.e., symbol file parsing software) in a computer device opens a symbol file, the symbol file parser reads each function in the symbol file, obtains the module name, version number, function name, function address, function length corresponding to each function, and generates corresponding records, and finally inserts each record into a function address mapping table, where each record in the function address mapping table represents a mapping.

[0088] In the function address mapping table generated above, after the module name and version number are determined, the function address corresponding to the function and the function length of the function can be determined according to the function name of the module reference function.

[0089] And in a possible implementation, since the text stored in the symbol file inside the computer device is distributed according to a certain pattern, when the computer device needs to extract the symbol file, it can perform word segmentation processing according to the rules corresponding to the symbol file (for example, each line of text is divided according to the number of characters, or according to the identifier inside the symbol file), thereby dividing the content in the symbol file into "module name, version number, function name, function address, function length".

[0090] Step 203: parse the executable file to obtain a string address mapping table.

[0091] The string address mapping table is used to indicate the mapping relationship between string data and function addresses. The string address mapping table is stored in the computer device as relational two-dimensional data. When the computer device obtains the executable file, it can extract the "process name, module name, address, length, string text and reference address" in the executable file and generate a function address mapping table according to the mapping relationship.

[0092] For example, Figure 4 A schematic diagram of a process of converting an executable file string to a mapping table according to an embodiment of the present application is shown. Figure 4 As shown, after the binary executable file parser (i.e., software used to parse binary specifiable files) loads the binary executable file, it can read each string resource (including string content, length, reference address, etc.) in the binary executable file and treat it as a record. Finally, each record (i.e., each string information) is inserted into the mapping table (i.e., the two-dimensional table record) as individual mappings in the mapping table.

[0093] That is, after the string address mapping table is generated, the computer device can search the string address mapping table according to the string data to obtain the reference address corresponding to the string data.

[0094] And in a possible implementation, since the text stored in the executable file inside the computer device is distributed according to a certain pattern, when the computer device needs to extract the executable file, it can perform word segmentation processing according to the rules corresponding to the symbol file (for example, each line of text is divided according to the number of characters, or according to the identifier inside the executable file), thereby dividing the content in the executable file into "process name, module name, address, length, string text and reference address".

[0095] Step 204: parse the log file to obtain a log content table.

[0096] The log content table is used to indicate the string data corresponding to each thread and time. When the computer device obtains the log file, for each line of text in the log file, it contains "module name, version number, time, process ID, thread ID, string text". The computer device can perform word segmentation on each line of text in the log file to obtain "module name", "version number", "process ID", "thread ID", and "string text" respectively, and establish the corresponding log content table to save the mapping relationship between the string text and "module name, version number, time, process ID, thread ID".

[0097] For example, Figure 5 A schematic diagram of a log file conversion mapping table process involved in an embodiment of the present application is shown. Figure 5 As shown, after the log content parser (i.e., software for parsing log content) opens the text file, the log content parser reads each line of text in the log content (i.e., each log), and extracts the time, process ID, thread ID, content and other fields contained in the log, and generates a record corresponding to the log. The log content parser inserts each record into the mapping table (i.e., the two-dimensional table record) as each mapping in the mapping table.

[0098] Step 205 , constructing a log content function table containing function information referenced by each thread at different times according to the log content table, the string address mapping table and the function address mapping table, so as to characterize the behavior of the application.

[0099] After the computer device obtains the log content table, the string address mapping table and the function address mapping table, the computer device can query the string address mapping table according to the string text recorded in the log content table to obtain the reference address corresponding to the string text, which is the address of the function referenced by the log content.

[0100] At this time, the computer device uses the reference address to query the function address mapping table. Since the function address mapping table records the function address and function length corresponding to each function name, the function address mapping table can determine the function interval in which the reference address falls and return the function name corresponding to the function interval.

[0101] At this time, the computer device obtains the function name of the function referenced by the log content. When the function name corresponding to each log content (i.e., each row in the log content table) is obtained, the log content function table can be constructed at this time, and each row of the log content function table includes {module name, version number, time, process ID, thread ID, string text, function name}. Therefore, the log content function table can represent each execution module, the process ID and thread ID occupied within the specified time, the string text used in the thread indicated by each thread ID, and the referenced function name, thereby more completely showing the running status of the application.

[0102] To sum up, when it is necessary to analyze the log, the computer device can read the symbol file set, executable file and log file corresponding to the application, and parse each file, so as to obtain the mapping relationship between function name and function address, the mapping relationship between string and function address, and string data related to time respectively; therefore, the computer device can establish the function name referenced by each string in the log file according to the above mapping relationship, so as to construct a log content function table indicating the function information referenced by each thread of the application at different times, so as to directly characterize the execution status of each thread of the application at different times, without the need for manual step-by-step searching in the log and source code, thereby improving the efficiency of log analysis.

[0103] Figure 6 is a flow chart of a log analysis method according to an exemplary embodiment. The method is executed by a server or a terminal, and the server or the terminal may be as follows Figure 1 The server or terminal in the log analysis system shown. Figure 6 As shown, the log analysis method may include the following steps:

[0104] Step 601, obtaining symbol files, executable files and log files.

[0105] Step 602, parse the symbol file to obtain a function address mapping table.

[0106] Step 603: parse the executable file to obtain a string address mapping table.

[0107] Step 604: parse the log file to obtain a log content table.

[0108] The specific implementations of step 601 and step 604 are similar to the specific implementations of step 201 to step 204 and are not described again here.

[0109] Step 605: In the string address mapping table, query the target function address corresponding to the target string data in the log content table.

[0110] For the target string data in the log content table, when the target string data needs to be analyzed, since the string address mapping table contains a mapping relationship between the string data and the function address, the target function address corresponding to the target string data can be found when searching the string address mapping table based on the target string data.

[0111] In a possible implementation, the target string data is segmented to obtain a target segmentation set of the target string data; and the function address corresponding to the string data with the highest similarity to the target segmentation set in the string address mapping table is determined as the target function address.

[0112] Since the format of the target string data in the log content table (i.e., the target string data extracted from the log file) may not be completely consistent with the format of the string data in the string address mapping table (i.e., the string data from the binary executable file), for example, the target string data in the log content table contains identifiers such as the percent sign, while the string in the string address mapping table does not contain identifiers, the target string data in the log content table may not find the same string data in the string address mapping table.

[0113] At this time, the computer device can perform word segmentation processing on the target string data to obtain a target word segmentation set of the target string data, and then compare the target word segmentation set with the string data in the string address mapping table for similarity, and determine the function address corresponding to the string data with the highest similarity as the target function address, thereby avoiding query failures due to inconsistent formats.

[0114] In another possible implementation, the target string data is segmented and the segmentation results are vectorized to obtain a target string vector; each string data in the string address mapping table is segmented and the segmentation results are vectorized to obtain each query vector corresponding to the string address mapping table; the function address indicated by the query vector having the smallest vector distance with the target string vector is determined as the target function address.

[0115] That is, when it is necessary to perform a query operation based on the target string data in the string address mapping table, the target string data and each string data in the string address mapping table can also be vectorized separately. For example, when the target string data is 012042045061, a one-dimensional vector value is generated for every three data (for example, an accumulation operation is performed) to generate a target string vector (3, 6, 9, 7); similarly, each string data in the string address mapping table can also be operated as follows to calculate the vector distance between vectors, thereby determining the target function address.

[0116] Step 606: Look up the target function name corresponding to the target function address in the function address mapping table.

[0117] After the target function address corresponding to the target string data is obtained, the target function name corresponding to the target function address can be queried in the function address mapping table to obtain the target function name corresponding to the target string data.

[0118] In one possible implementation, the function address mapping table stores the correspondence between the function name, function address, and function length, that is, for a certain function name, the function address mapping table contains the starting address of the function name and the length of the function name, so that the function interval corresponding to the function name can be obtained; when the function interval corresponding to the target function address is determined according to the function address mapping table, the function name corresponding to the function interval corresponding to the target function address is the target function name.

[0119] Step 607: insert the mapping relationship between the target function name and the target string data into the log content table to obtain the log content function table.

[0120] After obtaining the target function name corresponding to the target string data, it can be determined that there is a mapping relationship between the target string data and the target function name, and then the mapping relationship between the target function name and the target string is inserted into the log content table, thereby generating a new log content function table.

[0121] In the log content function table, in addition to the module name, process ID, thread ID, time, and string data contained in the original log content table, there is also a function name corresponding to the string data, so that the target process of the target module name can be determined, and the target string data is processed by the target function indicated by the target function name within the target time of the target thread.

[0122] Step 608: Display the time axis corresponding to each thread in the log analysis interface, and display the function name of the self-referenced function on the time axis corresponding to each thread in chronological order.

[0123] Please refer to Figure 7 , which shows a schematic diagram of a log analysis interface involved in an embodiment of the present application. When a log content function table of function information referenced by each thread at different times is constructed, the log content function table records the function names of the functions referenced by each thread according to time. For each thread, the computer device can generate a timeline and display the function names of the functions referenced by the thread (such as counter, obj.funct, etc.) on the timeline according to time.

[0124] In one possible implementation, a function control is displayed on the timeline corresponding to each of the threads; the function name of the referenced function is superimposed and displayed on the function control; in response to receiving a specified operation on the function control, operation information corresponding to the function control is displayed on the log analysis interface; the operation information includes at least local variables corresponding to the function name of the referenced function.

[0125] That is, Figure 7 On the log analysis interface shown, when the log analysis interface is displayed on a terminal device, the user can use the mouse or touch operation to click on the function controls displayed on the timeline corresponding to each thread, thereby displaying the specific operation information of the referenced function (such as the local variables used by the function).

[0126] In a possible implementation, the abnormal snapshot information table is queried to obtain the abnormal time point; and the abnormal time point is displayed on the log analysis interface.

[0127] The computer device also stores an exception snapshot information table of the application, which records the abnormal time point when the application exception occurs. After the computer device reads the abnormal time point, it can display the abnormal time point on the log analysis interface, so that the user can directly analyze the abnormal situation of the log according to the abnormal time point.

[0128] In a possible implementation, an abnormal time control with abnormal time information superimposed is displayed on the timeline in the log analysis interface; in response to receiving a second designated operation on the abnormal time control, stack information is displayed on the log analysis interface.

[0129] The abnormal snapshot information table can also save the thread ID of the application where the abnormality occurs. According to the thread ID corresponding to the abnormal thread, the log analysis interface can be used to determine the abnormality of the application. Figure 7 The exception thread shown in , and in Figure 7The time axis on the log analysis interface shown shows an abnormal time control, on which the abnormal time point "17.20" is superimposed. When the user clicks or touches the abnormal time control, the log analysis interface can also display the stack information read from the abnormal memory snapshot.

[0130] In a possible implementation, a memory process snapshot is read to obtain each process name, process ID, process version number, and exception occurrence time stored in the memory process snapshot, and an exception snapshot information table corresponding to the memory process snapshot is generated.

[0131] When an exception occurs, the application in the computer device will generate a snapshot of the memory at that time and dump it into an abnormal memory process snapshot file. The computer device can parse the abnormal memory process snapshot file (and the memory process snapshot) to obtain the process name, process ID, process version number and abnormal occurrence time stored in the memory process snapshot, and generate an abnormal snapshot information table corresponding to the memory process snapshot.

[0132] Please refer to Figure 8 , which shows a schematic diagram of a memory snapshot file conversion mapping involved in an embodiment of the present application.

[0133] During the process memory snapshot parsing process, the process memory snapshot parser in the computer device (i.e., software used to parse the process memory snapshot) opens the process snapshot file, reads the basic information in the process memory snapshot (i.e., process name, version number, exception occurrence time, process ID, etc.), and reads the call stack of each thread, thereby determining each function in the call stack (including function name, function address, etc.), and inserts the above basic information and the call stack into the process exception basic information table (i.e., exception snapshot information table).

[0134] In addition, the process snapshot file can also parse out the basic information of the modules used by each thread (including module name, version number, module loading address, module size, etc.), and combine the basic information of each module with the function information stored in the call stack to generate a module information table, thereby indicating the module running status and memory usage status of the application when an exception occurs.

[0135] To sum up, when it is necessary to analyze the log, the computer device can read the symbol file set, executable file and log file corresponding to the application, and parse each file, so as to obtain the mapping relationship between function name and function address, the mapping relationship between string and function address, and string data related to time respectively; therefore, the computer device can establish the function name referenced by each string in the log file according to the above mapping relationship, so as to construct a log content function table indicating the function information referenced by each thread of the application at different times, so as to directly characterize the execution status of each thread of the application at different times, without the need for manual step-by-step searching in the log and source code, thereby improving the efficiency of log analysis.

[0136] Please refer to Fig. 9 , which is a schematic diagram of an abnormal log analysis process according to an embodiment of the present application. Fig. 9 As shown, the steps of the abnormal log analysis are as follows.

[0137] Step 901, obtaining uploaded data.

[0138] 1) Get the uploaded symbol file set. Generally, symbol files have multiple formats, including binary structured storage file format and string format, triggering symbol file parsing (see Figure 7 ), after the upload is completed, the parsing action will be triggered to extract the function set and the address range of the function from the symbol file and store them in the relational database function address mapping table.

[0139] 2) Get the uploaded executable file and module file set. Generally, executable files will be in different binary file formats on different operating systems, including executable files and dynamic libraries. There will generally be an independent relocation table inside, which mainly contains string constants, from which the content can be extracted to the string address mapping table.

[0140] 3) Get the uploaded process memory snapshot file. When an exception occurs in the process, an exception context is generally generated. At the same time, the memory snapshot will dump the current memory of the process into a file. Relevant information can be parsed from the file and added to the snapshot information table.

[0141] 4) After obtaining the uploaded log file, the parsing action is triggered. The parser parses the text in the log file line by line, extracts each time, process ID, thread ID, and string content, and adds them to the log content table.

[0142] Step 902, the computer device starts the analysis. At this time, the string content in each log is extracted from the log content table, and the string is matched in the string address mapping table to obtain the address. That is, the function address of this log is printed out; then the string reference address is used to query the function address mapping table, the address range of each function (function address + function size), and if the reference address falls within the function address range, the function name is obtained.

[0143] Use the function name to associate with the corresponding log content record, obtain a log record containing the function name, and add it to a new log content function table.

[0144] Step 903: Generate a workflow diagram.

[0145] According to the log content function table, a multi-threaded operation diagram on the timeline can be generated.

[0146] Step 904: display the details of the workflow diagram.

[0147] The user can click on the above workflow diagram to see the details. The details can be extracted in the process snapshot. More fields are saved in the two-dimensional table, such as stack information, local variables, etc., to build a more detailed workflow diagram.

[0148] Please refer to Fig.10 , which shows a schematic diagram of the software module structure of a log analysis system involved in an embodiment of the present application. The abnormal log analysis system includes a data parser, a correlation analyzer, and a chart generator. The data parser, the correlation analyzer, and the chart generator are all modules abstracted by software, and actually perform corresponding functions through software or code.

[0149] Data parser: mainly parses the input data of different formats and converts it into a relational two-dimensional table. The data parser includes an exception log parser, a symbol file parser, a process memory snapshot parser, and a binary executable file parser to parse and process each type of file.

[0150] The association analyzer is used to establish association mapping relationships in a two-dimensional table to form a new relationship table. The association analyzer includes: process and thread extractors, which are used to obtain threads and processes corresponding to each mapping relationship; log string location analyzer, which is used to match the log content in the log file with the source code in the binary executable file; abnormal time interval location analysis, which is used to match the abnormal time point on the abnormal memory snapshot with the time point in the log content.

[0151] Chart generator: Through the above correlation analysis, a multi-threaded distribution chart of source code functions on the timeline is finally formed.

[0152] Fig.11The following is a block diagram of a log analysis device according to an exemplary embodiment. The device includes:

[0153] The file acquisition module 1101 is used to acquire symbol files, executable files and log files;

[0154] The symbol file parsing module 1102 is used to parse the symbol file to obtain a function address mapping table; the function address mapping table is used to indicate the mapping relationship between the function name and the function address;

[0155] The executable file parsing module 1103 is used to parse the executable file to obtain a string address mapping table; the string address mapping table is used to indicate the mapping relationship between string data and function addresses;

[0156] The log file parsing module 1104 is used to parse the log file to obtain a log content table; the log content table is used to indicate the character string data corresponding to each thread and time;

[0157] The mapping table construction module 1105 is used to construct a log content function table containing function information referenced by each thread at different times according to the log content table, the string address mapping table and the function address mapping table, so as to characterize the behavior of the application.

[0158] In a possible implementation, the mapping table construction module includes:

[0159] A function address query unit, used for querying the target function address corresponding to the target string data in the log content table in the string address mapping table;

[0160] A function name query unit, used to query the target function name corresponding to the target function address in the function address mapping table;

[0161] The mapping table acquisition unit is used to insert the mapping relationship between the target function name and the target string data into the log content table to obtain the log content function table.

[0162] In a possible implementation, the function address query unit is further configured to:

[0163] Perform word segmentation processing on the target character string data to obtain a target word segmentation set of the target character string data;

[0164] The function address corresponding to the character string data with the highest similarity to the target word segmentation set in the character string address mapping table is determined as the target function address.

[0165] In a possible implementation manner, the device further includes:

[0166] The timeline display module is used to display the timelines corresponding to each thread in the log analysis interface, and to display the function names of the functions referenced by each thread on the timelines corresponding to each thread in chronological order.

[0167] In a possible implementation, the timeline display module is further used to:

[0168] Displaying a function control on the time axis corresponding to each of the threads; and displaying the function name of the referenced function superimposed on the function control;

[0169] The device also includes:

[0170] An operation information display module is used to display the operation information corresponding to the function control on the log analysis interface in response to receiving a specified operation on the function control; the operation information at least includes the local variables corresponding to the function name of the referenced function.

[0171] The device also includes:

[0172] The abnormal time point acquisition module is used to query the abnormal snapshot information table and obtain the abnormal time point;

[0173] The abnormal time point display module is used to display the abnormal time point on the log analysis interface.

[0174] In a possible implementation manner, the device further includes:

[0175] The abnormal snapshot information table generation module is used to read the memory process snapshot, obtain the process name, process ID, process version number and abnormal occurrence time stored in the memory process snapshot, and generate the abnormal snapshot information table corresponding to the memory process snapshot.

[0176] To sum up, when it is necessary to analyze the log, the computer device can read the symbol file set, executable file and log file corresponding to the application, and parse each file, so as to obtain the mapping relationship between function name and function address, the mapping relationship between string and function address, and string data related to time respectively; therefore, the computer device can establish the function name referenced by each string in the log file according to the above mapping relationship, so as to construct a log content function table indicating the function information referenced by each thread of the application at different times, so as to directly characterize the execution status of each thread of the application at different times, without the need for manual step-by-step searching in the log and source code, thereby improving the efficiency of log analysis.

[0177] See also Fig.12, which is a schematic diagram of a computer device provided according to an exemplary embodiment of the present application, the computer device includes a memory and a processor, the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned log analysis method is implemented.

[0178] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0179] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above method embodiment.

[0180] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0181] In an exemplary embodiment, a computer-readable storage medium is also provided, which is used to store at least one computer program, and the at least one computer program is loaded and executed by a processor to implement all or part of the steps in the above method. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0182] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0183] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A log analysis method, characterized in that: The method comprises: Get symbol files, executable files, and log files; Parsing the symbol file to obtain a function address mapping table; the function address mapping table is used to indicate a mapping relationship between a function name and a function address; Parsing the executable file to obtain a string address mapping table; the string address mapping table is used to indicate a mapping relationship between string data and function addresses; Parsing the log file to obtain a log content table; the log content table is used to indicate character string data corresponding to each thread and time; According to the log content table, the string address mapping table and the function address mapping table, a log content function table including function information referenced by each thread at different times is constructed to characterize the behavior of the application program; The step of constructing a log content function table including function information referenced by each thread at different times according to the log content table, the string address mapping table, and the function address mapping table includes: In the string address mapping table, query the target function address corresponding to the target string data in the log content table; In the function address mapping table, query the target function name corresponding to the target function address; The mapping relationship between the target function name and the target character string data is inserted into the log content table to obtain the log content function table.

2. The method according to claim 1, characterized in that The querying of the target function address corresponding to the target string data in the log content table in the string address mapping table includes: Perform word segmentation processing on the target character string data to obtain a target word segmentation set of the target character string data; The function address corresponding to the character string data with the highest similarity to the target word segmentation set in the character string address mapping table is determined as the target function address.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: The time axis corresponding to each thread is displayed in the log analysis interface, and the function names of the functions referenced by each thread are displayed on the time axis corresponding to each thread in chronological order.

4. The method according to claim 3, characterized in that The function names of the self-referenced functions are displayed on the time axes corresponding to the threads, respectively, including: Displaying a function control on the time axis corresponding to each of the threads; and displaying the function name of the referenced function superimposed on the function control; The method further comprises: In response to receiving a designated operation on the function control, operation information corresponding to the function control is displayed on the log analysis interface; the operation information at least includes a local variable corresponding to the function name of the referenced function.

5. The method according to claim 3, characterized in that: The method further comprises: Query the abnormal snapshot information table to obtain the abnormal time point; The abnormal time point is displayed on the log analysis interface.

6. The method according to claim 5, characterized in that The querying of the abnormal snapshot information table to obtain the abnormal time point includes: The memory process snapshot is read to obtain each process name, process ID, process version number and exception occurrence time stored in the memory process snapshot, and an exception snapshot information table corresponding to the memory process snapshot is generated.

7. A log analysis device, characterized in that: The device comprises: File acquisition module, used to obtain symbol files, executable files and log files; A symbol file parsing module, used to parse the symbol file to obtain a function address mapping table; the function address mapping table is used to indicate the mapping relationship between function names and function addresses; An executable file parsing module, used to parse the executable file to obtain a string address mapping table; the string address mapping table is used to indicate a mapping relationship between string data and function addresses; A log file parsing module, used to parse the log file to obtain a log content table; the log content table is used to indicate the character string data corresponding to each thread and time; A mapping table construction module, used to construct a log content function table containing function information referenced by each thread at different times according to the log content table, the string address mapping table and the function address mapping table, so as to characterize the behavior of the application program; The mapping table construction module is used to query the target function address corresponding to the target string data in the log content table in the string address mapping table; In the function address mapping table, query the target function name corresponding to the target function address; The mapping relationship between the target function name and the target character string data is inserted into the log content table to obtain the log content function table.

8. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the log analysis method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the log analysis method according to any one of claims 1 to 6.

Images