Log analysis method and system, computer equipment and readable storage medium

Abstract

The invention provides a log analysis method and system, computer equipment and a readable storage medium, and the method comprises the steps: extracting a script character string from a selected script through a sub-thread, splicing a script method name and a script character string which are configured in advance or in real time, the data exchange format object of each piece of log data to be analyzed, the identification parameter of the first piece of log data and the identification parameter of the last piece of log data to generate a script method character string; the script method character string is converted into a script method to be executed, and an execution conclusion is called back to the main thread; and transmitting the execution conclusion to a front-end page for display through the main thread. According to the method, the sub-thread is arranged outside the main thread, the sub-thread and the main thread do not interfere with each other, each log executes the selected script once in the sub-thread, and after the code execution of the sub-thread is completed, the execution result is called back to the main thread to be processed and then transmitted to the front-end page to be rendered and displayed. And therefore, main thread blockage and page lagging cannot be caused.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a log analysis method, system, computer device and readable storage medium. Background Art

[0002] The browser Javascript (an advanced, interpreted or just-in-time compiled programming language, referred to as JS) is single-threaded. When JS performs high-load operations, the user interface (UI) rendering will be blocked, the page will be stuck, and the user experience will be poor. Even setTimeout and setInterval are only asynchronous and single-threaded, and cannot fundamentally solve the problem. Among them, setTimeout is a function in JavaScript that is used to execute a function or a specified section of code after a specified delay time. setInterval is a function in JavaScript that is used to repeatedly call a function or execute a section of code, once at a fixed time interval. Summary of the invention

[0003] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a log analysis method, system, computer device and readable storage medium to solve the technical problems existing in the prior art.

[0004] To achieve the above purpose and other related purposes, the present application provides a log analysis method, comprising the following steps:

[0005] Establishing multi-thread communication, wherein the multi-thread includes a main thread and a sub-thread;

[0006] Extracting a script string from the selected script through the child thread, and concatenating a pre- or real-time configured script method name with the script string, a data exchange format object of each log data to be analyzed, a first log data identification parameter, and a last log data identification parameter to generate a script method string; and converting the script method string into a script method for execution, and calling back the execution conclusion to the main thread;

[0007] The execution conclusion is transmitted to the front-end page for display through the main thread.

[0008] In one embodiment of the present application, before extracting the script string from the selected script by the child thread, the method further includes:

[0009] Obtaining pre- or real-time script list information and log data to be analyzed through the main thread; and selecting a script from the script list information to analyze all log data to be analyzed, extracting a data exchange format object for each log data to be analyzed, and transmitting the selected script, the data exchange format object for each log data to be analyzed, and log data identification parameters to the child thread through a preset thread mechanism;

[0010] The log data identification parameters include: the first log data identification parameter and the last log data identification parameter.

[0011] In one embodiment of the present application, before obtaining the script list information written in advance or in real time through the main thread, the method further includes:

[0012] Configure the editor on the front-end page;

[0013] A script written by a target object in the editor based on business logic is received, and when a script code corresponding to the script does not change within a preset time, the script list information is generated, and the script code is stored in a server.

[0014] In one embodiment of the present application, the process of converting the script method string into a script method for execution and calling back the execution conclusion to the main thread includes:

[0015] Convert the script method string into a script method by using a preset conversion function;

[0016] Loading the script method into a target storage area, and executing the script method in the target storage area to obtain an execution conclusion;

[0017] The execution conclusion is called back to the main thread through a callback function.

[0018] In one embodiment of the present application, the process of selecting a script from the script list information to analyze all log data to be analyzed includes:

[0019] Select a script from the script list information as a target script;

[0020] All log data to be analyzed are analyzed through the target script, and a scheduled task is started on the front-end page; wherein the scheduled task is used to extract the data exchange format object of each log data to be analyzed and the target script.

[0021] In one embodiment of the present application, the default thread mechanism includes a Web Worker mechanism.

[0022] The present application also provides a log analysis system, the system comprising:

[0023] A thread communication module, used to establish multi-thread communication, wherein the multi-thread includes a main thread and sub-threads;

[0024] A child thread communication module is used to extract a script string from the selected script through the child thread, and concatenate a pre- or real-time configured script method name with the script string, a data exchange format object of each log data to be analyzed, a first log data identification parameter, and a last log data identification parameter to generate a script method string; and convert the script method string into a script method for execution, and call back the execution conclusion to the main thread;

[0025] The front-end page module is used to transmit the execution conclusion to the front-end page for display through the main thread.

[0026] In one embodiment of the present application, the system also includes: a main thread communication module, which is used to obtain pre- or real-time script list information and log data to be analyzed through the main thread; and select a script from the script list information to analyze all log data to be analyzed, extract the data exchange format object of each log data to be analyzed, and transmit the selected script, the data exchange format object of each log data to be analyzed, and the log data identification parameters to the child thread through a preset thread mechanism; wherein the log data identification parameters include: the first log data identification parameter and the last log data identification parameter.

[0027] The present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the log analysis methods described above.

[0028] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the log analysis methods described above are implemented.

[0029] As described above, the present application provides a log analysis method, system, computer device and readable storage medium, which have the following beneficial effects: the present application extracts a script string from a selected script through a child thread, and splices the pre- or real-time configured script method name with the script string, the data exchange format object of each log data to be analyzed, the first log data identification parameter and the last log data identification parameter to generate a script method string; and, converts the script method string into a script method for execution, and calls back the execution conclusion to the main thread; and transmits the execution conclusion to the front-end page for display through the main thread. It can be seen from this that the present application can start a polling task after selecting a script, quickly scan a large number of user logs, and each log will execute the selected script once in the child thread, and finally call back to the main thread and output the display business conclusion on the page, so as to achieve the purpose of efficiently analyzing a large number of user logs on the front-end page according to different scripts, and displaying them on the front-end page. Therefore, this application sets up a child thread outside the main thread, and the operation of the child thread and the main thread do not interfere with each other. When the child thread code is executed, the execution result is called back to the main thread for processing and then transmitted to the front-end page for rendering and display. This will not cause the main thread to be blocked and the page to be stuck, thus solving the problem of page stuck during single-thread execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of an exemplary system architecture for applying the technical solutions in one or more embodiments of the present application;

[0031] Figure 2 A schematic diagram of the script generation provided in an embodiment of the present application;

[0032] Figure 3 A flow chart of a log analysis method provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of the hardware structure of a log analysis system provided in an embodiment of the present application;

[0034] Figure 5 The figure is a schematic diagram of the hardware structure of a computer device suitable for implementing one or more embodiments of the present application. DETAILED DESCRIPTION

[0035] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0036] It should be noted that the illustrations provided in this embodiment only illustrate the basic concept of the present application in a schematic manner, and therefore the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0037] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution in one or more embodiments of the present application can be applied is shown. Figure 1 As shown, the system architecture 100 may include a terminal device 110, a network 120, and a server 130. The terminal device 110 may include various electronic devices such as a smart phone, a tablet computer, a laptop computer, and a desktop computer. The server 130 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The network 120 may be a communication medium of various connection types that can provide a communication link between the terminal device 110 and the server 130, such as a wired communication link or a wireless communication link.

[0038] According to the implementation requirements, the system architecture in the embodiment of the present application can have any number of terminal devices, networks and servers. For example, the server 130 can be a server group composed of multiple server devices. In addition, the technical solution provided in the embodiment of the present application can be applied to the terminal device 110, can also be applied to the server 130, or can be implemented by the terminal device 110 and the server 130 together, and the present application does not make any special restrictions on this.

[0039] In one embodiment of the present application, the terminal device 110 or server 130 of the present application can extract the script string from the selected script through a child thread, and splice the script method name configured in advance or in real time with the script string, the data exchange format object of each log data to be analyzed, the first log data identification parameter and the last log data identification parameter to generate a script method string; and, convert the script method string into a script method for execution, and call back the execution conclusion to the main thread; and transmit the execution conclusion to the front-end page for display through the main thread. Using the terminal device 110 or the server 130 to execute the log analysis method, the polling task can be started after selecting a script, and a large number of user logs can be quickly scanned. Each log will execute the selected script once in the child thread, and finally call back to the main thread and output the display business conclusion on the page, so as to achieve the purpose of efficiently analyzing a large number of user logs on the front-end page according to different scripts and displaying them on the front-end page. Therefore, by setting a child thread outside the main thread, the operation of the child thread and the main thread does not interfere with each other. When the child thread code is executed, the execution result is called back to the main thread for processing and then transmitted to the front-end page for rendering and display, which will not cause the main thread to be blocked and the page to freeze.

[0040] The above section introduces the contents of an exemplary system architecture that applies the technical solution of the present application. Next, we will continue to introduce the log analysis method of the present application.

[0041] In an exemplary embodiment, the embodiment provides a log analysis method, the method comprising the following steps:

[0042] Establish multi-thread communication, multi-threading includes main thread and sub-thread;

[0043] Obtaining pre-written or real-time script list information and log data to be analyzed through the main thread; and selecting a script from the script list information to analyze all log data to be analyzed, extracting the data exchange format object of each log data to be analyzed, and transmitting the selected script, the data exchange format object of each log data to be analyzed, and the log data identification parameters to the child thread through the preset thread mechanism; wherein the log data identification parameters include: the first log data identification parameter and the last log data identification parameter. As an example, in this embodiment or other embodiments, the preset thread mechanism includes but is not limited to the Web Worker mechanism.

[0044] A script string is extracted from the selected script through a child thread, and the pre- or real-time configured script method name is concatenated with the script string, the data exchange format object of each log data to be analyzed, the first log data identification parameter, and the last log data identification parameter to generate a script method string; and the script method string is converted into a script method for execution, and the execution conclusion is called back to the main thread.

[0045] The execution conclusion is transmitted to the front-end page for display through the main thread.

[0046] It can be seen that this embodiment can start the polling task after selecting a certain script, quickly scan a large number of user logs, and each log will execute the selected script once in the child thread, and finally call back to the main thread and output the display business conclusion on the page, so as to achieve the visualization purpose of efficiently analyzing a large number of user logs on the front-end page according to different scripts and displaying them on the front-end page. It is equivalent to the child thread being responsible for script execution and log analysis, and the main thread being responsible for rendering and displaying the conclusions after the child thread analysis, which greatly reduces the burden of the main thread, and the page is still smooth when processing large amounts of logs. At the same time, since the user will call the logCallback function when writing the script, this function will send the analysis conclusion information that needs to be output in the script to the main thread through the Web Worker mechanism, and the main thread will listen to this callback event, obtain the analysis conclusion information in the callback event and display it in the front-end page, so as to achieve efficient analysis of large amounts of logs through front-end timing tasks and multi-threaded communication. Therefore, this embodiment sets up a child thread outside the main thread, and the operation of the child thread and the main thread do not interfere with each other. When the child thread code is executed, the execution result is called back to the main thread for processing and then transmitted to the front-end page for rendering and display. This will not cause the main thread to be blocked and the page to be stuck, thus solving the problem of page stuck during single-thread execution.

[0047] According to the above records, if Figure 2As shown, in an exemplary embodiment, before obtaining the script list information written in advance or in real time through the main thread, this embodiment may also include: configuring an editor on the front page, receiving the script written by the target object in the editor based on the business logic, and when the script code corresponding to the script does not change within the preset time, generating script list information, and storing the script code in the server. Specifically, in the editor preset on the front page, the target object can start writing the script of the analysis log according to the business logic and target parameters combed in advance, and when the script needs to be output to the front page, the logCallback function is called to call back the conclusion information, and the script code is saved to the server after the script is written. Wherein, the logCallback function is a callback function (CallbackFunction), which is usually designed to be called when a specific log event occurs. A callback function is a function passed as a parameter to another function, which is called when a certain operation is completed or a certain event occurs. In the context of log records, the logCallback function can be used to process or respond to log messages, such as sending log messages to a remote server, displaying them in a console, updating the user interface to reflect the log status, etc. In this embodiment or other embodiments, the target parameters may be the following four parameters preset in the editor: logInfo: a single log JSON (JavaScript Object Notation, JSON for short) object, isFirstLog: whether it is the first log data; isLastLog: whether it is the last log data; logCallback: callback function, used to call back the conclusion information after the single log analysis to the main thread. Among them, JSON is a lightweight data exchange format that is easy for people to read and write, and is also easy for machines to parse and generate. It can be seen that in this embodiment, in the front-end page, you can write a JS analysis script belonging to your own business logic in the preset editor according to the business logic of the target object itself, and upload the written script to the server for storage.

[0048] According to the above records, in an exemplary embodiment, the process of converting a script method string into a script method for execution and calling back the execution conclusion to the main thread includes: converting the script method string into a script method through a preset conversion function; loading the script method into a target storage area, and executing the script method in the target storage area to obtain an execution conclusion; calling back the execution conclusion to the main thread through a callback function. Specifically, in this embodiment or other embodiments, the script method string can be converted into a script method through an eval function, and the script method can be loaded into the target storage area, and the script method can be executed in the target storage area to obtain an execution conclusion.

[0049] According to the above records, in an exemplary embodiment, the process of selecting a script from the script list information to analyze all log data to be analyzed includes: selecting a script from the script list information as the target script; analyzing all log data to be analyzed through the target script, and starting a scheduled task on the front-end page; wherein the scheduled task is used to extract the data exchange format object and the target script for each log data to be analyzed. Therefore, this embodiment uses the main thread to start the scheduled task to dispatch each log analysis task, and the specific log analysis is communicated to the child process, and the conclusion is communicated back to the main thread after the child process is processed, so as to achieve the purpose of the child thread processing logic and the main thread rendering display, so that the front-end page will not be stuck. In addition, after the target script completes the log analysis, you can reselect a script for log analysis.

[0050] In another exemplary embodiment of the present application, the embodiment provides a log analysis method. Figure 3 FIG. 1 shows a flow chart of the log analysis method provided in this embodiment. Specifically, Figure 3 As shown, the steps of the log analysis method include:

[0051] Configure an editor that supports JS code on the front-end page. The following four parameters are preset in the editor: logInfo: a single log JSON object; isFirstLog: whether it is the first log data; isLastLog: whether it is the last log data; logCallback: callback function, used to call back the conclusion information after analyzing a single log to the main thread.

[0052] The target object customizes the script according to its own business logic and uploads it to the server. Specifically, in the editor preset on the front-end page, the target object starts to write the script for analyzing the log logInfo based on the pre-sorted business logic and the above four parameters. When the script needs to be output to the front-end page, the logCallback function is called to call back the conclusion information. After writing the script, the script code is saved to the server.

[0053] The call interface obtains a list of all scripts and all log data from the server, selects a script to start log analysis, and the front-end page starts a scheduled task through the setInterval function, passing each log JSON object, isFirstLog parameter, isLastLog parameter and the selected script to the child thread for processing through the Web Worker mechanism.

[0054] Sub-thread processing flow. Specifically, the sub-thread listens to the communication event of the main thread, extracts the log JSON object, isFirstLog parameter, isLastLog parameter and script string from the corresponding communication event parameters, and concatenates the script string and the preset JS method name excuOuterScript, as well as the extracted log JSON object, isFirstLog parameter and isLastLog parameter into a complete JS method string, which is then converted into a real JS method through the eval function, loaded into memory and executed immediately.

[0055] Script output display. Specifically, since the user will call the logCallback function when writing a script, this function will send the analysis conclusion information that needs to be output in the script to the main thread through the Web Worker mechanism. The main thread listens to this callback event, obtains the analysis conclusion information in the callback event and displays it on the front-end page, achieving efficient analysis of large amounts of log data through front-end scheduled tasks and multi-threaded communication.

[0056] As an example, the process of quickly analyzing a large amount of log data on a front-end page and displaying the analysis conclusion in this embodiment includes:

[0057] Step 1: Obtain the list of scripts written by all users and all log data that need to be analyzed from the server;

[0058] Step 2: Select the script to be executed to start analyzing the logs.

[0059] Step 3: The front-end page starts the scheduled task setInterval, takes out the log JSON objects one by one, and communicates whether the JSON objects are the first log data or the last log data to the child thread through the Web Worker mechanism;

[0060] Step 4: The child thread concatenates the script with the pre-known JS method name and parameter name into the final JS method string, converts the string into a JS method through the built-in eval function and passes it into a JSON object. It checks whether it is the first or last item, and the callback parameter logCallback is executed immediately.

[0061] Step 5: During the script execution, the analysis conclusion information will be called back through the logCallback function, and the logCallback function will communicate the conclusion information to the main thread through the Web Worker mechanism;

[0062] Step 6: The main thread receives the communication event of the child thread, and immediately extracts the event parameters (conclusion information of the log analysis) and displays them on the front-end page to form a visualization of the log analysis.

[0063] It can be seen that the JS script written by the user is essentially a JS method. The method name and method parameters of this JS method have been determined. The user only needs to use the method parameters to write the script code that conforms to his own business logic and upload it to the server (the script code is hosted on the server in the form of a string). When the user's script needs to be executed, the script list will be obtained from the server first, and a certain script will be selected for execution. The selected script string and the preset JS method method name and method parameters are string-concatenated to obtain a complete JS method string. The string method is converted into a JS method through the built-in eval function in JS and loaded into the memory for execution. Therefore, this embodiment realizes platformization on the front-end page, and presets script editing, script uploading, script acquisition, script concatenation, script execution, thread communication, and log display on the platform. The user only needs to write the script he needs according to the business logic. Each JS script can be compared to an independent atom, which can be inserted into the platform to run and output the conclusion. At the same time, this embodiment is based on the scheduled task setInterval and the WebWorker mechanism in the HTML5 standard, and the heavy JS processing logic is transferred to the child thread for processing, while the main thread is only responsible for rendering and displaying the page, so as to achieve the purpose of efficiently analyzing a large amount of data, and the page will not be stuck. It is equivalent to using the main thread to start the scheduled task to dispatch each log analysis task, and the specific log analysis is communicated to the child process, and the conclusion is communicated back to the main thread after the child process is processed, so as to achieve the purpose of the child thread processing logic and the main thread rendering and display, so that the page will not be stuck.

[0064] In summary, the present application provides a log analysis method, which extracts a script string from a selected script through a child thread, and splices the script method name configured in advance or in real time with the script string, the data exchange format object of each log data to be analyzed, the first log data identification parameter, and the last log data identification parameter to generate a script method string; and, converts the script method string into a script method for execution, and calls back the execution conclusion to the main thread; and transmits the execution conclusion to the front-end page for display through the main thread. It can be seen that this method can start a polling task after selecting a script, quickly scan a large number of user logs, and each log will execute the selected script once in the child thread, and finally call back to the main thread and output the display business conclusion on the page, so as to achieve the purpose of efficiently analyzing a large number of user logs on the front-end page according to different scripts, and displaying them on the front-end page. At the same time, since the user will call the logCallback function when writing the script, this function will send the analysis conclusion information that needs to be output in the script to the main thread through the Web Worker mechanism, and the main thread will listen to this callback event, obtain the analysis conclusion information in the callback event and display it on the front-end page, so as to achieve efficient analysis of large amounts of data logs through front-end timing tasks and multi-threaded communication. Therefore, this method sets up a child thread outside the main thread, and the operation of the child thread and the main thread do not interfere with each other. When the child thread code is executed, the execution result is called back to the main thread for processing and then transmitted to the front-end page for rendering and display. This will not cause the main thread to be blocked and the page to be stuck, solving the problem of page stuck during single-thread execution.

[0065] In another exemplary embodiment of the present application, Figure 4 As shown, this embodiment also provides a log analysis system, including:

[0066] Thread communication module, used to establish multi-thread communication, multi-thread includes main thread and sub-thread;

[0067] The main thread communication module is used to obtain the pre-written or real-time script list information and the log data to be analyzed through the main thread; and select a script from the script list information to analyze all the log data to be analyzed, extract the data exchange format object of each log data to be analyzed, and transmit the selected script, the data exchange format object of each log data to be analyzed, and the log data identification parameter to the child thread through the preset thread mechanism; wherein the log data identification parameter includes: the first log data identification parameter and the last log data identification parameter. As an example, in this embodiment or other embodiments, the preset thread mechanism includes but is not limited to the Web Worker mechanism.

[0068] The child thread communication module is used to extract a script string from the selected script through the child thread, and concatenate the script method name configured in advance or in real time with the script string, the data exchange format object of each log data to be analyzed, the first log data identification parameter, and the last log data identification parameter to generate a script method string; and convert the script method string into a script method for execution, and call back the execution conclusion to the main thread;

[0069] The front-end page module is used to transmit the execution conclusion to the front-end page for display through the main thread.

[0070] It can be seen that this embodiment can start the polling task after selecting a certain script, quickly scan a large number of user logs, and each log will execute the selected script once in the child thread, and finally call back to the main thread and output the display business conclusion on the page, so as to achieve the visualization purpose of efficiently analyzing a large number of user logs on the front-end page according to different scripts and displaying them on the front-end page. It is equivalent to the child thread being responsible for script execution and log analysis, and the main thread being responsible for rendering and displaying the conclusions after the child thread analysis, which greatly reduces the burden of the main thread, and the page is still smooth when processing large amounts of logs. At the same time, since the user will call the logCallback function when writing the script, this function will send the analysis conclusion information that needs to be output in the script to the main thread through the Web Worker mechanism, and the main thread will listen to this callback event, obtain the analysis conclusion information in the callback event and display it in the front-end page, so as to achieve efficient analysis of large amounts of logs through front-end timing tasks and multi-threaded communication. Therefore, this embodiment sets up a child thread outside the main thread, and the operation of the child thread and the main thread do not interfere with each other. When the child thread code is executed, the execution result is called back to the main thread for processing and then transmitted to the front-end page for rendering and display. This will not cause the main thread to be blocked and the page to be stuck, thus solving the problem of page stuck during single-thread execution.

[0071] According to the above records, in an exemplary embodiment, before obtaining the script list information written in advance or in real time through the main thread, the present embodiment may also include: configuring an editor on the front page, receiving the script written by the target object in the editor based on the business logic, and when the script code corresponding to the script does not change within the preset time, generating script list information, and storing the script code in the server. Specifically, in the editor preset by the front page, the target object can start writing the script of the analysis log according to the business logic and target parameters combed in advance, and when the script needs to be output to the front page, the logCallback function is called to call back the conclusion information, and the script code is saved to the server after the script is written. Wherein, the logCallback function is a callback function (Callback Function), which is usually designed to be called when a specific log event occurs. The callback function is a function passed as a parameter to another function, which is called when a certain operation is completed or a certain event occurs. In the context of log records, the logCallback function can be used to process or respond to log messages, such as sending log messages to a remote server, displaying them in a console, updating the user interface to reflect the log status, etc. In this embodiment or other embodiments, the target parameters may be the following four parameters preset in the editor: logInfo: a single log JSON (JavaScript Object Notation, JSON for short) object, isFirstLog: whether it is the first log data; isLastLog: whether it is the last log data; logCallback: callback function, used to call back the conclusion information after the single log analysis to the main thread. Among them, JSON is a lightweight data exchange format that is easy for people to read and write, and is also easy for machines to parse and generate. It can be seen that in this embodiment, in the front-end page, you can write a JS analysis script belonging to your own business logic in the preset editor according to the business logic of the target object itself, and upload the written script to the server for storage.

[0072] According to the above records, in an exemplary embodiment, the process of converting a script method string into a script method for execution and calling back the execution conclusion to the main thread includes: converting the script method string into a script method through a preset conversion function; loading the script method into a target storage area, and executing the script method in the target storage area to obtain an execution conclusion; calling back the execution conclusion to the main thread through a callback function. Specifically, in this embodiment or other embodiments, the script method string can be converted into a script method through an eval function, and the script method can be loaded into the target storage area, and the script method can be executed in the target storage area to obtain an execution conclusion.

[0073] According to the above records, in an exemplary embodiment, the process of selecting a script from the script list information to analyze all log data to be analyzed includes: selecting a script from the script list information as the target script; analyzing all log data to be analyzed through the target script, and starting a scheduled task on the front-end page; wherein the scheduled task is used to extract the data exchange format object and the target script for each log data to be analyzed. Therefore, this embodiment uses the main thread to start the scheduled task to dispatch each log analysis task, and the specific log analysis is communicated to the child process, and the conclusion is communicated back to the main thread after the child process is processed, so as to achieve the purpose of the child thread processing logic and the main thread rendering display, so that the front-end page will not be stuck. In addition, after the target script completes the log analysis, you can reselect a script for log analysis.

[0074] In another exemplary embodiment of the present application, the embodiment provides a log analysis system. Specifically, the log analysis system can perform the following steps:

[0075] Configure an editor that supports JS code on the front-end page. The following four parameters are preset in the editor: logInfo: a single log JSON object; isFirstLog: whether it is the first log data; isLastLog: whether it is the last log data; logCallback: callback function, used to call back the conclusion information after analyzing a single log to the main thread.

[0076] The target object customizes the script according to its own business logic and uploads it to the server. Specifically, in the editor preset on the front-end page, the target object starts to write the script for analyzing the log logInfo based on the pre-sorted business logic and the above four parameters. When the script needs to be output to the front-end page, the logCallback function is called to call back the conclusion information. After writing the script, the script code is saved to the server.

[0077] The call interface obtains a list of all scripts and all log data from the server, selects a script to start log analysis, and the front-end page starts a scheduled task through the setInterval function, passing each log JSON object, isFirstLog parameter, isLastLog parameter and the selected script to the child thread for processing through the Web Worker mechanism.

[0078] Sub-thread processing flow. Specifically, the sub-thread listens to the communication event of the main thread, extracts the log JSON object, isFirstLog parameter, isLastLog parameter and script string from the corresponding communication event parameters, and concatenates the script string and the preset JS method name excuOuterScript, as well as the extracted log JSON object, isFirstLog parameter and isLastLog parameter into a complete JS method string, which is then converted into a real JS method through the eval function, loaded into memory and executed immediately.

[0079] Script output display. Specifically, since the user will call the logCallback function when writing a script, this function will send the analysis conclusion information that needs to be output in the script to the main thread through the Web Worker mechanism. The main thread listens to this callback event, obtains the analysis conclusion information in the callback event and displays it on the front-end page, achieving efficient analysis of large amounts of log data through front-end scheduled tasks and multi-threaded communication.

[0080] As an example, the process of quickly analyzing a large amount of log data on a front-end page and displaying the analysis conclusion in this embodiment includes:

[0081] Step 1: Obtain the list of scripts written by all users and all log data that need to be analyzed from the server;

[0082] Step 2: Select the script to be executed to start analyzing the logs.

[0083] Step 3: The front-end page starts the scheduled task setInterval, takes out the log JSON objects one by one, and communicates whether the JSON objects are the first log data or the last log data to the child thread through the Web Worker mechanism;

[0084] Step 4: The child thread concatenates the script with the pre-known JS method name and parameter name into the final JS method string, converts the string into a JS method through the built-in eval function and passes it into a JSON object. It checks whether it is the first or last item, and the callback parameter logCallback is executed immediately.

[0085] Step 5: During the script execution, the analysis conclusion information will be called back through the logCallback function, and the logCallback function will communicate the conclusion information to the main thread through the Web Worker mechanism;

[0086] Step 6: The main thread receives the communication event of the child thread, and immediately extracts the event parameters (conclusion information of the log analysis) and displays them on the front-end page to form a visualization of the log analysis.

[0087] It can be seen that the JS script written by the user is essentially a JS method. The method name and method parameters of this JS method have been determined. The user only needs to use the method parameters to write the script code that conforms to his own business logic and upload it to the server (the script code is hosted on the server in the form of a string). When the user's script needs to be executed, the script list will be obtained from the server first, and a certain script will be selected for execution. The selected script string and the preset JS method method name and method parameters are string-concatenated to obtain a complete JS method string. The string method is converted into a JS method through the built-in eval function in JS and loaded into the memory for execution. Therefore, this embodiment realizes platformization on the front-end page, and presets script editing, script uploading, script acquisition, script concatenation, script execution, thread communication, and log display on the platform. The user only needs to write the script he needs according to the business logic. Each JS script can be compared to an independent atom, which can be inserted into the platform to run and output the conclusion. At the same time, this embodiment is based on the scheduled task setInterval and the WebWorker mechanism in the HTML5 standard, and the heavy JS processing logic is transferred to the child thread for processing, while the main thread is only responsible for rendering and displaying the page, so as to achieve the purpose of efficiently analyzing a large amount of data, and the page will not be stuck. It is equivalent to using the main thread to start the scheduled task to dispatch each log analysis task, and the specific log analysis is communicated to the child process, and the conclusion is communicated back to the main thread after the child process is processed, so as to achieve the purpose of the child thread processing logic and the main thread rendering and display, so that the page will not be stuck.

[0088] In summary, the present application provides a log analysis system, which extracts a script string from a selected script through a child thread, and splices the script method name configured in advance or in real time with the script string, the data exchange format object of each log data to be analyzed, the first log data identification parameter and the last log data identification parameter to generate a script method string; and, converts the script method string into a script method for execution, and calls back the execution conclusion to the main thread; and transmits the execution conclusion to the front-end page for display through the main thread. It can be seen that the system can start a polling task after selecting a script, quickly scan a large number of user logs, and each log will execute the selected script once in the child thread, and finally call back to the main thread and output the display business conclusion on the page, so as to achieve the purpose of efficiently analyzing a large number of user logs on the front-end page according to different scripts, and displaying them on the front-end page. At the same time, since the user will call the logCallback function when writing the script, this function will send the analysis conclusion information that needs to be output in the script to the main thread through the Web Worker mechanism, and the main thread will listen to this callback event, obtain the analysis conclusion information in the callback event and display it on the front-end page, so as to achieve efficient analysis of large amounts of logs through front-end timing tasks and multi-threaded communication. Therefore, this system sets up a child thread outside the main thread, and the operation of the child thread and the main thread do not interfere with each other. When the child thread code is executed, the execution result is called back to the main thread for processing and then transmitted to the front-end page for rendering and display. This will not cause the main thread to be blocked and the page to be stuck, solving the problem of page stuck during single-thread execution.

[0089] It should be noted that the log analysis system provided in the above embodiment and the log analysis method provided in the above embodiment belong to the same concept, wherein the specific manner in which the log analysis method performs the operation has been described in detail in the above method embodiment, and will not be repeated here. In actual application, the log analysis system provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the log analysis system into different functional modules, and then implement all or part of the functions of the corresponding functional modules through the log analysis method described in the above embodiment, and this is not specifically limited here.

[0090] The present application also provides a computer device, which may include a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program so that the computer device executes Figure 3 The steps of the log analysis method. Figure 5 FIG. 1 is a schematic diagram showing a structure of a computer device 1000. Figure 5As shown, the computer device 1000 includes: a processor 1010 , a memory 1020 , a power supply 1030 , a display unit 1040 , and an input unit 1060 .

[0091] The processor 1010 is the control center of the computer device 1000. It uses various interfaces and lines to connect various components, and executes various functions of the computer device 1000 by running or executing computer programs / instructions stored in the memory 1020, thereby monitoring the computer device 1000 as a whole. In the embodiment of the present application, when the processor 1010 calls the computer program stored in the memory 1020, the following is executed: Figure 3 The steps of the log analysis method. Optionally, the processor 1010 may include one or more processing units; preferably, the processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application, etc., and the modem processor mainly processes wireless communication. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips.

[0092] The memory 1020 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, various applications, etc.; the data storage area may store instruction data, etc. created according to the use of the computer device 1000. In addition, the memory 1020 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices, etc.

[0093] The computer device 1000 also includes a power source 1030 (such as a battery) for supplying power to various components. The power source can be logically connected to the processor 1010 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0094] The display unit 1040 may be used to display information input by the user or information provided to the user and various menus of the computer device 1000, and in the embodiment of the present application, is mainly used to display the display interface of each application in the computer device 1000 and objects such as text and pictures displayed in the display interface. The display unit 1040 may include a display panel 1050. The display panel 1050 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0095] The input unit 1060 may be used to receive information such as numbers or characters input by the user. The input unit 1060 may include a touch panel 1070 and other input devices 1080. The touch panel 1070, also known as a touch screen, may collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus, or any other suitable object or accessory on or near the touch panel 1070).

[0096] Specifically, the touch panel 1070 can detect the user's touch operation, detect the signal brought by the touch operation, convert these signals into touch point coordinates, send them to the processor 1010, and receive and execute the command sent by the processor 1010. In addition, the touch panel 1070 can be implemented in various types such as resistive, capacitive, infrared and surface acoustic wave. Other input devices 1080 can include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, a joystick, etc.

[0097] Of course, the touch panel 1070 can cover the display panel 1050. When the touch panel 1070 detects a touch operation on or near it, it is transmitted to the processor 1010 to determine the type of touch event, and then the processor 1010 provides a corresponding visual output on the display panel 1050 according to the type of touch event. Figure 5 In the embodiment, the touch panel 1070 and the display panel 1050 are two independent components to implement the input and output functions of the computer device 1000, but in some embodiments, the touch panel 1070 and the display panel 1050 can be integrated to implement the input and output functions of the computer device 1000.

[0098] The computer device 1000 may further include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity light sensor, etc. Of course, according to the needs of specific applications, the computer device 1000 may also include other components such as a camera.

[0099] The embodiment of the present application also provides a computer-readable storage medium, in which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the above-mentioned device can perform the above-mentioned Figure 3 The steps of the log analysis method.

[0100] It can be understood by those skilled in the art that Figure 5This is only an example of a computer device and does not constitute a limitation on the device. The device may include more or fewer components than shown in the figure, or combine certain components, or different components. For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0101] Those skilled in the art should understand that the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be applied to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0102] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A log analysis method, characterized in that: The method comprises the following steps: Establishing multi-thread communication, wherein the multi-thread includes a main thread and a sub-thread; Extracting a script string from the selected script through the child thread, and concatenating a pre- or real-time configured script method name with the script string, a data exchange format object of each log data to be analyzed, a first log data identification parameter, and a last log data identification parameter to generate a script method string; and converting the script method string into a script method for execution, and calling back the execution conclusion to the main thread; The execution conclusion is transmitted to the front-end page for display through the main thread.

2. The log analysis method according to claim 1, characterized in that: Before extracting the script string from the selected script by the child thread, the method further includes: Obtaining pre- or real-time script list information and log data to be analyzed through the main thread; and selecting a script from the script list information to analyze all log data to be analyzed, extracting a data exchange format object for each log data to be analyzed, and transmitting the selected script, the data exchange format object for each log data to be analyzed, and log data identification parameters to the child thread through a preset thread mechanism; The log data identification parameters include: the first log data identification parameter and the last log data identification parameter.

3. The log analysis method according to claim 2, characterized in that: Before obtaining the script list information written in advance or in real time through the main thread, the method further includes: Configure the editor on the front-end page; A script written by a target object in the editor based on business logic is received, and when a script code corresponding to the script does not change within a preset time, the script list information is generated, and the script code is stored in a server.

4. The log analysis method according to claim 2, characterized in that: The process of converting the script method string into a script method for execution and calling back the execution conclusion to the main thread includes: Convert the script method string into a script method by using a preset conversion function; Loading the script method into a target storage area, and executing the script method in the target storage area to obtain an execution conclusion; The execution conclusion is called back to the main thread through a callback function.

5. The log analysis method according to claim 2, characterized in that: The process of selecting a script from the script list information to analyze all log data to be analyzed includes: Select a script from the script list information as a target script; All log data to be analyzed are analyzed through the target script, and a scheduled task is started on the front-end page; wherein the scheduled task is used to extract the data exchange format object of each log data to be analyzed and the target script.

6. The log analysis method according to claim 2, characterized in that: The preset thread mechanism includes a WebWorker mechanism.

7. A log analysis system, characterized in that: The system comprises: A thread communication module, used to establish multi-thread communication, wherein the multi-thread includes a main thread and sub-threads; A child thread communication module is used to extract a script string from the selected script through the child thread, and concatenate a pre- or real-time configured script method name with the script string, a data exchange format object of each log data to be analyzed, a first log data identification parameter, and a last log data identification parameter to generate a script method string; and convert the script method string into a script method for execution, and call back the execution conclusion to the main thread; The front-end page module is used to transmit the execution conclusion to the front-end page for display through the main thread.

8. The log analysis system according to claim 7, characterized in that: The system also includes: A main thread communication module is used to obtain pre- or real-time script list information and log data to be analyzed through the main thread; and select a script from the script list information to analyze all log data to be analyzed, extract the data exchange format object of each log data to be analyzed, and transmit the selected script, the data exchange format object of each log data to be analyzed, and the log data identification parameter to the sub-thread through a preset thread mechanism; The log data identification parameters include: the first log data identification parameter and the last log data identification parameter.

9. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the log analysis method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the log analysis method described in any one of claims 1 to 6 are implemented.

Images