Log processing method, device and storage medium
The front-end and server logs are collected and asynchronously updated by event listening functions, which solves the problems of log archiving and association analysis in the information system, and realizes the integrity and security of the logs, which facilitates problem location and processing.
Patent Information
- Application Number
- CN202011503247.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-17
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-12-17
AI Technical Summary
In the information system, front-end logs are difficult to archive and the monitoring mechanism is incomplete, resulting in the inability to obtain complete front-end logs. It is difficult to analyze the association between front-end logs and back-end logs, and it is difficult for developers and operation and maintenance personnel to accurately locate and deal with problems.
The front-end page log is collected through event listening function and sent to the server through asynchronous update requests. The XMLHttpRequest object is used to write the request identifier at the head of the HTTP request message, realizing the association between the front-end log and the server log, and using local database storage and periodicity Report, reduce the frequency of requests, and increase security.
It realizes the association between front-end logs and server logs, provides complete log data, facilitates developers to locate and deal with problems, reduces labor costs and response delays, and ensures user privacy and security.
Smart Images

Figure CN114650236B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of computer data processing, and in particular to a log processing method, device and storage medium. Background Art
[0002] In an information system, logs play a vital role. Logs can record all the behaviors of the system. The behaviors recorded in the logs can be used to analyze the usage of the system, especially abnormal problems that occur during the use of the system. Developers can use these logs to locate and handle abnormal problems, and then analyze the direction in which the system can be optimized.
[0003] In information systems, there are many ways to record logs, and front-end logs are not easy to archive, and the log monitoring mechanism is not perfect, so it is impossible to obtain complete front-end logs. In addition, front-end logs and back-end logs cannot be well correlated and analyzed. Therefore, neither developers nor operation and maintenance personnel can accurately locate and reproduce problems. Summary of the invention
[0004] In view of this, embodiments of the present application provide a log processing method, device, and storage medium to solve at least one problem existing in the prior art.
[0005] To achieve the above purpose, the technical solution of the embodiment of the present application is implemented as follows:
[0006] In a first aspect, an embodiment of the present application provides a log processing method, the method comprising:
[0007] Collect front-end page logs through event listening functions;
[0008] Sending the front-end page log to the server;
[0009] An asynchronous update request is created and sent to a server, wherein the asynchronous update request is used by the server to associate the front-end page log with the server log.
[0010] In an optional implementation, the front-end page log includes a first field, and the first field is used to indicate a request identifier;
[0011] The step of creating an asynchronous update request and sending the asynchronous update request to a server includes:
[0012] Create an XMLHttpRequest object;
[0013] Calling the initial open function of the XMLHttpRequest object, wherein the open function is used to write the request identifier into the header of the HTTP request message;
[0014] The send function of the XMLHttpRequest object is called, and the send function is used to send the HTTP request message to the server.
[0015] In an optional implementation, the request identifier is the same as the page identifier in the front-end page log generated by the front-end page.
[0016] In an optional implementation, before collecting the front-end page logs through the event listening function, the method further includes:
[0017] Call the logger to obtain the log collection strategy;
[0018] The collecting of front-end page logs through the event monitoring function includes:
[0019] Based on the log collection strategy, the front-end page logs are collected through the event monitoring function.
[0020] In an optional implementation, before calling the logger to obtain the log collection strategy, the method further includes:
[0021] Configure the logger on the front-end page and mount the logger on the window object.
[0022] In an optional implementation, sending the front-end page log to the server includes:
[0023] Storing the front-end page log in a local database;
[0024] The log interface is called according to a preset period, and the front-end page log is sent to the server through the log interface.
[0025] In an optional implementation, the event monitoring function includes an error event monitoring function and an unhandledRejection event monitoring function.
[0026] In a second aspect, an embodiment of the present application provides a log processing method, the method comprising:
[0027] Receive the front-end page log sent by the front-end page;
[0028] An asynchronous update request sent by the front-end page is received, and the front-end page log and the server log are associated based on the asynchronous update request.
[0029] In an optional implementation, the front-end page log includes a first field, and the first field is used to indicate a request identifier;
[0030] The receiving the asynchronous update request sent by the front-end page, and associating the front-end page log and the server log based on the asynchronous update request, includes:
[0031] Receiving an HTTP request message sent by the front-end page, wherein the header of the HTTP request message carries the request identifier;
[0032] Based on the request identifier and the call chain identifier of the server, the front-end page log and the server log are associated.
[0033] In an optional embodiment, the method further includes:
[0034] The front-end page log and the server log that are associated with each other are aggregated to obtain an aggregate log, and the aggregate log is analyzed.
[0035] In a third aspect, an embodiment of the present application provides a log processing device, the device comprising:
[0036] The log collection unit is used to collect the front-end page logs through the event listening function;
[0037] A log sending unit, used for sending the front-end page log to a server;
[0038] The request unit is used to create an asynchronous update request and send the asynchronous update request to the server, wherein the asynchronous update request is used by the server to associate the front-end page log with the server log.
[0039] In an optional implementation, the front-end page log includes a first field, and the first field is used to indicate a request identifier;
[0040] The request unit is specifically used to create an XMLHttpRequest object;
[0041] Calling the initial open function of the XMLHttpRequest object, wherein the open function is used to write the request identifier into the header of the HTTP request message;
[0042] The send function of the XMLHttpRequest object is called, and the send function is used to send the HTTP request message to the server.
[0043] In an optional implementation, the request identifier is the same as the page identifier in the front-end page log generated by the front-end page.
[0044] In an optional embodiment, the device further comprises:
[0045] A calling unit is used to call the logger to obtain the log collection strategy;
[0046] The log collection unit is specifically used to collect front-end page logs through an event monitoring function based on the log collection strategy.
[0047] In an optional embodiment, the device further comprises:
[0048] The configuration unit is used to configure the logger on the front-end page and mount the logger on the window object.
[0049] In an optional implementation, the log sending unit is specifically used to store the front-end page log in a local database;
[0050] The log interface is called according to a preset period, and the front-end page log is sent to the server through the log interface.
[0051] In an optional implementation, the event monitoring function includes an error event monitoring function and an unhandledRejection event monitoring function.
[0052] In a fourth aspect, an embodiment of the present application provides a log processing device, the device comprising:
[0053] A receiving unit, used for receiving a front-end page log sent by a front-end page;
[0054] The association unit is used to receive the asynchronous update request sent by the front-end page, and associate the front-end page log and the server log based on the asynchronous update request.
[0055] In an optional implementation, the front-end page log includes a first field, and the first field is used to indicate a request identifier;
[0056] The association unit is specifically used to receive an HTTP request message sent by the front-end page, wherein the header of the HTTP request message carries the request identifier;
[0057] Based on the request identifier and the call chain identifier of the server, the front-end page log and the server log are associated.
[0058] In an optional embodiment, the device further comprises:
[0059] The aggregation analysis unit is used to aggregate the front-end page log and the server log that are associated together to obtain an aggregate log, and analyze the aggregate log.
[0060] In a fifth aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:
[0061] Network interface, used to achieve connection and communication between components;
[0062] A memory for storing executable instructions;
[0063] The processor is used to implement the log processing method described in the first aspect when executing the executable instructions stored in the memory.
[0064] In a sixth aspect, an embodiment of the present application provides a server, the server comprising:
[0065] Network interface, used to achieve connection and communication between components;
[0066] A memory for storing executable instructions;
[0067] The processor is used to implement the log processing method described in the second aspect when executing the executable instructions stored in the memory.
[0068] In a seventh aspect, an embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program. When the computer program is executed by at least one processor, the log processing method described in the first aspect or the log processing method described in the second aspect is implemented.
[0069] The technical solution of the embodiment of the present application first collects the front-end page logs generated by the front-end page through an event listening function, and then associates the front-end page logs and the server logs through an asynchronous update request. In this way, the logs generated by the user on the front-end page and the logs generated by the server can be associated, and the developers can obtain complete logs, which is convenient for the developers to locate and handle problems. BRIEF DESCRIPTION OF THE DRAWINGS
[0070] Figure 1 A flow chart of a log processing method provided in an embodiment of the present application;
[0071] Figure 2 The overall architecture diagram of the log processing method provided in the embodiment of the present application;
[0072] Figure 3 A flow chart of a log processing method provided in an embodiment of the present application;
[0073] Figure 4 The overall architecture diagram of the log processing method provided for a specific example of this application;
[0074] Figure 5 A schematic diagram of the structure of a log processing device provided in an embodiment of the present application;
[0075] Figure 6 A schematic diagram of the structure of a log processing device provided in an embodiment of the present application;
[0076] Figure 7 A specific hardware structure of an electronic device provided in an embodiment of the present application;
[0077] Figure 8 A specific hardware structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0078] The exemplary embodiments disclosed in the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the specific embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope disclosed in the present application to those skilled in the art.
[0079] In the following description, a large number of specific details are given to provide a more thorough understanding of the present application. However, it is obvious to those skilled in the art that the present application can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present application, some technical features known in the art are not described; that is, all features of the actual embodiments are not described here, and well-known functions and structures are not described in detail.
[0080] In this era of big front-end and serverless, the amount of code in front-end development is getting bigger and bigger, and the complexity is getting higher and higher. Problem location of front-end pages also consumes a lot of time and manpower. Usually, when there is a problem with online services, the problem can be located directly by viewing the service log. However, for problems with online front-end pages, the detailed process before and after the exception usually can only be obtained by communicating with users, and then developers can reproduce the problem through local simulation. For those problems that cannot be reproduced by local simulation, it is like a "kite with a broken string", which makes people helpless and unable to deal with it. If there is a way to obtain the complete log when the problem occurs and the context information of the user environment, it can help developers quickly understand and restore the scene of the problem, and can more effectively locate and troubleshoot the problem, so that the "kite" can be finally pulled back into the hands of developers. In view of the above problems in the process of front-end location, the front-end log solution came into being. In most of the existing front-end technical architectures, the front-end logs generally stay in the browser on the user side. Of course, some large systems will have a log collection platform to collect front-end logs. In recent years, many different front-end log processing methods have emerged:
[0081] (1) An existing front-end log processing method is for front-end developers to perform breakpoint debugging and positioning in the browser's built-in console, or to set relevant print output in the code and then view the corresponding output records in the console. This method of front-end problem positioning is obviously inefficient, and the labor cost required for problem positioning is also very high.
[0082] (2) Another existing front-end log processing method is to manually embed logs at some key process points and directly report log content through the log interface provided by the back-end server. This solution usually has the following problems: when the number of embedded logs at the front end increases, the log interface will be called frequently, resulting in a slow response of the front-end page; because the source data is not encrypted, it is easy to leak customer privacy.
[0083] To this end, the following technical solutions of the embodiments of the present application are proposed.
[0084] The present application embodiment provides a log processing method. Figure 1 A flow chart of a log processing method provided in an embodiment of the present application, the method mainly comprises the following steps:
[0085] Step 101: Collect front-end page logs through event monitoring functions.
[0086] Step 102: Send the front-end page log to the server.
[0087] In the embodiment of the present application, for the collection of front-end platform logs, in order to facilitate the use of developers, the logger feLogger is configured in the front-end page in the embodiment of the present application, and the logger feLogger is directly mounted on the window object: window.feLogger = feLogger, so that the logger feLogger can be directly called in each front-end page, and the logger feLogger is compatible with various front-end platform frameworks. The logger feLogger includes a custom log collection strategy, which includes an event trigger strategy and a means trigger strategy.
[0088] In an embodiment of the present application, the log recorder feLogger is called to obtain a log collection strategy; based on the log collection strategy, the front-end page log is collected through an event listening function. Here, the event listening function includes an error error event listening function and an unhandled rejection unhandledRejection event listening function. The front-end page log includes an exception log, an interface log, a custom log, etc. In the embodiment of the present application, in the collection strategy of the front-end page log, a global error event listening function and an unhandledRejection event listening function are added to collect the front-end page uncaught exceptions and Promise uncaught exceptions, so as to avoid the developer's omission of reporting the exception log (front-end page log). It should be noted that in addition to the event listening function, there is also a log collection function that comes with the front-end platform, such as the Promise function.
[0089] In an embodiment of the present application, the front-end page log is stored in a local database; the log interface is called according to a preset period, and the front-end page log is sent to the server through the log interface. In the embodiment of the present application, a strategy combining local log storage and periodic reporting is adopted to report the front-end page log, so as to avoid the problem of slow response of the page due to frequent calls to the log interface. In the embodiment of the present application, the front-end page log is first stored in the local database (such as Indexed DB) of the front-end page in a queue data structure, and then the log interface is called according to a preset period, and the front-end page log is reported to the server through the log interface, thus reducing the number of requests and the bandwidth used. In addition, the operation and maintenance monitoring platform and the server can also be docked to facilitate the rapid retrieval of the required front-end page log. Here, the server can be a log microservice, and the log interface is the interface of the log microservice.
[0090] In the embodiment of the present application, in order to prevent user information leakage, in the process of sending the front-end page log to the server, the Advanced Encryption Standard (AES) algorithm is used to encrypt and transmit the front-end page log. The AES algorithm has the characteristics of fast encryption speed and low resource consumption.
[0091] In the embodiment of the present application, several necessary fields are designed for the data structure of log reporting. The data types and descriptions of these fields are shown in Table 1:
[0092]
[0093] Table 1
[0094] Step 103: Create an asynchronous update request and send the asynchronous update request to the server, where the asynchronous update request is used by the server to associate the front-end page log with the server log.
[0095] In an embodiment of the present application, the asynchronous update request is an asynchronous JavaScript and XML (Asynchronous Javascript And XML, Ajax) request. Here, Ajax refers to a web page development technology for creating interactive web page applications. Among them, XML is Extensible Markup Language (XML). Ajax can achieve asynchronous updates of web pages by exchanging a small amount of data with the server in the browser background. The so-called "asynchronous" means that there is no need to wait for the server response after sending the request. The request is cached in the task list and waits for the response, and the next request is sent. This means that Ajax can update a part of a web page without reloading the entire web page. When updating a web page, there may be multiple Ajax requests. The traditional Ajax request call method adopts a static nested method, which is integrated with the business code to implement the Ajax request call. In order to associate the front-end page log with the server log in the embodiment of the present application, and to facilitate the aggregated association analysis of the entire business log, the embodiment of the present application rewrites the initial open function and the send function of the XMLHttpRequest object at the bottom of the Ajax request (corresponding to the step of creating an asynchronous update request in the embodiment of the present application) to write the request identifier into the header of the HTTP request message and send it to the server, so that the server can associate the front-end page log with the server log through the request identifier. XMLHttpRequest is a browser object that can be used to simulate HTTP GET and POST requests, and can realize the timed data update of page data without refreshing. The browser can retrieve all types of resource data through XMLHttpRequest.
[0096] In the embodiment of the present application, the specific process of creating an Ajax request is as follows: creating an XMLHttpRequest object; calling the open function of the XMLHttpRequest object, rewriting the open function, the open function enhances the function of writing the request identifier into the header of the HTTP request message, and the open function is used to write the request identifier into the header of the HTTP request message; calling the send function of the XMLHttpRequest object, rewriting the send function, the send function enhances the ability to temporarily store the interface log locally, and the send function is used to send the HTTP request message to the server. Here, the open function and the send function in the embodiment of the present application are both functions rewritten based on the Ajax request, so that the request identifier X-Fe-RequestId is written into the HTTP request message and sent to the server, so that the server can associate the front-end page log and the server log generated by the user on the front-end page according to the request identifier X-Fe-RequestId in the HTTP request message. Specifically, the front-end page log includes a first field, the first field is used to indicate the request identifier, and the request identifier is X-Fe-RequestId. The request identifier is the same as the log.messages request identifier generated by the user on this front-end page, so that the request identifier is the same as the page identifier in the front-end page log generated by this front-end page. In this way, the front-end page log of this front-end page and the Ajax request initiated by this front-end page can be associated through the request identifier (X-Fe-RequestId). In addition, on the server side, after the server receives the HTTP request message, it obtains the request identifier (X-Fe-RequestId) in the HTTP request message, associates the request identifier (X-Fe-RequestId) with the call chain identifier of the server, and after the above association, the front-end page log generated by the user on the front-end page and the server log are associated. Afterwards, the front-end page log reported by these front-end pages can also be entered into Elasticsearch to analyze the business log. Here, the call chain identifier of the server is the traceId (tracking ID) of the call chain. In some embodiments, associating the request identifier (X-Fe-RequestId) with the call chain identifier of the server can be to set the call chain identifier (traceId) of the server to the same identifier as the request identifier (X-Fe-RequestId). Here, the server may be a log microservice.
[0097] Figure 2 The overall architecture diagram of the log processing method provided in the embodiment of the present application is as follows: Figure 2As shown, the front-end platform (such as a browser client, an application APP client, an application applet client, etc.) in the embodiment of the present application collects the front-end platform logs (such as exception logs, interface logs, custom logs, etc.) through the log collection strategy (such as event trigger strategy, manual trigger strategy, etc.), and stores the front-end platform logs in a local database, encrypts and reports the front-end platform logs to the server according to a preset period, and connects the operation and maintenance monitoring platform to the server to facilitate the association, aggregation analysis, etc. of the front-end platform logs and server logs.
[0098] In the embodiment of the present application, the front-end page log is collected through the event listening function; the front-end page log is sent to the server; an Ajax request is created, and the Ajax request is sent to the server, and the Ajax request is used by the server to associate the front-end page log with the server log. In the embodiment of the present application, the front-end page log generated by the front-end page is collected through the event listening function, and the front-end page log and the server log are associated through the Ajax request. In this way, the log generated by the user on the front-end page and the log generated by the server can be associated, and the developer can get a complete log, which is convenient for the developer to locate and handle the problem.
[0099] In the embodiment of the present application, the log collection strategy of the front-end page log is encapsulated into the window object; and by rewriting XMLHttpRequest, the front-end page log and the server log are cleverly associated. In the embodiment of the present application, the front-end page log is also pre-stored and encrypted and reported to the server in a time-sharing manner.
[0100] The present application embodiment provides a log processing method. Figure 3 A flow chart of a log processing method provided in an embodiment of the present application, the method mainly comprises the following steps:
[0101] Step 301: Receive the front-end page log sent by the front-end page.
[0102] Step 302: Receive an asynchronous update request sent by the front-end page, and associate the front-end page log and the server log based on the asynchronous update request.
[0103] In an embodiment of the present application, the asynchronous update request is an Ajax request. The server receives an HTTP request message sent by the front-end page, and the header of the HTTP request message carries the request identifier. The server obtains the request identifier carried in the HTTP request message, and associates the front-end page log and the server log based on the request identifier and the call chain identifier of the server. Here, the request identifier is X-Fe-RequestId. The call chain identifier of the server is the traceId of the call chain. After associating the X-Fe-RequestId with the traceId of the call chain, all logs related to the Ajax request can be found on the server side through the traceId of the call chain, and all logs related to the Ajax request can be found on the front-end page side through the X-Fe-RequestId.
[0104] In an embodiment of the present application, the front-end page log and the server log that are associated together can be aggregated together to obtain an aggregate log, and the aggregate log can be analyzed. The specific implementation method is: the front-end page log and the server log reported by the front-end page are entered into Elasticsearch to obtain an aggregate log, so that the aggregate log can be analyzed using Elasticsearch. The aggregate logs stored in Elasticsearch can also be searched, analyzed and visualized through Kibana. Thus, users can search for all logs related to Ajax requests on the ELK (Elasticsearch, Logstash, Kibana) log platform through X-Fe-RequestId or traceId.
[0105] In the embodiment of the present application, the server receives the front-end page log sent by the front-end page, receives the Ajax request sent by the front-end page, and associates the front-end page log with the server log based on the Ajax request. In the embodiment of the present application, the front-end page log and the Ajax request are sent to the server, so that the server associates the front-end page log with the server log based on the Ajax request, so that the log generated by the user on the front-end page and the log generated by the server can be associated, and the developer can obtain a complete log, which is convenient for the developer to locate and handle the problem.
[0106] Figure 4 The overall architecture diagram of the log processing method provided for a specific example of this application is as follows: Figure 4As shown, the functions and plug-ins used in the above-mentioned log processing method are integrated in the FeLogSdk.js front-end architecture, the browser (front-end page) includes the React.js and Vue.js general front-end architecture and FeLogSdk.js front-end architecture required for the above-mentioned log processing method, the request sent by the front-end platform is forwarded by Nginx, and uploaded to the log microservice (server) by the gateway (Gateway), the log microservice enters the front-end platform log and the server log into the ELK log platform, and the developer or operation and maintenance personnel can realize the aggregation analysis and query of the log through the ELK log platform. The browser can be presented in the form of a web view (WebView), or it can be presented in the form of a client. The front-end page in the embodiment of the present application includes FeLogSdk.js, react.js and Vue.js front-end architecture, etc., compared to traditional js libraries, it is possible to realize the collection of front-end platform logs, and the front-end platform logs are stored in a local database, the front-end platform logs are encrypted and reported to the server according to the preset period, the operation and maintenance monitoring platform is docked with the server, and the front-end platform logs are associated with the server logs, aggregated analysis, etc.
[0107] Based on the same technical concept as the aforementioned log processing method, an embodiment of the present application provides a log processing device. In some embodiments, the log processing device can be implemented in the form of a software module. Figure 5 A schematic diagram of the structure of a log processing device provided in an embodiment of the present application is shown in FIG. Figure 5 , the log processing device 500 provided in the embodiment of the present application includes:
[0108] The log collection unit 501 is used to collect the front-end page logs through the event monitoring function;
[0109] The log sending unit 502 is used to send the front-end page log to the server;
[0110] The request unit 503 is used to create an asynchronous update request and send the asynchronous update request to the server. The asynchronous update request is used by the server to associate the front-end page log with the server log.
[0111] In other embodiments, the front-end page log includes a first field, and the first field is used to indicate a request identifier;
[0112] The request unit 503 is specifically used to create an XMLHttpRequest object;
[0113] Calling the open function of the XMLHttpRequest object, wherein the open function is used to write the request identifier into the header of the HTTP request message;
[0114] The send function of the XMLHttpRequest object is called, and the send function is used to send the HTTP request message to the server.
[0115] In other embodiments, the device further comprises:
[0116] The calling unit 504 is used to call the logger to obtain the log collection strategy;
[0117] The log collection unit 501 is specifically configured to collect front-end page logs through an event monitoring function based on the log collection strategy.
[0118] In other embodiments, the device further comprises:
[0119] The configuration unit 505 is used to configure the logger on the front-end page and mount the logger on the window object.
[0120] In other embodiments, the log sending unit 502 is specifically used to store the front-end page log in a local database;
[0121] The log interface is called according to a preset period, and the front-end page log is sent to the server through the log interface.
[0122] In other embodiments, the event monitoring function includes an error event monitoring function and an unhandledRejection event monitoring function.
[0123] Based on the same technical concept as the aforementioned log processing method, an embodiment of the present application provides a log processing device. In some embodiments, the log processing device can be implemented in the form of a software module. Figure 6 A schematic diagram of the structure of a log processing device provided in an embodiment of the present application is shown in FIG. Figure 6 , the log processing device 600 provided in the embodiment of the present application includes:
[0124] The receiving unit 601 is used to receive the front-end page log sent by the front-end page;
[0125] The associating unit 602 is used to receive the asynchronous update request sent by the front-end page, and associate the front-end page log and the server log based on the asynchronous update request.
[0126] In other embodiments, the front-end page log includes a first field, and the first field is used to indicate a request identifier;
[0127] The association unit 602 is specifically configured to receive an HTTP request message sent by the front-end page, wherein the header of the HTTP request message carries the request identifier;
[0128] Based on the request identifier and the call chain identifier of the server, the front-end page log and the server log are associated.
[0129] In other embodiments, the device further comprises:
[0130] The aggregation analysis unit 603 is used to aggregate the front-end page log and the server log that are associated together to obtain an aggregate log, and analyze the aggregate log.
[0131] The components in the embodiments of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software function modules.
[0132] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in the embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.
[0133] Therefore, an embodiment of the present application provides a storage medium, which stores a computer program. When the computer program is executed by at least one processor, the steps described in the above embodiment are implemented.
[0134] See also Figure 7 , shows a specific hardware structure of an electronic device 700 provided in an embodiment of the present application, including: a network interface 701, a memory 702 and a processor 703; each component is coupled together through a bus system 704. It can be understood that the bus system 704 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 704 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 7 Various buses are labeled as bus system 704 .
[0135] The network interface 701 is used to receive and send signals during the process of sending and receiving information with other external network elements;
[0136] A memory 702, used to store a computer program that can be run on the processor 703;
[0137] The processor 703 is used to execute the aforementioned log processing method when running the computer program.
[0138] See also Figure 8 , shows a specific hardware structure of a server 800 provided in an embodiment of the present application, including: a network interface 801, a memory 802 and a processor 803; each component is coupled together through a bus system 804. It can be understood that the bus system 804 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 804 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 8 Various buses are labeled as bus system 804 .
[0139] The network interface 801 is used to receive and send signals during the process of sending and receiving information with other external network elements;
[0140] A memory 802, used to store a computer program that can be run on the processor 803;
[0141] The processor 803 is used to execute the aforementioned log processing method when running the computer program.
[0142] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0143] The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor to be executed. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0144] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.
[0145] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.
[0146] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.
[0147] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0148] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0149] In addition, all functional units in the embodiments of the present application can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art can understand that all or part of the steps of the above method embodiments can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above method embodiments; and the above storage medium includes: mobile storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.
[0150] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.
[0151] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.
[0152] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.
[0153] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A log processing method, characterized in that: The method comprises: Collect front-end page logs through event listening functions; Sending the front-end page log to the server according to a preset period; Creating an asynchronous update request, and sending the asynchronous update request to a server, wherein the asynchronous update request is used by the server to associate the front-end page log with the server log; The front-end page log includes a first field, and the first field is used to indicate a request identifier; The step of creating an asynchronous update request and sending the asynchronous update request to a server includes: Create an XMLHttpRequest object; Calling the rewritten open function of the XMLHttpRequest object, wherein the open function enhances the function of writing the request identifier into the header of the HTTP request message, and the open function is used to write the request identifier into the header of the HTTP request message, wherein the request identifier is the same as the log.messages request identifier generated by the user on this front-end page; The rewritten send function of the XMLHttpRequest object is called, and the send function enhances the ability to temporarily store the interface log locally. The send function is used to send the HTTP request message to the server so that the server can obtain the request identifier in the HTTP request message, associate the request identifier with the call chain identifier of the server, and thus associate the front-end page log generated by the user on the front-end page with the server log.
2. The log processing method according to claim 1, characterized in that: The request identifier is the same as the page identifier in the front-end page log generated by the front-end page.
3. The log processing method according to claim 1, characterized in that: Before collecting the front-end page logs through the event monitoring function, the method further includes: Call the logger to obtain the log collection strategy; The collecting of front-end page logs through the event monitoring function includes: Based on the log collection strategy, the front-end page logs are collected through the event monitoring function.
4. The log processing method according to claim 3, characterized in that: Before calling the logger to obtain the log collection strategy, the method further includes: Configure the logger on the front-end page and mount the logger on the window object.
5. The log processing method according to claim 1, characterized in that: The sending of the front-end page log to the server includes: Storing the front-end page log in a local database; The log interface is called according to a preset period, and the front-end page log is sent to the server through the log interface.
6. The log processing method according to any one of claims 1 to 5, characterized in that: The event monitoring function includes an error event monitoring function and an unhandledRejection event monitoring function.
7. A log processing method, characterized in that: The method comprises: Receive the front-end page logs sent by the front-end page according to a preset period; Receiving an asynchronous update request sent by the front-end page, and associating the front-end page log and the server log based on the asynchronous update request; The receiving the asynchronous update request sent by the front-end page and associating the front-end page log and the server log based on the asynchronous update request include: Receive an HTTP request message sent by the front-end page, wherein the header of the HTTP request message carries a request identifier; wherein the front-end page log includes a first field, and the first field is used to indicate the request identifier; the front-end page creates an XMLHttpRequest object; calls the rewritten open function of the XMLHttpRequest object, wherein the open function enhances the function of writing the request identifier into the header of the HTTP request message, and the open function is used to write the request identifier into the header of the HTTP request message, and the request identifier is the same as the log.messages request identifier generated by the user on this front-end page; calls the rewritten send function of the XMLHttpRequest object, wherein the send function enhances the ability to temporarily store the interface log locally, and the send function is used to send the HTTP request message to the server; The request identifier is associated with the call chain identifier of the server, thereby associating the front-end page log generated by the user on the front-end page with the server log.
8. The log processing method according to claim 7, characterized in that: The method further comprises: The front-end page log and the server log that are associated with each other are aggregated to obtain an aggregate log, and the aggregate log is analyzed.
9. A log processing device, characterized in that: The device comprises: The log collection unit is used to collect the front-end page logs through the event listening function; A log sending unit, used to send the front-end page log to the server according to a preset period; A request unit, used for creating an asynchronous update request and sending the asynchronous update request to a server, wherein the asynchronous update request is used by the server to associate the front-end page log with the server log; The front-end page log includes a first field, and the first field is used to indicate a request identifier; The request unit is specifically used for: Create an XMLHttpRequest object; Calling the rewritten open function of the XMLHttpRequest object, wherein the open function enhances the function of writing the request identifier into the header of the HTTP request message, and the open function is used to write the request identifier into the header of the HTTP request message, wherein the request identifier is the same as the log.messages request identifier generated by the user on this front-end page; The rewritten send function of the XMLHttpRequest object is called, and the send function enhances the ability to temporarily store the interface log locally. The send function is used to send the HTTP request message to the server so that the server can obtain the request identifier in the HTTP request message, associate the request identifier with the call chain identifier of the server, and thus associate the front-end page log generated by the user on the front-end page with the server log.
10. A log processing device, characterized in that: The device comprises: A receiving unit, used for receiving a front-end page log sent by a front-end page according to a preset period; an associating unit, configured to receive an asynchronous update request sent by the front-end page, and associate the front-end page log with the server log based on the asynchronous update request; Wherein, the association unit is specifically used for: Receive an HTTP request message sent by the front-end page, wherein the header of the HTTP request message carries a request identifier; wherein the front-end page log includes a first field, and the first field is used to indicate the request identifier; the front-end page creates an XMLHttpRequest object; calls the rewritten open function of the XMLHttpRequest object, wherein the open function enhances the function of writing the request identifier into the header of the HTTP request message, and the open function is used to write the request identifier into the header of the HTTP request message; the request identifier is the same as the log.messages request identifier generated by the user on this front-end page; calls the rewritten send function of the XMLHttpRequest object, wherein the send function enhances the ability to temporarily store the interface log locally, and the send function is used to send the HTTP request message to the server; The request identifier is associated with the call chain identifier of the server, thereby associating the front-end page log generated by the user on the front-end page with the server log.
11. An electronic device, characterized in that: The electronic device comprises: Network interface, used to achieve connection and communication between components; A memory for storing executable instructions; The processor is used to implement the log processing method according to any one of claims 1 to 6 when executing the executable instructions stored in the memory.
12. A server, characterized in that: The server comprises: Network interface, used to achieve connection and communication between components; A memory for storing executable instructions; The processor is used to implement the log processing method described in claim 7 or 8 when executing the executable instructions stored in the memory.
13. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by at least one processor, the log processing method according to any one of claims 1 to 6, or the log processing method according to claim 7 or 8 is implemented.
Citation Information
Patent Citations
Log management method
CN104182506A
Method and system for displaying updated data on webpage
CN104657479A
Search log combination method and system
CN105808605A
Information collection system and Web application
CN107016106A
Data communication system and method based on B / S structure, Web server and monitoring system
CN107426233A
