Log presentation management method and device based on Web page

Abstract

The invention provides a log presentation management method and device based on a Web page, and the method comprises the steps: a log filter processes log table data and a log file based on a target search engine, constructs structured data suitable for the target search engine, obtains structured log index data and a log file, and transmits the structured log index data and the log file to a server; the structured log index data and the log file are sent to the distributed data storage system, a log data query operation is executed on a display page of the front-end browser in response to a user, the front-end browser sends a query request of the log data query operation to the background Web server, and the background Web server uses a target search engine to search the log data in the query request. And searching log data matched with the query request of the log data query operation from the distributed data storage system, and quickly returning the log data to a display interface of the front-end browser in a fragmentation form, so that the front-end browser displays the log data to a user.

Description

Technical Field

[0001] This application relates to the technical field of data processing. Specifically, it relates to a method and device for log presentation management based on a Web page. Background Art

[0002] System logs originate from various logs generated by multiple applications, and the data scale is huge. The system logs are stored in a MySQL database in the form of one table per day, and the information recorded includes the storage location, size, type, etc. of the log files.

[0003] Currently, in a computer system, users generally manage system logs through a Web application. During the use process, users need to quickly search for logs and be able to preview the files in the search results.

[0004] However, due to the huge scale of the logs, the search and query speed of the log data is slow, affecting the user experience and operation efficiency. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method and device for log presentation management based on a Web page. By constructing a log presentation management system with a hierarchical storage architecture, and constructing structured data of a search engine through a log filter and a distributed data storage system in the log presentation management system and storing it, the subsequent front-end browser can directly obtain and present the queried log data quickly from the distributed data storage system through interaction with the background Web server, improving the query speed of the log data and the user experience and operation efficiency.

[0006] In a first aspect, an embodiment of this application provides a method for log presentation management based on a Web page, which is applied to a log presentation management system; the log presentation management system includes a standard log processing system, a log filter, and a distributed data storage system; the standard log processing system includes a log file system, a log table database, a background Web server, and a front-end browser; the method includes:

[0007] The log filter periodically extracts log table data from the log table database and obtains the corresponding log files in the log file system based on the log table data;

[0008] The log filter processes the log table data and the log files based on a target search engine, constructs structured data applicable to the target search engine, obtains structured log index data and log files, and sends the structured log index data and log files to the distributed data storage system;

[0009] In response to a user performing a log data query operation on the display page of the front-end browser, the front-end browser sends a query request for the log data query operation to the back-end Web server; wherein, the log data represents the data of a log file.

[0010] The back-end Web server uses the target search engine to search the distributed data storage system for log data that matches the query request of the log data query operation, and quickly returns the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user.

[0011] In a possible implementation manner, the method further includes:

[0012] In response to the user's preview operation on the target large file in the log data queried by the front-end browser, the front-end browser shards the target large file into multiple target large file shards, and initiates a shard data request corresponding to the target large file shard to the back-end Web server.

[0013] After receiving the shard data request, the back-end Web server determines the starting position of the file content corresponding to the target large file shard, reads the corresponding file content in the distributed data storage system based on the starting position, and returns the read file shard content to the front-end browser, so that the display page of the front-end browser can display the already acquired file shard content in real time.

[0014] In a possible implementation manner, the method further includes:

[0015] In response to a sliding operation performed by the user using the mouse while previewing the file shard content on the display page of the front-end browser, the display page of the front-end browser continues to load the corresponding target number of file shard contents.

[0016] In a possible implementation manner, the front-end browser shards the target large file into multiple target large file shards and initiates a data request corresponding to the target large file shard to the back-end Web server, including:

[0017] The front-end browser sends a file preview request for the target large file to the back-end Web server, and the back-end Web server determines and returns the size information of the target large file to the front-end browser based on the file preview request.

[0018] The front-end browser calculates the display height of the total display area for displaying the target large file on the display page based on the size information, and generates a corresponding target placeholder element based on the display height.

[0019] The front - end browser determines the shard position of the target large - file shard in the current display area within the total display area based on the target placeholder element, and determines and initiates a corresponding shard data request to the back - end Web server based on the shard position.

[0020] In a possible implementation, the display page of the front - end browser continues to load the corresponding target number of file shard contents, including:

[0021] The front - end browser determines the scrolling direction of the mouse wheel, and determines the target placeholder element of the target display area based on the scrolling direction;

[0022] The front - end browser sends a corresponding shard data request to the Web server based on the target placeholder element of the target display area, and obtains and continues to load the target number of file shard contents.

[0023] In a possible implementation, the query request carries query conditions; the method further includes:

[0024] After receiving the query request, the back - end Web server obtains the corresponding total number of log data and the first log data within the first data range in the total number of log data in the distributed data storage system according to the query conditions based on the target search engine, and sends the first log data to the front - end browser; wherein, the first data range represents the range of page numbers;

[0025] After receiving the first log data, the front - end browser displays the first log data of the target page within the first data range to the user through the display page.

[0026] In a possible implementation, the method further includes:

[0027] In response to the user's paging operation on the first log data displayed on the display page of the front - end browser, the front - end browser obtains and displays the first log data of the next page within the first data range;

[0028] In response to the completion of the display of the first log data within the first data range, the front - end browser determines whether there is remaining data that has not been displayed among the queried log data based on the total number of log data;

[0029] If there is remaining data that has not been displayed, the front-end browser sends a corresponding instruction to the back-end Web server. After receiving the instruction, the back-end Web server sends a pre-request for obtaining the second log data within the second data range to the front-end browser, so that the front-end browser can obtain the second log data within the second data range and continue to display it.

[0030] In a second aspect, an embodiment of the present application further provides a log presentation management device based on a Web page, which is applied to a log presentation management system; the log presentation management system includes a standard log processing system, a log filter, and a distributed data storage system; the standard log processing system includes a log file system, a log table database, a back-end Web server, and a front-end browser; the device includes:

[0031] An extraction module, configured to enable the log filter to periodically extract log table data from the log table database and obtain corresponding log files in the log file system based on the log table data;

[0032] A construction module, configured to enable the log filter to process the log table data and the log files based on a target search engine, construct structured data applicable to the target search engine, obtain structured log index data and log files, and send the structured log index data and log files to the distributed data storage system;

[0033] A first query module, configured to, in response to a user performing a log data query operation on the display page of the front-end browser, the front-end browser sends a query request for the log data query operation to the back-end Web server; wherein, the log data represents data of a log file;

[0034] A second query module, configured to enable the back-end Web server to use the target search engine to search for log data in the distributed data storage system that matches the query request for the log data query operation, and quickly return the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user.

[0035] In a possible implementation manner, the log presentation management device based on a Web page of the present application further includes:

[0036] A preview module, configured to, in response to the user's preview operation on a target large file in the log data queried by the front-end browser, the front-end browser shards the target large file into multiple target large file shards and sends a sharded data request corresponding to the target large file shards to the back-end Web server;

[0037] The first acquisition module is used to determine the starting position of the file content corresponding to the target large file shard after the background Web server receives the shard data request, read the corresponding file content in the distributed data storage system based on the starting position, and return the read file shard content to the front-end browser, so that the display page of the front-end browser can display the acquired file shard content in real time.

[0038] In a possible implementation manner, the Web page-based log presentation management device of the present application further includes:

[0039] The sliding module is used to, in response to a sliding operation performed by the user using the mouse when previewing the file shard content on the display page of the front-end browser, cause the display page of the front-end browser to continue to load the corresponding target number of file shard contents.

[0040] In a possible implementation manner, the preview module is specifically used for:

[0041] The front-end browser sends a file preview request for the target large file to the background Web server, and the background Web server determines and returns the size information of the target large file to the front-end browser based on the file preview request;

[0042] The front-end browser calculates the display height of the display total area for displaying the target large file in the display page based on the size information, and generates a corresponding target placeholder element based on the display height;

[0043] The front-end browser determines the shard position of the current display area where the target large file shard is located in the display total area based on the target placeholder element, and determines and initiates a corresponding shard data request to the background Web server based on the shard position.

[0044] In a possible implementation manner, the sliding module is specifically used for:

[0045] The front-end browser determines the scrolling direction of the mouse wheel, and determines the target placeholder element of the target display area based on the scrolling direction;

[0046] The front-end browser sends a corresponding shard data request to the Web server based on the target placeholder element of the target display area, and obtains and continues to load the target number of file shard contents.

[0047] In a possible implementation manner, the query request carries query conditions; the Web page-based log presentation management device of the present application further includes:

[0048] A third query module, which is used to, after the background Web server receives the query request, obtain the total number of corresponding log data and the first log data in the first data range of the total number of log data in the distributed data storage system according to the query conditions based on the target search engine, and send the first log data to the front-end browser; wherein, the first data range represents the range of page numbers.

[0049] A display module, which is used to, after the front-end browser receives the first log data, display the first log data of the target page in the first data range to the user through the display page.

[0050] In a possible implementation manner, the Web page-based log presentation management device of the present application further includes:

[0051] A paging module, which is used to, in response to a paging operation of the user on the first log data displayed on the display page of the front-end browser, the front-end browser obtains the first log data of the next page within the first data range and displays it.

[0052] A judgment module, which is used to, in response to the completion of the display of the first log data in the first data range, the front-end browser judges whether there is remaining data that has not been displayed among the query log data based on the total number of log data.

[0053] A second acquisition module, which is used to, if there is remaining data that has not been displayed, the front-end browser sends a corresponding instruction to the background Web server, and the background Web server sends a pre-request for obtaining the second log data in the second data range to the front-end browser after receiving the instruction, so that the front-end browser obtains the second log data in the second data range and continues to display it.

[0054] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the Web page-based log presentation management method according to any one of the first aspects.

[0055] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the Web page-based log presentation management method according to any one of the first aspects.

[0056] A log presentation management method and device provided by an embodiment of the present application. The log filter processes log table data and log files based on a target search engine, constructs structured data suitable for the target search engine, obtains structured log index data and log files, and sends the structured log index data and log files to a distributed data storage system. In response to a user performing a log data query operation on the display page of the front-end browser, the front-end browser sends a query request for the log data query operation to the background Web server. The background Web server uses the target search engine to search the distributed data storage system for log data that matches the query request of the log data query operation, and quickly returns the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user. In this application, by constructing a log presentation management system with a hierarchical storage architecture, and constructing and storing structured data of the search engine through the log filter and the distributed data storage system in the log presentation management system, the subsequent front-end browser can directly obtain and present the queried log data quickly from the distributed data storage system through interaction with the background Web server, improving the speed of log data query and enhancing the user experience and operation efficiency.

[0057] To make the above objects, features, and advantages of the present application more obvious and understandable, the following specifically gives preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other relevant drawings can also be obtained based on these drawings.

[0059] Figure 1 is a flowchart of a Web page-based log presentation management method provided by an embodiment of the present application;

[0060] Figure 2 is a schematic diagram of the operation process of a standard log processing system;

[0061] Figure 3 is a schematic diagram of the operation process of a log presentation management system;

[0062] Figure 4 is a flowchart of a Web page-based log presentation management method provided by another embodiment of the present application;

[0063] Figure 5It is a schematic structural diagram of a log presentation management device based on a Web page provided by an embodiment of the present application;

[0064] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purposes of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the present application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical context relationships may be reversed or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present application.

[0066] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and illustrated in the accompanying drawings here may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but only represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0067] It should be noted that the term "including" will be used in the embodiments of the present application to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.

[0068] Considering that system logs are derived from various logs generated by multiple applications and the data scale is huge, the system logs are stored in a MySQL database in the form of one table per day, and the information recorded includes the storage location, size, type, etc. of the log files.

[0069] Currently, in a computer system, users generally manage system logs through a Web application. During use, users need to quickly search for logs and be able to preview the files in the search results. However, due to the huge scale of the logs, the search and query speed of the log data is slow, affecting the user experience and operation efficiency.

[0070] To address this issue, the present application provides a Web page-based log presentation management method and apparatus. By constructing a log presentation management system with a hierarchical storage architecture, and by constructing and storing structured data of a search engine through a log filter and a distributed data storage system in the log presentation management system, subsequent front-end browsers can directly obtain and present the queried log data quickly from the distributed data storage system through interaction with the back-end Web server, improving the speed of log data query and enhancing the user experience and operation efficiency.

[0071] Figure 1 It is a flowchart of the Web page-based log presentation management method provided by an embodiment of the present application.

[0072] The Web page-based log presentation management method of an embodiment of the present application is applied to a log presentation management system, which includes a standard log processing system, a log filter, and a distributed data storage system; the standard log processing system includes a log file system, a log table database, a back-end Web server, and a front-end browser. For example, as Figure 2 and Figure 3 shown, Figure 2 represents the operation process of the standard log processing system, Figure 3 represents the operation process of the log presentation management system, where the mysql data table represents the log table database, and the Elasticsearch storage system represents the distributed data storage system.

[0073] It should be noted that the standard log processing system is a conventional log processing system, the log filter is a preset log filter; the back-end Web server is a web server that implements a Web search query interface; the distributed data storage system is a distributed search and analysis engine storage system, such as the Elasticsearch storage system, which is used as an example in the present application for description, but does not constitute a limitation thereto. Additionally, Elasticsearch is a distributed search and analysis engine based on apache lucence.

[0074] Those skilled in the art can understand that in the standard log processing system, the log table database is classified by log type and records the location of each log file in the form of one table per day.

[0075] In short, the core of the log presentation management system lies in adding a log filter and a distributed data storage system on the basis of a standard log processing system. When the standard log processing system processes logs, it specifically records the locations of log files in the log file system through a MySQL database, and queries the logs required by the front-end browser according to the MySQL database. The log presentation management system of this application constructs structured data of the Elasticsearch storage system through a log filter, and subsequently can directly query the logs according to the Elasticsearch storage system and return them to the front-end browser through the background web server.

[0076] As Figure 1 shown, the Web page-based log presentation management method of the embodiments of this application may specifically include:

[0077] S101. The log filter periodically extracts log table data from the log table database and obtains the corresponding log files in the log file system based on the log table data.

[0078] S102. The log filter processes the log table data and log files based on the target search engine, constructs structured data applicable to the target search engine, obtains structured log index data and log files, and sends the structured log index data and log files to the distributed data storage system.

[0079] S103. In response to the user performing a log data query operation on the display page of the front-end browser, the front-end browser sends a query request for the log data query operation to the background Web server.

[0080] S104. The background Web server uses the target search engine to search for log data that matches the query request of the log data query operation from the distributed data storage system, and quickly returns the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user.

[0081] In the above Web page-based log presentation management method, by constructing a log presentation management system with a hierarchical storage architecture, and constructing and storing structured data of the search engine through the log filter and distributed data storage system in the log presentation management system, the front-end browser can subsequently directly obtain and present the queried log data quickly from the distributed data storage system through interaction with the background Web server, improving the speed of log data query and enhancing the user experience and operation efficiency.

[0082] The following will illustrate the above exemplary steps of the embodiments of this application with specific examples:

[0083] S101, The log filter periodically extracts log table data from the log table database and obtains the corresponding log files in the log file system based on the log table data.

[0084] In the embodiment of the present application, there are log files in the log file system. In the log table database, the log table database is classified according to the log types of the log files, and records the locations of each log file in the form of one table per day. The log filter periodically extracts log table data from the log table database and obtains the corresponding log files in the log file system for subsequent processing. For example, as Figure 3 shown, the log filter periodically obtains log table data from the MySQL database (the corresponding Figure 2 MySQL database) every day.

[0085] Optionally, extract the log table data from the log table database according to a preset scheduled task.

[0086] Thus, through the preset scheduled task, it is ensured that the newly generated log information can be obtained in a timely manner, providing a basis for subsequent data processing.

[0087] S102, The log filter processes the log table data and log files based on the target search engine, constructs structured data applicable to the target search engine, obtains structured log index data and log files, and sends the structured log index data and log files to the distributed data storage system.

[0088] In the embodiment of the present application, the target search engine is a preset search engine. For example, Elasticsearch. This application takes this as an example for description, but does not constitute a limitation thereto. The distributed data storage system represents the storage system of the target search engine. After obtaining the log table data and log files, the log filter processes the log table data and log files based on the target search engine, constructs structured data applicable to the target search engine, obtains structured log index data and log files, and sends the structured log index data and log files to the distributed data storage system for subsequent efficient retrieval. For example, as Figure 3 shown, after obtaining the log table data and log files, the log filter processes them, constructs structured data applicable to the target search engine, and stores them in the Elasticsearch storage system.

[0089] S103, In response to the user performing a log data query operation on the display page of the front-end browser, the front-end browser sends the query request of the log data query operation to the background Web server.

[0090] In the embodiments of the present application, the query request carries query conditions, which include query time, etc. The log data represents the data of the log file. When the user performs a log data query operation on the display page of the front-end browser, that is, when querying the log data, the front-end browser sends the query request of the log data query operation to the background Web server. For example, as Figure 3 shown, the front-end browser sends a request to query the log data to the background Web server.

[0091] S104. The background Web server uses the target search engine to search for the log data that matches the query request of the log data query operation from the distributed data storage system, and quickly returns the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user.

[0092] In the embodiments of the present application, after receiving the request to query the log data from the front-end browser, the background Web server uses the target search engine to search for the log data that matches the query request of the log data query operation from the distributed data storage system, and quickly returns the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user. For example, as Figure 3 shown, when the user performs a log data query operation on the front-end browser, the background Web server obtains the corresponding log data from the Elasticsearch storage system and feeds the result back to the front-end browser to achieve a quick response.

[0093] The log presentation management method based on a Web page provided by an embodiment of the present application. The log filter processes log table data and log files based on a target search engine, constructs structured data applicable to the target search engine, obtains structured log index data and log files, and sends the structured log index data and log files to a distributed data storage system. In response to a user performing a log data query operation on the display page of the front-end browser, the front-end browser sends a query request for the log data query operation to the background Web server. The background Web server uses the target search engine to search the distributed data storage system for log data that matches the query request of the log data query operation, and quickly returns the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user. The log presentation management method based on a Web page of the present application, by constructing a log presentation management system with a hierarchical storage architecture, and constructing and storing structured data of a search engine through the log filter and the distributed data storage system in the log presentation management system. Subsequently, the front-end browser can directly obtain and present the queried log data quickly from the distributed data storage system through interaction with the background Web server, improving the speed of log data query and enhancing the user experience and operation efficiency.

[0094] Further, as Figure 4 shown, the log presentation management method based on a Web page of an embodiment of the present application may further include the following steps:

[0095] S401, in response to a user's preview operation on a target large file in the log data queried by the front-end browser, the front-end browser shards the target large file into multiple target large file shards and sends a shard data request corresponding to the target large file shards to the background Web server.

[0096] In an embodiment of the present application, when the user previews a target large file in the log data queried by the above front-end browser, for example, clicks on the target large file, the front-end browser shards the target large file into multiple target large file shards and sends a shard data request corresponding to the target large file shards to the background Web server for subsequent processing.

[0097] Specifically, the display page of the front-end browser shards the target large file.

[0098] As a possible implementation, the front-end browser sends a file preview request for the target large file to the background Web server. The background Web server determines and returns the size information of the target large file to the front-end browser based on the file preview request. The front-end browser calculates the display height of the total display area for presenting the target large file in the display page based on the size information, and generates a corresponding target placeholder element based on the display height. The front-end browser determines the shard position of the current display area where the target large file shard is located in the total display area based on the target placeholder element, and determines and initiates a corresponding shard data request to the background Web server based on the shard position. Here, the total display area is the area in the display page for presenting the target large file, and the current display area is the display area where the current large file shard of the target large file is located. Optionally, the target placeholder element can be a div placeholder element, and this application takes this as an example.

[0099] Specifically, after receiving the file preview request from the front-end browser, the background Web server first returns the size information of the file content. The front-end calculates the height of the total display area of the page based on the obtained file size and generates a corresponding div placeholder element. Subsequently, the front-end browser determines and initiates a corresponding shard data request based on the position of the div placeholder element of the current display area in the total display area.

[0100] It should be noted that the front-end browser determines the hidden area in the total display area based on the current display area and clears the div data in the hidden area. The hidden area represents the area corresponding to the shards of the target large file that are not being displayed (other shards except the currently displayed shard of the target large file).

[0101] In short, the front-end clears the div data in the hidden area, thereby avoiding the problem of page freezing caused by the front-end browser occupying memory due to loading too much data.

[0102] S402. After receiving the shard data request, the background Web server determines the starting position of the file content corresponding to the target large file shard, reads the corresponding file content in the distributed data storage system based on the starting position, and returns the read file shard content to the front-end browser so that the display page of the front-end browser can display the already obtained file shard content in real time.

[0103] In the embodiment of this application, after receiving the shard data request sent by the front-end browser, the background Web server reads the corresponding file content in the distributed data storage system according to the starting position of the file content corresponding to the target large file shard, thereby obtaining the content of the target large file shard, and returns the file shard content (i.e., shard data) to the front-end browser to display the already obtained file shard content in real time on the front-end browser.

[0104] Specifically, if the user needs to preview a large file in the query result of the log data, after clicking on the target large file, the display page sends a data request to the background Web server in the form of shards. After receiving the request, the background Web server reads the file content by shards and returns the data to the display page, and the display page can immediately present the obtained shard data.

[0105] Thus, by initiating and displaying data requests for the target large file in the form of shards, the problem of lag caused by overly large files is effectively solved, improving the user experience and operation efficiency.

[0106] Furthermore, in response to the sliding operation performed by the user using the mouse while previewing the shard content of the file on the display page of the front-end browser, the display page of the front-end browser continues to load the corresponding target number of file shard contents.

[0107] Specifically, when the user previews the shard content of the file on the display page of the front-end browser, the user can use the mouse wheel to slide. Under the mouse slide, the display page of the front-end browser can continue to load more file shard contents, that is, continue to load the file shard contents of the next page. In short, by scrolling the mouse, more shard data can be loaded, enhancing the preview experience.

[0108] It should be noted that after receiving the shard data request, the background Web server reads and returns the corresponding file shard content data to the current display area of the front-end browser for display. When the user scrolls the mouse, the front-end browser determines and decides that the div of the relevant display area requests the corresponding shard data from the background Web server according to the scrolling direction of the mouse wheel.

[0109] Optionally, the front-end browser determines the scrolling direction of the mouse wheel, determines the target placeholder element of the target display area based on the scrolling direction; the front-end browser sends a corresponding shard data request to the Web server based on the target placeholder element of the target display area, and obtains and continues to load the target number of file shard contents. Among them, the target display area is the display area corresponding to the file shard content that needs to be continued to be loaded.

[0110] Further, the query request carries query conditions. After receiving the query request, the background Web server obtains the total number of corresponding log data and the first log data within the first data range in the total number of log data in the distributed data storage system based on the target search engine, and sends the first log data to the front-end browser. After receiving the first log data, the front-end browser displays the first log data of the target page within the first data range to the user through the display page. Herein, the first data range represents the range of page numbers. For example, the first 10 pages; the first log data represents at least one page of log data, that is, the first log data within the first data range is at least one page of log data; the target page is the target page within the first data range, for example, one of the first 10 pages.

[0111] Specifically, after receiving the query request from the front-end browser, the background Web server uses the target search engine to execute the retrieval task according to the established query conditions, obtains the total number of data and the first 10 pages of data, and returns them to the front-end browser. After receiving these 10 pages of data, the front-end browser first displays the content of one page to the user.

[0112] Further, in response to the user's paging operation on the first log data displayed on the display page of the front-end browser, the front-end browser obtains the first log data of the next page within the first data range and displays it; in response to the completion of the display of the first log data within the first data range, the front-end browser determines whether there is remaining data that has not been displayed among the retrieved log data based on the total number of log data; if there is remaining data that has not been displayed, the front-end browser sends a corresponding instruction to the background Web server, and after receiving the instruction, the background Web server sends a pre-request for obtaining the second log data within the second data range to the front-end browser, so that the front-end browser obtains the second log data within the second data range and continues to display it.

[0113] It should be noted that when the front-end browser receives 10 pages of data and first displays the content of one page to the user, when the user performs a paging operation, within the range of these 10 pages of data, the front-end browser can achieve page switching without interacting with the background Web server until these 10 pages of data are about to be displayed completely. The front-end browser determines whether there is remaining data based on the total number of data obtained. If there is, the background Web server sends a pre-request for the next 10 pages of data to the front-end browser.

[0114] Thus, through this mechanism, the interaction frequency between the front end and the back end is effectively reduced, thereby achieving an efficient paging function.

[0115] Generally speaking, this application has carried out multiple optimization designs in log presentation management. First, at the data storage level, a hierarchical storage architecture and an efficient indexing mechanism are adopted. Through a reasonable data partitioning strategy and a multi-level index structure, the fast retrieval function of log data is realized. Secondly, in terms of the performance of large file preview, a sharding loading technology is adopted. The system can dynamically adjust the request strategy according to factors such as file size, and obtain data in a way of loading on demand, thus effectively reducing resource consumption while ensuring the user experience. These optimization measures have significantly improved the overall performance and scalability of the system, providing users with an efficient and stable operating environment.

[0116] It should be noted that a log presentation management method based on a Web page in this application, that is, a method for fast log presentation on a Web page.

[0117] Figure 5 is a schematic structural diagram of a log presentation management device based on a Web page provided by an embodiment of this application; the log presentation management device provided by an embodiment of this application is applied to a log presentation management system; the log presentation management system includes a standard log processing system, a log filter, and a distributed data storage system; the standard log processing system includes a log file system, a log table database, a background Web server, and a front-end browser; as Figure 5 shown, the log presentation management device 500 based on a Web page in an embodiment of this application may specifically include:

[0118] An extraction module 501, configured to periodically extract log table data from the log table database by the log filter, and obtain the corresponding log file in the log file system based on the log table data;

[0119] A construction module 502, configured to process the log table data and the log file by the log filter based on a target search engine, construct structured data applicable to the target search engine, obtain structured log index data and log files, and send the structured log index data and log files to the distributed data storage system;

[0120] A first query module 503, configured to respond to a user's log data query operation on the display page of the front-end browser, and the front-end browser sends a query request for the log data query operation to the background Web server; wherein, the log data represents the data of the log file;

[0121] A second query module 504, configured to use the target search engine by the background Web server to search for log data that matches the query request of the log data query operation from the distributed data storage system, and quickly return the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user.

[0122] In a possible implementation, the Web page-based log presentation management device of the present application further includes:

[0123] A preview module, which is used to respond to a preview operation of a target large file in the log data queried by the user for the front-end browser. The front-end browser slices the target large file into multiple target large file slices and sends a slice data request corresponding to the target large file slice to the background Web server;

[0124] A first acquisition module, which is used to determine the starting position of the file content corresponding to the target large file slice after the background Web server receives the slice data request, read the corresponding file content in the distributed data storage system based on the starting position, and return the read file slice content to the front-end browser so that the display page of the front-end browser can display the already acquired file slice content in real time.

[0125] In a possible implementation, the Web page-based log presentation management device of the present application further includes:

[0126] A sliding module, which is used to respond to a sliding operation performed by the user using the mouse when previewing the file slice content on the display page of the front-end browser, and the display page of the front-end browser continues to load the corresponding target number of file slice contents.

[0127] In a possible implementation, the preview module is specifically used for:

[0128] The front-end browser sends a file preview request of the target large file to the background Web server, and the background Web server determines and returns the size information of the target large file to the front-end browser based on the file preview request;

[0129] The front-end browser calculates the display height of the display total area for displaying the target large file in the display page based on the size information, and generates a corresponding target placeholder element based on the display height;

[0130] The front-end browser determines the slice position of the current display area where the target large file slice is located in the display total area based on the target placeholder element, and determines and sends a corresponding slice data request to the background Web server based on the slice position.

[0131] In a possible implementation, the sliding module is specifically used for:

[0132] The front-end browser determines the scrolling direction of the mouse wheel, and determines the target placeholder element of the target display area based on the scrolling direction;

[0133] The front-end browser sends a corresponding shard data request to the Web server based on the target placeholder element in the target display area, obtains and continues to load the target number of file shard contents.

[0134] In a possible implementation manner, the query request carries query conditions; the log presentation management device based on a Web page of the present application further includes:

[0135] A third query module, configured to, after the background Web server receives the query request, obtain the corresponding total number of log data and the first log data in the first data range in the total number of log data in the distributed data storage system based on the target search engine according to the query conditions, and send the first log data to the front-end browser; wherein, the first data range represents the range of page numbers;

[0136] A display module, configured to, after the front-end browser receives the first log data, display the first log data of the target page in the first data range to the user through the display page.

[0137] In a possible implementation manner, the log presentation management device based on a Web page of the present application further includes:

[0138] A paging module, configured to, in response to a paging operation of the user on the first log data displayed on the display page of the front-end browser, the front-end browser obtains and displays the first log data of the next page within the first data range;

[0139] A judgment module, configured to, in response to the completion of the display of the first log data in the first data range, the front-end browser determines whether there is remaining data that has not been displayed in the queried log data based on the total number of log data;

[0140] A second obtaining module, configured to, if there is remaining data that has not been displayed, the front-end browser sends a corresponding instruction to the background Web server, and the background Web server sends a pre-request for obtaining the second log data in the second data range to the front-end browser after receiving the instruction, so that the front-end browser obtains the second log data in the second data range and continues to display it.

[0141] The log presentation management device based on a Web page provided by an embodiment of the present application processes log table data and log files based on a target search engine by a log filter, constructs structured data applicable to the target search engine, obtains structured log index data and log files, and sends the structured log index data and log files to a distributed data storage system. In response to a user performing a log data query operation on a display page of a front-end browser, the front-end browser sends a query request for the log data query operation to a background Web server. The background Web server uses the target search engine to search for log data that matches the query request of the log data query operation from the distributed data storage system, and quickly returns the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user. The log presentation management device based on a Web page of the present application constructs a log presentation management system with a hierarchical storage architecture, and constructs and stores structured data of a search engine through a log filter and a distributed data storage system in the log presentation management system. Subsequently, the front-end browser can directly obtain and present the queried log data quickly from the distributed data storage system through interaction with the background Web server, improving the speed of log data query and enhancing the user experience and operation efficiency.

[0142] As Figure 6 shown, an electronic device 600 provided by an embodiment of the present application includes: a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device runs, communication is carried out between the processor 601 and the memory 602 through the bus. The processor 601 executes the machine-readable instructions to perform the steps of the log presentation management method based on a Web page as described above.

[0143] Specifically, the above-mentioned memory 602 and processor 601 can be general-purpose memory and processor, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, it can execute the log presentation management method based on a Web page as described above.

[0144] Corresponding to the log presentation management method based on a Web page as described above, an embodiment of the present application also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, it executes the steps of the log presentation management method based on a Web page as described above.

[0145] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, and will not be elaborated herein. In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical or other forms.

[0146] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0147] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0148] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the deployment methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks or optical discs that can store program codes.

[0149] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A log presentation management method based on a Web page, which is applied to a log presentation management system; the log presentation management system includes a standard log processing system, a log filter, and a distributed data storage system; The described standard log processing system includes a log file system, a log table database, a background Web server, and a front-end browser; characterized in that, The method includes: The log filter periodically extracts log table data from the log table database and obtains the corresponding log file in the log file system based on the log table data; The log filter processes the log table data and the log file based on a target search engine, constructs structured data applicable to the target search engine, obtains structured log index data and log files, and sends the structured log index data and log files to the distributed data storage system; In response to a user performing a log data query operation on the display page of the front-end browser, the front-end browser sends a query request for the log data query operation to the background Web server; wherein, the log data represents the data of the log file; The background Web server uses the target search engine to search the distributed data storage system for log data that matches the query request of the log data query operation, and quickly returns the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user.

2. The method according to claim 1, characterized in that The method further includes: In response to the user's preview operation on a target large file in the log data queried by the front-end browser, the front-end browser shards the target large file into multiple target large file shards and sends a shard data request corresponding to the target large file shard to the background Web server; After receiving the shard data request, the background Web server determines the starting position of the file content corresponding to the target large file shard, reads the corresponding file content in the distributed data storage system based on the starting position, and returns the read file shard content to the front-end browser, so that the display page of the front-end browser can display the already obtained file shard content in real time.

3. The method according to claim 2, wherein The method further includes: In response to a sliding operation performed by the user using the mouse while previewing the file shard content on the display page of the front-end browser, the display page of the front-end browser continues to load the corresponding target number of file shard contents.

4. The method according to claim 3, characterized in that, The front-end browser shards the target large file into multiple target large file shards and sends a data request corresponding to the target large file shard to the background Web server, including: The front-end browser sends a file preview request for the target large file to the background Web server, and the background Web server determines and returns the size information of the target large file to the front-end browser based on the file preview request; The front-end browser calculates the display height of the total display area for displaying the target large file on the display page based on the size information, and generates a corresponding target placeholder element based on the display height; The front-end browser determines the shard position of the target large file shard in the current display area in the total display area based on the target placeholder element, and determines and initiates a corresponding shard data request to the back-end Web server based on the shard position.

5. The method according to claim 4, wherein The display page of the front-end browser continues to load the corresponding target number of file shard contents, including: The front-end browser determines the scrolling direction of the mouse wheel and determines the target placeholder element of the target display area based on the scrolling direction; The front-end browser sends a corresponding shard data request to the Web server based on the target placeholder element of the target display area, and obtains and continues to load the target number of file shard contents.

6. The method according to claim 5, wherein The query request carries query conditions; the method further includes: After receiving the query request, the back-end Web server obtains the corresponding total number of log data and the first log data in the first data range in the total log data from the distributed data storage system according to the query conditions based on the target search engine, and sends the first log data to the front-end browser; wherein, the first data range represents the range of page numbers; After receiving the first log data, the front-end browser displays the first log data of the target page in the first data range to the user through the display page.

7. The method according to claim 6, wherein The method further includes: In response to the paging operation of the user on the first log data displayed on the display page of the front-end browser, the front-end browser obtains and displays the first log data of the next page within the first data range; In response to the completion of the display of the first log data in the first data range, the front-end browser determines whether there is remaining data that has not been displayed in the queried log data based on the total number of log data; If there is remaining data that has not been displayed, the front-end browser sends a corresponding instruction to the back-end Web server, and the back-end Web server sends a pre-request for obtaining the second log data in the second data range to the front-end browser after receiving the instruction, so that the front-end browser obtains the second log data in the second data range and continues to display it.

8. A log presentation management device based on a Web page, which is applied to a log presentation management system; the log presentation management system includes a standard log processing system, a log filter, and a distributed data storage system; The standard log processing system includes a log file system, a log table database, a back-end Web server, and a front-end browser; characterized in that The device includes: An extraction module, configured to extract log table data from the log table database by the log filter at regular intervals, and obtain corresponding log files in the log file system based on the log table data; A construction module, configured to process the log table data and the log files by the log filter based on a target search engine, construct structured data applicable to the target search engine, obtain structured log index data and log files, and send the structured log index data and log files to the distributed data storage system; A first query module, configured to respond to a user's log data query operation performed on a display page of a front-end browser, and the front-end browser sends a query request of the log data query operation to the background Web server; wherein, the log data represents data of a log file. A second query module, configured to enable the background Web server to use the target search engine to search for log data in the distributed data storage system that matches the query request of the log data query operation, and quickly return the log data to the display interface of the front-end browser in a sharded form, so that the front-end browser can display the log data to the user.

9. An electronic device, characterized in that, Comprising: A processor, a memory and a bus, where the memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the Web page-based log presentation management method according to any one of claims 1 to 7 are executed.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is run by the processor, the steps of the Web page-based log presentation management method according to any one of claims 1 to 7 are executed.

Images