Log retrieval method and device, equipment and storage medium

Abstract

The invention provides a log retrieval method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: receiving a log query request, wherein the log query request comprises a query time period and an identifier of a first service; obtaining identifiers of second services which can be provided in the server, identifiers of a plurality of first log files corresponding to each second service, time information of each first log file, and start time and line numbers corresponding to a plurality of groups of log data in each first log file; on the basis of the query time period, the identifier of the first service, the identifier of each second service and the multiple pieces of time information, determining the identifier of a second log file in the identifiers of the multiple first log files; determining a target line number corresponding to the query time period based on the start time and the line number corresponding to the multiple groups of log data in the second log file; and on the basis of the target line number, log data corresponding to the log query request is obtained in the second log file, so that the log retrieval efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a log retrieval method and device, equipment and storage medium. BACKGROUND

[0002] In recent years, distributed system architecture is widely used, such as system usually consists of hundreds of nodes, each node will produce a large number of logs in the running process, these logs record the system state, error information, user operation and other key data.

[0003] At present, the log retrieval method usually depends on full-text retrieval or time sequence retrieval, however, for some simple retrieval needs of the user, for example: in a large amount of log data, the retrieval efficiency is low when retrieving the log content of a specific time period. SUMMARY

[0004] The present application provides a log retrieval method, device, equipment and storage medium, to improve the efficiency of log retrieval.

[0005] In a first aspect, the present application provides a log retrieval method, the method comprising:

[0006] receiving a log query request, the log query request comprising a query time period and an identifier of a first service, the first service generating log data during execution;

[0007] obtaining identifiers of second services available in a server, identifiers of a plurality of first log files corresponding to each second service, time information of each first log file, start times and line numbers corresponding to a plurality of groups of log data in each first log file;

[0008] based on the query time period, the identifier of the first service, the identifiers of the second services, and the plurality of time information, determining an identifier of a second log file from the identifiers of the plurality of first log files;

[0009] based on the start times and line numbers corresponding to the plurality of groups of log data in the second log file, determining a target line number corresponding to the query time period;

[0010] based on the target line number, obtaining log data corresponding to the log query request from the second log file.

[0011] In a possible implementation, obtaining the identifiers of the second services available in the server, the identifiers of the plurality of first log files corresponding to each second service, the time information of each first log file, the start times and line numbers corresponding to the plurality of groups of log data in each first log file comprises:

[0012] obtaining a first table and a second table in a preset database;

[0013] The first table includes an identification of the second services available in the server, an identification of a plurality of first log files corresponding to each second service, and time information of each first log file.

[0014] The second table includes a start time and a line number corresponding to each group of log data in each first log file.

[0015] In a possible implementation, based on the query time period, the identification of the first service, the identification of each second service, and the plurality of time information, the identification of the second log file is determined from the identification of the plurality of first log files, including:

[0016] The identification of the first service is matched with the identification of each second service, and based on the matching result, a target identification is determined from the identification of each second service, the target identification being the same as the identification of the first service;

[0017] Based on the first table and the target identification, the identification of the plurality of first log files corresponding to the target identification is determined, and based on the identification of the plurality of first log files, the plurality of time information of the plurality of first log files is determined;

[0018] Based on the plurality of time information of the plurality of first log files and the query time period, the identification of the second log file is determined.

[0019] In a possible implementation, based on the plurality of time information of the plurality of first log files corresponding to the target identification and the query time period, the identification of the second log file is determined, including:

[0020] Based on the time information of the first log file corresponding to the target identification, the first log file having an intersection with the query time period is determined, and the second log file is obtained;

[0021] Based on the second log file, the identification of the second log file is determined.

[0022] In a possible implementation, based on the start time and the line number corresponding to each group of log data in the second log file, the target line number corresponding to the query time period is determined, including:

[0023] Based on the start time of the query time period and the plurality of groups of log data in the second log file, the start line number corresponding to the query time period is determined;

[0024] Based on the end time of the query time period and the plurality of groups of log data in the second log file, the end line number corresponding to the query time period is determined;

[0025] The start line number and the end line number are determined as the target line number corresponding to the query time period.

[0026] In one possible implementation, the starting line number corresponding to the query period is determined based on the start time of the query period and multiple sets of log data in the second log file, including:

[0027] Based on the start time of the query period, in the second log file, determine the last group of log data whose start time is less than or equal to the start time among multiple groups of log data;

[0028] The row number corresponding to the last set of log data is determined as the starting row number of the query period.

[0029] In one possible implementation, the end line number corresponding to the query period is determined based on the end time of the query period and multiple sets of log data in the second log file, including:

[0030] Based on the end time of the query period, in the second log file, determine the first set of log data whose start time is greater than or equal to the end time from multiple sets of log data;

[0031] The row number corresponding to the next set of log data in the first set of log data is determined as the end row number of the query period.

[0032] In one possible implementation, based on the target line number, the log data corresponding to the log query request is obtained from the second log file, including:

[0033] Based on the identifier of the second log file, the second log file is determined in the preset path corresponding to the first service;

[0034] Based on the starting line number, log data is retrieved line by line in the second log file until the end line number of the second log file is reached.

[0035] Secondly, this application provides a log retrieval device, comprising: a receiving module, a first acquiring module, a first determining module, a second determining module, and a second acquiring module, wherein:

[0036] The receiving module is used to receive log query requests. The log query request includes the query period and the identifier of the first service. Log data can be generated during the execution of the first service.

[0037] The first acquisition module is used to acquire the identifier of the second service that can be provided in the server, the identifiers of multiple first log files corresponding to each second service, the time information of each first log file, and the start time and line number corresponding to multiple sets of log data in each first log file.

[0038] The first determination module is used to determine the identifier of the second log file from among the identifiers of multiple first log files based on the query time period, the identifier of the first service, the identifiers of each second service, and multiple time information.

[0039] The second determination module is used to determine the target row number corresponding to the query period based on the start time and row number corresponding to multiple sets of log data in the second log file.

[0040] The second acquisition module is used to retrieve the log data corresponding to the log query request from the second log file based on the target line number.

[0041] In one possible implementation, the first acquisition module is specifically used for:

[0042] Retrieve the first and second tables from the preset database;

[0043] The first table includes the identifier of the second service available on the server, the identifiers of multiple first log files corresponding to each second service, and the time information of each first log file;

[0044] The second table includes the start time and line number corresponding to multiple sets of log data in each of the first log files.

[0045] In one possible implementation, the first determining module is specifically used for:

[0046] The identifier of the first service is matched with the identifiers of each second service. Based on the matching results, the target identifier is determined from the identifiers of each second service. The target identifier is the same as the identifier of the first service.

[0047] Based on the first table and the target identifier, the identifiers of multiple first log files corresponding to the target identifier are determined, and based on the identifiers of multiple first log files, multiple time information of multiple first log files are determined;

[0048] The identifier of the second log file is determined based on multiple time information and query periods from multiple first log files.

[0049] In one possible implementation, the first determining module is specifically used for:

[0050] Based on the time information of the first log file corresponding to the target identifier, the first log file that intersects with the query time period is determined, and the second log file is obtained;

[0051] Based on the second log file, determine the identifier of the second log file.

[0052] In one possible implementation, the second determining module is specifically used for:

[0053] Based on the start time of the query period and multiple sets of log data in the second log file, determine the starting row number corresponding to the query period;

[0054] Based on the end time of the query period and multiple sets of log data in the second log file, determine the end line number corresponding to the query period;

[0055] The starting and ending row numbers are determined as the target row numbers corresponding to the query period.

[0056] In one possible implementation, the second determining module is specifically used for:

[0057] Based on the start time of the query period, in the second log file, determine the last group of log data whose start time is less than or equal to the start time among multiple groups of log data;

[0058] The row number corresponding to the last set of log data is determined as the starting row number of the query period.

[0059] In one possible implementation, the second determining module is specifically used for:

[0060] Based on the end time of the query period, in the second log file, determine the first set of log data whose start time is greater than or equal to the end time from multiple sets of log data;

[0061] The row number corresponding to the next set of log data in the first set of log data is determined as the end row number of the query period.

[0062] In one possible implementation, the second acquisition module is specifically used for:

[0063] Based on the identifier of the second log file, the second log file is determined in the preset path corresponding to the first service;

[0064] Based on the starting line number, log data is retrieved line by line in the second log file until the end line number of the second log file is reached.

[0065] Thirdly, embodiments of this application provide an electronic device, including: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, causing the at least one processor to perform the log retrieval method as described in the first aspect above and any of the first aspects that may be involved.

[0066] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the log retrieval method described in the first aspect above and any of the first aspects that may be involved.

[0067] This application provides a log retrieval method, apparatus, device, and storage medium. The method receives a log query request, which includes a query period and an identifier of a first service. Log data can be generated during the execution of the first service. It obtains the identifier of a second service available on a server, the identifiers of multiple first log files corresponding to each second service, the time information of each first log file, and the start time and line number corresponding to multiple sets of log data in each first log file. Based on the query period, the identifier of the first service, the identifiers of each second service, and the multiple time information, it determines the identifier of a second log file from among the identifiers of the multiple first log files. Based on the start time and line number corresponding to the multiple sets of log data in the second log file, it determines the target line number corresponding to the query period. Based on the target line number, it retrieves the log data corresponding to the log query request from the second log file. In this method, by first determining the identifier of the second log file after receiving the log query request, then determining the target line number based on the identifier of the second log file, and then retrieving the log data corresponding to the log query request based on the identifier of the second log file and the target line number, the log retrieval process does not need to traverse all log files or perform full-text parsing of the log content, significantly reducing the amount of invalid data processed, thereby improving the efficiency of log retrieval. Attached Figure Description

[0068] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0069] Figure 1 A flowchart illustrating a log retrieval method provided in an embodiment of this application;

[0070] Figure 2 A flowchart illustrating a method for determining the identifier of a second log file, provided in an embodiment of this application;

[0071] Figure 3 A flowchart illustrating a method for determining the target row number corresponding to a query time period, provided in an embodiment of this application;

[0072] Figure 4 A schematic diagram illustrating the architecture of a log retrieval method provided in an embodiment of this application;

[0073] Figure 5 A schematic diagram illustrating a service running in another electronic device, provided as an embodiment of this application;

[0074] Figure 6 This is a schematic diagram illustrating a process for setting configuration information according to an embodiment of this application;

[0075] Figure 7 A schematic diagram illustrating a user authentication method provided in an embodiment of this application;

[0076] Figure 8 A schematic diagram illustrating the structure for determining the first table and the second table, provided as an embodiment of this application;

[0077] Figure 9 This is a schematic diagram of the structure of a log retrieval device provided in an embodiment of this application;

[0078] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0079] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0080] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0081] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0082] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0083] In recent years, distributed system architecture has been widely adopted. Such systems typically consist of hundreds or thousands of nodes, and each node generates a large number of logs during operation. These logs record key data such as system status, error messages, and user operations.

[0084] Currently, log retrieval methods typically rely on full-text search or chronological search. However, for some simple retrieval needs of users, such as retrieving log content for a specific time period from a large amount of log data, the retrieval efficiency is low.

[0085] Full-text search refers to: traversing multiple log files, and for each log file, the system will check the timestamp corresponding to its data and determine whether it is within the time period input by the user. If it is, the system will extract and return the data corresponding to that timestamp.

[0086] Searching by time order refers to: traversing multiple log files, sorting the data in each log file by timestamp, and then filtering out the data whose timestamps fall within the time period input by the user from the sorted results.

[0087] To address the aforementioned issues, this application provides a log retrieval method. The method receives a log query request, which includes a query period and an identifier of a first service. Log data can be generated during the execution of the first service. It then obtains the identifier of a second service available on the server, the identifiers of multiple first log files corresponding to each second service, the time information of each first log file, and the start time and line number corresponding to multiple sets of log data in each first log file. Based on the query period, the identifier of the first service, the identifiers of each second service, and the multiple time information, it determines the identifier of a second log file from among the identifiers of the multiple first log files. Based on the start time and line number corresponding to the multiple sets of log data in the second log file, it determines the target line number corresponding to the query period. Based on the target line number, it retrieves the log data corresponding to the log query request from the second log file. In this method, by first determining the identifier of the second log file after receiving the log query request, then determining the target line number based on the identifier of the second log file, and finally retrieving the log data corresponding to the log query request based on the identifier of the second log file and the target line number, the log retrieval process does not need to traverse all log files or perform full-text parsing of the log content, significantly reducing the amount of invalid data processed, thereby improving the efficiency of log retrieval.

[0088] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0089] Figure 1 This is a flowchart illustrating a log retrieval method provided in an embodiment of this application. Please refer to... Figure 1 As shown, the method may include the following steps:

[0090] S101. Receive a log query request. The log query request includes the query period and the identifier of the first service. Log data can be generated during the execution of the first service.

[0091] The execution entity in this application embodiment can be an electronic device or a log retrieval device installed in an electronic device. The log retrieval device can be implemented by software or by a combination of software and hardware.

[0092] In one possible implementation, the log query request can be a log query request sent by another electronic device, and the query period can be a time period determined by the user through input or selection on another electronic device. The query period can include the start time and end time of the query period. For example, the start time of the query period is time A, and the end time of the query period is time B. The query period can be represented as time A - time B. It should be noted that the start time and end time of the query period can also include specific year, month, and day. For example, the start time of the query period is represented by the year, month, and day of the m-th day - time A, and the end time of the query period is represented by the year, month, and day of the n-th day - time B. Here, m and n can be the same or different. For simplicity, the query period in the following embodiments will be described using a time-time example.

[0093] In one possible implementation, the first service refers to the service of the component that the user specifies in the log query request to retrieve their logs in a distributed system (such as a Hadoop cluster).

[0094] In this context, a component refers to a functional module in a distributed system, while a service refers to the actual process instances of these components running on cluster nodes. A component can include multiple services. Examples of components include: Hadoop Distributed File System (HDFS), Yet Another Resource Negotiator (YARN), and MapReduce. Examples of services include: NameNode, DataNode, and JournalNode in HDFS; ResourceManager and NodeManager in YARN; and Zookeeper.

[0095] In one possible implementation, the service identifier is used to uniquely identify the service. The service identifier can be a string or an identifier, for example, the service identifier is the name of the service (such as "JournalNode", "NodeManager", etc.), or the service identifier is a combination of the component name and the service name (such as "HDFS-NameNode", "YARN-ResourceManager", etc.). This application embodiment does not limit this.

[0096] S102. Obtain the identifier of the second service that can be provided in the server, the identifiers of multiple first log files corresponding to each second service, the time information of each first log file, and the start time and line number corresponding to multiple sets of log data in each first log file.

[0097] In one possible implementation, the log file identifier is used to uniquely identify the log file. The log file identifier can be the inode of the log file on the server. Each log file has a different inode, and the inode can be a number, such as 6031.

[0098] In one possible implementation, the time information of the first log file may include the start time and end time of the log data in the first log file. The start time of the log data in the first log file refers to the occurrence time of the log data with the earliest timestamp among the log data contained in the first log file, and the end time of the log data in the first log file refers to the occurrence time of the log data with the latest timestamp among the log data contained in the first log file.

[0099] In one possible implementation, the log data in the first log file can be grouped according to a preset time interval to obtain multiple groups of log data. The start time corresponding to the multiple groups of log data in each first log file refers to the timestamp corresponding to the first data in each group of log data, and the line number corresponding to the multiple groups of log data in each first log file refers to the line number corresponding to the first data in each group of log data.

[0100] The preset time interval, for example, is 5 minutes (Minute), which indicates that each log record in the file is assigned to a continuous time interval of 5 minutes according to its timestamp.

[0101] In some implementations, obtaining the identifier of the second service available on the server, the identifiers of multiple first log files corresponding to each second service, the time information of each first log file, and the start time and line number corresponding to multiple sets of log data in each first log file may include:

[0102] Retrieve the first table and the second table from the preset database; the first table includes the identifier of the second service that can be provided on the server, the identifier of the multiple first log files corresponding to each second service, and the time information of each first log file; the second table includes the start time and line number corresponding to multiple sets of log data in each first log file.

[0103] In one possible implementation, the identifiers of the second services available on the server, the identifiers of the multiple first log files corresponding to each second service, and the time information of each first log file can be obtained from the first table using Structured Query Language (SQL) statements. Alternatively, the start time and line number corresponding to multiple sets of log data in each first log file can be obtained from the second table using SQL statements.

[0104] In one possible implementation, the default database is, for example, a MySQL database. It should be noted that the default database needs to be deployed in a network location accessible to both the log retrieval device (or the electronic device in which it is located) and all electronic devices in the cluster, and the database connection parameters (such as Internet Protocol (IP) address, port, username, password, etc.) should be correctly configured so that each electronic device can have stable and efficient data interaction with the database when building log indexes and retrieving logs.

[0105] In one possible implementation, the first table and the second table are determined by each electronic device in the distributed system based on its own log file. The electronic devices in the distributed system share the same first table and the same second table. That is, the first table includes the identifiers of each second service that each server can provide, the identifiers of multiple first log files corresponding to each second service, and the time information of each first log file; the second table includes the start time and line number corresponding to multiple sets of log data in each first log file of each server.

[0106] In one possible implementation, the identifiers of the various second services provided by the servers included in the first table can be different, and the identifiers of the multiple first log files corresponding to the various second services included in the first table can also be different. That is, the same second service can generate multiple log files during its operation, and the time information of each first log file included in the first table can include the start and end times of the log data in each first log file.

[0107] For example, the first table is Table 1. It should be noted that the start time and end time of the log data can also include specific year, month, and day. The start time of the log data is, for example, the year, month, and day of the j-th day - time C, and the end time of the log data is, for example, the year, month, and day of the k-th day - time D. Here, j and k can be the same or different. For the sake of simplicity, the start time and end time of the log data in the following embodiments will be described using a certain time - a certain time as an example.

[0108] Table 1

[0109]

[0110] In one possible implementation, in addition to including the start time and line number corresponding to multiple sets of log data in each of the first log files, the second table may also include the identifier of the second service and the identifier of the first log file.

[0111] For example, the first table is Table 2. It should be noted that Table 2 is used as an example to illustrate multiple sets of log data in a first log file.

[0112] Table 2

[0113]

[0114] If the preset time interval is 5 minutes, then there is a 5-minute interval between time K and time L, and a 5-minute interval between time L and time M. For example, time L is time K plus 5 minutes, time M is time L plus 5 minutes, and the line number of the log data corresponding to time K is 980336.

[0115] S103. Based on the query period, the identifier of the first service, the identifiers of each second service, and multiple time information, determine the identifier of the second log file from among the identifiers of multiple first log files.

[0116] In one possible implementation, the identifier of the second service corresponding to the second log file is the same as the identifier of the first service.

[0117] In one possible implementation, there can be one or more identifiers for the second log file. That is, based on the query period, the identifier of the first service, the identifiers of each second service, and multiple time information, it can be determined that one or more second log files include log data corresponding to the log query request.

[0118] S104. Based on the start time and line number corresponding to multiple sets of log data in the second log file, determine the target line number corresponding to the query period.

[0119] In one possible implementation, the target row number can be represented as a continuous range of row numbers, such as from row A to row D; or, the target row number can be represented as a discrete set of row numbers, such as row A, row B, row C, and row D.

[0120] In one possible implementation, the target line number can be determined for each of the second log files identified in S103.

[0121] S105. Based on the target line number, retrieve the log data corresponding to the log query request in the second log file.

[0122] In one possible implementation, the target line number is used to locate the data in the corresponding second log file, and the data at the target line number in the second log file is read directly. This line number-based reading method avoids a full scan of multiple log files from beginning to end, thus improving the efficiency of log retrieval.

[0123] In some implementations, retrieving the log data corresponding to the log query request in the second log file, based on the target line number, may include:

[0124] Based on the identifier of the second log file, the second log file is determined in the preset path corresponding to the first service; based on the starting line number, log data is obtained line by line in the second log file until the ending line number of the second log file is read.

[0125] In one possible implementation, log files for different services can be stored in different paths, and the names of the log files for different services can be different. Based on preset rules, a preset path for the first service can be determined, and a second log file can be determined based on the name of the log file for the first service within that preset path.

[0126] In this embodiment, a log query request is received, including a query period and an identifier of a first service. Log data can be generated during the execution of the first service. The method obtains the identifier of a second service available on the server, the identifiers of multiple first log files corresponding to each second service, the time information of each first log file, and the start time and line number corresponding to multiple sets of log data in each first log file. Based on the query period, the identifier of the first service, the identifiers of each second service, and the multiple time information, the identifier of the second log file is determined from among the identifiers of the multiple first log files. Based on the start time and line number corresponding to the multiple sets of log data in the second log file, the target line number corresponding to the query period is determined. Based on the target line number, the log data corresponding to the log query request is obtained from the second log file. In this method, by first determining the identifier of the second log file after receiving the log query request, then determining the target line number based on the identifier of the second log file, and then obtaining the log data corresponding to the log query request based on the identifier of the second log file and the target line number, the log retrieval process does not need to traverse all log files or perform full-text parsing of the log content, significantly reducing the amount of invalid data processed, thereby improving the efficiency of log retrieval and enabling a faster response to user requests for log queries.

[0127] Furthermore, in this embodiment of the application, by using a database as the underlying storage for the log index, resource consumption can be reduced.

[0128] Based on the above embodiments, when the identifier of the second log file is determined, the identifier of the second log file, the identifier of the service of the second log file, and the time information of the second log file can be stored in a temporary cache space. Then, the electronic device reads this information in the temporary cache space to determine the target row number corresponding to the query period. In this way, the efficiency of the electronic device in reading information can be improved.

[0129] Based on any of the above embodiments, the following, in conjunction with Figure 2 The detailed explanation is as follows: based on the query period, the identifier of the first service, the identifiers of each second service, and multiple time information, the identifier of the second log file is determined from among the identifiers of multiple first log files. Figure 1 The method in S103 of the embodiment.

[0130] Figure 2 A flowchart illustrating a method for determining the identifier of a second log file, provided in this application embodiment, is available for detailed reference. Figure 2 The method may include:

[0131] S201. Match the identifier of the first service with the identifiers of each second service, and based on the matching results, determine the target identifier from the identifiers of each second service. The target identifier is the same as the identifier of the first service.

[0132] In one possible implementation, the target identifier is the identifier of the service, and the target identifier is used to uniquely identify the service in the server.

[0133] In one possible implementation, the identifiers of each second service that the server can provide can be extracted from the first table to determine the service list. The identifier of the first service in the user query request is matched with the identifiers of each second service in the service list (one-to-one comparison). If the identifier of the second service is found to be exactly the same as the identifier of the first service, the identifier of the second service (or the identifier of the first service) is determined as the target identifier, and S202 continues to be executed.

[0134] In one possible implementation, if no identifier of a second service that is exactly the same as the identifier of the first service is found, the log retrieval process can be terminated early, and the log data corresponding to the log query request that was not retrieved can be returned to the sender of the log query request.

[0135] This allows invalid requests to be filtered out early in the log retrieval process, or it can determine whether log data corresponding to a log query request exists early in the log retrieval process. If not, the log retrieval process can be terminated in advance without executing subsequent methods, thus saving computing resources.

[0136] S202. Based on the first table and the target identifier, determine the identifiers of multiple first log files corresponding to the target identifier, and based on the identifiers of multiple first log files, determine multiple time information of multiple first log files.

[0137] In one possible implementation, an SQL statement can be used to query the first table to find log records whose identifiers match the target identifier for each second service, and then determine the identifier of the corresponding first log file from the matched log records.

[0138] In one possible implementation, multiple time information of multiple first log files can be identified in the first table by using the identifiers of multiple first log files.

[0139] S203. Based on multiple time information and query time periods of multiple first log files, determine the identifier of the second log file.

[0140] In some implementations, the specific methods for determining the identifier of the second log file based on multiple time information and query time periods from multiple first log files are as follows:

[0141] Based on the time information of the first log file corresponding to the target identifier, the first log file that intersects with the query period is determined, and the second log file is obtained; based on the second log file, the identifier of the second log file is determined.

[0142] In one possible implementation, there can be multiple first log files corresponding to the target identifier.

[0143] In one possible implementation, the time information of the first log file overlaps with the query period. The first log file whose time information overlaps with the query period is determined as the second log file. The overlap between the time information of the first log file and the query period can indicate that the first log file includes log data corresponding to at least one log query request.

[0144] In one possible implementation, the start time of the log data in the time information of the first log file is time File_start, the end time of the log data in the time information of the first log file is time File_end, the start time of the query period in the query period is time Query_start, and the end time of the query period in the query period is time Query_end. For example, the association between the time information of the first log file and the query period can include the following situations, and the corresponding method for determining whether there is an intersection between the time information of the first log file and the query period is as follows:

[0145] Scenario 1: The time information in the first log file completely covers the query period. Determine whether the time File_start is less than the time Query_end, and at the same time determine whether the time File_end is greater than the time Query_start. If the time File_start is less than the time Query_end, and the time File_end is greater than the time Query_start, then the time information in the first log file and the query period overlap.

[0146] Scenario 2: The query period completely covers the time information of the first log file. Determine whether the time File_start is less than the time Query_end, and at the same time determine whether the time File_end is greater than the time Query_start. If the time File_start is less than the time Query_end, and the time File_end is greater than the time Query_start, then the time information of the first log file and the query period overlap.

[0147] Scenario 3: The time information of the first log file partially overlaps with the query period. Determine whether the time File_start is less than the time Query_end, and at the same time determine whether the time File_end is greater than the time Query_start. If the time File_start is less than the time Query_end, and the time File_end is greater than the time Query_start, then the time information of the first log file overlaps with the query period.

[0148] In this embodiment, by matching the service identifier, the search scope is narrowed from all services to the service corresponding to the target identifier. Then, by judging the intersection of the query period and the time information of the log file, the search scope is further narrowed from the service corresponding to the target identifier to the log file related to the query period. In this way, the log file corresponding to the log query request is determined in memory before performing disk input / output (I / O) operations (reading log data), which can improve the log retrieval efficiency.

[0149] Below, in conjunction with Figure 3 The detailed explanation is based on the start time and line number corresponding to multiple sets of log data in the second log file to determine the target line number corresponding to the query period. Figure 1 The method in S104 of the embodiment.

[0150] Figure 3 A flowchart illustrating a method for determining the target row number corresponding to a query time period, provided in this application embodiment, is available for detailed reference. Figure 3 The method may include:

[0151] S301. Based on the start time of the query period and multiple sets of log data in the second log file, determine the starting line number corresponding to the query period.

[0152] In some implementations, the starting line number corresponding to the query period is determined based on the start time of the query period and multiple sets of log data in the second log file, including:

[0153] Based on the start time of the query period, in the second log file, the last group of log data whose start time is less than or equal to the start time is identified from multiple groups of log data; the line number corresponding to the last group of log data is determined as the starting line number corresponding to the query period.

[0154] In one possible implementation, algorithms such as binary search can be used to determine the last group of log data in the second table whose start time is less than or equal to the start time of the query period.

[0155] For example, the second table is Table 3. It should be noted that Table 3 is used as an example with a preset time interval of 5 minutes. The start time of the query period is 19:43 on day U. Therefore, the start time of the last group of log data whose start time is less than or equal to the start time is 19:40 on day U. The starting row number of the query period is 610336.

[0156] Table 3

[0157]

[0158]

[0159] S302. Based on the end time of the query period and multiple sets of log data in the second log file, determine the end line number corresponding to the query period.

[0160] In some implementations, the end line number corresponding to the query period is determined based on the end time of the query period and multiple sets of log data in the second log file, including:

[0161] Based on the end time of the query period, in the second log file, the first group of log data whose start time is greater than or equal to the end time is determined from multiple groups of log data; the line number corresponding to the next group of log data is determined as the end line number corresponding to the query period.

[0162] In one possible implementation, algorithms such as binary search can be used to determine the log data in the second table whose start time is greater than or equal to the end time of the query period.

[0163] For example, the second table is Table 3. The end time of the query period is 19:47 on day U. Then, the start time of the first group of log data whose start time is greater than or equal to the end time is 19:50 on day U. The end row number of the query period is 611009.

[0164] S303. Determine the starting row number and ending row number as the target row number corresponding to the query period.

[0165] In one possible implementation, the starting line number and the ending line number can be combined to form a continuous range of line numbers, and each line number in this continuous range of line numbers can be determined as the target line number.

[0166] In this embodiment, the row number corresponding to the last group of log data whose start time is less than or equal to the start time is determined as the starting row number of the query period, and the row number corresponding to the first group of log data whose start time is greater than or equal to the end time is determined as the ending row number of the query period. The target row number is determined based on the starting and ending row numbers. The entire log retrieval process does not involve reading and parsing the log file content. The electronic device performs two fast lookups (such as binary search) on the second table (a lightweight index table) to accurately locate the row range of the log data to be read. This reduces disk I / O operations.

[0167] Based on the above embodiments, when there is a large amount of retrieved log data, electronic device 1 can store the retrieved log data in memory. Since the content displayed on the front end is limited, electronic device 1 can display a portion of the retrieved log data. Then, in response to the user's swiping operation on electronic device 1, electronic device 1 can continue to read log data from memory from the end line of the displayed portion of log data until the user has completely browsed the log data in memory.

[0168] Next, combined Figure 4 The overall architecture of a log retrieval method provided in this application embodiment will be described in detail, for example, Figure 4 This is a schematic diagram of the architecture of a log retrieval method provided in an embodiment of this application.

[0169] Reference Figure 4 Users can configure information on the World Wide Web (WEB) page of electronic device 1 by typing or inputting. This configuration information can include: data source, target address, configuration rules, filtering rules, and advanced configuration items. After the user completes the configuration, electronic device 1 can send the configuration information and program to other electronic devices in the distributed system (electronic devices 2, 3, and 4), allowing these other electronic devices to determine and store the first and second tables based on the configuration information.

[0170] The data source can include the address and service name of other electronic devices; the target address refers to the address where the log file is stored on other electronic devices; the configuration rules refer to the file names of log files for different services; and the filtering rules refer to the format of log data for different services.

[0171] Advanced configuration options can include a first configuration sub-item and a second configuration sub-item. The first configuration sub-item can be used to implement self-awareness, which means that the program will monitor changes in log files on other electronic devices in the distributed system (Hadoop cluster) and monitor changes in other electronic devices (nodes). For nodes that are being expanded, the program will send the program and the configuration selected or entered by the user on the web page of electronic device 1 to the newly expanded node; for nodes that are being shrunk, the program will stop running the program on the shrunk node and delete the data about the shrunk node in the first table and the second table.

[0172] The second configuration sub-item can be used to achieve adaptive behavior. Adaptive behavior refers to real-time sending of configuration information and dynamic adjustment of threads. Real-time sending of configuration information means that after the user modifies the configuration information on the web page of electronic device 1, the program can immediately send the updated configuration information to each electronic device (electronic device 2, electronic device 3, and electronic device 4) in the distributed system to ensure that the configuration information of each electronic device in the distributed system is consistent. Dynamic adjustment of threads means that when there are many services in other electronic devices or the log files are large, the time to determine the first table and the second table based on the log files is long. The program can automatically start other threads to determine the first table and the second table in parallel based on the log files, thereby achieving automatic task splitting and efficient determination of the first table and the second table.

[0173] In this embodiment of the application, log retrieval is implemented by using front-end and back-end processes with minimal resources, which reduces resource consumption.

[0174] In one possible implementation, users can send configuration information and programs with a single click on the front-end web page, making it easier and faster for users to retrieve logs.

[0175] In one possible implementation, the way electronic device 1 sends configuration information and programs to other electronic devices in the distributed system can include the following two methods:

[0176] Implementation Method 1: Electronic device 1 can send configuration information and programs to other electronic devices respectively. That is, electronic device 1 can send configuration information and programs to electronic devices 2, 3 and 4 respectively. Users using the web page need to fill in the username and password of electronic devices 2, 3 and 4 respectively to trigger the sending.

[0177] Implementation Method 2: When electronic devices 2, 3, and 4 are all configured with passwordless login, electronic device 1 can send configuration information and programs to one of the other electronic devices (such as electronic device 2). Before sending, the user needs to fill in the username and password of electronic device 2 on electronic device 1, and the user also needs to enter the usernames of electronic devices 2 and 3.

[0178] It should be noted that the program can query each node in the distributed system through the Application Programming Interface (API), and then remotely copy the program and configuration information to each node.

[0179] In one possible implementation, the configuration information sent by electronic device 1 may include component name, program deployment path, information of each electronic device in the cluster, whether the first configuration sub-item is enabled, whether the second configuration sub-item is enabled, multiple services, target address of log files corresponding to each service, configuration rules corresponding to each service (file name of log files corresponding to each service), filtering rules corresponding to each service (format of log data in log files corresponding to each service), database type (MySQL, etc.) stored in the first and second tables, database address, database username, database password, database port, and database name, etc.

[0180] When other electronic devices receive the program and configuration information sent by electronic device 1, the other electronic devices can obtain information from the log files of different services in the electronic device based on the configuration information, determine the first table and the second table, and store the first table and the second table in the corresponding database according to the configuration information.

[0181] It should be noted that the program has the ability to monitor task status and supports breakpoint restart. In other words, since the database records the start time and line number corresponding to each group of log data in the log file, if other electronic devices exit the determination of the first and second tables due to system failure or other issues, the program can continue the determination of the first and second tables from the previous breakpoint when it restarts the determination of the first and second tables.

[0182] It should be noted that electronic devices 2, 3, and 4 can run different services on their respective devices. This application embodiment does not limit this; it is merely an example. Figure 5 A schematic diagram illustrating a service running in another electronic device as provided in this application embodiment, with reference to... Figure 5 The services running on electronic device 2 include: Zookeeper, NameNode, ResourceManager, NodeManager, and DataNode; the services running on electronic device 3 include: zookeeper and NameNode; and the services running on electronic device 4 include: NameNode, ResourceManager, and NodeManager.

[0183] Based on the above embodiments, the configuration information may also include the name of the component. When a user selects a component, the multiple services included in that component can be selected by default, which can make the user's operation more convenient.

[0184] Based on the above embodiments, electronic device 1 can also provide users with pre-set configuration combinations on the WEB page. The configuration combination includes default values ​​of configuration information. Users can directly use the configuration combination, or modify the default values ​​based on the configuration combination and then trigger the sending of the modified configuration information to other electronic devices. This makes the user's operation more convenient.

[0185] In this embodiment, the user only needs to fill in basic parameters such as the target address of the log file and the database connection information, and the electronic device can automatically generate a configuration file; the electronic device also provides preset configuration combination templates, which the user can use or modify directly, reducing the difficulty and complexity of configuration.

[0186] Based on the above embodiments, users can retrieve logs through a web page. Electronic device 1 sends a log query request to the server corresponding to the log query request (e.g., electronic device 2). Electronic device 2 then obtains the log data using the log retrieval method provided in this application embodiment and returns the log data to electronic device 1 so that electronic device 1 can display the log data to the user.

[0187] Based on the above embodiments, the following will be combined with... Figure 6 This document provides a detailed explanation of the process for users to set configuration information on a web page, with examples provided. Figure 6 This is a schematic diagram illustrating a process for setting configuration information according to an embodiment of this application. Users can operate on a web page, selecting components (such as HDFS, YARN, MapReduce, etc.) and obtaining configuration combinations. All services within the components are selected by default. Users can select the service corresponding to the logs they wish to retrieve. Afterwards, users can modify the configuration. For example, users can access advanced configuration items, including a first configuration sub-item and a second configuration sub-item. After completing the configuration modification, users can send the configuration information and program to other nodes in the distributed system by clicking buttons, thereby enabling the retrieval of log data from services on each node in the distributed system.

[0188] Based on the above embodiments, user authentication and user authorization can be performed on the WEB page of electronic device 1. User authentication refers to the fact that when a user logs into the WEB page on electronic device 1, electronic device 1 can authenticate the user to determine whether the user can read the log file service. If the user searches for the log file of a service that the user cannot read, electronic device 1 can return information such as "no permission to search" to the user.

[0189] In one possible implementation, a service that allows tenants to read their log files can be pre-configured. When a user logs into the web page, the user's tenant can be identified first, and then the service that allows the tenant to read their log files can be identified by determining that the user can read their log files.

[0190] For example, Figure 7 A schematic diagram of a user authentication method provided in an embodiment of this application is shown below. Figure 7 Tenant 1 can retrieve log data from the log files corresponding to Zookeeper, NameNode, DataNode, ResourceManager, and NodeManager. Tenant 2 can also retrieve log data from the log files corresponding to Zookeeper, DataNode, and NodeManager. Users 1 and 2 belong to Tenant 1, and User 3 belongs to Tenant 2. Therefore, Users 1 and 2 can both retrieve log data from the log files corresponding to Zookeeper, NameNode, DataNode, ResourceManager, and NodeManager, and User 3 can retrieve log data from the log files corresponding to Zookeeper, DataNode, and NodeManager. This allows for the verification of user access to specific services.

[0191] Based on the above embodiments, user authorization or deauthorization can be performed on the web page of electronic device 1. For example, user 3 cannot retrieve the log file corresponding to the service named NameNode. By authorizing user 3, user 3 can retrieve the log file corresponding to the service named NameNode. User 1 can retrieve the log file corresponding to the service named Zookeeper. By deauthorizing user 1 for Zookeeper, user 1 cannot retrieve the log file corresponding to the service named Zookeeper.

[0192] In this embodiment of the application, user authentication is performed so that only authorized users can retrieve logs. Furthermore, by establishing a fine-grained authorization system, access and operation permissions for different services can be restricted based on user roles and permissions, thereby preventing the leakage of sensitive log information.

[0193] Furthermore, in this embodiment of the application, by building a unified platform including a web page, centralized management and monitoring of multiple log synchronization tasks can be achieved. On the web page, users can view the running status, performance indicators, error information, etc. of each task in real time, thereby improving management efficiency and reducing operation and maintenance costs.

[0194] Next, combined Figure 8 The structure of the first and second tables is explained, for example. Figure 8 A schematic diagram illustrating the structure of a first table and a second table provided in this application embodiment, referring to... Figure 8 When the user sets the configuration information on the web page of electronic device 1, electronic device 1 can send the configuration information and program to electronic devices 2, 3 and 4. Then, electronic devices 2, 3 and 4 combine the log files of different services in their respective devices to determine the first table and the second table, and store the first table and the second table in the database.

[0195] The following describes in detail the method for determining the first and second tables of electronic device 2, using electronic device 2 as an example:

[0196] First, a table creation operation is performed to create the first and second tables in the database indicated by the configuration information. Then, the services of electronic device 2 are obtained. Next, each service is traversed, and during the traversal, the log files of each service are obtained. After that, the existence of the log file in the cache is determined based on the identifier of the log file.

[0197] If the log file is not in the cache, retrieve the detailed information of the log file, such as the log file modification time, log file time information, the maximum line number of the log file, and whether the log file is archived, and insert this information into the first table; at the same time, insert the start time and line number corresponding to each group of log data in the log file into the second table, and update the cache;

[0198] If the log file is in the cache, determine whether the log data in the log file has been updated based on the modification time of the log file; if the log data has been updated, for the newly added log data, update the start time and line number of the log data group corresponding to the newly added log data to the second table, and at the same time update the maximum line number of the log file and the modification time of the log file to the first table; after all services have been traversed, electronic device 2 determines that the first table and the second table are complete.

[0199] In this embodiment of the application, by using a pre-built first table and a second table, the log retrieval method is changed from a full scan method to a line number-based retrieval method. In this way, when the user performs log retrieval, the electronic device does not need to traverse a large amount of log file content. By querying the lightweight first table and the second table, the electronic device can locate the log file and the precise line number range where the target log data is located, which greatly reduces the disk input / output (I / O) and computational overhead and speeds up the log retrieval response speed.

[0200] Figure 9 This is a schematic diagram of a log retrieval device provided in an embodiment of this application. Please refer to... Figure 9 The log retrieval device 900 includes: a receiving module 901, a first acquisition module 902, a first determination module 903, a second determination module 904, and a second acquisition module 905, wherein:

[0201] The receiving module 901 is used to receive log query requests. The log query requests include the query period and the identifier of the first service. Log data can be generated during the execution of the first service.

[0202] The first acquisition module 902 is used to acquire the identifier of the second service that can be provided in the server, the identifiers of multiple first log files corresponding to each second service, the time information of each first log file, and the start time and line number corresponding to multiple sets of log data in each first log file.

[0203] The first determining module 903 is used to determine the identifier of the second log file from among the identifiers of multiple first log files based on the query time period, the identifier of the first service, the identifiers of each second service, and multiple time information.

[0204] The second determination module 904 is used to determine the target row number corresponding to the query period based on the start time and row number corresponding to multiple sets of log data in the second log file.

[0205] The second acquisition module 905 is used to acquire the log data corresponding to the log query request in the second log file based on the target line number.

[0206] The log retrieval device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0207] In one possible implementation, the first acquisition module 902 is specifically used for:

[0208] Retrieve the first and second tables from the preset database;

[0209] The first table includes the identifier of the second service available on the server, the identifiers of multiple first log files corresponding to each second service, and the time information of each first log file;

[0210] The second table includes the start time and line number corresponding to multiple sets of log data in each of the first log files.

[0211] In one possible implementation, the first determining module 903 is specifically used for:

[0212] The identifier of the first service is matched with the identifiers of each second service. Based on the matching results, the target identifier is determined from the identifiers of each second service. The target identifier is the same as the identifier of the first service.

[0213] Based on the first table and the target identifier, the identifiers of multiple first log files corresponding to the target identifier are determined, and based on the identifiers of multiple first log files, multiple time information of multiple first log files are determined;

[0214] The identifier of the second log file is determined based on multiple time information and query periods from multiple first log files.

[0215] In one possible implementation, the first determining module 903 is specifically used for:

[0216] Based on the time information of the first log file corresponding to the target identifier, the first log file that intersects with the query time period is determined, and the second log file is obtained;

[0217] Based on the second log file, determine the identifier of the second log file.

[0218] In one possible implementation, the second determining module 904 is specifically used for:

[0219] Based on the start time of the query period and multiple sets of log data in the second log file, determine the starting row number corresponding to the query period;

[0220] Based on the end time of the query period and multiple sets of log data in the second log file, determine the end line number corresponding to the query period;

[0221] The starting and ending row numbers are determined as the target row numbers corresponding to the query period.

[0222] In one possible implementation, the second determining module 904 is specifically used for:

[0223] Based on the start time of the query period, in the second log file, determine the last group of log data whose start time is less than or equal to the start time among multiple groups of log data;

[0224] The row number corresponding to the last set of log data is determined as the starting row number of the query period.

[0225] In one possible implementation, the second determining module 904 is specifically used for:

[0226] Based on the end time of the query period, in the second log file, determine the first set of log data whose start time is greater than or equal to the end time from multiple sets of log data;

[0227] The row number corresponding to the next set of log data in the first set of log data is determined as the end row number of the query period.

[0228] In one possible implementation, the second acquisition module 905 is specifically used for:

[0229] Based on the identifier of the second log file, the second log file is determined in the preset path corresponding to the first service;

[0230] Based on the starting line number, log data is retrieved line by line in the second log file until the end line number of the second log file is reached.

[0231] The log retrieval device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0232] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 10 As shown, the electronic device 1000 may include: a transceiver 1001, a processor 1002, and a memory 1003.

[0233] Processor 1002 executes computer execution instructions stored in memory, causing processor 1002 to perform the scheme in the above embodiments. Processor 1002 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0234] The memory 1003 is connected to the processor 1002 via the system bus and completes communication between them. The memory 1003 is used to store computer program instructions.

[0235] Transceiver 1001 can be used to obtain the task to be run and its configuration information.

[0236] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0237] The electronic device provided in this application embodiment can be any device with on-device computing capabilities. For example, the electronic device can be a server, computer, or other such device, and this application embodiment does not limit this.

[0238] This application also provides a chip for executing instructions, which is used to execute the log retrieval method described in the above embodiments.

[0239] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a computer, the computer performs the log retrieval method described in the above embodiments.

[0240] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0241] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0242] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0243] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0244] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0245] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0246] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0247] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0248] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0249] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0250] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A log retrieval method characterized by comprising: The method comprises: receiving a log query request, the log query request comprising a query period and an identification of a first service, the first service generating log data during execution; obtaining, in a server, an identification of a second service available in the server, an identification of a plurality of first log files corresponding to each second service, time information of each first log file, and a start time and a line number corresponding to each group of log data in each first log file; based on the query period, the identification of the first service, the identification of each second service, and the plurality of time information, determining, in the identification of the plurality of first log files, an identification of a second log file; based on the start time and the line number corresponding to each group of log data in the second log file, determining a target line number corresponding to the query period; based on the target line number, obtaining, in the second log file, log data corresponding to the log query request.

2. The method of claim 1, wherein, The obtaining, in the server, of the identification of the second service available in the server, the identification of the plurality of first log files corresponding to each second service, the time information of each first log file, and the start time and the line number corresponding to each group of log data in each first log file comprises: obtaining a first table and a second table in a preset database; the first table comprises the identification of the second service available in the server, the identification of the plurality of first log files corresponding to each second service, and the time information of each first log file; the second table comprises the start time and the line number corresponding to each group of log data in each first log file.

3. The method of claim 2, wherein, The determining, based on the query period, the identification of the first service, the identification of each second service, and the plurality of time information, of the identification of the second log file in the identification of the plurality of first log files comprises: matching the identification of the first service with the identification of each second service, and determining, according to a matching result, a target identification in the identification of each second service, the target identification being the same as the identification of the first service; based on the first table and the target identification, determining the identification of the plurality of first log files corresponding to the target identification, and based on the identification of the plurality of first log files, determining a plurality of time information of the plurality of first log files; based on the plurality of time information of the plurality of first log files and the query period, determining the identification of the second log file.

4. The method of claim 3, wherein, The determining, based on the plurality of time information of the plurality of first log files corresponding to the target identification and the query period, of the identification of the second log file comprises: based on the time information of the first log file corresponding to the target identification, determining a first log file having an intersection with the query period, to obtain the second log file; based on the second log file, determining the identification of the second log file.

5. The method of claim 2, wherein, The determining, based on the start time and the line number corresponding to each group of log data in the second log file, of a target line number corresponding to the query period comprises: based on a start time of the query period and each group of log data in the second log file, determining a start line number corresponding to the query period; determine, based on the end time of the query time period and the multiple sets of log data in the second log file, an end line number corresponding to the query time period; determine the start line number and the end line number as a target line number corresponding to the query time period.

6. The method of claim 5, wherein, The determination of the start line number corresponding to the query time period based on the start time of the query time period and the multiple sets of log data in the second log file comprises: determine, based on the start time of the query time period, the last set of log data in the multiple sets of log data in the second log file, for which the start time of the log data is less than or equal to the start time; determine, as the start line number corresponding to the query time period, the line number corresponding to the last set of log data.

7. The method of claim 5, wherein, The determination of the end line number corresponding to the query time period based on the end time of the query time period and the multiple sets of log data in the second log file comprises: determine, based on the end time of the query time period, the first set of log data in the multiple sets of log data in the second log file, for which the start time of the log data is greater than or equal to the end time; determine, as the end line number corresponding to the query time period, the line number corresponding to the next set of log data of the first set of log data.

8. The method of claim 5, wherein, The obtaining of the log data corresponding to the log query request in the second log file based on the target line number comprises: determine the second log file under a preset path corresponding to the first service based on the identifier of the second log file; obtain the log data in the second log file line by line based on the start line number until the end line number of the second log file is read.

9. A log retrieval apparatus characterized by comprising: comprise: a receiving module, a first obtaining module, a first determining module, a second determining module, and a second obtaining module, wherein: the receiving module is configured to receive a log query request, the log query request comprising a query time period and an identifier of a first service, the first service being capable of generating log data during execution; the first obtaining module is configured to obtain identifiers of second services available in a server, identifiers of a plurality of first log files corresponding to the second services, time information of the first log files, and start times and line numbers of the multiple sets of log data in the first log files; the first determining module is configured to determine, based on the query time period, the identifier of the first service, the identifiers of the second services, and the time information, an identifier of a second log file in the identifiers of the first log files; the second determining module is configured to determine, based on the start times and line numbers of the multiple sets of log data in the second log file, a target line number corresponding to the query time period; the second obtaining module is configured to obtain, based on the target line number, log data corresponding to the log query request in the second log file.

10. An electronic device comprising: a processor and a memory in communication connection with the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the log retrieval method according to any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are used for realizing the log retrieval method in any one of claims 1 to 8 when executed by the processor.

Images