Log retrieval method and apparatus

Abstract

The application provides a log retrieval method and device. The method comprises the following steps: obtaining retrieval information; retrieving a target log package from log packages of a plurality of servers corresponding to an application program according to the retrieval information, wherein each log package of each server comprises log data generated by a corresponding link of a corresponding service of the application program, an identifier of a thread used by the application program when the log data is generated, and input data and output data of a called method; retrieving an extended log package from the log packages of the plurality of servers according to the identifier, the input data and the output data in the target log package; and determining the target log package and the extended log package as log packages corresponding to the retrieval information. In this way, when log data is retrieved based on the retrieval information, associated log data can be found according to the thread identifier and the input data and output data of the called method carried by the retrieved log data, comprehensive searching of log data required by the retrieval information is realized, and the completeness of log retrieval is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a log retrieval method, log retrieval device, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Logs are an integral part of applications. Recording application logs provides data support when troubleshooting application anomalies. Retrieving recorded logs allows for precise identification of the specific logs needed, improving problem-solving efficiency.

[0003] Logs are typically stored in a distributed manner. This means that different logs generated by an application are stored on different servers. When it's necessary to retrieve these application logs, the logs from different servers are first moved to the same location, and then a search is performed within that location using keywords or key phrases to obtain the desired logs.

[0004] However, logs generated at different stages of the same business process within an application may not share common parameters. Take the payment process in a shopping app as an example. The payment process involves transaction, settlement, and clearing stages. Logs generated during the transaction stage contain the order number, while logs generated during the settlement and clearing stages do not. Searching the logs using the order number will only retrieve logs generated during the transaction stage of the payment process. Logs generated during the settlement and clearing stages cannot be retrieved using the order number. This reduces the completeness of log retrieval. Summary of the Invention

[0005] The purpose of this application is to provide a log retrieval method, log retrieval device, electronic device, computer-readable storage medium, and computer program product to improve the completeness of log retrieval.

[0006] To address the aforementioned technical problems, this application provides the following technical solutions:

[0007] The first aspect of this application provides a log retrieval method, the method comprising: obtaining retrieval information; retrieving a target log package from log packages of multiple servers corresponding to an application based on the retrieval information, wherein each log package of each server includes log data generated by a corresponding link of the application's corresponding business, an identifier of the thread used by the application when the log data was generated, and input and output data of the called method; retrieving an extended log package from the log packages of the multiple servers based on the identifier, input data, and output data in the target log package; and determining the target log package and the extended log package as the log packages corresponding to the retrieval information.

[0008] Compared to existing technologies, the log retrieval method provided in the first aspect of this application stores the thread identifier that generated the log data, as well as the input and output data of the method call, along with the log data itself. This allows for the retrieval of log data based on search information, enabling the identification of the thread and the input and output data of the called method carried in the retrieved log data to identify related log data. This achieves a comprehensive search of the log data required for the search information, improving the completeness of log retrieval.

[0009] In some modified embodiments of the first aspect of this application, before obtaining the retrieval information, the method further includes: obtaining target log data generated by the target stage of the target business of the application; obtaining the identifier of the thread used by the application when the target log data is generated, as well as the input data and output data of the called method, wherein different stages of the same business in the application use the same thread when running, and the output data of the method called in the previous stage is the same as the input data of the method called in the next stage; encapsulating the target log data, identifier, input data and output data into a log package; and storing the log package on the server corresponding to the target log data.

[0010] When a certain step in a business process of an application generates log data, the generated log data, along with the thread identifier, input data, and output data of the called method, are immediately encapsulated into a log package for storage. This ensures that the log data, thread identifier, and related data of the called method are stored completely together, avoiding data errors or loss, and thus improving the accuracy of log retrieval.

[0011] In some modified embodiments of the first aspect of this application, before encapsulating the target log data, identifier, input data, and output data into a log package, the method further includes: storing the input data and output data in the thread set corresponding to the identifier, so that the input data and output data can be encapsulated together when the log data generated in the next stage of the target stage is encapsulated; encapsulating the target log data, identifier, input data, and output data into a log package includes: encapsulating the target log data, identifier, and input data and output data in the thread set into a log package.

[0012] While storing the input and output data of the currently called method, the log package also stores the input and output data of the methods called in previous stages of the same business process. This allows the log package to provide a comprehensive representation of the business processes involved to the greatest extent possible, thereby improving the accuracy of log retrieval.

[0013] In some modified embodiments of the first aspect of this application, before encapsulating the target log data, identifier, input data, and output data into a log package, the method further includes: obtaining the characterization information of the server corresponding to the storage of the target log data; encapsulating the target log data, identifier, input data, and output data into a log package, including: encapsulating the target log data, identifier, input data, output data, and characterization information into a log package.

[0014] While storing thread identifiers and data related to method calls, it also stores server representation information. This allows for the determination of the server where the log data is stored when the log data is found. When tracing based on log data, the server can be quickly located, improving the efficiency of problem solving based on log data.

[0015] In some modified embodiments of the first aspect of this application, obtaining target log data generated by the target stage of the target business of the application includes: in response to calling the log append method, obtaining the log data corresponding to the log append method, and determining the obtained log data as the target log data generated by the target stage of the target business of the application.

[0016] When the log append method is detected to be called, the log data written to the calling method can be obtained quickly. The log data and its thread identifier can then be quickly encapsulated into a log package, thereby improving the storage efficiency of the log package.

[0017] In some modified embodiments of the first aspect of this application, retrieving a target log package from log packages of multiple servers corresponding to the application based on retrieval information includes: if the retrieval information matches words in a preset custom dictionary, then retrieving a target log package containing the retrieval information from log packages of multiple servers corresponding to the application, wherein the words in the custom dictionary consist of method names called by the application; if the retrieval information does not match words in the preset custom dictionary, then performing word segmentation on the retrieval information to obtain multiple words, and retrieving a target log package containing each word from log packages of multiple servers corresponding to the application.

[0018] Before performing a search based on the search information, it is first matched against words in a custom dictionary. If a match is found, the search information is used directly for the search, avoiding the need to split the search information into multiple words. Searching using each word individually not only requires multiple searches, leading to low efficiency, but also retrieves irrelevant content, reducing search accuracy. Therefore, when the search information matches words in the custom dictionary, directly using the search information improves both the efficiency and accuracy of log retrieval.

[0019] In some modified embodiments of the first aspect of this application, before retrieving the extended log packet from log packets of multiple servers based on the identifier, input data, and output data in the target log packet, the method further includes: if the number of target log packets exceeds a preset threshold, grouping target log packets with the same identifier into one category to obtain and output multiple target log packet sets; selecting a specified target log packet set from the multiple target log packet sets based on the selection result of the target log packet sets; retrieving the extended log packet from log packets of multiple servers based on the identifier, input data, and output data in the target log packet includes: retrieving the extended log packet from log packets of multiple servers based on the identifier, input data, and output data in each target log packet in the specified target log packet set.

[0020] When a large number of log packets are initially retrieved, log packets for the same business can be categorized based on the thread identifier in the log packets and displayed to the user. Based on the user's selection, the log packets for a specific business can be output, which can improve the accuracy of log packet retrieval.

[0021] A second aspect of this application provides a log retrieval apparatus, comprising: an acquisition module for acquiring retrieval information; a retrieval module for retrieving a target log package from log packages of multiple servers corresponding to an application based on the retrieval information, wherein each log package of each server includes log data generated by a corresponding step of the application's corresponding business, an identifier of the thread used by the application when the log data was generated, and input and output data of the called method; an extension module for retrieving an extended log package from log packages of multiple servers based on the identifier, input data, and output data in the target log package; and a determination module for determining the target log package and the extended log package as the log package corresponding to the retrieval information.

[0022] A third aspect of this application provides an electronic device, which includes a processor, a memory, and a bus. The processor and the memory communicate with each other via the bus. The processor is used to call program instructions in the memory to execute the method in the first aspect.

[0023] A fourth aspect of this application provides a computer-readable storage medium including a stored program that, when executed, controls the device containing the computer-readable storage medium to perform the method of the first aspect.

[0024] The fifth aspect of this application provides a computer program product, which includes a computer program or instructions that, when executed by a device, implement the method of the first aspect.

[0025] The log retrieval device provided in the second aspect, the electronic device provided in the third aspect, the computer-readable storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect of this application have the same or similar beneficial effects as the log retrieval method provided in the first aspect. Attached Figure Description

[0026] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein:

[0027] Figure 1 This is a schematic diagram of the scenario architecture for the log retrieval method in the embodiments of this application. Figure 1 ;

[0028] Figure 2 This is a schematic diagram of the scenario architecture for the log retrieval method in the embodiments of this application. Figure 2 ;

[0029] Figure 3 This is a flowchart illustrating the log retrieval method in the embodiments of this application. Figure 1 ;

[0030] Figure 4 This is a flowchart illustrating the log retrieval method in the embodiments of this application. Figure 2 ;

[0031] Figure 5 This is a schematic diagram of the log retrieval device in the embodiments of this application. Figure 1 ;

[0032] Figure 6 This is a schematic diagram of the log retrieval device in the embodiments of this application. Figure 2 ;

[0033] Figure 7 This is a schematic diagram of the structure of the electronic device in the embodiments of this application. Detailed Implementation

[0034] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0035] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0036] Log data generated by different stages of the same business in an application may not have common parameters. When log data is stored in a distributed manner, it may not be possible to retrieve all log data generated by each stage of the same business based on retrieval information, thereby reducing the completeness of log retrieval.

[0037] In view of this, embodiments of this application provide a log retrieval method, a log retrieval device, an electronic device, a computer-readable storage medium, and a computer program product. While storing log data, the method also stores the identifier of the thread that generated the log data, as well as the input and output data of the method. This allows for the retrieval of log data based on retrieval information, enabling the retrieval of data from other parts of the same business process based on the thread identifier and the input and output data carried in the log data, thus improving the completeness of log retrieval.

[0038] First, the application scenarios of the log retrieval method provided in the embodiments of this application will be described.

[0039] Figure 1 This is a schematic diagram of the scenario architecture for the log retrieval method in the embodiments of this application. Figure 1 See Figure 1 As shown, the architecture may include: application 11, log appender 12, multiple servers 13 and a preset location 14.

[0040] Application 11 can be any known program capable of providing the relevant services. After application 11 generates log data, log appender 12 obtains the log data, retrieves relevant data according to the log appending rules, and stores the relevant data together with the log data on server 13.

[0041] The log data here can be data representing user behavior in application 11.

[0042] The log appender 12 here can be any known tool that extends the log data output by application 11 without modifying the original program code. In practical applications, log appender 12 can be an appender. The specific type of log appender 12 is not limited here.

[0043] The log appending rules here can be specific rules for expanding the log data output by application 11. For example, the identifiers of the threads called before and after the output log data is generated. The specific content of the log appending rules can be determined according to the actual log content to be appended, and no specific restrictions are made here.

[0044] The relevant data here refers to the data obtained based on log appending rules.

[0045] After obtaining the relevant data, the log appender 12 stores the log data and related data together in the corresponding server 13, according to the server 13 where the log data was originally to be stored.

[0046] Log data generated by different business processes and stages within application 11 may be stored on different servers 13 depending on different storage requirements. For example, log data generated by business process a1 in application 11 may be stored on server A, log data generated by business process a2 in application 11 may be stored on server B, log data generated by business process b1 in application 11 may be stored on server B, and so on. Therefore, when searching for log data based on retrieval information, it is necessary to first store the log data stored in each server 13 in a preset location 14, and then retrieve the log data based on the retrieval information in that preset location 14.

[0047] In practical applications, the preset location 14 can be any one of the multiple servers 13 or a specified server, or it can be other storage media besides the multiple servers 13, such as other servers, the cloud, etc.

[0048] The log retrieval method provided in this application mainly includes two parts. One part is the storage of log data after the application generates it. The other part is the retrieval of the stored log data based on retrieval information.

[0049] For the storage of log data, this can be achieved by dynamically injecting a packet appender module into the log appender (Spring) runtime proxy. This requires no modification to the original code by developers and is compatible with various mainstream logging frameworks. In practical applications, the packet appender module can be any module capable of retrieving relevant data and packaging and storing it along with the log data. For example, the PacketAppenderProxy module.

[0050] Figure 2 This is a schematic diagram of the scenario architecture for the log retrieval method in the embodiments of this application. Figure 2 See Figure 2As shown, a normalized log packet proxy (PacketAppenderProxy) is dynamically injected into the Appender. Appenders include FileAppender, ConsoleAppender, and CustomAppender. Before FileAppender, ConsoleAppender, or CustomAppender appends log data generated by the application, the asynchronous Appender base interface (UnsynchronizedAppenderBase) defines that PacketAppenderProxy calls the proxy method, namely the packet() method, to normalize and packetize the log data before FileAppender, ConsoleAppender, or CustomAppender executes the append() method. That is, the log data, thread identifier, and method input / output data are first packetized, then the packetized data is appended to the log, and finally stored on the server.

[0051] It should be noted that the relevant data, processing methods, software, and hardware devices involved in the embodiments of this application have all been authorized in advance and are legal and compliant.

[0052] The log retrieval method provided in the embodiments of this application will be described in detail below.

[0053] Figure 3 This is a flowchart illustrating the log retrieval method in the embodiments of this application. Figure 1 See Figure 3 As shown, the method may include:

[0054] S31: Obtain search information.

[0055] The retrieval information here refers to the information entered by relevant personnel when they need to retrieve specific log data from an application. In practical applications, the retrieval information can be determined based on the actual log data required. Taking a shopping app as an example, when maintenance personnel need to search for all log data related to a particular order, the retrieval information could be the order number of that order. The specific content and format of the retrieval information are not limited here.

[0056] S32: Retrieve the target log packet from the log packets of multiple servers corresponding to the application based on the retrieval information.

[0057] Each log packet from each server includes log data generated by the corresponding business process of the application, the identifier of the thread used by the application when the log data was generated, and the input and output data of the called method.

[0058] When log data is generated at a specific stage of a business process within the application, the packetization module obtains the identifier of the thread invoked during that stage's runtime, as well as the input and output data of the called method. It then encapsulates the log data, thread identifier, and input and output data into a log packet. The log appender then appends log data to the log packet and stores the processed log packet on the server where the original log data was stored. In this way, multiple servers corresponding to the application store log packets for different stages of different business processes within the application.

[0059] A thread identifier can refer to a thread number or other identifier that can uniquely represent different threads, such as the data characteristics processed by the thread. The specific type of thread identifier is not limited here.

[0060] The input and output data for calling a method can refer to the data input and output at the current stage. Taking the discount stage of a payment process in a shopping app as an example, the input data is the order amount, and the output data is the amount to be paid.

[0061] Log packets from each server are pre-stored in a preset location for later retrieval. This preset location can be a database. The specific content of the preset location is not limited here.

[0062] After obtaining the retrieval information, it can be matched sequentially across all stored log packets. The log packet that matches successfully is the target log packet retrieved. Matching here can mean that the retrieval information and the log packet are identical in content or contain the same fields. Specifically, the retrieval information can be matched against the log data within the log packet, or against the thread identifier, input data of the called method, and output data within the log packet.

[0063] The log packet that is successfully matched is the target log packet. When multiple log packets are matched, all of them are target log packets.

[0064] In some cases, the search results may not match all the required log packets. For example, if you need to retrieve log data from a specific order payment, and the search information is the order number, but the payment process includes three stages: transaction, settlement, and clearing, the order number might only be present in the transaction log data. Searching by order number would only retrieve transaction log data and log data from non-payment processes (such as order inquiries), leading to incomplete or incorrect log data retrieval. In such cases, it's necessary to further search all log packets based on the content of the target log packet.

[0065] S33: Retrieve extended log packets from log packets of multiple servers based on the identifier, input data, and output data in the target log packet.

[0066] When different stages of the same business logic run within an application, they all invoke the same thread. After retrieving the target log packet based on the search information, other log packets with the same thread identifier recorded in the target log packet may be other stages within the same business logic corresponding to the target log packet. Therefore, log packets whose thread identifiers match the thread identifiers in all log packets based on the target log packet can be used as extended log packets.

[0067] Furthermore, within the same business logic within the application, the output and input data are identical. That is, the output data of the previous stage becomes the input data of the next stage. Therefore, by matching the input data in the target log packet with the output data of all log packets, the log packet that successfully matches is the log packet of the previous stage corresponding to the target log packet, i.e., the extended log packet. Similarly, by matching the output data in the target log packet with the input data of all log packets, the log packet that successfully matches is the log packet of the next stage corresponding to the target log packet, i.e., the extended log packet.

[0068] After matching the extended log packet based on the input and output data in the target log packet, you can continue to match log packets based on the input or output data in the extended log packet. The matched log packets are also extended log packets, until no more log packets can be matched.

[0069] The log packets matched based on the thread identifier, input data, and output data in the target log packet may contain some redundant packets that are not actually needed. The required log packets will match the thread identifier and either the input or output data in the target log packet. Therefore, the final extended log packet can be selected from the extended log packets matched based on the thread identifier and those matching based on the input and output data, thus achieving precise expansion based on the target log packet.

[0070] To improve the retrieval efficiency of extended log packets, and thus improve overall log retrieval efficiency, we can first determine the pending log packets based on the input and output data in the target log packet. Then, we can determine the extended log packets from the pending log packets based on the thread identifier in the target log packet. This reduces the number of log packet matches during retrieval, improving matching efficiency. Alternatively, we can first determine the pending log packets based on the thread identifier in the target log packet, and then determine the extended log packets from the pending log packets based on the input and output data in the target log packet. The order of thread identifier matching versus input and output data matching is not specified here. The ultimate goal is to quickly match the extended log packets.

[0071] S34: Identify the target log packet and the extended log packet as the log packets corresponding to the retrieved information.

[0072] The target log packet is the log packet directly retrieved from all log packets based on the search information; generally, it represents the log packet actually needed for the search. The extended log packet, on the other hand, is the log packet indirectly retrieved from all log packets based on the target log packet, supplementing the target log packet. Therefore, identifying the target and extended log packets as the log packets corresponding to the search information enables a complete retrieval of the required logs.

[0073] As described above, the log retrieval method provided in this application stores the thread identifier that generated the log data, as well as the input and output data of the method call, along with the log data itself. This allows for the retrieval of log data based on search information, enabling the search for related log data using the thread identifier carried by the retrieved log data and the input and output data of the called method. This achieves a comprehensive search for the log data required for the search information and improves the completeness of log retrieval.

[0074] Furthermore, as a response to Figure 3 In a refinement and extension of the method shown, this application also provides a log retrieval method.

[0075] Figure 4 This is a flowchart illustrating the log retrieval method in the embodiments of this application. Figure 2 See Figure 4 As shown, this method can include two parts: log storage and log retrieval.

[0076] I. Log Storage

[0077] S41: Obtain the target log data generated by the target process of the application's target business.

[0078] When each business function within an application runs, it outputs corresponding log data based on the configurations configured by the developers. The log data generated by a specific business function within an application based on user actions constitutes the target log data generated by the target business function's target stage.

[0079] On the application's server side, a log appender is configured to append logs without modifying the original code. However, to achieve more complete log retrieval, it's necessary to store thread identifiers, method inputs, and outputs along with the log data. This requires further improvement to the log appender: dynamically injecting a packetization module. This module performs standardized packetization of log data generated by the application, before the log appender appends it. Specifically, it packets the log data, thread identifiers, and method inputs and outputs before appending and storing the logs.

[0080] Specifically, step S41 above may include: in response to calling the log append method, obtaining the log data corresponding to the log append method, and determining the obtained log data as the target log data generated by the target process of the application's target business.

[0081] After the log appender receives the log data generated by the application, it begins calling the log append method to append the data. The packetization module, injected into the log appender, monitors the calls to the log append method. Once a call to the log append method is detected, it indicates that the log appender has received the log data generated by the application. At this point, the packetization module intercepts the log data. The intercepted log data is the target log data to be normalized and packetized. The packetization module adds a thread identifier and the input and output data of the called method to the log data, resulting in a log packet. This log packet is then sent back to the log appender, which calls the log append method to append log data to the packet, ultimately storing the appended log packet on the appropriate server.

[0082] As for the monitoring of log appending method calls and the interception of log data by the packet module, known monitoring and interception methods are used, which will not be elaborated here.

[0083] S42: Obtain the identifier of the thread used by the application when the target log data was generated, as well as the input and output data of the called method.

[0084] S43: Encapsulate the target log data, identifier, input data, and output data into a log package.

[0085] In this application, different stages of the same business logic run using the same thread. The output data of the method called in the previous stage is the same as the input data of the method called in the next stage.

[0086] Generally, a specific business function within an application calls a thread during a particular execution. This same thread is used in all stages of that business process. For example, in a shopping app, when a user makes a payment, the transaction, settlement, and clearing stages all use the same thread. Therefore, the thread called when the application generates log data is the thread of the stage to which the log data belongs, and also the thread of the business function. By encapsulating and storing the thread's identifier along with the log data, logs belonging to the same business function can be found through the thread identifier during log retrieval.

[0087] After the packet module obtains the target log data, it searches for the corresponding thread in the application using the target log data. The identifier of the found thread is the identifier of the thread used by the application when the target log data to be obtained was generated.

[0088] In addition to obtaining the thread identifier, it is also necessary to obtain the input and output data of the methods called by the application when the target log data is generated.

[0089] Specifically, after the packet module obtains the target log data, it searches for the corresponding calling method in the application using the target log data. The input and output data corresponding to the found calling method are the input and output data of the method called by the application when the target log data was generated.

[0090] Taking a shopping app as an example, when a user makes a payment, the payment process involves a discount. This discount process generates log data. After the packet module obtains this log data, it locates the thread responsible for the discount process based on the log data, and then determines through this thread that the discount method was called (which can be understood as using a coupon). The input data of the discount method is the order amount, and the output data is the actual amount to be paid. In this way, the input and output data of the methods called when the application generates log data are obtained. By packetizing and storing the input and output data of the called methods along with the log data, log retrieval can be performed to find the logs of the preceding and following stages belonging to the same business process using the input and output data.

[0091] At this point, the input and output data serve as the output data of the previous stage and the input data of the next stage. In log retrieval, they can only be associated with adjacent stages. If a business involves many stages, a single association may not be able to retrieve logs from all stages involved. Therefore, the input and output data of method calls from different stages of the same business can be stored together. This way, when the log data of each stage is packaged, the input and output data of all previous stages can be packaged together, and the logs of all stages can be retrieved with a single association during retrieval.

[0092] Specifically, step S42 may further include: storing the input data and output data in the thread set corresponding to the identifier, so that the input data and output data can be encapsulated together when the log data generated by the next stage of the target stage is encapsulated.

[0093] Accordingly, step S43 above may include: encapsulating the target log data, identifier, input data and output data in the thread set into a log package.

[0094] For example, suppose an application's business logic includes stages a, b, c, and d in sequence. Stage a generates log data 'a', with data 1 as input and data 2 as output when calling its method. Stage b generates log data 'b', with data 2 as input and data 3 as output when calling its method. Stage c generates log data 'c', with data 3 as input and data 4 as output when calling its method. Stage d generates log data 'd', with data 4 as input and data 5 as output when calling its method.

[0095] First, the packetization module obtains log data 'a'. Based on log data 'a', the packetization module determines thread 'x', as well as input data 1 and output data 2. While packetizing and storing log data 'a', thread 'x', and data 1 and data 2, the packetization module creates a set for thread 'x' and stores data 1 and data 2 into the set for thread 'x'.

[0096] Then, the packetization module obtains log data b. Based on log data b, the packetization module determines thread x, as well as data 2 and data 3. While packetizing and storing data based on log data b, thread x, and data 1, 2, and 3, the packetization module also stores data 2 and data 3 into the set of thread x.

[0097] Next, the packetization module obtains log data c. Based on log data c, the packetization module determines thread x, as well as data 3 and data 4. While packetizing and storing log data c, thread x, and data 1, data 2, data 3, and data 4, the packetization module also stores data 3 and data 4 into the set of thread x.

[0098] Finally, the packetization module obtains the log data d. Based on the log data d, the packetization module determines thread x, as well as data 4 and data 5. While packetizing and storing the log data d, thread x, and data 1, data 2, data 3, data 4, and data 5, the packetization module also stores data 4 and data 5 into the set of thread x.

[0099] At this time, the server stores log package a (log data a, thread x, data 1 and data 2), log package b (log data b, thread x, data 1, data 2 and data 3), log package c (log data c, thread x, data 1, data 2, data 3 and data 4), and log package d (log data d, thread x, data 1, data 2, data 3, data 4 and data 5).

[0100] When the search information includes data 4, although a direct search can only retrieve log packages c and d, it is possible to indirectly retrieve log packages a, b, and d through thread x, data 1, data 2, data 3, or data 5 in log packages c and d, thereby achieving complete log retrieval for all aspects of the same business.

[0101] While encapsulating the input and output data of the called method along with the log data, the key server parameters of the current code execution point of the application can also be encapsulated together. This allows for direct identification of the corresponding server when tracing through the logs, thereby improving problem-solving efficiency.

[0102] Specifically, step S42 may further include: obtaining the representation information of the server corresponding to the storage of the target log data.

[0103] Accordingly, step S43 above may include: encapsulating the target log data, identifier, input data, output data, and representation information into a log package.

[0104] The representation information here can be the server's Internet Protocol (IP) address, actual name, distributed node name, database name, table name, transaction name, etc., as long as it can uniquely represent the server. The specific content is not limited here.

[0105] Since different code snippets in an application execute on different servers, the code-server mapping can be pre-configured and stored in a specific location. After obtaining the target log data, the packetization module determines the current code execution point based on the target log data. Then, it uses the stored mapping to determine the server corresponding to the current code execution point and retrieves the server's representation information from its configuration. Finally, the packetization module sequentially packages the target log data, its corresponding thread identifier, the input and output data of the called method, and the server's representation information.

[0106] Once the packetization module obtains the thread identifier, thread set, and server representation information corresponding to the target log data, it packets the target log data, its corresponding thread identifier, all data in the thread set, and the server representation information to obtain a log packet.

[0107] In practical applications, logs can be packaged into a standard JSON format. The specific format of the log package can be determined according to actual needs; no specific limitations are made here.

[0108] S44: Store the log packet on the server corresponding to the target log data.

[0109] After the packet module systematically encapsulates the log packets, the log packets are appended to the logs by the log appender. The appended log packets are then stored by the log appender on the server where the target log data was originally configured to be stored.

[0110] This completes the standardized packaging, log appending, and storage of the target log data, enabling subsequent retrieval of complete logs for a specific business function based on the retrieved information.

[0111] II. Log Retrieval

[0112] S45: Obtain search information.

[0113] When relevant personnel need to retrieve logs, they enter the retrieval information so that the required log data can be retrieved.

[0114] The specific search information needs to be set by relevant personnel based on actual needs. The specific content of the search information is not limited here.

[0115] Log packets from multiple servers are pre-stored in a predetermined location. After obtaining the retrieval information, a search can be performed within all log packets in that predetermined location based on the retrieval information. Specifically, the retrieval information can be matched against the log data in the log packets; if a match is found, the log packet corresponding to the matched log data is the target log packet.

[0116] When performing searches within log packets based on search information, it is sometimes necessary to segment the search information into words first. However, if the segmentation is too fine, it can lead to a large search volume and numerous retrieved results, some of which may not be relevant to the actual search. Therefore, the completeness of the search information should be assessed before segmentation.

[0117] S46: Determine whether the search information matches the words in the preset custom dictionary. If yes, proceed to S47; otherwise, proceed to S48.

[0118] The words in the custom dictionary consist of the names of methods called by the application. During the operation of each business function in the application, corresponding methods are called, and the names of these methods are extracted and stored in the custom dictionary. In this way, the words stored in the custom dictionary are all words with complete semantic meaning from the application's runtime. Using these words to search for log data avoids both overly narrow search terms, which would increase the number of searches and the amount of retrieved content, and overly broad search terms, which would lead to omissions in the search results.

[0119] When determining consistency, if the search information is identical to a word in a custom dictionary, then the search information is considered consistent with the words in the preset custom dictionary. For example, the search information is "NetEase Cloud Music e-coupon," and the custom dictionary includes the word "NetEase Cloud Music e-coupon." If part of the search information is identical to a word in the custom dictionary, then the search information is considered inconsistent with the words in the preset custom dictionary. For example, the search information is "NetEase Cloud Music e-coupon," and the custom dictionary includes the word "NetEase Cloud Music." In this case, the search information can be split into "NetEase Cloud Music" and "e-coupon." If part of the search information is identical to a word in the custom dictionary, then the search information is considered inconsistent with the words in the preset custom dictionary. For example, the search information is "NetEase Cloud Music," and the custom dictionary includes the word "NetEase Cloud Music e-coupon." In this case, the search information is split according to conventional semantics, resulting in "NetEase," "Cloud," and "Music."

[0120] As can be seen, the search results here match the words in the preset custom dictionary with exact match.

[0121] S47: Retrieve the target log packet containing the retrieval information from the log packets of multiple servers corresponding to the application.

[0122] If the search result matches a word in the custom dictionary, it indicates that the search information was accurate, and the word was neither too long or too specific, nor too short or too general. The entire search result can then be used to match against log data in all stored log packets. The log packet that matches successfully is the target log packet.

[0123] S48: Perform word segmentation on the retrieved information to obtain multiple words, and retrieve the target log package containing each word from the log packages of multiple servers corresponding to the application.

[0124] If the search results are different from all the words in the custom dictionary, it indicates that there is a certain difference between the search results and the log data. In this case, it is necessary to first perform word segmentation on the search results according to semantics to obtain multiple words, and then match each word with the log data in all log packets. The log packet corresponding to the successfully matched log data is the target log packet.

[0125] After retrieving the target log package, since the retrieval information may not be able to directly retrieve all the required log packages, we can continue to search through all the stored log packages again using the thread identifier in the target log package and the input and output data of the called method to obtain the extended log package.

[0126] In some cases, the search results may retrieve log packages for multiple business processes, not just one. However, only the log package for a specific business process is needed. For example, if the search result is an order number, it might retrieve log packages for payment, query, and refund processes simultaneously. In reality, the user only needs the payment log package based on the order number. In this situation, relevant information can be displayed to allow for filtering by designated personnel.

[0127] S49: Determine if the number of target log packets exceeds the preset threshold. If yes, execute S410-S412; otherwise, execute S413.

[0128] S410: Group target log packets with the same identifier into one category, obtain and output multiple sets of target log packets.

[0129] S411: Select the specified target log package set from multiple target log package sets based on the selection result of the target log package set.

[0130] S412: Retrieve extended log packets from log packets of multiple servers based on the identifier, input data, and output data in each target log packet in the specified target log packet set.

[0131] S413: Retrieve extended log packets from log packets of multiple servers based on the identifier, input data, and output data in the target log packet.

[0132] The preset threshold can be determined based on the average or maximum number of steps included in different services; no specific limit is set here.

[0133] If the number of target log packets exceeds a preset threshold after obtaining the target log packets, and the number of corresponding links in a business is limited, thus the number of corresponding log packets is limited, it indicates that the target log packets involve multiple businesses, and it is possible that log packets beyond what is actually needed have been obtained.

[0134] At this point, based on the thread identifier in the target log package, and considering that different services use different threads, the target log packages corresponding to the same service with the same thread identifier are grouped into one category, resulting in multiple sets of target log packages.

[0135] Next, multiple target log package sets can be directly output to relevant personnel so that they can select the target log package corresponding to their actual business needs. Alternatively, multiple target log package sets can be used to obtain the corresponding business, and then the names of each business can be output to relevant personnel so that they can select from the displayed business names. Finally, the target log package set can be output based on the selections of the relevant personnel.

[0136] If the number of target log packets does not exceed the preset threshold, it means that the target log packets do not involve a lot of business and are likely all log packets that are actually needed.

[0137] At this point, the extended log packet with the same thread identifier, input data, and output data can be directly retrieved from all log packets based on the thread identifier, input data, and output data in the target log packet.

[0138] S414: Identify the target log packet and extended log packet as the log packets corresponding to the retrieved information.

[0139] The target log package is the log package directly retrieved from the log data of all stored log packages based on the retrieval information, thus meeting the retrieval requirements. The extended log package is the log package retrieved from the thread identifier, input data, and output data of all stored log packages based on the thread identifier, input data, and output data of the called method within the target log package. It serves as a supplementary retrieval of information, compensating for any deficiencies in the target log package based on the retrieval requirements. Both the target and extended log packages provide a more complete retrieval response to the retrieval information, thus serving as the corresponding log packages for the retrieved information.

[0140] In practical applications, after retrieving the target log packet based on the search information, and retrieving the extended log packet based on the target log packet, the target log packet and the extended log packet can be displayed separately in a preset template so that relevant personnel can clearly see which log packets are obtained directly from the search information and which log packets are obtained from the extended search.

[0141] Furthermore, the template can display log data, thread parameters, input and output data of called methods, server parameters, etc. in the target log package, as well as log data, thread parameters, input and output data of called methods, server parameters, etc. in the extended log package, to comprehensively and clearly present the search results.

[0142] This concludes the description of the log retrieval method provided in the embodiments of this application.

[0143] Based on the same inventive concept, as an implementation of the above method, this application also provides a log retrieval device.

[0144] Figure 5 This is a schematic diagram of the log retrieval device in the embodiments of this application. Figure 1 See Figure 5 As shown, the device may include: an acquisition module 51, a retrieval module 52, an expansion module 53, and a determination module 54.

[0145] The acquisition module 51 is used to acquire search information.

[0146] The retrieval module 52 is used to retrieve the target log package from the log packages of multiple servers corresponding to the application based on the retrieval information. Each log package of each server includes log data generated by the corresponding link of the application's corresponding business, the identifier of the thread used by the application when the log data was generated, and the input and output data of the called method.

[0147] Extension module 53 is used to retrieve extended log packets from log packets of multiple servers based on the identifier, input data, and output data in the target log packet.

[0148] The determination module 54 is used to determine the target log packet and the extended log packet as the log packets corresponding to the retrieved information.

[0149] Furthermore, as a response to Figure 5 In a refinement and extension of the illustrated device, this application also provides a log retrieval device.

[0150] Figure 6 This is a schematic diagram of the log retrieval device in the embodiments of this application. Figure 2 See Figure 6 As shown, the device may include: a log acquisition module 61, a data acquisition module 62, a packet acquisition module 63, a storage module 64, an information acquisition module 65, a retrieval module 66, an interaction module 67, an extension module 68, and a determination module 69.

[0151] The log acquisition module 61 is used to acquire target log data generated by the target process of the application's target business.

[0152] The log acquisition module 61 is specifically used to respond to the call to the log append method, acquire the log data corresponding to the log append method, and determine the acquired log data as the target log data generated by the target process of the application's target business.

[0153] The data acquisition module 62 is used to acquire the identifier of the thread used by the application when the target log data is generated, as well as the input and output data of the called method. In this case, different stages of the same business in the application use the same thread, and the output data of the method called in the previous stage is the same as the input data of the method called in the next stage.

[0154] The data acquisition module 62 is also used to store the input data and output data in the thread set corresponding to the identifier, so that the input data and output data can be encapsulated together when the log data generated by the next stage of the target stage is encapsulated.

[0155] The data acquisition module 62 is also used to acquire the characterization information of the server corresponding to the storage of the target log data.

[0156] Packetization module 63 is used to encapsulate target log data, identifiers, input data, and output data into log packets.

[0157] The encapsulation module 63 is also used to encapsulate target log data, identifiers, input data and output data in the thread set into log packets.

[0158] The encapsulation module 63 is also used to encapsulate target log data, identifiers, input data, output data, and characterization information into log packets.

[0159] Storage module 64 is used to store log packets on the server corresponding to the target log data.

[0160] The information acquisition module 65 is used to acquire search information.

[0161] The retrieval module 66 is used to retrieve the target log package from the log packages of multiple servers corresponding to the application based on the retrieval information. Each log package of each server includes log data generated by the corresponding link of the application's corresponding business, the identifier of the thread used by the application when the log data was generated, and the input and output data of the called method.

[0162] The retrieval module 66 is specifically used to retrieve the target log package containing the retrieval information from the log packages of multiple servers corresponding to the application if the retrieval information matches the words in the preset custom dictionary. The words in the custom dictionary consist of the names of the methods called by the application. If the retrieval information does not match the words in the preset custom dictionary, the retrieval information is segmented to obtain multiple words, and the target log package containing each word is retrieved from the log packages of multiple servers corresponding to the application.

[0163] The interaction module 67 is used to group target log packets with the same identifier into one category if the number of target log packets exceeds a preset threshold, obtain and output multiple target log packet sets; and select a specified target log packet set from multiple target log packet sets based on the selection result of the target log packet set.

[0164] Extension module 68 is used to retrieve extended log packets from log packets of multiple servers based on the identifier, input data and output data in the target log packet.

[0165] Extension module 68 is also used to retrieve extended log packets from log packets of multiple servers based on the identifier, input data and output data in each target log packet in a specified set of target log packets.

[0166] The determination module 69 is used to identify the target log packet and the extended log packet as the log packets corresponding to the retrieved information.

[0167] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0168] Based on the same inventive concept, embodiments of this application also provide an electronic device.

[0169] Figure 7 This is a schematic diagram of the electronic device in an embodiment of this application. See also... Figure 7 As shown, the electronic device may include: a processor 71, a memory 72, and a bus 73. The processor 71 and the memory 72 communicate with each other through the bus 73. The processor 71 is used to call program instructions in the memory 72 to execute the methods in one or more of the above embodiments.

[0170] It should be noted that the descriptions of the above electronic device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the electronic device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0171] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium, which may include: a stored program that, when the program is running, controls the device where the storage medium is located to execute the methods described in one or more of the above embodiments.

[0172] It should be noted that the description of the above computer-readable storage medium embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the computer-readable storage medium embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0173] Based on the same inventive concept, this application also provides a computer program product, which includes a computer program or instructions that, when executed by the device, implement the methods in one or more of the above embodiments.

[0174] It should be noted that the descriptions of the above computer program product embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the computer program product embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0175] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A log retrieval method, characterized in that, The method includes: Obtain the target log data generated by the target process of the application's target business; Obtain the identifier of the thread used by the application when the target log data is generated, as well as the input and output data of the called method. In this case, different stages of the same business in the application use the same thread, and the output data of the method called in the previous stage is the same as the input data of the method called in the next stage. The target log data, the identifier, the input data, and the output data are encapsulated into a log package; The log package is stored on the server corresponding to the target log data; Obtain search information; The target log package is retrieved from the log packages of multiple servers corresponding to the application based on the retrieval information. Each log package of each server includes log data generated by the corresponding link of the application's corresponding business, the identifier of the thread used by the application when the log data is generated, and the input and output data of the called method. The extended log packet is retrieved from the log packets of the plurality of servers based on the identifier, input data and output data in the target log packet; The target log package and the extended log package are identified as the log packages corresponding to the retrieved information.

2. The method according to claim 1, characterized in that, Before encapsulating the target log data, the identifier, the input data, and the output data into a log packet, the method further includes: The input and output data are stored in the thread set corresponding to the identifier, so that the input and output data can be encapsulated together when the log data generated by the next stage of the target stage is encapsulated. The step of encapsulating the target log data, the identifier, the input data, and the output data into a log package includes: The target log data, the identifier, and the input and output data from the thread set are encapsulated into a log package.

3. The method according to claim 1, characterized in that, Before encapsulating the target log data, the identifier, the input data, and the output data into a log packet, the method further includes: Obtain the characterization information of the server corresponding to the target log data; The step of encapsulating the target log data, the identifier, the input data, and the output data into a log package includes: The target log data, the identifier, the input data and output data, and the representation information are encapsulated into a log package.

4. The method according to claim 1, characterized in that, The acquisition of target log data generated by the target process of the application's target business includes: In response to the call to the log append method, the log data corresponding to the log append method is obtained, and the obtained log data is determined as the target log data generated by the target process of the target business of the application.

5. The method according to any one of claims 1 to 4, characterized in that, The step of retrieving the target log packet from log packets of multiple servers corresponding to the application based on the retrieval information includes: If the search information matches the words in the preset custom dictionary, then the target log package containing the search information is retrieved from the log packages of multiple servers corresponding to the application. The words in the custom dictionary are composed of the method names called by the application. If the search information does not match the words in the preset custom dictionary, the search information is segmented to obtain multiple segments, and the target log package containing each segment is retrieved from the log packages of multiple servers corresponding to the application.

6. The method according to any one of claims 1 to 4, characterized in that, Before retrieving the extended log packet from the log packets of the plurality of servers based on the identifier, input data, and output data in the target log packet, the method further includes: If the number of target log packets exceeds a preset threshold, target log packets with the same identifier will be grouped into one category, and multiple sets of target log packets will be obtained and output. Based on the selection result of the target log package set, select the specified target log package set from multiple target log package sets; The step of retrieving the extended log packet from the log packets of the plurality of servers based on the identifier, input data, and output data in the target log packet includes: Extended log packets are retrieved from the log packets of the plurality of servers based on the identifier, input data, and output data of each target log packet in the specified target log packet set.

7. A log retrieval device, characterized in that, The device includes: The log acquisition module is used to acquire target log data generated by the target process of the application's target business. The data acquisition module is used to acquire the identifier of the thread used by the application when the target log data is generated, as well as the input and output data of the called methods. Different stages of the same business in the application use the same thread, and the output data of the method called in the previous stage is the same as the input data of the method called in the next stage. The encapsulation module is used to encapsulate target log data, identifiers, input data, and output data into log packets; The storage module is used to store log packets on the server corresponding to the target log data; The acquisition module is used to retrieve search information; The retrieval module is used to retrieve the target log package from the log packages of multiple servers corresponding to the application based on the retrieval information. Each log package of each server includes log data generated by the corresponding link of the application's corresponding business, the identifier of the thread used by the application when the log data is generated, and the input and output data of the called method. An extension module is used to retrieve an extended log packet from the log packets of the plurality of servers based on the identifier, input data, and output data in the target log packet; The determination module is used to determine the target log package and the extended log package as the log package corresponding to the retrieval information.

8. An electronic device, characterized in that, The electronic device includes: a processor, a memory, and a bus; wherein the processor and the memory communicate with each other via the bus; the processor is used to call program instructions in the memory to execute the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium includes: a stored program; wherein, when the program is executed, it controls the device where the storage medium is located to perform the method as described in any one of claims 1 to 6.

Images