Log Processing Method, Device, Terminal Device and Storage Medium

Abstract

This application is applicable to the field of computer technology and provides a log processing method, including: obtaining a log file query request, where the log file query request carries a first identifier and a query keyword; obtaining a preset association table corresponding to the first identifier according to the first identifier, where the preset association table contains multiple second identifiers; reading the second identifier corresponding to the query keyword in the preset association table; and retrieving a first log file corresponding to the second identifier in the storage layer according to the second identifier. By quickly obtaining the preset association table through the first identifier, finding the corresponding second identifier according to the query keyword, and loading the corresponding log file according to the second identifier, it is not necessary to load all log files according to the first identifier, reducing the loading time of log files, and quickly locating log files according to the query keyword, improving the query efficiency of log files.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and particularly relates to a log processing method, apparatus, terminal device, and storage medium. Background Art

[0002] When a problem occurs in the system, it is generally necessary to rely on the logs during system operation to verify the system problem. However, currently, when querying logs based on the log service of the ARK architecture (a lightweight distributed plug-in framework written in C++), the target logs cannot be quickly found, resulting in the inability to quickly and effectively find the system problem, making the system problem unable to be solved in a timely manner. Summary of the Invention

[0003] Embodiments of this application provide a log processing method, apparatus, terminal device, and storage medium, which can solve the problem of not being able to quickly find target logs.

[0004] In a first aspect, embodiments of this application provide a log processing method, including:

[0005] Obtain a log file query request, where the log file query request carries a first identifier and a query keyword;

[0006] According to the first identifier, obtain a preset association table corresponding to the first identifier, where the preset association table contains multiple second identifiers;

[0007] Read the second identifier corresponding to the query keyword in the preset association table;

[0008] According to the second identifier, retrieve a first log file corresponding to the second identifier in the storage layer.

[0009] Embodiments of this application quickly obtain a preset association table through the first identifier, find the corresponding second identifier according to the query keyword, and load the corresponding log file according to the second identifier. Thus, only the preset association table needs to be loaded to find the second identifier, without the need to load all log files according to the first identifier, nor to find the corresponding first log file from all log files, reducing the loading time of log files, and achieving quick positioning of log files according to the query keyword, improving the query efficiency of log files.

[0010] In a second aspect, embodiments of this application provide a log processing apparatus, including:

[0011] A first obtaining module, configured to obtain a log file query request, where the log file query request carries a first identifier and a query keyword;

[0012] A second obtaining module, configured to obtain a preset association table corresponding to the first identifier according to the first identifier, where the preset association table contains multiple second identifiers;

[0013] A reading module, configured to read the second identifier corresponding to the query keyword in the preset association table;

[0014] An extraction module, configured to extract, according to the second identifier, a first log file corresponding to the second identifier in the storage layer.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the log processing method according to any one of the above first aspects is implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the log processing method according to any one of the above first aspects is implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, and when the computer program product runs on a terminal device, the terminal device is enabled to execute the log processing method according to any one of the above first aspects.

[0018] It can be understood that the beneficial effects of the above second aspect to the fifth aspect can refer to the relevant descriptions in the above first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 is a schematic flowchart of a log processing method provided by an embodiment of the present application;

[0021] Figure 2 is a schematic flowchart of a log processing method provided by another embodiment of the present application;

[0022] Figure 3 is a schematic flowchart of a log processing method provided by another embodiment of the present application;

[0023] Figure 4 is a schematic flowchart of a log processing method provided by another embodiment of the present application;

[0024] Figure 5It is a schematic flowchart of a log processing method provided by another embodiment of the present application;

[0025] Figure 6 It is a schematic structural diagram of a log processing device provided by an embodiment of the present application;

[0026] Figure 7 It is a schematic structural diagram of a terminal device provided by an embodiment of the present application. Detailed implementation manners

[0027] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures, technologies, etc. are presented to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0028] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0029] It should also be understood that the term "and / or" used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0030] As used in the specification of the present application and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if detecting [the described condition or event]" can be interpreted as meaning "once determined", "in response to determining", "once detecting [the described condition or event]", or "in response to detecting [the described condition or event]" depending on the context.

[0031] In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0032] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but rather mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0033] As introduced in the related background art, when a problem occurs in the system, it is generally necessary to rely on the logs during system operation to verify the system problem. However, currently, when querying logs using the log service based on the ARK architecture (a lightweight distributed plug-in framework written in C++), the target logs cannot be quickly found, and the log files of some log levels (such as info) cannot be printed, resulting in the inability to quickly and effectively find out the system problem and thus the system problem cannot be solved in time.

[0034] Therefore, the embodiments of this application provide a log processing method, which realizes the process of quickly obtaining a preset association table through a first identifier, finding a corresponding second identifier from the preset association table according to a query keyword, and loading a corresponding log file according to the second identifier, reduces the loading time of the log file, improves the query efficiency of the log file, and solves the problem of being unable to quickly and effectively find the target log.

[0035] Among them, the above ARK framework is a lightweight distributed plug-in framework written in C++ language, which can create application services more easily and quickly. The above log processing method is based on multiple microservices under the ARK framework, and this method can track the call relationships of each microservice and query the log files generated by each microservice.

[0036] The log processing method provided by the embodiments of this application can be applied to terminal devices such as mobile phones, tablet computers, in-vehicle devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), desktop computers, servers, etc. The embodiments of this application do not impose any restrictions on the specific types of terminal devices.

[0037] Figure 1 The schematic flowchart of the log processing method provided by this application is shown. By way of example and not limitation, this method can be applied to the above terminal devices. As Figure 1The method shown above includes steps S101 to S104, and the specific implementation principles of each step are as follows.

[0038] S101, obtain a log file query request, where the log file query request carries a first identifier and a query keyword;

[0039] In the above S101, the first identifier is a global trace identifier, that is, a trace ID, which can be unique. Specifically, the first identifier can be a business flow number, or other unique trace IDs generated by random algorithms, hash algorithms, etc. The query keyword includes but is not limited to a time keyword, a microservice name keyword, and a log level keyword, which are used to quickly locate the log file that the user needs to query and improve the query efficiency. In this embodiment, after the user enters the first identifier (such as a business flow number) and the query keyword in the front end and clicks to confirm, the front end sends a log query request carrying the first identifier and the query keyword to the back end, and the back end obtains the log query request.

[0040] S102, according to the first identifier, obtain a preset association table corresponding to the first identifier, where the preset association table contains multiple second identifiers;

[0041] In the above S102, the preset association table is a data table stored in a database, and the database can be redis. The second identifier is a call identifier generated respectively when each microservice is called to execute a business processing request, and it is used to identify the log file generated by calling the corresponding microservice. For example, the back end calls the corresponding microservice according to the business processing request to execute the business, then generates the second identifier corresponding to the microservice, writes the second identifier into the log file generated by calling the microservice to execute this business request, and associates the second identifier with the first identifier and records it in the preset association table. It should be understood that under the same business processing request, a second identifier corresponding to each called microservice is generated.

[0042] By obtaining the preset association table, it is convenient to query the second identifier and load the corresponding log file according to the second identifier, rather than loading all the log files according to the first identifier, which can reduce the file loading time.

[0043] S103, read the second identifier corresponding to the query keyword in the preset association table;

[0044] In the above S103, there is a corresponding relationship between the above query keyword and the second identifier. Optionally, the corresponding relationship can be recorded in a preset association table. For example, the preset association table also records the microservices corresponding to each second identifier, the time point when added to the preset association table, and the log level of the log file corresponding to the second identifier. Optionally, the corresponding relationship can also be recorded in other data tables. The relationship between the second identifier and the query keyword is queried from other data tables, and then one or more second identifiers corresponding to the query keyword are obtained from the preset association table according to the above relationship.

[0045] S104, according to the second identifier, retrieve the first log file corresponding to the second identifier in the storage layer.

[0046] In the above S104, the above storage layer is a database for storing log files. This database can be MySQL, Oracle, Mongdb, etc., preferably Mongdb. Log data is unstructured data, and there is no need to consider changes in the data structure when storing it in Mongdb, which is more convenient for storage. Since the second identifier has been written into the log file when the log file is generated, the first log file containing the second identifier in the storage layer is found according to the second identifier, and the first log file is loaded, instead of loading all log files and then finding the corresponding first log file from all log files, which reduces the loading time of the log file and realizes fast positioning of the log file according to the query keyword, improving the query efficiency of the log file.

[0047] In Figure 1 Based on the above-described embodiments, the present application provides an embodiment of another log processing method. The above S102 specifically includes S1021. It should be noted that the steps identical to Figure 1 the embodiments are not described herein again. Please refer to the foregoing.

[0048] S1021, according to the first identifier, read the preset mapping table in the database, and obtain the preset association table corresponding to the first identifier in the preset mapping table. The preset mapping table contains the mapping relationship between multiple third identifiers and multiple preset association tables, and the first identifier is one of the multiple third identifiers.

[0049] In the above S1021, when microservices based on the ARK architecture process the same business processing request, a global tracking identifier (the first identifier) for this request is generated, and the first identifier and a preset association table ID recording the inter-microservice call relationship of this request are written into a preset mapping table. The writing method can be to write the first identifier as the Key and the preset association table ID as the value into the preset mapping table. Specifically, the snowflake model can be used to implement the inter-table association between the preset association table, the preset mapping table, and other data tables, or the star model can be used to implement the inter-table association between the preset association table and the preset mapping table.

[0050] Through the preset mapping table, it is possible to quickly find the preset association table corresponding to the first identifier from multiple preset association tables corresponding to multiple business processing requests, and then directly read the corresponding preset association table, without having to traverse multiple preset association tables in the database to find the corresponding one, thus improving the query efficiency.

[0051] Optionally, the above query keywords include one or more of a time keyword, a microservice name keyword, and a log level keyword.

[0052] When the query keyword is a time keyword, one or more of the second identifiers corresponding to the time point or time period of the time keyword are obtained, where the preset association table also contains the time point when each of the second identifiers was added to the preset association table. The above time keyword is the keyword for querying the log file for the corresponding time point or time period. For example, 17:30 - 17:32, or 15:40. When it is a time keyword, all second identifiers within the corresponding time period or all second identifiers at the corresponding time point are obtained, so that all log files corresponding to the time point or time period can be obtained according to the second identifiers.

[0053] When the query keyword is a microservice name keyword, one or more of the second identifiers corresponding to the microservice name of the microservice name keyword are obtained, where the preset association table also contains the microservice name corresponding to each of the second identifiers. The above microservice name keyword is the keyword for querying the log file according to the microservice name. When it is a microservice keyword, when the microservice name verification is correct, all second identifiers corresponding to the microservice name are obtained, so that all log files generated when the microservice executes the corresponding business processing request can be obtained according to the corresponding second identifiers. It can be understood that under the same business processing request, each time a microservice is called, a second identifier is generated according to the UUID to avoid log data confusion caused by multiple calls to the same microservice.

[0054] When the query keyword is a log level keyword, obtain one or more of the second identifiers corresponding to the log level of the log level keyword, where the preset association table further includes the log level of the log file corresponding to each of the second identifiers. The above log level keyword is a keyword for querying the log file of the corresponding log level, such as error, warn. When it is a log level keyword, obtain the second identifier corresponding to the log level, so as to obtain all the log files of the log level pair according to the corresponding second identifier. Querying the log file through the log level keyword can verify system problems more quickly.

[0055] It should be understood that the query keyword can also be any combination of a time keyword, a microservice name keyword, and a log level keyword to find the log files that meet the time, microservice, and / or log level at the same time.

[0056] Optionally, the above log levels include but are not limited to the info, warn, and error levels.

[0057] In this embodiment, the above info level is the log level indicating the normal state of the system, the warn level is the log level indicating that the system has minor unreasonableness but does not affect operation and use, and the error level is the log level indicating that the system has errors and exceptions and cannot complete the target operation normally. The log levels from low to high are info < warn < error. In a general log printing system, after defining the high-level log, the low-level log cannot be printed. For example, if the system defines the warn level, the log data of the info level cannot be printed, and the warn level and error level can be printed.

[0058] In Figure 1 Based on the shown embodiment, Figure 2 It is a schematic flowchart of another log processing method provided by an embodiment of the present application. After the above S104, S201 and S202 are further included. It should be noted that the steps the same as those in Figure 1 the embodiment are not described herein again, please refer to the foregoing.

[0059] S201, obtain the associated link corresponding to the second identifier in the preset association table. The preset association table includes multiple associated links, and each associated link includes an association relationship between one of the first identifiers and one or more fourth identifiers. The second identifier is one of the fourth identifiers;

[0060] In the above S201, since a business processing request may require multiple microservices to cooperate to complete, there is a call relationship between some microservices, that is, there is a topological relationship between some second identifiers. The above association link is a link for recording the above topological relationship. For example, after the backend receives a business processing request, it calls microservices A and B to cooperate in processing. When microservice A processes the business, it calls microservices C and D. Then there are three association links: backend - microservice A - microservice C, backend - microservice A - microservice D, and backend - microservice B. The three association links have a common node which is the backend. Then this node of the backend forms a complete association link with the three association links. Among them, the backend corresponds to the first identifier, and microservices A, B, C, and D respectively correspond to a second identifier.

[0061] S202. According to the association link, retrieve the second logs corresponding to all the fourth identifiers in the association link.

[0062] In the above S202, in order to facilitate verifying system problems, retrieve the context of the second identifier corresponding to the query keyword input by the user, that is, retrieve all the log files of the association link where the second identifier is located.

[0063] In Figure 1 the embodiment shown, Figure 3 is a schematic flowchart of another log processing method provided by an embodiment of the present application. After the above S101, S301 to S303 are further included. It should be noted that the steps the same as those in Figure 1 the embodiment will not be described in detail here. Please refer to the foregoing.

[0064] S301. Obtain a business processing request and generate the first identifier corresponding to the business processing request;

[0065] In the above S301, the above business processing request includes requests during the operation of the system. The first identifier is used to track all the log files under the same business processing request. For example, the backend receives a business request sent by the frontend, such as a request to query the account balance, a request for fund transfer, etc.; the backend generates a unique first identifier corresponding to this business request, which can be the business serial number of this business request, and stores it in the form of taking traceID as the key and the business serial number as the vakue. The first identifier serves as a global index for querying the log files generated by this business request.

[0066] S302. Obtain a number of log data generated when calling a number of microservices to process the business processing request, and generate the second identifier corresponding to each log data;

[0067] In the above S302, the second identifier is unique in the same preset association table to ensure that each second identifier associated with the first identifier is unique. Preferably, the second identifier is unique globally to further ensure that the second identifier is unique in all log files.

[0068] S303. Establish the preset association table corresponding to the first identifier and each of the second identifiers.

[0069] In the above S303, optionally, all the second identifiers can be used as keys and the first identifier as the value and stored in Redis, thereby establishing the mapping relationship between each second identifier and the first identifier, enabling the system to completely track the log data between all microservices, and thus realizing obtaining all the log data generated by the same task request according to the first identifier. Optionally, according to the call relationship between each microservice, the second identifier is associated with the first identifier and other second identifiers in a tree structure to establish the mapping relationship between the first identifier and the second identifier and the mapping relationship between the second identifiers with a call relationship, facilitating the user to locate system problems and improving the efficiency of verifying system problems.

[0070] In Figure 3 the basis of the illustrated embodiment, Figure 4 is a schematic flowchart of another log processing method provided by an embodiment of the present application. After the above S303, S401 to S403 are further included. It should be noted that the steps identical to those in Figure 3 the embodiment will not be repeated here. Please refer to the foregoing.

[0071] S401. Write each of the log data into the corresponding preset buffer queue;

[0072] In the above S401, the log data includes but is not limited to log data of info level, warn level, and error level. The log level is defined as the info level, and preset buffer queues corresponding to the info level, warn level, and error level are provided respectively for temporarily caching log data of different level types. Further, the system is in a normal state in most cases, and the log data of the info level is more than that of other log levels. Therefore, the preset buffer queue of the info level is more than those of the warn level and the error level. For example, there are 3 preset buffer queues of the info level, and 1 each of the warn level and the error level.

[0073] S402. Invoke the log consumer corresponding to the preset buffer queue to process each of the log data in the preset buffer queue to obtain the log file corresponding to each log data;

[0074] S403. Store the log file in the storage layer.

[0075] In the above S402 and S403, the log consumer is the consumer in the producer - consumer model of log processing. Each buffer queue corresponds to one or more log consumers, and the log consumers consume the log data into the database and store it in the log library corresponding to the corresponding log level, so as to facilitate querying the log files of the corresponding log level in the later stage.

[0076] In Figure 4 the basis of the illustrated embodiment, Figure 5 is a schematic flowchart of another log processing method provided by an embodiment of the present application. The above method further includes S501 and S502. It should be noted that the steps identical to those in Figure 4 the embodiment will not be repeated here. Please refer to the foregoing.

[0077] S501. Periodically detect the preset buffer queue;

[0078] S502. When there is no such log data in the preset buffer queue, control the log consumer corresponding to the buffer queue to sleep for a preset time.

[0079] In the above S501, periodically detect whether the log consumer corresponding to the buffer queue is in an idle state. When the log consumer corresponding to the cache queue is in an idle state, detect whether there is log data in the cache queue. If there is, consume the log data to store the log data in the database; if not, loop the detection until the number of detections reaches the preset number, and if the log consumer still has no log data, control the log consumer to sleep for a preset time to reduce thread occupation.

[0080] It should be understood that the magnitudes of the sequence numbers of the above steps do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0081] Corresponding to the log processing method described in the above - mentioned embodiment, Figure 6 shows a structural block diagram of a log processing device 600 provided by an embodiment of the present application. For the sake of convenience of description, only the parts related to the embodiment of the present application are shown.

[0082] Referring to Figure 6 , the device includes:

[0083] A first acquisition module 601, configured to acquire a log file query request, where the log file query request carries a first identifier and a query keyword;

[0084] A second acquisition module 602, configured to obtain a preset association table corresponding to the first identifier according to the first identifier, where the preset association table includes a plurality of second identifiers;

[0085] A reading module 603, configured to read the second identifier corresponding to the query keyword in the preset association table;

[0086] An invocation module 604, configured to invoke a first log file corresponding to the second identifier in the storage layer according to the second identifier.

[0087] It should be noted that, for the information interaction, execution process, etc. between the above-mentioned device / units, since they are based on the same concept as the method embodiments of the present application, their specific functions and the technical effects brought about can be specifically referred to in the method embodiment part, and will not be elaborated here.

[0088] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments and will not be elaborated here.

[0089] Figure 7 It is a schematic structural diagram of a terminal device provided in an embodiment of the present application. As Figure 7 shown, the terminal device 7 of this embodiment includes: at least one processor 70 ( Figure 7 only one is shown in the figure), a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70. When the processor 70 executes the computer program 72, the steps in any of the above-mentioned method embodiments for log processing are implemented.

[0090] The terminal device 7 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art can understand, Figure 7The above is only an example of the terminal device 7, which does not constitute a limitation on the terminal device 7. It may include more or fewer components than those shown in the figure, or combine some components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0091] The so-called processor 70 may be a central processing unit (CPU). The processor 70 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0092] In some embodiments, the memory 71 may be an internal storage unit of the terminal device 7, such as the hard disk or memory of the terminal device 7. In some other embodiments, the memory 71 may also be an external storage device of the terminal device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 7. Further, the memory 71 may also include both the internal storage unit and the external storage device of the terminal device 7. The memory 71 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0093] The embodiments of the present application also provide a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0094] The embodiments of the present application provide a computer program product, and when the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned various method embodiments when executed.

[0095] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a portable hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0096] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0097] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0098] In the embodiments provided in this application, it should be understood that the disclosed device / network device and method can be implemented in other ways. For example, the device / network device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical, or other form.

[0099] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A log processing method, characterized in that, it includes: Obtain a log file query request, where the log file query request carries a first identifier and a query keyword, and the query keyword includes a time keyword, a microservice name keyword, and a log level keyword; According to the first identifier, obtain a preset association table corresponding to the first identifier, and the preset association table contains multiple second identifiers; Read the second identifier corresponding to the query keyword in the preset association table; According to the second identifier, retrieve a first log file corresponding to the second identifier in the storage layer; Wherein, before obtaining the log query request, it further includes: Obtain a service processing request and generate the first identifier corresponding to the service processing request; Obtain a number of log data generated when calling a number of microservices to process the service processing request, and generate the second identifier corresponding to each log data; Establish the preset association table corresponding to the first identifier and each second identifier, wherein, according to the call relationship between each microservice, associate the second identifier with the first identifier and other second identifiers in a tree structure to establish the mapping relationship between the first identifier and the second identifier and the mapping relationship between the second identifiers with call relationships, and obtain the preset association table; After retrieving the first log corresponding to the second identifier in the storage layer according to the second identifier, it further includes: Obtain the association link corresponding to the second identifier in the preset association table. The preset association table contains multiple association links, and each association link contains the association relationship between one first identifier and one or more fourth identifiers; According to the association link, retrieve the second logs corresponding to all the fourth identifiers in the association link.

2. The log processing method according to claim 1, characterized in that, The step of obtaining the preset association table corresponding to the first identifier according to the first identifier includes: According to the first identifier, read a preset mapping table in the database and obtain the preset association table corresponding to the first identifier in the preset mapping table. The preset mapping table contains the mapping relationship between multiple third identifiers and multiple preset association tables, and the first identifier is one of the multiple third identifiers.

3. The log processing method according to claim 1, characterized in that, The second identifier is one of the fourth identifiers.

4. The log processing method according to claim 1, characterized in that, After establishing the preset association table corresponding to the first identifier and each second identifier, it further includes: Write each log data into a corresponding preset buffer queue; Call the log consumer corresponding to the preset buffer queue to process each log data in the preset buffer queue to obtain the log file corresponding to each log data; Store the log file in the storage layer.

5. The log processing method according to claim 4, characterized in that, The method further includes: Periodically detect the preset buffer queue; When the preset buffer queue does not have the log data, the log consumer corresponding to the preset buffer queue is controlled to sleep for a preset time.

6. A log processing device, characterized in that it includes: A first acquisition module, configured to acquire a log file query request, the log file query request carrying a first identifier and a query keyword, the query keyword including a time keyword, a microservice name keyword, and a log level keyword; A second acquisition module, configured to acquire a preset association table corresponding to the first identifier according to the first identifier, and the preset association table contains multiple second identifiers; A reading module, configured to read the second identifier corresponding to the query keyword in the preset association table; An invocation module, configured to invoke, according to the second identifier, a first log file corresponding to the second identifier in the storage layer; Wherein, before acquiring the log query request, a service processing request is acquired, and the first identifier corresponding to the service processing request is generated; Acquire a plurality of log data generated when a plurality of microservices are called to process the service processing request, and generate a second identifier corresponding to each log data; Establish the preset association table corresponding to the first identifier and each second identifier, wherein, according to the call relationship between each microservice, the second identifier is associated with the first identifier and other second identifiers in a tree structure, so as to establish the mapping relationship between the first identifier and the second identifier and the mapping relationship between the second identifiers with a call relationship, and obtain the preset association table; After invoking the first log corresponding to the second identifier in the storage layer according to the second identifier, obtain the association link corresponding to the second identifier in the preset association table, and the preset association table contains a plurality of association links, and each association link includes an association relationship between a first identifier and one or more fourth identifiers; according to the association link, invoke the second logs corresponding to all the fourth identifiers in the association link.

7. A terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program, the method described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, the computer-readable storage medium stores a computer program, characterized in that when the computer program is executed by a processor, the method described in any one of claims 1 to 5 is implemented.

Images