Log monitoring method, electronic device, and readable storage medium

Abstract

The present application provides a log monitoring method, electronic device, and readable storage medium. The log monitoring method comprises: obtaining logs of a microservice within a preset time period and at a predetermined level, wherein the logs at the predetermined level include exception stack information and log text; extracting information indicating the exception category from the exception stack information; extracting a text summary from the log text; and monitoring the logs based on the information indicating the exception category and the text summary. The log monitoring method of the present application can obtain log data in a unified format using unified log extraction rules, and when an exception occurs, quickly count and analyze the exception.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a log monitoring method, an electronic device, and a readable storage medium. Background Art

[0002] With the rapid development of network business complexity and globalization, the deployment and monitoring methods of existing software systems are facing challenges, and service architecture represented by microservices has become a new trend.

[0003] The core concepts of microservices are fine-grained module division, service-oriented interface encapsulation, and lightweight communication interactions. Microservices inherently offer excellent scalability and are gradually becoming the mainstream architecture for building internet applications.

[0004] It should be understood that this background technology section is intended in part to provide a useful background for understanding the technology, however, these contents are not necessarily what are known or understood by those skilled in the art before the filing date of this application. Summary of the Invention

[0005] The present application provides a log monitoring method, comprising: obtaining a log of a microservice within a preset time period and having a predetermined level, the log of the predetermined level including exception stack information and log text; extracting information indicating the exception category from the exception stack information; extracting a text summary from the log text; and monitoring the log based on the information indicating the exception category and the text summary.

[0006] In one embodiment of the present application, the predetermined level includes an error level.

[0007] In one embodiment of the present application, the method further includes: determining the total number of requests received by the microservice within a preset time period; and determining the number of abnormal requests within the preset time period based on the information indicating the abnormality category.

[0008] In one embodiment of the present application, monitoring the log includes: sending an alarm in response to a ratio of the determined number of abnormal requests to the total number of requests being greater than a preset threshold.

[0009] In one embodiment of the present application, extracting information indicating the exception category from the exception stack information includes: extracting information indicating the exception category from the exception stack information using a regular rule; and removing duplicate logs from the logs according to the exception category.

[0010] In one embodiment of the present application, the monitoring granularity includes at least one of a service name, an interface address, a method name, an interface response code, or an interface response time, and determining the number of abnormal requests within the preset time period based on the information representing the abnormal category also includes: identifying the monitoring granularity corresponding to the abnormal category from the information representing the abnormal category; and determining the number of abnormal requests corresponding to each monitoring granularity within the preset time period.

[0011] In one embodiment of the present application, extracting a text summary from the log text includes:

[0012] removing characters of a preset format from the log text; and extracting the Chinese characters from the log text in response to the log text including Chinese characters.

[0013] In one embodiment of the present application, extracting a text summary from the log text includes:

[0014] removing characters of a preset format from the log text; and in response to the log text only including English characters, intercepting the English characters.

[0015] In one embodiment of the present application, the method further includes: storing the information indicating the exception category and the log of the call link of the corresponding same request as a core log.

[0016] In one embodiment of the present application, the method further includes: determining a data type of the core log based on the importance weight; and determining at least one of a storage method and a storage location of the core log according to the data type.

[0017] In one embodiment of the present application, the method also includes: determining the data type of the core log based on the importance weight includes: calculating the importance weight of the core log based on the data generation time, log level and at least one of the user-defined logs of the core log; and determining the core log as hot data, warm data or cold data based on the importance weight.

[0018] Another aspect of the present application provides an electronic device, comprising: a memory for storing computer instructions; and a processor for communicating with the memory to execute the computer instructions, thereby implementing any of the above methods.

[0019] Another aspect of the present application provides a readable storage medium, wherein the readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, any of the above methods is implemented.

[0020] The log monitoring method of the present application can obtain log data in a unified format through unified log extraction rules, and when an anomaly occurs, the anomaly can be quickly counted and analyzed. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings, in which:

[0022] Figure 1 Flowchart of a log monitoring method according to an embodiment of the present application;

[0023] Figure 2 Schematic diagram of monitoring data identification in the log monitoring method according to an embodiment of the present application;

[0024] Figure 3 Schematic diagram of a log monitoring device according to an embodiment of the present application;

[0025] Figure 4 The present invention is a block diagram of an electronic device according to a log monitoring method according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] It should be noted that the terms "first", "second" etc. described in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, or product comprising a series of steps is not necessarily limited to the units of those steps or products clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, or products.

[0028] As mentioned earlier, microservices are a widely used service architecture. Different microservices typically have different technology stacks, including databases and data models. When running large and complex systems, each microservice generates a large amount of logs.

[0029] Researchers discovered that, on the one hand, microservices using different technology stacks output logs in different formats, making it difficult to uniformly process and troubleshoot issues after collection. Typically, problems are discovered and reported by users and testers during system usage, and operations and maintenance personnel or developers then use the logs to locate the issues. Consequently, microservice problem discovery is inefficient and slow.

[0030] On the other hand, the database stores a large number of logs generated by microservices, which can reach terabytes per day, but the database storage space is limited.

[0031] The present application embodiment proposes a log monitoring method to address at least one of the above problems. Figure 1 FIG. 2 shows a flow chart of a log monitoring method 200 according to an embodiment of the present application. Figure 1 As shown, the log monitoring method 200 may include:

[0032] S210: Obtaining logs of the microservice within a preset time period and with a predetermined level, where the logs of the predetermined level include exception stack information and log text;

[0033] S220: extracting information indicating the exception type (exception_type) from the exception stack information;

[0034] S230: extracting a text summary from the log text;

[0035] S240: Monitor the log based on the information and text summary representing the abnormal category. It should be noted that the order of steps listed in the implementation method of the present application is only one way of executing the steps among many, and does not represent the only execution order. In actual operation, the method of the implementation method of the present application can be executed sequentially or in parallel. In some embodiments, the steps shown in the log monitoring method 200 are not exhaustive, and other steps can be executed before, after, or between the described methods, and the present application does not limit this.

[0036] The following combination Figure 1-Figure 4 The log monitoring method 200 of the present application is described in detail. Figure 1 As shown, method 200 starts at step S210. In step S210, logs of the microservice within a preset time period and with a predetermined level may be obtained. The logs of the predetermined level include exception stack information and log text.

[0037] Specifically, a call request can initiate multiple microservice calls at multiple layers. Each microservice provides a monitoring data identifier. The structure of the monitoring data identifier is as follows: Figure 2 As shown. Figure 2In the example shown, each monitoring data identifier includes TraceId and spanld. The multi-layer microservices of the same call request may have the same Traceld. For example, the first-layer microservice A and the second-layer microservices B, C, D and the third-layer microservices E and F may have the same TraceId (for example, oa1234) to connect the entire call request in series. Therefore, all logs of the same call request can be obtained through TraceId. When an abnormal request occurs, the logs of all links of the same abnormal call request can be obtained and stored for subsequent troubleshooting and analysis. In addition, in the same call request link, different microservices have different spanld, which can be used as a unique identifier for each microservice.

[0038] In some examples, a predetermined level of log can be configured through the Log4j2 server. Logs configured with Log4j2 can have multiple log levels, including ALL, DEBUG, INFO, WARN, ERROR, FATAL, OFF, etc. Logs of predetermined levels can be printed as needed. Taking logs at the WARN or ERROR levels as an example, the printed log files of these levels may include exception stack traces generated by abnormal access requests to the microservice. The exception stack traces include information indicating the exception category. Based on this information indicating the exception category, the cause of the exception can be located.

[0039] The logs at the predetermined level may also include log text describing the event information based on the access request, such as the time, date, user, and operation. The operational status of the microservice can be recorded by analyzing and inspecting the log text.

[0040] Reference again Figure 1 , the method 200 continues to step S220 , in which information indicating the exception category may be extracted from the exception stack information.

[0041] Taking error-level logs as an example, regular rules can be used to extract information indicating the exception category from the exception stack. In related technologies, regular rules, also known as regular expression rules, are filtering or matching rules based on regular expressions. Regular expressions use a single string to describe and match a series of strings that conform to a certain syntactic rule.

[0042] Typically, information representing an exception category includes the exception category, its class, parent class, and line. The exception stack generated by each microservice for the same exception request contains a large amount of duplicate information. Optionally, logs can be deduplicated based on the class, parent class, and line to facilitate statistical analysis.

[0043] In other examples, the information containing the exception category in the exception stack information is often only located in the first two lines of the exception stack information. For example, the first two lines of information may include the exception category (Service Exception) identified by the system, the exception cause customized by the developer (for example, the input parameter is incorrect and the corresponding account cannot be found), the location where the exception occurred (for example, ServiceListService.java:262), and the class and cause under the exception category (for example, findServiceByRoleWithOld) and other information. Therefore, the exception stack information can be intercepted, such as the first two lines, for storage and analysis. In some examples, the log of the call link of the same request corresponding to the information representing the exception category can also be stored as a core log.

[0044] In some examples, after extracting the first two lines of information from the exception stack information, information indicating the exception category can be extracted from the two lines using regular rules. Alternatively, the information indicating the exception category can be extracted using regular rules. For example, the first two lines of information can be split by a line break character (\n). Alternatively, the first and second elements can be extracted after the split to form the first and second extracted lines of information.

[0045] Optionally, the first line of extracted information can be used to extract the exception category identified by the system and the exception cause customized by the developer. For example, the first line of extracted information can be further segmented, for example, using a ":" to further segment the first line of extracted information. After segmentation, information representing the exception category identified by the system can be extracted. For example, the first element can be extracted as the exception category identified by the system. Optionally, information representing the exception cause customized by the developer can also be extracted. For example, the last element can be extracted as the exception cause customized by the developer.

[0046] Optionally, the second row of extracted information can be further segmented to extract the location, class, and cause of the exception. For example, "." can be used for further segmentation, and the last element can be extracted as the location, class, and cause of the exception.

[0047] Continue to refer Figure 1 , the method 200 proceeds to step S230 , in which a text summary may be extracted from the log text.

[0048] In some examples, a text summary can be extracted from the log text included in the logs of the predetermined level, and the logs can be monitored daily through statistical analysis of the text summary to understand the operating status of the microservice.

[0049] In some cases, in order to facilitate troubleshooting, the text summary usually includes redundant information such as the input parameters of the abnormal request and the response results. When the amount of response result data is very large, saving all the data will cause unnecessary storage resource occupation. Exemplarily, the text summary can be extracted by regular rules, such as removing the characters of the preset format of the log text. Removing the above-mentioned preset format characters includes: extracting the information containing statusMessage, and then removing the json format information included therein. Optionally, when the log text includes Chinese characters, Chinese characters can be further extracted from the information containing statusMessage. In other examples, when the log text includes only English characters, the English characters of the statusMessage field can be truncated, for example, the first 100 characters can be truncated.

[0050] Taking some application scenarios of vehicle management as an example, the extracted log text summary may include "the input parameters are incorrect and the corresponding account cannot be queried", "the synchronous lock obtains the lock and sets the timeout period", the device grouping information cannot be found according to the device, "the owner authentication fails to add points and the service call fails", etc.

[0051] By extracting text summaries and information representing abnormal categories, the ranking of high-frequency abnormal log texts can be obtained and presented in a visual form.

[0052] Alternatively, by clicking on a text summary in the visualization interface, you can display the entire call chain log and the call relationships between each microservice. Alternatively, you can view information such as the time when each abnormal request first occurred and the number of abnormal requests in the visualization interface.

[0053] The implementation method of the present application can obtain log data in a unified format through unified log extraction rules. On the one hand, it can quickly count and analyze abnormal situations when abnormalities occur, and on the other hand, it can save storage space.

[0054] Reference again Figure 1 , the method 200 continues to step S240 , in which the log may be monitored based on the information indicating the abnormality category and the text summary.

[0055] Optionally, within a preset time period, the total number of requests received by the microservice can be determined, and the number of abnormal requests can be determined based on the information representing the abnormal category. For example, after deduplicating the information representing the abnormal category, the number of information representing the abnormal category is counted as the number of abnormal requests.

[0056] Optionally, the number of abnormal requests can be compared with the total number of requests. If the ratio of the number of abnormal requests to the total number of requests is greater than a preset threshold, which can be set to 30%, for example, then an alarm is sent. Optionally, the preset time period can include, for example, the past hour or the past day.

[0057] As an option, the log analysis system can be used to count and record the number of abnormal requests as the data volume of the error log, and then the total number of requests can be counted as the total data volume of the log. The error ratio can be calculated by calculating the ratio of the data volume of the error log to the total data volume of the log. When the preset error rate is exceeded, the log analysis system can alarm and automatically notify the operation and maintenance personnel through text messages, etc., so that the operation and maintenance personnel can quickly understand, troubleshoot and solve the problem.

[0058] In some examples, the database storing enterprise microservices includes, for example, an ES (Elasticsearch) database. The Elasticsearch database can be used as a real-time distributed search and analysis engine, and its data logic organization is similar to that of a relational database. For example, Elasticsearch indexes data, each index has a type, and each type has a corresponding mapping. The Elasticsearch database provides a specific data query API interface (such as a RESTful API or a Java client API). Therefore, the total number of requests received, the number of abnormal requests, and their changing trends can be queried regularly and in real time through the API interface, and comparisons with yesterday, last week, etc. can be calculated.

[0059] In some embodiments, the number of abnormal requests can be counted according to monitoring granularity classifications such as service name, interface address, method name, interface response code, and interface response time. Optionally, the monitoring granularity corresponding to the abnormal category can be identified from the information representing the abnormal category, and the number of abnormal requests corresponding to each monitoring granularity within a preset time period can be determined. In some examples, the extracted information representing the abnormal category includes, for example, {ServiceException: The input parameters are incorrect and the corresponding account number cannot be found}, and the monitoring granularity corresponding to the service name of the abnormal category can be determined. After deduplication of the information representing the abnormal category, the number of occurrences of {ServiceException: The input parameters are incorrect and the corresponding account number cannot be found} can be counted as the number of abnormal requests of the monitoring granularity. The number of abnormal requests of the monitoring granularity is compared with the number of requests of the monitoring granularity to obtain the service name error rate, thereby performing fine-grained monitoring of the log.

[0060] In some examples, the number of abnormal requests at the monitoring granularity of the interface address can be counted and compared with the number of requests of the interface address to obtain the interface error rate; for example, the number of error logs can be filtered according to the method name to obtain the method error rate; for example, the number of unsuccessful response codes can be filtered according to the interface response code to obtain the interface business error rate; for example, the number of data greater than a preset number of seconds can be filtered according to the interface response time to obtain the number of request timeouts.

[0061] In some examples, when the error rate at each of the above monitoring granularities is greater than the corresponding reference value, an alarm reminder may be sent.

[0062] In some application scenarios, for example, if microservice A's bb interface (e.g., http interface) reports an error at some point, you can analyze whether the error rate of microservice A's bb interface in the past hour exceeds a threshold. If so, you can send a text message to alert operations and maintenance personnel of the A service anomaly at that point. Alternatively, you can subscribe to alerts for microservice A's bb interface anomalies. That is, when collecting error log data next time, if the same problem occurs and the number of times exceeds the set threshold, a message push notification will be sent.

[0063] In some embodiments, the log monitoring method of the present application further includes storing core log data. For example, the above-mentioned information indicating the exception category and the logs of all call links corresponding to the same call request can be stored as core logs.

[0064] When the storage resources of the server are limited, the importance of the stored core logs can be evaluated, and different data types can be obtained based on the importance. As an option, the data type of the core log can be determined based on the importance weight; and the storage method of the core log data can be dynamically adjusted according to its data type, or the storage location of the core log data can be dynamically adjusted according to the data type. In some examples, the above-mentioned determination of the data type of the core log based on the importance weight includes: calculating the importance weight of the core log based on the time when the core log data is generated, the log level, or at least one of the user-defined logs; and determining the data type of the core log as hot data, warm data, or cold data based on the calculated importance weight, and different importance weights represent different data types.

[0065] As shown in formula (1), the importance weight (important) is mainly determined by three factors: timeliness (days, also known as data generation time), log level (levels), and user customization (customize). α1, α2, and α3 represent timeliness, log level, and user customization weights, respectively. For example, a demarcation threshold can be set for the importance weights of cold data, warm data, and hot data, respectively. As an option, the importance weight of hot data can be greater than that of warm data and cold data, respectively. By dynamically adjusting α1, α2, and α3, the data type of the core log can be regularly adjusted. The core log can be converted within the above-mentioned demarcation thresholds. Based on the dynamically converted data type, the data storage method or data storage location can be dynamically adjusted.

[0066] important=α l days+α2level+α3customize (1)

[0067] Optionally, the storage method and storage location of the data may be adjusted simultaneously. For example, the cold data may be moved out of the stored database and saved to the hard disk in the form of files such as excel, txt or .sql.

[0068] In some implementations, a corresponding storage method can be selected based on characteristics such as the query frequency of the log. For example, core logs with low query frequency are removed from the storage database and saved to the hard disk in the form of files such as Excel, txt, or .sql, thereby improving storage space utilization.

[0069] The log monitoring method provided by the embodiments of this application, on the one hand, stores the logs of all call links of the same call request that represent information of the exception category as core logs, making it easier to quickly determine the cause of the exception through analysis and comparison when an exception occurs later. On the other hand, by categorizing and storing the core logs, the utilization rate of storage space can be improved.

[0070] like Figure 3 As shown, the embodiment of the present application further provides a log monitoring device 400, which includes a program module framework for implementing the above method and carrying the following program code for implementing the method of the present application.

[0071] In some embodiments, the device 400 includes: an acquisition module 410, used to obtain logs of the microservice within a preset time period and with a predetermined level, the logs of the predetermined level include exception stack information and log text; a first extraction module 420, used to extract information representing the exception category from the exception stack information; a second extraction module 430, used to extract a text summary from the log text; and a monitoring module 440, used to monitor the log based on the information representing the exception category and the text summary.

[0072] In some examples, the first extraction module 420 may include a first extraction unit 421 for extracting information indicating an exception category from the exception stack information using regular rules; and a deduplication unit 422 for removing duplicate logs from the logs according to the exception category.

[0073] In some examples, the second extraction module 430 may include a formatting removal unit 431 for removing characters in a preset format from the log text; and a second extraction unit 432 for extracting Chinese characters from the log text when the log text includes Chinese characters. Optionally, the second extraction module 430 may also include a third extraction unit 433 for extracting English characters when the log text includes only English characters.

[0074] In some embodiments, the monitoring module 440 includes: a total request determination unit 441, which is used to count the total number of log requests received by the microservice within a preset time period; an abnormal request statistics unit 442, which determines the number of abnormal requests within a preset time period based on information representing the abnormal category to monitor the log; and an alarm unit 443, which is used to send an alarm reminder when the ratio of the number of abnormal requests to the total number of requests is greater than a preset threshold.

[0075] Exemplarily, the abnormal request statistics unit 442 includes: a monitoring granularity identification subunit 444 for identifying the monitoring granularity corresponding to the abnormal category from the information representing the abnormal category, wherein the monitoring granularity includes at least one of a service name, an interface address, a method name, an interface response code, or an interface response time; and an abnormal request statistics subunit 445 for determining the number of abnormal requests corresponding to each of the monitoring granularities within the preset time period. As an example, when the ratio of the number of abnormal requests corresponding to each monitoring granularity to the number of requests for each monitoring granularity is greater than a reference value, an alarm reminder can be sent through the alarm unit 443.

[0076] In some examples, the apparatus 400 may further include: a storage module 450 for storing the information representing the abnormal category extracted by the first extraction module 420 and the log of the call link corresponding to the same request as a core log.

[0077] In some examples, apparatus 400 may further include a data type confirmation module 460 that determines the data type of the core log based on the importance weight and determines at least one of a storage method and a storage location for the core log based on the data type. The core log may be stored in storage module 450 using different storage methods based on the confirmed data type. Alternatively, the core log may be categorized and stored in different storage locations in storage module 450 based on the confirmed data type.

[0078] In some examples, the data type confirmation module 460 includes an importance weight calculation unit 461, which is used to calculate the importance weight of the core log based on the data generation time, log level and at least one of the user-defined logs of the core log; and a data type confirmation unit 472, which is used to determine the core log as hot data, warm data or cold data based on the importance weight.

[0079] The embodiments of the present application also provide an electronic device and a readable storage medium for a log. Figure 4 A block diagram of an electronic device 600 according to a log monitoring method according to an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0080] like Figure 4 As shown, the electronic device 600 includes a processor 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a memory 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 can also be stored. The processor 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An I / O interface (input / output interface) 605 is also connected to the bus 604.

[0081] In some embodiments, if desired, multiple processors 601 and / or multiple buses 604 can be used along with multiple memories 608. Similarly, multiple electronic devices 600 can be connected, with each electronic device 600 providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system).

[0082] Multiple components in the electronic device 600 are connected to the I / O interface 605, including an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a memory 608, such as a magnetic disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0083] The processor 601 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 601 executes the various methods described above, such as the log monitoring method. For example, in some embodiments, the log monitoring method can be implemented as a computer software program that is tangibly contained in a readable storage medium, such as the memory 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the processor 601, one or more steps of the log monitoring method described above can be performed. Alternatively, in other embodiments, the processor 601 can be configured to execute the log monitoring method in any other appropriate manner (e.g., by means of firmware).

[0084] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0085] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. The above program code can be packaged into a computer program product. These program codes or computer program products can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that the program code, when executed by the processor 601, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or completely on a remote machine or server.

[0086] In the context of the present application, a readable storage medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A readable storage medium can be a machine-readable signal storage medium. A readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0087] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0088] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0089] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship is established by computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within a cloud computing service system that addresses the management difficulties and poor business scalability of traditional physical hosts and VPS services ("Virtual Private Servers," or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0090] The above-described specific embodiments further illustrate the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above description is merely a specific embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. Log monitoring methods, including: Obtain logs of the microservice within a preset time period and with a predetermined level, where the logs of the predetermined level include exception stack information and log text; extracting information indicating the exception category from the exception stack information; extracting a text summary from the log text; as well as monitoring the log based on the information indicating the abnormality category and the text summary; The information indicating the exception category and the log of the call link corresponding to the same request are stored as core logs; Determining the data type of the core log based on the importance weight; as well as Determining at least one of a storage method and a storage location of the core log according to the data type; The determining of the data type of the core log based on the importance weight includes: Calculating the importance weight of the core log according to at least one of the data generation time, log level, and user-defined log of the core log; and The core log is determined as hot data, warm data, or cold data based on the importance weight.

2. The method according to claim 1, wherein The predetermined level includes an error level.

3. The method according to claim 1, wherein monitoring the log comprises: Determine the total number of requests received by the microservice within a preset time period; as well as The number of abnormal requests within the preset time period is determined based on the information indicating the abnormality category.

4. The method according to claim 3, wherein: Monitoring the log includes: In response to the determined ratio of the number of abnormal requests to the total number of requests being greater than a preset threshold, an alarm reminder is sent.

5. The method according to claim 3, wherein: The monitoring granularity includes at least one of a service name, an interface address, a method name, an interface response code, or an interface response time, and determining the number of abnormal requests within the preset time period based on the information indicating the abnormality category further includes: Identifying a monitoring granularity corresponding to the abnormality category from the information indicating the abnormality category; and Determine the number of abnormal requests corresponding to each monitoring granularity within the preset time period.

6. The method according to claim 1, wherein extracting information indicating an exception category from the exception stack information comprises: extracting information indicating the exception category from the exception stack information by using regular rules; as well as Remove duplicate logs from the logs according to the exception category.

7. The method according to claim 1, wherein extracting a text summary from the log text comprises: Removing characters in a preset format from the log text; as well as In response to the log text including Chinese characters, the Chinese characters of the log text are extracted.

8. The method according to claim 1, wherein extracting a text summary from the log text comprises: Removing characters in a preset format from the log text; as well as In response to the log text including only English characters, the English characters are truncated.

9. An electronic device comprising: Memory, for storing computer instructions; as well as A processor, configured to communicate with the memory to execute the computer instructions, thereby implementing the method according to any one of claims 1 to 8.

10. A readable storage medium, wherein computer instructions are stored in the readable storage medium, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images