Log monitoring method and device, equipment and medium

Through automated log data collection and screening, monitoring is carried out in the order of region and application priorities, solving the problems of low efficiency and poor reliability of log data monitoring in the existing technology, and achieving fast response and efficient log monitoring.

CN118964141APending Publication Date: 2024-11-15SHUI YOU INFORMATION TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411170551.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the prior art, log data monitoring is inefficient and has poor reliability, and manual monitoring is required to lead to low efficiency.

Method used

By collecting log data from each area, filtering target log data based on preset core fields, monitoring it in the order of region and application priorities, automatically generating early warning signals, reducing the amount of redundant data, and improving monitoring efficiency and reliability.

Benefits of technology

It realizes automated log monitoring, improves monitoring efficiency and reliability, can quickly lock abnormal log data and generate early warning signals, reduces manual intervention, and is suitable for complex hybrid cloud deployment scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964141A_ABST
    Figure CN118964141A_ABST
Patent Text Reader

Abstract

The invention discloses a log monitoring method and device, equipment and a medium, and relates to the technical field of computers, and the method comprises the following steps: collecting log data of various application programs in each region; screening out target log data of each region from the log data based on a preset core field; sequentially determining each region as a current region according to a region priority sequence; monitoring the target log data of various application programs in the current area based on an application program priority order, so as to screen out abnormal log data with abnormal codes from the target log data, and generating an early warning signal of the current area according to the application program priority sequence and the abnormal log data. The target log data is automatically monitored according to the area priority sequence and the application program priority sequence, the log data of the areas and the application programs with high priorities are monitored as soon as possible, and the reliability and efficiency of log monitoring are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a log monitoring method, device, equipment and medium. Background Art

[0002] Logs have become the most important part of modern applications. More and more applications on the market use key log information to locate problems, handle runtime exceptions, record key processes, and collect potential user behavior trajectories. Real-time tracking of log information and monitoring of key information in logs through various alarm mechanisms have also become an important part of modern applications.

[0003] At present, it is still necessary to manually monitor various log data and identify abnormal data in order to determine the operation status of various applications. This method has the problems of low monitoring efficiency and poor reliability.

[0004] From the above, it can be seen that how to improve the monitoring efficiency of log data is a problem to be solved in this field. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a log monitoring method, device, equipment and medium to improve the monitoring efficiency of log data. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a log monitoring method, comprising:

[0007] Collect log data from various applications in various regions;

[0008] Filtering target log data of each of the areas from the log data based on a preset core field;

[0009] Determine each of the regions as the current region in turn according to the region priority order;

[0010] The target log data of various types of applications in the current area are monitored based on the application priority order so as to filter out abnormal log data with abnormal codes from the target log data, and generate an early warning signal for the current area according to the application priority order and the abnormal log data.

[0011] Optionally, before collecting log data of various types of applications in various areas, the method further includes:

[0012] Set the cloud server as the first region and the local server or physical device as the second region;

[0013] Classify the applications deployed in the first area and the second area according to the access layer application, the forwarding layer application, and the data layer application to obtain various types of applications in the first area and various types of applications in the second area;

[0014] Accordingly, the log data of various types of applications in various areas are collected, including:

[0015] The original log data of various types of applications in the first area and the second area are collected respectively, the original log data are pre-processed to obtain processed log data, and the processed log data are stored in the log server according to the application category.

[0016] Optionally, the preset core fields include a status code field for representing the application response status and a time-consuming field for representing the application response time;

[0017] Accordingly, the step of filtering out target log data of each area from the log data based on the preset core field includes:

[0018] Filter the target log data of each area from the processed log data based on the status code field and the time-consuming field, and store the target log data in the statistics server.

[0019] Optionally, the application priority order is data layer application, forwarding layer application, access layer application;

[0020] Accordingly, monitoring the target log data of various types of applications in the current area based on the application priority order includes:

[0021] If the current area has a data layer application and a forwarding layer application, after monitoring the target log data of the data layer application in the current area, the target log data of the forwarding layer application in the current area is monitored;

[0022] If the current area has a data layer application and an access layer application, after monitoring the target log data of the data layer application in the current area, the target log data of the access layer application in the current area is monitored;

[0023] If there are forwarding layer applications and access layer applications in the current area, the target log data of the access layer applications in the current area are monitored after the target log data of the forwarding layer applications in the current area are monitored.

[0024] Optionally, determining each of the regions as the current region in order of region priority includes:

[0025] Obtaining the category evaluation value of each area, and determining the regional priority order according to the size of the category evaluation value;

[0026] The regions are determined as current regions in sequence according to the region priority order.

[0027] Optionally, obtaining the category evaluation value of each area includes:

[0028] Determine the current area to be evaluated from each of the areas, and obtain each preset application program index of the current area to be evaluated; wherein the preset application program index includes the total number of nodes, the number of service interfaces, the explosion radius after the accident, and the number of times users access the application program in the current area to be evaluated;

[0029] The weight coefficients of the preset application indicators are determined, and the weight coefficients are used to perform weighted summation on the preset application indicators to obtain the category evaluation value of the current area to be evaluated.

[0030] Optionally, the total number of nodes is the total number of nodes of various types of applications, the number of service interfaces is the number of service interfaces provided externally by various types of applications, the post-accident explosion radius is the size of the impact area after an accident occurs in the current area to be assessed, and the number of times a user accesses an application is a value determined based on the frequency with which the user accesses various types of applications.

[0031] In a second aspect, the present application discloses a log monitoring device, comprising:

[0032] The log collection module is used to collect log data of various applications in various areas;

[0033] A log screening module, used for screening out target log data of each of the areas from the log data based on preset core fields;

[0034] A region determination module, used for determining each of the regions as a current region in sequence according to the region priority order;

[0035] The log monitoring module is used to monitor the target log data of various types of applications in the current area based on the application priority order, so as to filter out abnormal log data with abnormal codes from the target log data, and generate an early warning signal for the current area according to the application priority order and the abnormal log data.

[0036] In a third aspect, the present application discloses an electronic device, comprising:

[0037] Memory, used to store computer programs;

[0038] The processor is used to execute the computer program to implement the steps of the aforementioned disclosed log monitoring method.

[0039] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned disclosed log monitoring method are implemented.

[0040] The beneficial effects of this application are as follows: this application collects log data of various types of applications in various regions; based on the preset core fields, the target log data of each region is filtered out from the log data; each region is determined as the current region in turn according to the region priority order; based on the application priority order, the target log data of various types of applications in the current region is monitored, so as to filter out the abnormal log data with abnormal codes from the target log data, and generate the warning signal of the current region according to the application priority order and the abnormal log data. It can be seen that this application collects log data by region and application category, and filters out the target log data of each region from the log data based on the preset core fields, that is, the redundant log data is eliminated, and the amount of log data to be monitored later is reduced; further, the target log data is monitored according to the region priority order and the application priority order, so that the log data of the region and application with higher priority can be monitored as soon as possible, so as to lock the abnormal log data and generate the warning signal as soon as possible, and the emergency response can be made to the region and application with higher priority at the fastest speed, so as to improve the reliability of log monitoring; this application does not need manual log monitoring, and automated log monitoring can effectively improve monitoring efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0042] Figure 1 A flow chart of a log monitoring method disclosed in this application;

[0043] Figure 2 A specific log data processing schematic diagram disclosed in this application;

[0044] Figure 3 A schematic diagram for obtaining a specific regional priority order disclosed in this application;

[0045] Figure 4 A specific log monitoring process diagram disclosed in this application;

[0046] Figure 5 This is a structural schematic diagram of a log monitoring device disclosed in this application;

[0047] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0049] Logs have become the most important part of modern applications. More and more applications on the market use key log information to locate problems, handle runtime exceptions, record key processes, and collect potential user behavior trajectories. Real-time tracking of log information and monitoring of key information in logs through various alarm mechanisms have also become an important part of modern applications.

[0050] At present, it is still necessary to manually monitor various log data and identify abnormal data in order to determine the operation status of various applications. This method has the problems of low monitoring efficiency and poor reliability.

[0051] To this end, this application provides a log monitoring solution to improve the monitoring efficiency of log data.

[0052] See also Figure 1 As shown, the embodiment of the present application discloses a log monitoring method, including:

[0053] Step S11: Collect log data of various types of applications in various areas.

[0054] In this embodiment, before collecting log data of various types of applications in each area, it also includes: setting the cloud server as the first area, and setting the local server or physical device as the second area; classifying the applications deployed in the first area and the second area according to access layer applications, forwarding layer applications, and data layer applications to obtain various types of applications in the first area and various types of applications in the second area.

[0055] Set the cloud server as the first area, and set the local server or physical device as the second area, that is, collect log data for applications on the cloud and off the cloud respectively, where the cloud application refers to the application service deployed on the cloud, and the off-cloud application refers to the application service deployed on the local server or physical device, so as to ensure that log data collection in various application deployment scenarios is covered. Further, the applications deployed in the first area and the second area are classified according to the access layer application, forwarding layer application, and data layer application to obtain various types of applications in the first area and various types of applications in the second area. It should be noted that the first area and the second area will not contain all categories of applications, that is, the first area contains at most two types of applications in the access layer application, forwarding layer application, and data layer application, and the application categories contained in the second area are categories other than the application categories contained in the first area. For example, if the first area contains access layer applications and forwarding layer applications, then the second area contains data layer applications.

[0056] In this embodiment, the log data of various types of applications in various areas are collected, including: collecting the original log data of various types of applications in the first area and the second area respectively, preprocessing the original log data to obtain processed log data, and storing the processed log data in the log server according to the application category. After the collected log data is preprocessed, it is classified and stored in the corresponding log server according to the different application categories of each type of application. For example, each application such as e-commerce application, social application, game application, etc. can be stored in the corresponding log server according to the application category to which it belongs, so as to facilitate the subsequent log data processing and analysis.

[0057] Step S12: Filtering target log data of each area from the log data based on a preset core field.

[0058] In this embodiment, the preset core fields include a status code field for characterizing the response status of the application and a time-consuming field for characterizing the response time of the application; accordingly, filtering out the target log data of each area from the log data based on the preset core fields includes: filtering out the target log data of each area from the processed log data based on the status code field and the time-consuming field, and storing the target log data to the statistical server.

[0059] For example Figure 2A specific log data processing diagram is shown, in which log data is processed according to preset core fields, specifically, the target log data of each area can be screened out from the processed log data based on the status code field and the time-consuming field, and the target log data can be output to the statistical server for further analysis and monitoring. The statistical server can perform real-time monitoring, generate reports, and provide early warning notifications to ensure that the operating status of the application system is managed and controlled in a timely and effective manner. Among them, the status code field is, for example, 2XX, 3XX, 4XX, and 5XX. These status codes represent the status of the application response. By counting the frequency of occurrence of different status codes, the service quality and stability of the application can be evaluated; the time-consuming field is, for example, >3000ms, >5000ms, >10000ms, and >30000ms. The response efficiency of the application is evaluated based on the response time recorded in the log data.

[0060] Step S13: determining each of the regions as the current region in turn according to the region priority order.

[0061] In this embodiment, determining each of the regions as the current region in order of region priority includes: obtaining a category evaluation value of each region, and determining the region priority order according to the size of the category evaluation value; and determining each of the regions as the current region in order of region priority. Figure 3 A specific schematic diagram of obtaining the regional priority order is shown, obtaining the category evaluation value M of each region, and sorting the category evaluation value of each region from large to small to obtain the regional priority order, that is, the higher the ranking, the higher its priority level. For example, the category evaluation value of the first region is M1, the category evaluation value of the first region is M2, and M1>M2, then the first region is first determined as the current region, and when the target log data of the first region is completed, the second region is determined as the new current region.

[0062] In this embodiment, the acquisition of the category evaluation value of each area includes: determining the current area to be evaluated from each of the areas, and acquiring each preset application indicator of the current area to be evaluated; wherein the preset application indicators include the total number of nodes, the number of service interfaces, the explosion radius after the accident, and the number of times users access the application in the current area to be evaluated; determining the weight coefficient of each of the preset application indicators, and using the weight coefficients to perform weighted summation on each of the preset application indicators to obtain the category evaluation value of the current area to be evaluated. It can be understood that in the process of determining the priority order of the areas, the category evaluation value of each area is determined first, and the category evaluation value acquisition formula is specifically as follows:

[0063] M=aP+bK+cT+dD;

[0064] In the formula, P represents the explosion radius after the accident in the current area to be evaluated, K represents the number of times users in the current area to be evaluated access the application, T represents the number of service interfaces in the current area to be evaluated, D represents the total number of nodes in the current area to be evaluated, a, b, c, d are the weight coefficients of the preset application indicators, respectively. In a specific case, a, b, c, d are 0.3, 0.4, 0.2, 0.1, respectively. That is, the formula for obtaining the category evaluation value is:

[0065] M=0.3P+0.4K+0.2T+0.1D.

[0066] In this embodiment, the node sum is the total number of nodes of various types of applications, the number of service interfaces is the number of service interfaces provided by various types of applications, the explosion radius after the accident is the size of the affected area after the accident occurs in the current area to be evaluated, and the number of times the user accesses the application is a value determined based on the frequency of user access to various types of applications. The node sum D is the sum of nodes of various types of application systems in the region, including the number of deployment nodes of cloud and off-cloud applications; the number of service interfaces T is the number of service interfaces provided by various types of application systems; the explosion radius after the accident P is the range that may be affected after the accident occurs, which is used to evaluate the importance of the affected area; the number of user accesses K is the frequency statistics of user access to various types of application systems.

[0067] Step S14: Monitor the target log data of various types of applications in the current area based on the application priority order, so as to filter out abnormal log data with abnormal codes from the target log data, and generate an early warning signal for the current area according to the application priority order and the abnormal log data.

[0068] In this embodiment, the application priority order is data layer application, forwarding layer application, access layer application; accordingly, the target log data of each type of application in the current area is monitored based on the application priority order, including: if there are data layer applications and forwarding layer applications in the current area, then the target log data of the data layer application in the current area is monitored, and then the target log data of the forwarding layer application in the current area is monitored; if there are data layer applications and access layer applications in the current area, then the target log data of the data layer application in the current area is monitored, and then the target log data of the access layer application in the current area is monitored; if there are forwarding layer applications and access layer applications in the current area, then the target log data of the forwarding layer application in the current area is monitored, and then the target log data of the access layer application in the current area is monitored.

[0069] It is understandable that the monitoring points of the application access layer, forwarding layer and data set are set separately to ensure adequate coverage. Not all application categories are included in the same area. The pre-set application priority order is specifically data layer applications first, forwarding layer applications second, and access layer applications last. Therefore, there are the following situations:

[0070] 1) If there are data layer applications and forwarding layer applications in the current region, the target log data of the data layer applications in the current region is monitored first, and then the target log data of the forwarding layer applications in the current region is monitored;

[0071] 2) If there are data layer applications and access layer applications in the current area, the target log data of the data layer applications in the current area is monitored first, and then the target log data of the access layer applications in the current area is monitored;

[0072] 3) If there are forwarding layer applications and access layer applications in the current area, the target log data of the forwarding layer applications in the current area is monitored first, and then the target log data of the access layer applications in the current area is monitored;

[0073] Furthermore, data layer monitoring specifically implements data quality monitoring, detects and processes abnormal or invalid data in data sets, analyzes access patterns and frequencies of data sets, optimizes data storage and access performance, monitors the security and compliance of data sets, and ensures that data is not accessed or tampered with by unauthorized persons. When monitoring the data layer, data quality monitoring, data access pattern analysis, data security, and compliance can be considered. In terms of data quality monitoring, data quality monitoring strategies are implemented to detect and process abnormal or invalid data in data sets. In terms of data access pattern analysis, access patterns and frequencies of data sets are analyzed to help optimize data storage and access performance. In terms of data security and compliance, the security and compliance of data sets are monitored to ensure that data is not accessed or tampered with by unauthorized persons.

[0074] The forwarding layer monitoring specifically monitors data traffic and load conditions to ensure that the system runs stably under high load; regularly checks the availability and health status of the forwarding service, and tracks changes in forwarding rules and configurations. Monitoring of the forwarding layer can include data traffic and load balancing, forwarding service availability, forwarding rules, and configuration changes. In terms of data traffic and load balancing, real-time monitoring of the data traffic and load status of the forwarding layer ensures that the system runs stably under high loads; in terms of forwarding service availability, regularly checks the availability and health status of the forwarding service, and promptly discovers and handles service failures or delays; in terms of forwarding rules and configuration changes, track changes in forwarding rules and configurations to ensure that the changes do not affect the normal operation of the system.

[0075] Application access layer monitoring specifically implements access log collection strategies, monitors interface response time and abnormal requests, for example, records the response time of each user request, and detects requests with frequent access and abnormal data formats. When monitoring the application access layer, attention is paid to access log collection, interface response time monitoring, and abnormal request detection. In terms of access log collection, a log collection strategy for the access layer is implemented to ensure that all request information entering the system is captured; in terms of interface response time monitoring, the response time of the access layer interface is recorded and analyzed to promptly discover and resolve potential performance issues; in terms of abnormal request detection, a real-time detection and alarm mechanism for abnormal requests is implemented, such as frequent access, abnormal data formats, etc.

[0076] Confirm whether each application is running normally, that is, determine whether there is an abnormal code in the target log data. If there is, it means that the corresponding application is not running normally. Filter out the abnormal log data with abnormal codes from the target log data, and generate an early warning signal for the current area according to the application priority order and the abnormal log data. Send the early warning signal to the operation and maintenance personnel so that the operation and maintenance personnel can respond and handle the problem quickly; further, the monitoring alarm rules can be pre-configured, that is, the rules for generating early warning signals are set. For example, when the proportion of abnormal codes exceeds the preset threshold, the application is judged to be running abnormally and an early warning signal is generated. When the proportion of abnormal codes does not exceed the preset threshold, the application will not be judged to be running abnormally, and no early warning signal will be generated. It should be noted that it is also necessary to regularly review and optimize the monitoring strategy, and adjust the monitoring indicators and alarm rules according to the actual operation situation.

[0077] This embodiment adopts a regional approach to collect log data from each application system in the corresponding area, and improves monitoring efficiency through priority order calling and confirmation methods, which can avoid the problems of low monitoring efficiency and poor reliability caused by manual monitoring. Furthermore, the application scenarios are greatly realized in manufacturing, intelligent transportation, smart agriculture and other scenarios with mobile devices and Internet of Things (IoT) devices.

[0078] The beneficial effects of this application are as follows: this application collects log data of various types of applications in various regions; based on the preset core fields, the target log data of each region is filtered out from the log data; each region is determined as the current region in turn according to the region priority order; based on the application priority order, the target log data of various types of applications in the current region is monitored, so as to filter out the abnormal log data with abnormal codes from the target log data, and generate the warning signal of the current region according to the application priority order and the abnormal log data. It can be seen that this application collects log data by region and application category, and filters out the target log data of each region from the log data based on the preset core fields, that is, the redundant log data is eliminated, and the amount of log data to be monitored later is reduced; further, the target log data is monitored according to the region priority order and the application priority order, so that the log data of the region and application with higher priority can be monitored as soon as possible, so as to lock the abnormal log data and generate the warning signal as soon as possible, and the emergency response can be made to the region and application with higher priority at the fastest speed, so as to improve the reliability of log monitoring; this application does not need manual log monitoring, and automated log monitoring can effectively improve monitoring efficiency.

[0079] Below Figure 4Taking a specific log monitoring process diagram as an example, this application is explained. The various applications to be monitored include e-commerce platforms, offline transaction analysis systems, etc., which are deployed on cloud servers and local servers respectively. The cloud server is pre-set as the first area, and the local server or physical device is set as the second area, and the applications in each area are classified. Next, the log data of each application is continuously monitored. The specific process is as follows:

[0080] 1) Log data collection by region: On-cloud applications: By setting up log collection modules by region, log data of e-commerce platforms deployed on the cloud are collected. These log data include request logs, error logs, user behavior logs, etc. Off-cloud applications: On local servers, log collection modules are deployed to collect log data of transaction systems. These log data include server performance logs, database access logs, etc.

[0081] 2) Log data screening and uploading: From the collected log data, filter out key target log data, such as logs with response time exceeding 5000ms, error logs with 5XX status codes, etc.

[0082] 3) Upload to regional server: Upload the filtered target log data to the corresponding regional server by region. The log data on the cloud is uploaded to the cloud regional server, and the log data off the cloud is uploaded to the local regional server.

[0083] 4) Determine the regional priority order:

[0084] 4.1) Obtain indicators for each preset application: For the application systems in each region, collect and calculate the following key indicators: the total number of nodes D, the number of service interfaces T, the explosion radius range P after the accident, and the number of user accesses K to the application. Among them, the total number of application nodes on the cloud D is 50, the total number of application nodes off the cloud D is 30, the number of service interfaces of applications on the cloud is 20, the number of service interfaces of applications off the cloud is 15, the explosion radius range of applications on the cloud is 5, the explosion radius range of applications off the cloud is 3, the number of user accesses to applications on the cloud is 10,000, and the number of user accesses to applications off the cloud is 5000.

[0085] 4.2) Calculate the category evaluation value: Use the category evaluation value acquisition formula to calculate the category evaluation value of each area, where the category evaluation value acquisition formula is M=0.3P+0.4K+0.2T+0.1D, as follows:

[0086] On-cloud area: M = 0.3\times 5+0.4\times 10000+0.2\times 20+0.1\times 50=4005.5;

[0087] Off-cloud applications: M = 0.3\times 3+0.4\times 5000+0.2\times 15+0.1\times 30=2003.9;

[0088] Determine the priority order: Based on the calculation results, on-cloud applications have a higher priority than off-cloud applications.

[0089] 5) Regional-level log data call: Call log data according to priority: The monitoring server calls the target log data of the application on the cloud for monitoring first, and then calls the log data of the application under the cloud in order of priority.

[0090] 6) Application-level log data monitoring: In the same area, confirm whether the application system is running normally according to the different priority orders of the applications. For example, the payment interface of the e-commerce platform is monitored first, followed by the product display interface, and finally the user comment interface.

[0091] 7) Send warning signals: When the monitoring server confirms that an application is not running properly, such as an abnormal status code or a long response time, a warning signal is immediately sent to the operation and maintenance personnel to quickly respond and handle the problem.

[0092] Through the above steps, the efficiency and reliability of log monitoring of multiple application systems can be effectively improved, especially in complex hybrid cloud deployment scenarios, which can achieve efficient problem location, that is, through real-time log monitoring and early warning mechanisms, abnormal problems in the application system can be quickly discovered and located, reducing the troubleshooting time. Further, the system stability is improved, that is, through detailed log analysis and monitoring, performance bottlenecks and potential risks can be discovered in a timely manner, improving the stability and reliability of the application system. And scientific resource allocation is achieved, and the specific priority order based on comprehensive evaluation indicators makes resource allocation and emergency response more scientific and reasonable, ensuring the priority protection of key applications.

[0093] See also Figure 5 As shown, the embodiment of the present application discloses a log monitoring device, including:

[0094] The log collection module 11 is used to collect log data of various applications in various areas;

[0095] A log screening module 12, configured to screen target log data of each of the areas from the log data based on a preset core field;

[0096] The region determination module 13 is used to determine each region as the current region in sequence according to the region priority order;

[0097] The log monitoring module 14 is used to monitor the target log data of various types of applications in the current area based on the application priority order, so as to filter out abnormal log data with abnormal codes from the target log data, and generate an early warning signal for the current area according to the application priority order and based on the abnormal log data.

[0098] The beneficial effects of this application are as follows: this application collects log data of various types of applications in various regions; based on the preset core fields, the target log data of each region is filtered out from the log data; each region is determined as the current region in turn according to the region priority order; based on the application priority order, the target log data of various types of applications in the current region is monitored, so as to filter out the abnormal log data with abnormal codes from the target log data, and generate the warning signal of the current region according to the application priority order and the abnormal log data. It can be seen that this application collects log data by region and application category, and filters out the target log data of each region from the log data based on the preset core fields, that is, the redundant log data is eliminated, and the amount of log data to be monitored later is reduced; further, the target log data is monitored according to the region priority order and the application priority order, so that the log data of the region and application with higher priority can be monitored as soon as possible, so as to lock the abnormal log data and generate the warning signal as soon as possible, and the emergency response can be made to the region and application with higher priority at the fastest speed, so as to improve the reliability of log monitoring; this application does not need manual log monitoring, and automated log monitoring can effectively improve monitoring efficiency.

[0099] Furthermore, an embodiment of the present application also provides an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0100] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the log monitoring method performed by the electronic device disclosed in any of the aforementioned embodiments.

[0101] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0102] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0103] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.

[0104] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows, Unix, Linux, etc. In addition to including a computer program that can be used to complete the log monitoring method performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. In addition to data received by the electronic device and transmitted from an external device, the data 223 can also include data collected by its own input and output interface 25, etc.

[0105] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned disclosed log monitoring method. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0106] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0107] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly with hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), memory, read-only memory (ROM), electrically programmable EPROM (Erasable Programmable Read Only Memory), electrically erasable programmable EEPROM (Electrically Erasable Programmable read only memory), register, hard disk, removable disk, CD-ROM (Compact Disc Read-Only Memory), or any other form of storage medium known in the technical field.

[0108] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0109] The above is a detailed introduction to a log monitoring method, device, equipment and medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A log monitoring method, It is characterized in that include: Collect log data from various applications in various regions; Filtering target log data of each of the areas from the log data based on a preset core field; Determine each of the regions as the current region in turn according to the region priority order; The target log data of various types of applications in the current area are monitored based on the application priority order so as to filter out abnormal log data with abnormal codes from the target log data, and generate an early warning signal for the current area according to the application priority order and the abnormal log data.

2. The log monitoring method according to claim 1, It is characterized in that Before collecting log data of various applications in various regions, the process also includes: Set the cloud server as the first region and the local server or physical device as the second region; Classify the applications deployed in the first area and the second area according to the access layer application, the forwarding layer application, and the data layer application to obtain various types of applications in the first area and various types of applications in the second area; Accordingly, the log data of various types of applications in various areas are collected, including: The original log data of various types of applications in the first area and the second area are collected respectively, the original log data are pre-processed to obtain processed log data, and the processed log data are stored in the log server according to the application category.

3. The log monitoring method according to claim 2, It is characterized in that The preset core fields include a status code field for representing the application response status and a time-consuming field for representing the application response time; Accordingly, the step of filtering out target log data of each area from the log data based on the preset core field includes: Filter the target log data of each area from the processed log data based on the status code field and the time-consuming field, and store the target log data in the statistics server.

4. The log monitoring method according to claim 2, It is characterized in that The priority order of the applications is data layer application, forwarding layer application, access layer application; Accordingly, monitoring the target log data of various types of applications in the current area based on the application priority order includes: If the current area has a data layer application and a forwarding layer application, after monitoring the target log data of the data layer application in the current area, the target log data of the forwarding layer application in the current area is monitored; If the current area has a data layer application and an access layer application, after monitoring the target log data of the data layer application in the current area, the target log data of the access layer application in the current area is monitored; If there are forwarding layer applications and access layer applications in the current area, the target log data of the access layer applications in the current area are monitored after the target log data of the forwarding layer applications in the current area are monitored.

5. The log monitoring method according to claim 1, It is characterized in that The step of determining each of the regions as the current region in order of region priority includes: Obtaining the category evaluation value of each area, and determining the regional priority order according to the size of the category evaluation value; The regions are determined as current regions in sequence according to the region priority order.

6. The log monitoring method according to claim 5, It is characterized in that The obtaining of the category evaluation value of each area includes: Determine the current area to be evaluated from each of the areas, and obtain each preset application program index of the current area to be evaluated; wherein the preset application program index includes the total number of nodes, the number of service interfaces, the explosion radius after the accident, and the number of times users access the application program in the current area to be evaluated; The weight coefficients of the preset application indicators are determined, and the weight coefficients are used to perform weighted summation on the preset application indicators to obtain the category evaluation value of the current area to be evaluated.

7. The log monitoring method according to claim 6, It is characterized in that The total number of nodes is the total number of nodes of various types of applications, the number of service interfaces is the number of service interfaces provided by various types of applications to the outside world, the explosion radius after the accident is the size of the impact area after the accident occurs in the current area to be evaluated, and the number of times users access the application is a value determined based on the frequency of users accessing various types of applications.

8. A log monitoring device, It is characterized in that include: The log collection module is used to collect log data of various applications in various areas; A log screening module, used for screening out target log data of each of the areas from the log data based on preset core fields; A region determination module, used for determining each of the regions as a current region in sequence according to the region priority order; The log monitoring module is used to monitor the target log data of various types of applications in the current area based on the application priority order, so as to filter out abnormal log data with abnormal codes from the target log data, and generate an early warning signal for the current area according to the application priority order and the abnormal log data.

9. An electronic device, It is characterized in that include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the log monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, It is characterized in that Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the log monitoring method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Abnormal operation monitoring method and device, equipment and storage medium

    CN117273703A

  • An abnormal operation monitoring method, device, equipment and storage medium

    CN117273703B