Log management analysis operation and maintenance fault method, system and equipment and storage medium

Abstract

The invention provides a method, a system and equipment for analyzing operation and maintenance faults through log management, and a storage medium, and the method comprises the steps: carrying out the multi-log integration query of received operation and maintenance fault feedback data, so as to obtain a corresponding log record, carrying out the log display, and querying a link log and an operation log through building a traceID index; performing data association on the operation log, the link log and the running log by taking the traceID as an association condition; and through visual analysis, key index data and a service party topological relation are extracted from log records so as to locate a system fault reason. According to the method, the problems of inconvenience in log query, difficulty in association, low visualization degree and the like in the prior art are fundamentally solved, so that the efficiency and the accuracy of operation and maintenance fault analysis are remarkably improved, and stable and reliable operation of the system is guaranteed.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of operation and maintenance management, and particularly relates to a log management and analysis operation and maintenance fault method, system, device and storage medium. BACKGROUND

[0002] In the field of operation and maintenance management today, log management and fault analysis play a crucial role in ensuring the stable and efficient operation of the system. However, the existing system log processing method has many limitations. In most systems, operation logs, link logs and running logs are independently stored and managed, and their query and display functions are also separated from each other.

[0003] Taking a traditional enterprise resource planning (ERP) system as an example, operation logs are only used to record various operation behaviors of users, and a special operation log management module is responsible for storage. When querying operation logs, only simple time range retrieval within the module is usually available, and the flexibility is poor. Link logs are mainly used to monitor the status of network links in real time, and are generally generated and viewed by specific network management tools, lacking direct association with other types of logs. Running logs focus on recording various events and state changes during system operation, and their query function is limited to their own independent management interface, making it difficult to use them in conjunction with other logs. The existing method mainly has the following problems: (1) Low efficiency of log query: Since various logs are managed separately, operation and maintenance personnel need to query operation logs, link logs and running logs in multiple different systems or modules when troubleshooting system faults. This process not only consumes a lot of time and effort, but also easily leads to information omission due to frequent system switching operations, greatly reducing the efficiency of fault troubleshooting.

[0004] (2) Difficult log association: The existing systems generally lack a unified association identifier, making it difficult to establish effective contact between different types of logs. When a system fails, operation and maintenance personnel cannot quickly and accurately obtain complete information related to the fault from numerous logs, which seriously affects the accuracy of fault location and the depth of analysis, thereby prolonging the time to repair the fault and increasing the loss caused by system downtime.

[0005] (3) Serious lack of visualization: Currently, the display of logs is mostly in the form of simple text lists. For complex system operation data and fault information, this display method cannot intuitively present the running status of the system and potential fault risks. Operation and maintenance personnel cannot quickly grasp the trend of changes in key indicators and the relationship between different parts of the system from these text lists, which is not conducive to timely discovering problems in the system and making scientific and reasonable decisions. SUMMARY

[0006] In view of this, the present application aims to provide a log management analysis operation and maintenance fault method, system, device and storage medium to solve at least one of the above problems.

[0007] To achieve the above purpose, the technical scheme of the present application is as follows: In a first aspect, the present application provides a log management analysis operation and maintenance fault method, comprising: The received operation and maintenance fault feedback data is subjected to multi-log integrated query to obtain corresponding log records and perform log display, wherein traceID index is established to query link logs and running logs; The operation logs, link logs and running logs are data-associated with traceID as the association condition; Key indicator data and service topology relationship are extracted from the log records through visual analysis to locate the system fault cause, wherein the key indicator data is displayed through a line chart to show the indicator data trend, and the service topology relationship is displayed through a topology graph to show the system state.

[0008] In a second aspect, based on the same inventive concept, the present application further provides a log management analysis operation and maintenance fault system, comprising: A log query module configured to subject the received operation and maintenance fault feedback data to multi-log integrated query to obtain corresponding log records and perform log display, wherein traceID index is established to query link logs and running logs; A data association module configured to data-associate the operation logs, link logs and running logs with traceID as the association condition; A visual analysis module configured to extract key indicator data and service topology relationship from the log records through visual analysis to locate the system fault cause, wherein the key indicator data is displayed through a line chart to show the indicator data trend, and the service topology relationship is displayed through a topology graph to show the system state.

[0009] In a third aspect, based on the same inventive concept, the present application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to realize the method of the first aspect.

[0010] In a fourth aspect, based on the same inventive concept, the present application further provides a non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores computer instructions for causing the computer to execute the method of the first aspect.

[0011] Compared with the prior art, the log management analysis operation fault method, system, device and storage medium provided by the application have the following beneficial effects: The log management analysis operation fault method provided by the application fundamentally solves the problems of inconvenient log query, difficult association and low visualization degree in the prior art by organically integrating the query and display functions of operation logs, link logs and running logs, using a unique traceID to realize accurate association between different types of logs, and creating an intuitive and easy-to-understand visual interface and a convenient and efficient operation mode, thereby significantly improving the efficiency and accuracy of operation fault analysis and guaranteeing the stable and reliable operation of the system. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and the illustrative embodiments of the present application and their description serve the purpose of explaining the present application. The accompanying drawings should not be regarded as a limitation of the present application. In the drawings: Figure 1 A log management analysis operation fault method flowchart described in an embodiment of the present application; Figure 2 A log management analysis operation fault system structure schematic diagram described in an embodiment of the present application; Figure 3 An electronic device hardware structure schematic diagram described in an embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below with reference to the embodiments and the accompanying drawings.

[0014] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood as the usual meaning understood by those skilled in the art to which the present application belongs. The terms "first", "second" and the like used in the embodiments of the present application do not represent any order, number or importance, but are only used to distinguish different components. The terms "include" or "contain" and the like mean that the elements or objects before the terms cover the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and the like do not mean physical or mechanical connection, but can include electrical connection, whether direct or indirect. The terms "up", "down", "left", "right" and the like only represent relative positional relationships, which can change accordingly when the absolute positions of the described objects change.

[0015] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0016] Please refer toFigure 1 As shown, the embodiment provides a log management analysis operation and maintenance fault method, which specifically includes the following steps: Step S101, multi-log integration query is performed on the received operation and maintenance fault feedback data to obtain corresponding log records and perform log display, wherein traceID index is established to query link logs and running logs.

[0017] Specifically, in the embodiment, a unified query entrance is constructed to provide operation and maintenance personnel with a convenient log query experience. In terms of database design, for the operation log table, multiple field indexes are carefully established, covering key fields such as login account, module name, and operation time. This design optimizes the execution efficiency of query statements. When operation and maintenance personnel input these query conditions, the system can quickly retrieve matching records from the operation log table. For the query of link logs and running logs, an efficient traceID index is established in the log database. When the traceID is input, the system can quickly locate and query the related link logs and running log records by means of the index, realizing efficient and unified query of different types of logs.

[0018] The query method described in this step not only supports multi-condition flexible query of operation logs, but also can conveniently query the other two types of logs based on traceID, completely breaking through the barriers of traditional log query and providing operation and maintenance personnel with an efficient query means. Compared with the traditional decentralized log query method, the unified query function of the embodiment significantly saves the time of operation and maintenance personnel in switching between multiple systems for query. Through actual testing, in a typical system containing 10,000 operation logs, 5,000 link logs, and 8,000 running logs, using the query function of the invention for fault troubleshooting, the average query time is greatly shortened from 30 minutes to within 5 minutes, greatly improving the efficiency of fault troubleshooting and reducing the downtime of the system caused by faults.

[0019] In terms of link log information display, the system defaults to filtering out and displaying failed logs. In the design of the front-end interface, by setting reasonable initial display logic, when the page is loaded, the front-end will automatically send a request to the back-end to query failed link logs. After receiving the request, the back-end extracts data from the link log table according to the preset filtering conditions and returns it to the front-end for display.

[0020] When the user clicks the switching button, the front end sends a corresponding operation log or running log query request to the back end by listening to the click event of the button. The back end obtains data from the corresponding log table according to the request type and returns the data to the front end, thereby realizing seamless switching of different types of log display. When displaying the running log, the system filters and displays by judging whether the traceID field exists in the log record.

[0021] When the user clicks the "Query more" button, the front end obtains the time range of the currently displayed log, and calculates the time boundaries of the previous and next one minute according to the time range. Then, the front end sends a query request containing the new time range to the back end, and the back end queries and returns 30 running log records within the previous and next one minute from the running log table according to the time range, thereby providing more rich log information for the operation and maintenance personnel.

[0022] In step S102, the operation log, the link log and the running log are associated with the traceID as the association condition.

[0023] Specifically, in the log generation phase, the system assigns a unique traceID to each operation, link event and running state change, ensuring the uniqueness and traceability of log information. During log storage, the traceID field is ensured to be included in different types of log tables, laying a foundation for subsequent association operations. During data query and processing, the operation log table, the link log table and the running log table are accurately associated by using the JOIN operation in the SQL query statement with the traceID as the association condition. This mechanism enables the system to provide complete and accurate data support for fault analysis, greatly improving the reliability of fault analysis.

[0024] In this step, the traceID is used as the core association identifier throughout the entire process of log generation, storage and query. In this way, accurate association between different types of logs is ensured, providing a solid data foundation for comprehensive and in-depth fault analysis.

[0025] With the aid of the traceID association mechanism, the present application can present various types of log information related to faults in a complete and accurate manner, effectively avoiding fault misjudgment caused by missing information. In multiple simulated fault tests, the accuracy of fault positioning has been greatly improved from 60% to more than 95%, which enables operation and maintenance personnel to more accurately locate the root cause of the fault, thereby quickly taking effective repair measures and reducing the impact of the fault on system operation.

[0026] Step S103, through visual analysis, and extracting key indicator data and service topology relationship from log records to locate system failure causes, wherein the key indicator data is displayed by generating a line chart to show the trend of the indicator data, and the service topology relationship is displayed by generating a topology graph to show the system state.

[0027] Specifically, in this embodiment, in terms of key indicator display, the system extracts relevant data from the log database. For example, the calculation of operation frequency is achieved by counting the number of operation records in the operation log within a certain time period, and the failure occurrence time distribution is extracted from the time information of the failure-related log records and statistically analyzed, and professional data visualization libraries (such as Echarts) are used to visually present these data in the form of a line chart, in which the horizontal coordinate is set as time and the vertical coordinate is the corresponding indicator value. The operation and maintenance personnel can clearly observe the trend of the key indicators.

[0028] In the display of topology relationship, the system extracts the association information between services from the log to construct a complex topology graph data structure. For normal nodes and abnormal failure nodes, the node state field is distinguished, and different colors or icons are used in the topology graph for clear identification, so that the operation and maintenance personnel can quickly identify abnormal conditions in the system and provide intuitive basis for troubleshooting.

[0029] This embodiment uses advanced visualization methods such as line charts and topology graphs to intuitively and clearly display log key indicators and service topology relationships. This visualization design enables operation and maintenance personnel to quickly understand the system's running status and failure information, greatly improving the efficiency of troubleshooting and analysis, and reflecting the advantages of this method in user experience and data analysis.

[0030] The method described in this embodiment integrates the query and display functions of operation logs, link logs, and running logs, uses a unique traceID to accurately associate different types of logs, and creates an intuitive and easy-to-understand visualization interface and a convenient and efficient operation method, fundamentally solving the problems of inconvenient log query, difficult association, and low visualization in the prior art, thereby significantly improving the efficiency and accuracy of operation and maintenance failure analysis, and ensuring the stable and reliable operation of the system.

[0031] Embodiment one In the actual operation and maintenance scene of an e-commerce system, a user feedback encountered an abnormal situation when placing an order. After receiving the feedback, the operation and maintenance personnel immediately queried the operation log in the log query module according to the user's login account. The system quickly retrieved an operation record triggered by the user when placing an order, and obtained the corresponding traceID “001”. Subsequently, the operation and maintenance personnel input the traceID to query the link log and running log. In the log display module, the initial display of the link log information shows that a short delay occurred in the network link of the order submission. Then, the operation and maintenance personnel click the switch button to view the running log and find that the running log record with traceID “001” is displayed in the order processing module. Due to database connection exception, the order data insertion fails. Through the line chart of the visual analysis module, the operation and maintenance personnel find that the operation frequency in this time period does not show obvious abnormality, but the fault occurrence time distribution shows that other users also encountered similar problems at the same time point. When viewing the topology diagram, it is found that the order processing service node has an abnormal identifier. Based on the above information, the operation and maintenance personnel quickly locate the fault reason to be that the temporary load of the database server is too high, causing part of the order processing to fail. Subsequently, the operation and maintenance personnel optimize the database server in time, successfully solve the fault, and restore the normal operation of the system.

[0032] Embodiment Two In the daily operation and maintenance process of an online education platform, a teacher feedback encountered operation failure when uploading course materials. After receiving the feedback, the operation and maintenance personnel queried the operation log in the log query module according to the module name “course material upload module” and the operation time range. The system quickly obtained the relevant operation record and its traceID “002”. Through the traceID, the operation and maintenance personnel queried the link log and running log, and in the log display module, they saw that the link log showed that the network connection during the upload process was normal. After switching to the running log, they found that the running log record with traceID “002” showed that the file storage service had an error. The operation and maintenance personnel clicked the “query more” button to obtain the running log within the previous and next one minute, and from it they found that the file storage service had multiple disk space shortage warning information in the same time period. Through the topology diagram of the visual analysis module, the operation and maintenance personnel found that the file storage service node was in an abnormal state, and other services related to the node were also affected to a certain extent. Combined with the change of operation frequency in the line chart, the operation and maintenance personnel judged that the recent increase in course material upload volume caused the disk space shortage. Subsequently, the operation and maintenance personnel cleaned up the disk space in time, successfully solved the problem of the teacher uploading course materials, and ensured the normal teaching order of the online education platform.

[0033] It is to be understood that the foregoing description is descriptive only. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order and still achieve desirable results. Additionally, the processes depicted in the attached figures do not necessarily require the particular order shown or sequential order to achieve desirable results. In certain implementations, multitasking and parallel processing can be advantageous.

[0034] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the embodiments of the present application also provide a log management analysis operation and maintenance fault system.

[0035] As shown in Figure 2 The log management analysis operation and maintenance fault system comprises: The log query module 11 is configured to perform multi-log integrated query on the received operation and maintenance fault feedback data to obtain corresponding log records and perform log display, wherein the link log and the running log are queried through the establishment of traceID index; The data association module 12 is configured to associate the operation log table, the link log table and the running log table with traceID as the association condition; The visual analysis module 13 is configured to locate the system fault cause by visual analysis and extracting key indicator data and service topology relationship from the log records, wherein the key indicator data is displayed by generating a line chart to show the indicator data trend, and the service topology relationship is displayed by generating a topology graph to show the system state.

[0036] For the convenience of description, the above system is described in various modules by function. Of course, the functions of each module can be implemented in the same or multiple software and / or hardware when implementing the embodiments of the present application.

[0037] The system of the above embodiments is used to implement the corresponding method of any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here.

[0038] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the embodiments of the present application also provide an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to realize the method according to any one of the above embodiments.

[0039] Figure 3A more specific electronic device hardware structure schematic diagram provided by the embodiment is shown, and the device can include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected through the bus 1050 for communication between each other inside the device.

[0040] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present specification.

[0041] The memory 1020 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 1020 and called and executed by the processor 1010.

[0042] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0043] The communication interface 1040 is used to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0044] The bus 1050 includes a channel to transmit information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0045] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, but in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary to implement the embodiments of the present application, and does not have to contain all the components shown in the figure.

[0046] The electronic device of the above embodiment is used to implement the corresponding method in any of the preceding embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here.

[0047] Based on the same inventive concept, the present application also provides a non-transitory computer readable storage medium, which stores computer instructions for causing the computer to execute the method of any of the above embodiments.

[0048] The computer readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be realized by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0049] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the method of any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which are not described here.

[0050] Those skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to limit the scope (including claims) of the present application to these examples; the technical features of the above embodiments or different embodiments can also be combined under the idea of the present application, the steps can be implemented in any order, and there are many other changes of different aspects of the embodiments of the present application as described above. In order to be brief, they are not provided in detail.

[0051] Additionally, to simplify the description and discussion, and so as not to obscure the embodiments of the application being presented, the well-known functions or constructions of integrated circuit (IC) chips and other components can or can not be shown in the figures and will be omitted as not to unnecessarily obscure the embodiments of the application being presented. Moreover, the apparatus can be shown in block diagram form in order to avoid obscuring the embodiments of the application, and in view of the fact that the details in how such block diagram apparatus are to be implemented are highly dependent upon the platform within which the embodiments of the application are to be implemented (i.e., these details should be readily apparent to those of ordinary skill in the art having the benefit of this disclosure). Where specific details are set forth in order to describe an illustrative embodiment of the application, it will be apparent to one of ordinary skill in the art that the embodiment of the application can be practiced without these specific details. In other instances, detailed descriptions of well-known methods, procedures, components, circuits and the like can omit so as not to unnecessarily obscure aspects of the application.

[0052] Although the application has been described in conjunction with specific embodiments thereof, numerous alternatives, modifications, and variations will be readily apparent to those of ordinary skill in the art. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0053] It is intended that the embodiments of the application cover all such alternatives, modifications and variations as fall within the scope of the appended claims. Accordingly, any and all such modifications, variations or equivalents that fall within the scope of the application should be intended to be embraced by the appended claims.

Claims

1. A log management analysis operation and maintenance fault method, characterized in that, Comprising: Multi-log integration query is performed on the received operation and maintenance fault feedback data to obtain corresponding log records and perform log display, wherein traceID index is established to query link logs and running logs; Data correlation is performed on operation logs, link logs and running logs by taking traceID as a correlation condition; Visual analysis is performed, and key indicator data and service party topology relationship are extracted from log records to locate system fault causes, wherein key indicator data is displayed by generating a line chart to show indicator data trend, and service party topology relationship is displayed by generating a topology graph to show system status.

2. The method of claim 1, wherein, Further comprising: A multi-field index is established to query operation logs, wherein the multi-field index at least includes login account, module name and operation time.

3. The method of claim 1, wherein: In response to displaying link logs, the system defaults to filtering and displaying failure logs; In response to a user clicking a switching button, the front end sends a corresponding operation log or running log query request to the back end by listening to the click event of the button, and the back end returns data to the front end according to the request type.

4. The method of claim 1, wherein: In the log generation phase, the system assigns a unique traceID to each operation, link event and running state change; In the log storage phase, the traceID field is included in different types of log tables; In the data query and processing phase, the operation logs, link logs and running logs are correlated by using the JOIN operation in the SQL query statement and taking the traceID as the correlation condition.

5. The method of claim 1, wherein: The extracted operation frequency and fault time distribution are displayed by a line chart to show the trend of key indicators; The correlation information between service parties is extracted, a topology graph data structure is constructed, normal nodes and abnormal failure nodes are distinguished according to the node state field, and different colors or icons are used in the topology graph to identify them. 6.A system for log management and analysis operation and maintenance fault, characterized in that, Comprising: A log query module configured to perform multi-log integration query on the received operation and maintenance fault feedback data to obtain corresponding log records and perform log display, wherein traceID index is established to query link logs and running logs; A data correlation module configured to correlate operation logs, link logs and running logs by taking traceID as a correlation condition; A visual analysis module configured to perform visual analysis and extract key indicator data and service party topology relationship from log records to locate system fault causes, wherein key indicator data is displayed by generating a line chart to show indicator data trend, and service party topology relationship is displayed by generating a topology graph to show system status.

7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1-5 when executing the program.

8. A non-transitory computer-readable storage medium, comprising: Wherein, The non-transitory computer readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1-5.

Images