Log monitoring method and system, electronic equipment and storage medium

Abstract

The invention discloses a log monitoring method and system, electronic equipment and a storage medium. The method comprises the following steps: collecting log data in an industrial production process; reporting the log data to a buffer service so as to process the log data, and storing the processed log information into a buffer queue; the log information is consumed from the cache queue, a retrieval request based on the log information is received, and retrieval information is displayed. According to the embodiment of the invention, the log data is collected and processed, and the retrieval request based on the log information is provided, so that a large amount of industrial log data can be processed and displayed, and the log processing efficiency is improved. The method can be widely applied to the technical field of Internet of Things.

Description

Technical Field

[0001] The present application relates to the technical field of Internet of Things, and in particular to a log monitoring method, system, electronic device and storage medium. Background Art

[0002] In recent years, industry has developed rapidly. In industrial production activities, the normal operation and regular maintenance of industrial equipment are of great significance to ensure that production tasks are completed on schedule. The log data of the equipment plays a very important role here. The industrial log system is responsible for storing and analyzing the log data output by the equipment during the production process. At the same time, industrial big data is the key to the future industry's advantage in the global market competition, and the equipment logs stored in the industrial log system are an important source of industrial big data. Therefore, the industrial log system has become one of the hot issues studied by many scholars and technicians.

[0003] In the related technology, in industrial production scenarios, industrial equipment generates log data at a fast rate and in large amounts. It is very inefficient for technicians to enter the system and search for equipment logs using tools such as vim, and they are unable to cope with the large amount of log data in industrial scenarios. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide an efficient log monitoring method, system, electronic device and storage medium.

[0005] To achieve the above-mentioned purpose, one aspect of an embodiment of the present application proposes a log monitoring method, the method comprising: collecting log data in an industrial production process; reporting the log data to a buffer service to process the log data, and storing the processed log information in a cache queue; consuming the log information from the cache queue, receiving a retrieval request based on the log information, and displaying the retrieval information. By collecting and processing log data and providing a retrieval request based on log information, the embodiment of the present application can process and display a large amount of industrial log data, thereby improving the efficiency of log processing.

[0006] In some embodiments, the method provided by the embodiments of the present application further includes:

[0007] Establish a log monitoring framework, which includes:

[0008] The collection layer is used to implement log storage requests;

[0009] A buffer layer, used to process the log data and decouple the log collection service and the log retrieval service;

[0010] The retrieval layer is used to provide log retrieval services and a visual interface.

[0011] In some embodiments, the method provided by the embodiments of the present application, wherein the log data collected during the industrial production process includes:

[0012] Collect the log data through kafka;

[0013] The log data is written to a disk for storage.

[0014] In some embodiments, the method provided by the embodiments of the present application, reporting the log data to a buffer service to process the log data, and storing the processed log information in a cache queue, includes:

[0015] Processing the log data through logstash to obtain the log information;

[0016] The log information is stored in a cache queue through a copy mechanism.

[0017] In some embodiments, the method provided by the embodiments of the present application, consuming the log information from the cache queue, receiving a retrieval request based on the log information, and displaying the retrieval information, includes:

[0018] Receiving a search request based on the log information, performing log search through elasticsearch, and obtaining search information;

[0019] The search information is displayed through LOGAI.

[0020] In some embodiments, the method provided by the embodiments of the present application further includes:

[0021] Access LOGAI;

[0022] The log data is formatted and the format-converted log data is input into the LOGAI to analyze the log data and generate alarm information.

[0023] In some embodiments, the method provided by the embodiments of the present application further includes:

[0024] Determine the output data format and output endpoint of the LOGAI;

[0025] Create an input configuration in the logstash to receive the data of the LOGAI;

[0026] Configure the logstash to output to the elasticsearch.

[0027] To achieve the above object, another aspect of the embodiment of the present application provides a log monitoring system, the system comprising:

[0028] The first module is used to collect log data during industrial production;

[0029] The second module is used to report the log data to the buffer service to process the log data and store the processed log information into the buffer queue;

[0030] The third module is used to consume the log information from the cache queue, receive a search request based on the log information, and display the search information.

[0031] To achieve the above objective, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above method when executing the computer program.

[0032] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0033] The embodiments of the present application include at least the following beneficial effects: The method provided by the embodiments of the present application includes: collecting log data in the industrial production process; reporting the log data to a buffer service to process the log data, and storing the processed log information in a cache queue; consuming the log information from the cache queue, receiving a retrieval request based on the log information, and displaying the retrieval information. The embodiments of the present application can process and display a large amount of industrial log data by collecting and processing log data and providing a retrieval request based on log information, thereby improving the efficiency of log processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of an embodiment of the log monitoring method provided by the present application;

[0035] Figure 2 It is a schematic diagram of an embodiment of the log monitoring architecture provided by the present application;

[0036] Figure 3 is a schematic diagram of another embodiment of the log monitoring architecture provided by the present application;

[0037] Figure 4 This is a flow chart of an embodiment of the log collection process provided by the present application;

[0038] Figure 5 is a flow chart of an embodiment of the log data buffering process provided by the present application;

[0039] Figure 6is a flow chart of an embodiment of the log data retrieval process provided by the present application;

[0040] Figure 7 is a flow chart of an embodiment of the log analysis process provided by the present application;

[0041] Figure 8 This is a schematic diagram of an interface of an embodiment of the log analysis process provided by the present application;

[0042] Fig. 9 It is a structural diagram of a log monitoring system provided in an embodiment of the present application;

[0043] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0045] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0046] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0048] In recent years, industry has developed rapidly. In industrial production activities, the normal operation and regular maintenance of industrial equipment are of great significance to ensure that production tasks are completed on schedule. The log data of the equipment plays a very important role here. The industrial log system is responsible for storing and analyzing the log data output by the equipment during the production process. At the same time, industrial big data is the key to the future industry's advantage in the global market competition, and the equipment logs stored in the industrial log system are an important source of industrial big data. Therefore, the industrial log system has become one of the hot issues studied by many scholars and technicians.

[0049] People use Internet information technology to regulate industrial production activities, which means that enterprises can save more manpower and carry out larger-scale industrial production activities. However, in the current environment, there are more production equipment, and traditional manual search for equipment logs is not advisable. At the same time, the automation level of industrial equipment continues to improve, and the speed of equipment log output increases accordingly. Therefore, we need to consider the high-concurrency log storage requests of industrial equipment. For example, in the injection molding industry, multiple production stages may generate thousands of logs per minute, and the traditional industrial log system cannot bear high-concurrency log storage requests.

[0050] In the related technologies, traditional log systems lack real-time data analysis. The data processing methods of traditional log systems are relatively simple. In industrial production environments, industrial log systems also need to have the characteristics of high concurrency, low latency, and high data availability, but traditional log systems cannot have all these characteristics.

[0051] In view of this, a log monitoring method is provided in an embodiment of the present application, aiming to improve the analysis capability of log data and provide a log analysis system with high concurrency, low latency and high data availability. The present invention can be used in the field of industrial Internet of Things and belongs to Internet operation and maintenance technology.

[0052] The log monitoring method provided in the embodiment of the present application relates to the field of Internet of Things technology. The log monitoring method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the log monitoring method, etc., but is not limited to the above forms.

[0053] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0054] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0055] Figure 1 This is an optional flow chart of the log monitoring method provided in the embodiment of the present application; Figure 1 The method may include but is not limited to steps S100 to S300.

[0056] Step S100, collecting log data in the industrial production process;

[0057] Step S200, reporting the log data to the buffer service to process the log data and storing the processed log information in the buffer queue;

[0058] Step S300, consuming log information from the cache queue, receiving a search request based on the log information, and displaying the search information.

[0059] In response to the shortcomings of traditional log systems, the embodiments of the present application design and implement a distributed industrial log collection system based on ELK and Kafka technologies. The system supports real-time log data analysis, has the characteristics of high concurrency, low latency and high availability, and is connected to LOGAI log analysis, supports automatic log parsing, and automatically identifies error log keywords, applying large model-based AI analysis technology to the field of industrial logs.

[0060] In actual industrial production scenarios, industrial equipment generates log data at a high rate and in large amounts. It is very inefficient for technicians to enter the system and search for device logs using tools such as vim. However, this application designs and implements a real-time log data retrieval module based on elasticsearch at the log retrieval layer. This module can help technicians quickly retrieve the required device logs and provides a visual interface for log retrieval.

[0061] In response to the high concurrency, low latency and high availability characteristics required by industrial log systems, this application designs and implements an industrial log collection system at the log collection layer that can process high-concurrency log storage requests with low latency and high data availability.

[0062] Elk is a popular log collection, analysis, and visualization system, mainly composed of Elasticsearch, Logstash, and Kibana.

[0063] Elasticsearch is a distributed search and analysis engine for storing and retrieving log data. It can quickly process large amounts of data and provide powerful search capabilities. For example, it can quickly find records containing specific error codes in massive server logs.

[0064] Logstash is used to collect, filter, and convert log data. It supports multiple input sources, such as reading files and receiving network data. It can also pre-process logs, such as parsing log formats and extracting key information.

[0065] Kibana is a data visualization platform that can display log data stored in Elasticsearch in intuitive charts and graphs. For example, a bar chart can be used to display the number of visits in different time periods. These three components work together to help enterprises efficiently manage and utilize log data.

[0066] For log analysis, LOGAI is a free log analysis and intelligence library that supports a variety of log analysis and intelligence tasks. It is compatible with multiple log formats and has an interactive graphical user interface, making interactive AI log monitoring possible. LOGAI provides a unified model interface for popular statistical, time series, and deep learning models, making it easy to benchmark deep learning algorithms based on log anomaly detection. LOGAI uses deep learning models such as convolutional neural networks (CNN), long short-term memory networks (LSTM), and Transformer for log anomaly detection. At the same time, it supports a variety of log analysis tasks such as log summarization, clustering, and anomaly detection, and is compatible with OpenTelemetry data models to ensure seamless integration with a variety of log management platforms. Compared with traditional machine learning log monitoring, which still requires manual keyword screening of logs before classification monitoring, LOGAI integrates the Hugging Face framework to utilize the most advanced natural language processing model, which not only supports automatic log parsing and automatic identification of error log keywords, but also includes a full spectrum of log anomaly detection algorithms from traditional machine learning to deep learning to meet customized needs in different scenarios. LOGAI also opened a new interface based on the latest AI big model and connected to ChatGPT4 to realize the automatic generation of alarm information.

[0067] In some embodiments, reference Figure 2 and Figure 3 As shown, the method provided in the embodiment of the present application further includes:

[0068] Establish a log monitoring architecture, which includes:

[0069] The collection layer is used to implement log storage requests;

[0070] The buffer layer is used to process log data and decouple log collection services from log retrieval services.

[0071] The retrieval layer is used to provide log retrieval services and a visual interface.

[0072] In some embodiments, reference Figure 4 As shown, the method provided in the embodiment of the present application collects log data in an industrial production process, including:

[0073] Step S110, collecting log data through Kafka;

[0074] Step S120, writing the log data to a disk for storage.

[0075] In some possible implementations, Kafka is used to transfer log data to achieve high throughput and scalability.

[0076] In some embodiments, reference Figure 5 As shown, the method provided in the embodiment of the present application reports log data to a buffer service to process the log data, and stores the processed log information in a cache queue, including:

[0077] Step S210, processing the log data through logstash to obtain log information;

[0078] Step S220, storing the log information into the cache queue through the copy mechanism.

[0079] In some embodiments, reference Figure 6 As shown, the method provided by the embodiment of the present application consumes log information from the cache queue, receives a retrieval request based on the log information, and displays the retrieval information, including:

[0080] Step S310, receiving a search request based on log information, performing log search through elasticsearch, and obtaining search information;

[0081] Step S320: display the search information through LOGAI.

[0082] In some possible implementations, the present application can also display the search information through Kibana, and the present application does not limit the display of the search process. The present application provides a log search interface and rich search elements to meet the user's multi-faceted search needs.

[0083] In other embodiments, the present application can also receive a retrieval request based on log information, perform log retrieval through elasticsearch combined with LOGAI, and obtain retrieval information. That is, in some embodiments, the intelligent analysis function of LOGAI is used to enhance the analysis capability of ELK. LOGAI can perform semantic understanding and pattern recognition on logs, integrate these advanced analysis results into the display and query functions of ELK, and provide users with deeper log insights.

[0084] It is understandable that replication is a common strategy for achieving high availability. It refers to the replication of data or services. Taking the database as an example, a database replica is a copy of the database data. When the primary database fails, you can quickly switch to the replica database to allow the system to continue to work normally. At the same time, replicas can also be used to share the pressure of read operations, such as processing user query requests on multiple replicas at the same time to improve system performance.

[0085] In some embodiments, reference Figure 7 As shown, the method provided in the embodiment of the present application further includes:

[0086] Step S400, accessing LOGAI;

[0087] Step S500 , converting the format of the log data, and inputting the converted log data into LOGAI to analyze the log data and generate alarm information.

[0088] In some possible implementations, the alarm information is fed back to Elasticsearch and visualized through Kibana.

[0089] In some embodiments, the method provided by the embodiments of the present application further includes:

[0090] Determine the output data format and output endpoint of LOGAI;

[0091] Create input configuration in logstash to receive LOGAI data;

[0092] Configure logstash to output to elasticsearch.

[0093] The embodiment of the present application realizes data transmission between ELK and LOGAI by setting an interface between ELK and LOGAI, improves the accuracy of log data processing, and improves the comprehensiveness of search information display.

[0094] In some embodiments, the present application configures Kafka through the following steps:

[0095] Create a topic for log data;

[0096] Configure the producer, define key parameters and the format of log data;

[0097] Configure consumers and set consumer parameters to receive log data.

[0098] The following is a detailed description and explanation of the solution of the embodiment of the present invention in conjunction with a specific application example:

[0099] The present invention can be used in the industrial Internet of Things management platform to monitor the logs of some industrial production equipment during the industrial production process. The implementation scheme of the method is as follows:

[0100] 1) The entire log framework consists of log retrieval layer, buffer layer, collection layer, and device layer. See the attached for the specific architecture. Figure 2 and Figure 3 .

[0101] 2) The log collection layer of the framework focuses on low-latency processing of high-concurrency log storage requests and reporting log data to the buffer layer; the log buffer layer focuses on processing and buffering log data, decoupling log collection services from log retrieval services; the log retrieval layer focuses on real-time retrieval of log data and provides a visual retrieval interface for log data. In terms of high data availability, the collection layer ensures that data is not lost by writing data to disk, and the log buffer layer and log retrieval layer ensure high data availability through a replica mechanism.

[0102] 3) Taking the industrial Internet of Things platform as an example, the log collection service first collects the device log data and reports the log data to the log cache service. The log cache service receives the device logs from the collection layer, processes the original device logs, and adds the log data to the cache queue, waiting for the log calculation center implemented by the industrial Internet of Things platform to consume it, and provides real-time retrieval services for log data and a visual retrieval interface for data. The final log retrieval screen is attached. Figure 8 . Since it involves sensitive data of the real environment, the pictures have been desensitized.

[0103] At present, the traditional open source log system has the following problems: 1. The traditional log system lacks real-time data analysis. 2. The data processing method of the traditional log system is relatively simple. 3. In the industrial production environment, the industrial log system also needs to have the characteristics of high concurrency, low latency and high data availability, but the traditional log system cannot have all the characteristics. In this regard, this application implements a real-time retrieval module for log data based on elasticsearch. Logstash is used in the log buffer layer to process log data, and there is no need for the log generator to add additional fields to adapt to the log processing rules of the collection end. Its rich plug-in ecosystem provides convenience for users to write log data processing rules. In the log collection layer, an industrial log collection system that can process high-concurrency log storage requests with low latency and high data availability is designed and implemented. LOGAI log analysis is connected to apply AI analysis technology based on large models to the field of industrial logs. It is compatible with multiple log formats and has an interactive graphical user interface. LOGAI supports automatic log parsing and automatic identification of error log keywords. At the same time, LOGAI also opens a new interface based on the latest AI large model and connects to ChatGPT4 to realize automatic generation of alarm information.

[0104] In view of the shortcomings of traditional log systems, a distributed industrial log collection system based on ELK and Kafka technology was designed and implemented, thus solving the problems of traditional log systems lacking real-time data analysis, relatively simple data processing methods, and not having the characteristics of high concurrency, low latency, and high data availability. Compared with traditional machine learning log monitoring, which simply accesses machine learning models, this application also accesses LOGAI, supports automatic log parsing, and automatically identifies error log keywords. At the same time, LOGAI also opens a new interface based on the latest AI large model and accesses ChatGPT4 to realize automatic generation of alarm information, truly realizing the needs of AI log industrialization and intelligence.

[0105] Possible application scenarios of this application in the future include: Industrial log systems are widely used in the industrial industry. For possible application scenarios in the future, they are mainly based on management platforms connected to industrial equipment.

[0106] This application can realize the industrial log collection system with low latency and high concurrent log storage requests and high data availability in the industrial industry, and perform real-time retrieval and display of log data through the processing of log data. It adopts the emerging AI big model technology, connects to LOGAI to combine the automatic log parsing and automatic identification of error log keywords with the AI ​​big model ChatGPT4, and applies it to industrial logs, realizing the needs of AI log industrialization and intelligence.

[0107] See also Fig. 9 The embodiment of the present application also provides a log monitoring system, which can implement the above-mentioned log monitoring method, and the system includes:

[0108] The first module 810 is used to collect log data in the industrial production process;

[0109] The second module 820 is used to report the log data to the buffer service to process the log data and store the processed log information into the buffer queue;

[0110] The third module 830 is used to consume log information from the cache queue, receive a search request based on the log information, and display the search information.

[0111] In some embodiments, the system provided by the embodiments of the present application further includes a fourth module for:

[0112] Establish a log monitoring architecture, which includes:

[0113] The collection layer is used to implement log storage requests;

[0114] The buffer layer is used to process log data and decouple log collection services from log retrieval services.

[0115] The retrieval layer is used to provide log retrieval services and a visual interface.

[0116] In some embodiments, the system provided by the embodiments of the present application further includes a fifth module for:

[0117] Collect log data through Kafka;

[0118] Write log data to disk for storage.

[0119] In some embodiments, the system provided by the embodiments of the present application further includes a sixth module for:

[0120] Process the log data through logstash to obtain log information;

[0121] Log information is stored in the cache queue through the copy mechanism.

[0122] In some embodiments, the system provided by the embodiments of the present application further includes a seventh module for:

[0123] Receive retrieval requests based on log information, perform log retrieval through elasticsearch, and obtain retrieval information;

[0124] Display the search information through LOGAI.

[0125] In some embodiments, the system provided by the embodiments of the present application further includes an eighth module for:

[0126] Access LOGAI;

[0127] The log data is formatted and input into LOGAI to analyze the log data and generate alarm information.

[0128] In some embodiments, the system provided by the embodiments of the present application further includes a ninth module for:

[0129] Determine the output data format and output endpoint of LOGAI;

[0130] Create input configuration in logstash to receive LOGAI data;

[0131] Configure logstash to output to elasticsearch.

[0132] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0133] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above log monitoring method when executing the computer program. The electronic device can be any smart terminal including a tablet computer, a car computer, etc.

[0134] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0135] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0136] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0137] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the log monitoring method of the embodiment of this application;

[0138] Input / output interface 903, used to implement information input and output;

[0139] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0140] A bus 905 that transmits information between the various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0141] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0142] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned log monitoring method is implemented.

[0143] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0144] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0145] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0146] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0147] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0148] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0149] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0150] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0152] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0155] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A log monitoring method, characterized in that: The method comprises: Collect log data from industrial production processes; Reporting the log data to a buffer service to process the log data and storing the processed log information in a cache queue; The log information is consumed from the cache queue, a search request based on the log information is received, and the search information is displayed.

2. The method according to claim 1, characterized in that The method further comprises: Establish a log monitoring framework, which includes: The collection layer is used to implement log storage requests; A buffer layer, used to process the log data and decouple the log collection service and the log retrieval service; The retrieval layer is used to provide log retrieval services and a visual interface.

3. The method according to claim 1, characterized in that The log data collected during the industrial production process includes: Collect the log data through kafka; The log data is written to a disk for storage.

4. The method according to claim 1, characterized in that: The reporting the log data to the buffer service to process the log data and storing the processed log information in a buffer queue includes: Processing the log data through logstash to obtain the log information; The log information is stored in a cache queue through a copy mechanism.

5. The method according to claim 1, characterized in that The consuming the log information from the cache queue, receiving a search request based on the log information, and displaying the search information includes: Receiving a search request based on the log information, performing log search through elasticsearch, and obtaining search information; The search information is displayed through LOGAI.

6. The method according to claim 1, characterized in that The method further comprises: Access LOGAI; The log data is formatted and the format-converted log data is input into the LOGAI to analyze the log data and generate alarm information.

7. The method according to claim 5, characterized in that The method further comprises: Determine the output data format and output endpoint of the LOGAI; Create an input configuration in the logstash to receive the data of the LOGAI; Configure the logstash to output to the elasticsearch.

8. A log monitoring system, characterized in that: The system comprises: The first module is used to collect log data from industrial production processes; The second module is used to report the log data to the buffer service to process the log data and store the processed log information into the buffer queue; The third module is used to consume the log information from the cache queue, receive a search request based on the log information, and display the search information.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images