Microservice-based log real-time monitoring method and system

Abstract

The invention discloses a log real-time monitoring method and system based on micro-service, and relates to the technical field of micro-service real-time monitoring, and the method comprises the steps: collecting equipment operation logs in real time, carrying out the data transmission through a Kafka message queue, storing the collected log data in a distributed database, and carrying out the real-time monitoring of the equipment operation logs. And training a fault prediction model of a convolutional neural network by using the extracted features, performing fault prediction by using the fault prediction model of the convolutional neural network, dynamically adjusting an alarm threshold according to real-time monitoring data and a model prediction result, establishing a communication mechanism between micro-services, and transmitting a monitoring result and a threshold adjustment instruction. And the monitoring data is converted into a visual format and is displayed through an interface, and operation and maintenance personnel carry out equipment monitoring and analyze the state of the power grid according to an interface display result. The problems of insufficient real-time performance, low fault prediction accuracy, low data processing efficiency, insufficient service collaboration, insufficient visual display and low resource utilization rate in a log monitoring system are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of microservice real-time monitoring, and in particular to a microservice-based log real-time monitoring method and system. Background Art

[0002] In today's context of rapid development of information technology, microservice architecture has been widely used in many fields due to its high cohesion and low coupling characteristics, especially in the field of log monitoring. Traditional log processing methods often have problems such as low processing efficiency and poor real-time performance. In recent years, with the continuous maturity of big data technology, microservice-based log monitoring technology has gradually become a research hotspot. In the existing technology, a log collection system is usually used to collect device operation logs in real time, and various message queue technologies are used for data transmission and storage. However, there are still limitations in data processing, fault prediction and service collaboration.

[0003] However, the existing related technologies have shortcomings in the following aspects: First, although the existing technologies can realize the real-time collection and transmission of log data, they often rely on traditional machine learning methods in feature extraction and the construction of fault prediction models. When these methods process large-scale, high-dimensional log data, the prediction accuracy and real-time performance are difficult to meet actual needs; secondly, the existing log monitoring system is relatively fixed in the setting of alarm thresholds, and cannot be dynamically adjusted according to real-time monitoring data and model prediction results, which affects the accuracy of fault warning; finally, in terms of communication mechanism and visual display between microservices, the existing technology has not yet formed a complete solution, which makes it difficult for operation and maintenance personnel to make decisions quickly when performing equipment monitoring and analyzing power grid status. In view of the above problems, the present invention proposes a real-time log monitoring method and system based on microservices, which is expected to bring significant beneficial effects in fault prediction, dynamic adjustment of thresholds and service collaboration. Summary of the invention

[0004] In view of the above-mentioned existing problems, the present invention provides a microservice-based real-time log monitoring method and system to solve the problems in the prior art in fault prediction, dynamic adjustment of thresholds and service collaboration.

[0005] To solve the above technical problems, a microservice-based log real-time monitoring method is proposed, including:

[0006] Collect the operation log data of the first device and store it in the database, perform feature extraction to obtain the second feature data, and use the second feature data to train the fault model; analyze the operation log data of the first device, and use the fault model to output the third prediction data, and adjust the alarm threshold according to the third prediction data; transmit the third prediction data, and collaborate between microservices to analyze the power grid status through the monitoring interface.

[0007] As a preferred solution of the microservice-based log real-time monitoring method described in the present invention, wherein: the collection of the first device operation log includes real-time collection of device status data, voltage data, current data, frequency data, historical fault data, operation logs, environmental data and time series data;

[0008] The equipment status data includes the equipment's operating status, load conditions, temperature and humidity; the fault history data includes fault type, occurrence time and duration; the operation log includes power on and off time, maintenance records and configuration changes; the environmental data includes temperature, humidity and air pressure.

[0009] As a preferred solution of the microservice-based real-time log monitoring method described in the present invention, the storage in the database includes transmitting the collected first device operation data through a message queue and storing the data in a distributed database.

[0010] The distributed database includes data nodes, data sharding, data replication, distributed transaction management, data consistency guarantee, query processing, fault recovery, load balancing, data synchronization and network communication;

[0011] The data nodes include storage nodes and processing nodes; the data sharding includes horizontal sharding and vertical sharding; the data replication includes master-slave replication, multi-master replication, synchronous replication and asynchronous replication; the distributed transaction management includes transaction coordinator and two-phase commit; the data consistency guarantee includes strong consistency and eventual consistency; the query processing includes query optimizer and distributed execution plan; the fault recovery includes fault detection and failover; the load balancing includes data load balancing and query load balancing; the network communication includes internal network and external network.

[0012] As a preferred solution of the microservice-based log real-time monitoring method described in the present invention, the feature extraction to obtain the second feature data includes extracting time series data, structured data, unstructured data, numerical features, category features, time features, statistical features and text features from a distributed database, preprocessing the extracted data, analyzing the statistical characteristics of the data and using visualization tools to understand the structure and characteristics of the data, using correlation analysis to select the features with the highest correlation with the target variable, and automatically selecting features through a recursive feature elimination algorithm, constructing new features according to domain knowledge and business logic and using time window technology to create time series-based features, using normalization to unify feature scales, encoding category features, and using the processed features to train a convolutional neural network fault prediction model.

[0013] As a preferred solution of the microservice-based log real-time monitoring method described in the present invention, wherein: the use of the second feature data to train the fault model includes: using the extracted second feature data to construct a convolutional neural network model, selecting a convolutional layer, a pooling layer and a fully connected layer, initializing weights, and training the convolutional neural network model;

[0014] The preprocessing includes data cleaning and data conversion; wherein data conversion refers to converting non-numerical data into numerical data using unique coding;

[0015] The correlation analysis includes collecting data sets of two variables to be compared, converting them into numerical values, and calculating the correlation using the Pearson correlation coefficient;

[0016] The Pearson correlation coefficient calculation formula is expressed as:

[0017]

[0018] Among them, r xy is the correlation coefficient between x and y, x i and i is a single observation in x and y, is the sample mean of x, is the sample mean of y, and n is the number of observations;

[0019] The recursive feature elimination algorithm formula is expressed as:

[0020]

[0021] Among them, Z t is the feature set retained after the tth iteration, is the least important feature eliminated in the t+1th iteration, Z t+1 is the feature set retained after the t+1th iteration.

[0022] As a preferred solution of the microservice-based real-time log monitoring method described in the present invention, wherein: the outputting of the third prediction data using the fault model and adjusting the alarm threshold according to the third prediction data includes analyzing the first log data collected in real time, and performing fault prediction using the fault prediction model of the convolutional neural network, and obtaining the third prediction data, and dynamically adjusting the alarm threshold according to the third prediction result;

[0023] According to the set initial alarm threshold T0, F(c) is calculated at each time point c, and F(c) is compared with the current alarm threshold T x , when F(c)>T x When F(c) <T xWhen the alarm threshold is lowered, the threshold adjustment should follow the dynamic adjustment rules:

[0024] T new =T x +α·(F(c)-T x )

[0025] Among them, T new is the alarm threshold after dynamic adjustment, T x is the current alarm threshold, α is the dynamic adjustment factor, and F(c) is the alarm threshold at time point c.

[0026] As a preferred solution of the microservice-based log real-time monitoring method described in the present invention, the analysis of the power grid status through the monitoring interface includes using a message queue as an intermediary for communication between microservices, defining a message format and a protocol, and establishing a communication mechanism. The log monitoring service sends the collected data and the calculated F(c) to the message queue, the alarm threshold adjustment service listens to the message queue, receives the monitoring results, and sends the threshold adjustment instructions to the remaining microservices. The microservice updates the internal alarm threshold according to the received threshold adjustment instructions. The data visualization service subscribes to the message queue, receives real-time monitoring data and threshold adjustment instructions, and converts the data into a visualization format. The front-end interface obtains updated data from the data visualization service through a polling mechanism, and the operation and maintenance personnel monitor the equipment status and analyze the power grid status through the interface.

[0027] Another object of the present invention is to provide a microservice-based real-time log monitoring system, which optimizes the scalability, flexibility, stability and data analysis capabilities of the system; the system of the present invention obtains real-time log data, fault prediction, monitoring and early warning in real time, and dynamically adjusts the alarm threshold, thereby solving the problems of insufficient real-time performance, low fault prediction accuracy, static alarm threshold, poor system scalability, low data processing efficiency, insufficient service collaboration, insufficient visualization display and low resource utilization in the log monitoring system, and significantly improves the real-time performance, prediction accuracy, system flexibility and operation and maintenance efficiency of log monitoring.

[0028] As a preferred solution of the microservice-based log real-time monitoring system described in the present invention, it is characterized by comprising a log data collection and transmission module, a feature extraction and preprocessing module, a fault prediction model training module, a fault prediction and threshold adjustment module, and an inter-microservice communication and collaboration module;

[0029] The log data collection and transmission module is used to collect device operation logs in real time and transmit the data through a message queue, store the collected log data in a distributed database, and perform data management;

[0030] The feature extraction and preprocessing module is used to extract features from the distributed database and perform data preprocessing;

[0031] The fault prediction model training module is used to train a convolutional neural network model with the extracted feature data;

[0032] The fault prediction and threshold adjustment module is used for fault prediction and dynamic adjustment of alarm thresholds;

[0033] The inter-microservice communication and collaboration module is used to establish a communication mechanism between microservices, transmit monitoring results and threshold adjustment instructions, convert monitoring data into a visual format, and display it through an interface for operation and maintenance personnel to monitor and analyze.

[0034] A computer device includes a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of a method described in real-time log monitoring based on microservices are implemented.

[0035] A computer-readable storage medium stores a computer program thereon, wherein when the computer program is executed by a processor, the steps of a method described in real-time log monitoring based on microservices are implemented.

[0036] Beneficial effects of the present invention: The present invention collects equipment operation logs in real time and uses Kafka message queues for data transmission, thereby ensuring the immediacy and transmission efficiency of monitoring data, improving the timeliness of the system and the stability of data transmission; storing log data in a distributed database enhances the reliability and processing capability of data management; feature extraction and convolutional neural network model training significantly improve the accuracy of fault prediction and realize intelligent fault warning; dynamically adjust the alarm threshold to make the alarm system more adaptive and intelligent; establish a communication mechanism between microservices to promote service collaboration and optimize system operation efficiency; provide an intuitive monitoring interface for operation and maintenance personnel through data visualization, and improve the rapidity and accuracy of decision-making. The present invention has achieved significant beneficial effects in improving monitoring real-time, prediction accuracy, system flexibility and operation and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0038] Figure 1 An overall flow chart of a microservice-based real-time log monitoring method provided by an embodiment of the present invention.

[0039] Figure 2 A system solution flow chart of a microservice-based real-time log monitoring system provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0042] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it an embodiment that is mutually exclusive with other embodiments, either individually or selectively.

[0043] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0044] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0045] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0046] Example 1, reference Figure 1 , which is the first embodiment of the present invention, and provides a microservice-based log real-time monitoring method, comprising:

[0047] S1: Collect equipment operation logs in real time and transmit data through Kafka message queues. The collected log data is stored in a distributed database and features are extracted from it. The extracted features are used to train the fault prediction model of the convolutional neural network.

[0048] The real-time collection of equipment operation logs includes real-time collection of equipment status data, voltage data, current data, frequency data, historical fault data, operation logs, environmental data and time series data;

[0049] The equipment status data includes the equipment's operating status, load conditions, temperature and humidity; the fault history data includes fault type, occurrence time and duration; the operation log includes power on and off time, maintenance records and configuration changes; the environmental data includes temperature, humidity and air pressure.

[0050] Storing the collected log data in the distributed database includes transmitting the collected device operation data through the Kafka message queue and storing the data in the distributed database;

[0051] The distributed database includes data nodes, data sharding, data replication, distributed transaction management, data consistency guarantee, query processing, fault recovery, load balancing, data synchronization and network communication;

[0052] The data nodes include storage nodes and processing nodes; the data sharding includes horizontal sharding and vertical sharding; the data replication includes master-slave replication, multi-master replication, synchronous replication and asynchronous replication; the distributed transaction management includes transaction coordinator and two-phase commit; the data consistency guarantee includes strong consistency and eventual consistency; the query processing includes query optimizer and distributed execution plan; the fault recovery includes fault detection and failover; the load balancing includes data load balancing and query load balancing; the network communication includes internal network and external network.

[0053] The feature extraction includes extracting time series data, structured data, unstructured data, numerical features, category features, time features, statistical features and text features from a distributed database, preprocessing the extracted data, analyzing the statistical characteristics of the data and using visualization tools to understand the structure and characteristics of the data, using correlation analysis to select the features with the highest correlation with the target variable, and automatically selecting features through a recursive feature elimination algorithm, constructing new features based on domain knowledge and business logic and using time window technology to create time series-based features, using normalization to unify feature scales, and encoding category features, and using the processed features to train a convolutional neural network fault prediction model;

[0054] The preprocessing includes data cleaning and data conversion; wherein data conversion refers to converting non-numerical data into numerical data using unique coding;

[0055] The correlation analysis includes collecting data sets of two variables to be compared, converting them into numerical values, and calculating the correlation using the Pearson correlation coefficient;

[0056] The Pearson correlation coefficient calculation formula is expressed as:

[0057]

[0058] Among them, r xy is the correlation coefficient between x and y, x i and i is a single observation in x and y, is the sample mean of x, is the sample mean of y, and n is the number of observations;

[0059] The recursive feature elimination algorithm formula is expressed as:

[0060]

[0061] Among them, Z t is the feature set retained after the tth iteration, is the least important feature eliminated in the t+1th iteration, Z t+1 is the feature set retained after the t+1th iteration.

[0062] The training of the convolutional neural network fault prediction model includes constructing a convolutional neural network model using the extracted feature data, selecting a convolutional layer, a pooling layer, and a fully connected layer, initializing weights, and training the convolutional neural network model;

[0063] The training of the convolutional neural network model includes forward propagation, loss calculation, back propagation and weight update;

[0064] The forward propagation formula is expressed as:

[0065] Y=f(WX+b)

[0066] Among them, Y is the output data, W is the weight, X is the input data, b is the bias, and f is the ReLU activation function;

[0067] Among them, the ReLU activation function is expressed as:

[0068] ReLU(a)=max(0,a)

[0069] Among them, ReLU(a) is the output value of the activation function, and a is the input value;

[0070] The formula for calculating the loss is expressed as:

[0071]

[0072] Among them, L is the loss value, y i is the true label, p i is the predicted probability;

[0073] The formula for back propagation is expressed as:

[0074]

[0075] Among them, L is the loss value, Y is the output data, Z is the input of the ReLU activation function, and W is the weight;

[0076] The weight update formula is expressed as:

[0077]

[0078] Among them, W′ is the updated weight, W is the weight, α is the learning rate, is the gradient of loss with respect to the weights.

[0079] S2: Analyze the log data collected in real time, and use the fault prediction model of the convolutional neural network to predict faults, and dynamically adjust the alarm threshold based on the real-time monitoring data and model prediction results;

[0080] The fault prediction includes analyzing the log data collected in real time, and using the fault prediction model of the convolutional neural network to predict the fault, and dynamically adjusting the alarm threshold according to the real-time monitoring data and the model prediction results;

[0081] The formula for obtaining the alarm threshold is:

[0082]

[0083] Among them, F(c) is the alarm threshold, c is the real-time monitoring time, l is the number of log data samples, m is the number of real-time monitoring data samples, q f is the fth log data sample, θ is the parameter of the convolutional neural network model, s o is the oth real-time monitoring data sample, g(q f ,θ) is the convolutional neural network fault prediction function, λ is the attenuation factor, h(q f ,μ) is the information filtering function, μ is the mean of the data set, σ is the standard deviation of the real-time monitoring data, z(s o ,σ) is the normalization function, exp is the exponential function;

[0084] The h(q f ,μ) The information filtering function is expressed as:

[0085]

[0086] Among them, q f is the fth log data sample, μ is the mean of the data set, and β is the standard deviation of the data;

[0087] According to the set initial alarm threshold T0, F(c) is calculated at each time point c, and F(c) is compared with the current alarm threshold T x , when F(c)>T x When F(c) <T x When the alarm threshold is lowered, the threshold adjustment should follow the dynamic adjustment rules:

[0088] T new =T x +α·(F(c)-T x )

[0089] Among them, T new is the alarm threshold after dynamic adjustment, T x is the current alarm threshold, α is the dynamic adjustment factor, and F(c) is the alarm threshold at time point c.

[0090] S3: Establish a communication mechanism between microservices to transmit monitoring results and threshold adjustment instructions, perform service collaboration, and convert monitoring data into a visual format. Display it through the interface, and operation and maintenance personnel can monitor equipment and analyze the power grid status based on the results displayed on the interface.

[0091] The device monitoring and power grid status analysis includes: using a message queue as an intermediary for communication between microservices, defining a message format and a protocol, and establishing a communication mechanism; the log monitoring service sends the collected data and the calculated F(c) to the message queue; the alarm threshold adjustment service monitors the message queue, receives monitoring results, and sends threshold adjustment instructions to the remaining microservices; the microservice updates the internal alarm threshold according to the received threshold adjustment instructions; the data visualization service subscribes to the message queue, receives real-time monitoring data and threshold adjustment instructions, and converts the data into a visualization format; the front-end interface obtains updated data from the data visualization service through a polling mechanism; and the operation and maintenance personnel monitor the device status and analyze the power grid status through the interface.

[0092] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0093] Embodiment 2, the second embodiment of the present invention, is different from the previous embodiment in that:

[0094] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0095] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0096] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0097] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0098] Example 3, reference Figure 2 , which is the third embodiment of the present invention, and provides a microservice-based log real-time monitoring system, including a log data collection and transmission module 100, a feature extraction and preprocessing module 200, a fault prediction model training module 300, a fault prediction and threshold adjustment module 400, and a microservice communication and collaboration module 500;

[0099] The log data collection and transmission module 100 is used to collect device operation logs in real time and transmit the data through a message queue, store the collected log data in a distributed database, and perform data management;

[0100] The feature extraction and preprocessing module 200 is used to extract features from the distributed database and perform data preprocessing;

[0101] The fault prediction model training module 300 is used to train a convolutional neural network model with the extracted feature data;

[0102] The fault prediction and threshold adjustment module 400 is used for fault prediction and dynamic adjustment of alarm thresholds;

[0103] The inter-microservice communication and collaboration module 500 is used to establish a communication mechanism between microservices, transmit monitoring results and threshold adjustment instructions, convert monitoring data into a visual format, and display it through an interface for operation and maintenance personnel to monitor and analyze.

[0104] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A real-time log monitoring method based on microservices, characterized by: include, Collecting the operation log data of the first device and storing it in a database, performing feature extraction to obtain second feature data, and using the second feature data to train a fault model; Analyze the operation log data of the first device, output third prediction data using the fault model, and adjust the alarm threshold according to the third prediction data; The third prediction data is transmitted, and the microservices work together to analyze the power grid status through the monitoring interface.

2. A microservice-based log real-time monitoring method as claimed in claim 1, characterized in that: The collecting of the first device operation log includes collecting device status data, voltage data, current data, frequency data, historical fault data, operation log, environmental data and time series data in real time; The equipment status data includes the equipment's operating status, load conditions, temperature and humidity; the fault history data includes fault type, occurrence time and duration; the operation log includes power on and off time, maintenance records and configuration changes; the environmental data includes temperature, humidity and air pressure.

3. A microservice-based log real-time monitoring method as claimed in claim 2, characterized in that: The storing in the database includes transmitting the collected first device operation data through a message queue and storing the data in a distributed database; The distributed database includes data nodes, data sharding, data replication, distributed transaction management, data consistency guarantee, query processing, fault recovery, load balancing, data synchronization and network communication; The data nodes include storage nodes and processing nodes; the data sharding includes horizontal sharding and vertical sharding; the data replication includes master-slave replication, multi-master replication, synchronous replication and asynchronous replication; the distributed transaction management includes transaction coordinator and two-phase commit; the data consistency guarantee includes strong consistency and eventual consistency; The query processing includes a query optimizer and a distributed execution plan; the fault recovery includes fault detection and fault transfer; the load balancing includes data load balancing and query load balancing; and the network communication includes an internal network and an external network.

4. A microservice-based log real-time monitoring method as claimed in claim 3, characterized in that: The feature extraction to obtain the second feature data includes extracting time series data, structured data, unstructured data, numerical features, category features, time features, statistical features and text features from a distributed database, preprocessing the extracted data, analyzing the statistical characteristics of the data and using visualization tools to understand the structure and characteristics of the data, using correlation analysis to select the features with the highest correlation with the target variable, and automatically selecting features through a recursive feature elimination algorithm, constructing new features based on domain knowledge and business logic and using time window technology to create time series-based features, using normalization to unify feature scales, encoding category features, and using the processed features to train a convolutional neural network fault prediction model; The preprocessing includes data cleaning and data conversion; wherein data conversion refers to converting non-numerical data into numerical data using unique coding; The correlation analysis includes collecting data sets of two variables to be compared, converting them into numerical values, and calculating the correlation using the Pearson correlation coefficient; The Pearson correlation coefficient calculation formula is expressed as: Among them, r xy is the correlation coefficient between x and y, x i and i is a single observation in x and y, is the sample mean of x, is the sample mean of y, and n is the number of observations; The recursive feature elimination algorithm formula is expressed as: Among them, Z t is the feature set retained after the tth iteration, is the least important feature eliminated in the t+1th iteration, Z t+1 is the feature set retained after the t+1th iteration.

5. A microservice-based log real-time monitoring method as claimed in claim 4, characterized in that: The method of training the fault model using the second feature data includes constructing a convolutional neural network model using the extracted second feature data, selecting a convolutional layer, a pooling layer and a fully connected layer, initializing weights, and training the convolutional neural network model.

6. A microservice-based log real-time monitoring method as claimed in claim 5, characterized in that: The method of outputting the third prediction data using the fault model and adjusting the alarm threshold according to the third prediction data includes analyzing the first log data collected in real time, performing fault prediction using the fault prediction model of the convolutional neural network, obtaining the third prediction data, and dynamically adjusting the alarm threshold according to the third prediction result; According to the set initial alarm threshold T0, F(c) is calculated at each time point c, and F(c) is compared with the current alarm threshold T x , when F(c)>T x When F(c) <T x When the alarm threshold is lowered, the threshold adjustment should follow the dynamic adjustment rules: T new =T x +α·(F(c)-T x ) Among them, T new is the alarm threshold after dynamic adjustment, T x is the current alarm threshold, α is the dynamic adjustment factor, and F(c) is the alarm threshold at time point c.

7. A microservice-based log real-time monitoring method as claimed in claim 6, characterized in that: The analysis of the power grid status through the monitoring interface includes using a message queue as an intermediary for communication between microservices, defining a message format and protocol, and establishing a communication mechanism. The log monitoring service sends the collected data and the calculated F(c) to the message queue. The alarm threshold adjustment service monitors the message queue, receives monitoring results, and sends threshold adjustment instructions to the remaining microservices. The microservice updates the internal alarm threshold according to the received threshold adjustment instructions. The data visualization service subscribes to the message queue, receives real-time monitoring data and threshold adjustment instructions, and converts the data into a visualization format. The front-end interface obtains updated data from the data visualization service through a polling mechanism. The operation and maintenance personnel monitor the equipment status and analyze the power grid status through the interface.

8. A system using a microservice-based real-time log monitoring method as claimed in any one of claims 1 to 7, characterized in that: It includes log data collection and transmission module, feature extraction and preprocessing module, fault prediction model training module, fault prediction and threshold adjustment module, and microservice communication and collaboration module; The log data collection and transmission module is used to collect device operation logs in real time and transmit the data through a message queue, store the collected log data in a distributed database, and perform data management; The feature extraction and preprocessing module is used to extract features from the distributed database and perform data preprocessing; The fault prediction model training module is used to train a convolutional neural network model with the extracted feature data; The fault prediction and threshold adjustment module is used for fault prediction and dynamic adjustment of alarm thresholds; The inter-microservice communication and collaboration module is used to establish a communication mechanism between microservices, transmit monitoring results and threshold adjustment instructions, convert monitoring data into a visual format, and display it through an interface for operation and maintenance personnel to monitor and analyze.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a microservice-based real-time log monitoring method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a microservice-based real-time log monitoring method according to any one of claims 1 to 7 are implemented.

Images