Log collection method and log collection system

Abstract

The present application discloses a log collection method and a log collection system, the method comprising: receiving a start request for a target service, starting the target service with a log topic corresponding to the target service; outputting the log generated during the operation of the target service to a preset log storage directory corresponding to the target service; each log includes service operation unit information, and the service operation unit information includes the identification of the log topic corresponding to the target service; according to the service operation unit information of the newly added log in the preset log storage directory, the identification of the log topic and log data of the newly added log are parsed; and the parsed log data and the identification of the log topic are sent to the Kafka message system. Collecting logs using the identification of the log topic is conducive to reducing the complexity of subsequent use and processing of the logs, and reducing the difficulty of applying the logs. In addition, the PaaS platform can directly connect to the Kafka message system, so the log collection method has high practicality.

Description

Technical Field

[0001] The present application relates to the field of operation and maintenance technology, and in particular to a log collection method and a log collection system. Background Art

[0002] Logs refer to some information related to the operation of an application that is recorded during the operation of the application. In the function service, the application is only responsible for the business logic, and does not consider the server and related operation and maintenance. Therefore, the PaaS platform needs to collect logs for the application. In the existing technology, the logs generated by the operation of different applications are mixed together, which makes the subsequent use and processing of the logs very complicated. It brings great difficulties to the application of log retrieval, query, analysis, etc. In addition, it is difficult to achieve clear and definite log permission management.

[0003] Among the existing log collection technologies, the Knative log system uses the ElasticSearch plug-in at the sending end, and the Knative log system can only send logs to the ElasticSearch engine. In other words, the Knative log system can only be connected to the ElasticSearch engine. However, in actual applications, it is often necessary to collect logs using the Kafka messaging system. Since the existing Knative log system cannot be connected to the Kafka messaging system, the practicality of this collection solution is poor. Summary of the invention

[0004] Based on the above problems, the present application provides a log collection method and a log collection system to reduce the complexity of subsequent use and processing of logs, reduce the difficulty of log application, and improve the practicality of log collection.

[0005] The embodiments of the present application disclose the following technical solutions:

[0006] In a first aspect, the present application provides a log collection method, which is applied to a PaaS platform and includes:

[0007] Receive a start request for a target service, and start the target service with a log topic corresponding to the target service;

[0008] Output the logs generated during the operation of the target service to the preset log storage directory corresponding to the target service; each log includes service operation unit information, and the service operation unit information includes the identifier of the log subject corresponding to the target service;

[0009] According to the service operation unit information of the newly added log in the preset log storage directory, the log subject identifier and log data of the newly added log are parsed;

[0010] Send the parsed log data and the log topic identifier to the Kafka messaging system.

[0011] Optionally, the target service is started on a target server node; a log agent module is deployed on the target server node;

[0012] According to the service operation unit information of the newly added log in the preset log storage directory, the log subject identifier and log data of the newly added log are parsed, including:

[0013] The preset log storage directory is monitored by the log proxy module. When a new log is detected in the preset log storage directory, the log proxy module parses the log subject identifier and log data of the new log according to the service operation unit information of the new log.

[0014] Send the parsed log data and log topic identifier to the Kafka messaging system, including:

[0015] The parsed log data and log topic identifier are sent to the Kafka message system through the log agent module.

[0016] Optionally, the log agent module is implemented by a Fluentd program; the Fluentd program uses the Kafka message system as a log output terminal.

[0017] Optionally, the PaaS platform is implemented through the Kubernetes engine; the PaaS platform includes at least one server node, and the at least one server node includes a target server node; a replica service running unit corresponding to the DemeonSet resource is deployed on each of the at least one server node; the replica service running unit serves as a log agent module on the server node, and is used to monitor the logs of all service running units on the server node.

[0018] Optionally, before receiving the start request for the target service, the method further includes:

[0019] Receiving configuration information of a log topic corresponding to a target service; the configuration information of the log topic includes an identifier of the log topic; the identifier indicates a target log topic;

[0020] Configure the target log topic as the log topic corresponding to the target service according to the configuration information;

[0021] Start the target service with the log topic corresponding to the target service, including:

[0022] Start the target service with the target log topic.

[0023] Optionally, before receiving the start request for the target service, the method further includes:

[0024] Receive deployment parameters for a target service; the deployment parameters include: first identification information of an artificial intelligence model;

[0025] Storing an image of the artificial intelligence model; the image of the artificial intelligence model is one-to-one with the first identification information;

[0026] The start request includes: second identification information; after receiving the start request for the target service, it also includes:

[0027] Determine whether the second identification information is consistent with the first identification information. If so, run the image of the artificial intelligence model; the image is used to provide various services of the PaaS platform during operation.

[0028] Optionally, the method further comprises:

[0029] Receive a business request for the target service.

[0030] Optionally, the log topic received by the Kafka messaging system is used as the basis for performing any of the following operations on the log data:

[0031] Storage, query, analysis or permission control.

[0032] In a second aspect, the present application provides a log collection system, including: a PaaS platform and a Kafka message system, wherein the PaaS platform and the Kafka message system are in communication connection;

[0033] The PaaS platform is used to receive a start request for a target service, start the target service with a log topic corresponding to the target service; output the logs generated during the operation of the target service to a preset log storage directory corresponding to the target service; each log includes service operation unit information, and the service operation unit information includes an identifier of the log topic corresponding to the target service; parse the identifier of the log topic and log data of the newly added log according to the service operation unit information of the newly added log in the preset log storage directory; and send the parsed log data and the identifier of the log topic to the Kafka message system;

[0034] The Kafka message system is used to receive log data and log topic identifiers sent by the PaaS platform.

[0035] Optionally, the PaaS platform includes at least one server node; a log agent module is deployed on each of the at least one server node; at least one server node includes a target server node; the target service is started on the target server node;

[0036] The log agent module deployed on the target server node is specifically used to monitor the preset log storage directory. When it is monitored that there is a new log in the preset log storage directory, the log topic identifier and log data of the new log are parsed according to the service operation unit information of the new log, and the parsed log data and log topic identifier are sent to the Kafka message system.

[0037] Compared with the prior art, this application has the following beneficial effects:

[0038] The present application provides a log collection method and a log collection system, which are applied to the PaaS platform, including: receiving a start request for a target service, starting the target service with a log topic corresponding to the target service; outputting the logs generated during the operation of the target service to the preset log storage directory corresponding to the target service; each log includes service operation unit information, and the service operation unit information includes the identification of the log topic corresponding to the target service; according to the service operation unit information of the newly added log in the preset log storage directory, the identification of the log topic of the newly added log and the log data are parsed; and the parsed log data and the identification of the log topic are sent to the Kafka message system. In the present application, since the target service is started with its corresponding log topic, the logs generated during the operation of the target service can carry the identification of the log topic corresponding to the target service, and the identification of the log data and the log topic can be obtained by parsing the service operation unit information of the log. The log data is sent to the Kafka message system for collection together with the representation of the log topic, so that Kafka can collect the topic identification of the log generated by the target service and build the association relationship between the log of the target service and the log topic identification. Collecting logs using the identification of the log topic is conducive to reducing the complexity of subsequent use and processing of logs and reducing the difficulty of log application. In addition, the PaaS platform can be directly connected to the Kafka messaging system, so the log collection method is highly practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 A schematic diagram of an application scenario of the log collection method provided in an embodiment of the present application;

[0041] Figure 2 A flow chart of a log collection method provided in an embodiment of the present application;

[0042] Figure 3 A flowchart of another log collection method provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the structure of a log collection system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to facilitate understanding of the technical solution of the present application, the meanings of some technical terms involved in the present application are first introduced.

[0045] PaaS: refers to Platform as a Service, which provides infrastructure (including network, server, operating system or storage) as a service to users. Users do not need to manage and control the cloud infrastructure.

[0046] Kubernetes: A container orchestration engine open sourced by Google.

[0047] Pod, Node and Kubernetes: Kubernetes is an open source container orchestration engine from Google. Pod is the unit used by Kubernetes to run containers. Pod always runs on Node. Node is a working machine in Kubernetes, usually a virtual machine or physical machine.

[0048] DemeonSet: A type of resource in Kubernetes. Kubernetes ensures that only one replica Pod corresponding to the DemeonSet resource runs on each server node. In this patent, the replica is responsible for collecting logs of all Pods on this node.

[0049] Fluentd: An open source data collector for a unified log layer. Fluentd uses a pluggable architecture, and the community provides dozens of data sources and data output plug-ins to connect to different data sources and data storage systems.

[0050] Kafka: It is a distributed messaging system developed by Linked In. It is widely used for its horizontal scalability and high throughput.

[0051] ElasticSearch: A distributed, highly scalable, and highly real-time search and data analysis engine.

[0052] As described above, the Knative log system used in the current log collection technology solution can usually only be connected to ElasticSearch, and cannot be connected to the Kafka messaging system, so there is a problem of insufficient practicality. In addition, the current log collection technology solution has the problem of inconvenient post-processing and use of log collection.

[0053] Based on the above problems, the inventor has provided a log collection method and a log collection system in this application after research. By configuring the log topic, the log data generated during the service operation process and the log topic are associated with each other, thereby reducing the complexity of post-processing and use of log collection. In addition, the PaaS platform that implements the log collection method can be connected to the Kafka messaging system, which improves the practicality of the technical solution.

[0054] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] Method Embodiment

[0056] Figure 1 A schematic diagram of an application scenario of the log collection method provided in the embodiment of the present application. Figure 1 As shown in FIG. 1 , the application scenario includes: developer users, end users, PaaS platform and Kafka system. The PaaS platform deploys at least one server node, including a target server node that supports the startup and operation of the target service.

[0057] The PaaS platform can provide an interactive interface for developers to use. Developers can configure the log topics corresponding to each service on the PaaS platform. In one possible implementation, the same type of log topics are configured for each service of an application. In actual applications, the log topics of the service can be configured based on the developer's experience in classifying log data, or they can be set by the application providing the service. There are restrictions on the types of configured log topics here.

[0058] After the developer user completes the configuration of the log topic of the target service, he can send a start request to the PaaS platform to request the start of the target service. After receiving the start request for the target service, the PaaS platform can start the target service using the log topic corresponding to the target service.

[0059] The terminal user can communicate with the PaaS platform through his own terminal device. The terminal user can send a business request for the target service to the PaaS platform through his own terminal device. The target server node of the PaaS platform runs the target service according to the business request. Logs are generated when the target service runs. The target server node can output the logs generated by the operation of the target service to the preset log storage directory corresponding to the target service in the target server node. In one possible implementation, each service has a corresponding preset log storage directory; in another possible implementation, a server node has only one preset log storage directory, and the logs of all services running on the server node are output to the preset log storage directory.

[0060] Each server node on the PaaS platform is deployed with a log agent module. The log agent module is used to monitor the preset log storage directory on the server node. When a new log is added to the directory, the log agent module parses the log topic identifier and log data. Finally, the log agent module sends the parsed identifier and log data to the Kafka messaging system so that the Kafka messaging system can collect log data according to the identifier classification.

[0061] Figure 2 A flow chart of a log collection method provided in an embodiment of the present application. The method uses the PaaS platform as the execution subject.

[0062] like Figure 2 As shown, the log collection method includes:

[0063] S201: receiving a request to start a target service, and starting the target service with a log topic corresponding to the target service.

[0064] Developer users can enter or select deployment parameters for the target service on the PaaS platform, and enter or select configuration information for the log topic of the target service. The configuration information includes an identifier of the log topic, which indicates a target log topic. For example, the log topic identifier included in the configuration information is A, indicating that the selected target log topic is Topic A, and Topic A is used as the target log topic. The logs generated by the subsequent startup of the target service and the operation of the target service all use Topic A as the log topic.

[0065] The deployment parameters may include the first identification information of the artificial intelligence model. An image of the artificial intelligence model is pre-stored. The first identification information also corresponds one-to-one to the image of the artificial intelligence model. If the second identification information included in the startup request is consistent with the first identification information, it can be determined based on the second identification information that it corresponds to the stored image of the artificial intelligence model. That is, when identification information (such as an ID) that matches the image of the artificial intelligence model is provided in the startup request, the image of the artificial intelligence model can be run on the PaaS platform. When the image is running, various services of the PaaS platform can be provided.

[0066] After the PaaS platform receives the developer user's request to start the target service, based on the log topic previously configured by the developer user for the target service and after the model image is running, it can use the log topic corresponding to the target service to start the target service requested by the developer user.

[0067] S202: Outputting the logs generated during the operation of the target service to a preset log storage directory corresponding to the target service.

[0068] Each log generated includes service operation unit information (Pod information), and the service operation unit information includes the identification of the log subject corresponding to the target service. Since the log is output to the preset log storage directory, the Pod information of the log is also located in the preset log storage directory.

[0069] S203: According to the service operation unit information of the newly added log in the preset log storage directory, parse out the log subject identifier and log data of the newly added log.

[0070] This step can be performed in real time, for example, S203 is executed once for each new log. Of course, it can also be performed regularly, for example, S203 is executed once every 30 minutes. S203 can also be executed once for each new N logs, where N is a positive integer. The execution frequency of S203 is not limited here.

[0071] Since the Pod information includes the identifier of the log topic corresponding to the target service, the identifier of the log topic can be parsed according to the Pod information, and the log data can be parsed.

[0072] S204: Send the parsed log data and the log topic identifier to the Kafka message system.

[0073] In a possible implementation, the parsed log data and the log topic identifier of the log may be packaged and sent or bound and then sent to a Kafka message system.

[0074] The above is a log collection method provided by an embodiment of the present application. In the log collection method provided by the present application, since the target service is started with its corresponding log topic, the logs generated during the operation of the target service can carry the identification of the log topic corresponding to the target service, and the identification of the log data and the log topic can be obtained by parsing the service operation unit information of the log. The log data is sent to the Kafka message system for collection together with the representation of the log topic, so that Kafka can collect the topic identification of the log generated by the target service and build the association relationship between the log of the target service and the log topic identification. Collecting logs using the identification of the log topic is conducive to reducing the complexity of the subsequent use and processing of the log and reducing the application difficulty of the log. In addition, the PaaS platform can directly connect to the Kafka message system, so the log collection method has high practicality.

[0075] In a possible implementation of the present application, a log proxy module is deployed on each server node of the PaaS platform. The log proxy module is used to monitor, parse and send new logs in the directory.

[0076] Therefore, the aforementioned S203 specifically includes: monitoring the preset log storage directory through the log proxy module, when monitoring the presence of a new log in the preset log storage directory, the log proxy module parses the log subject identifier and log data of the new log according to the service operation unit information of the new log;

[0077] The aforementioned S204 specifically includes: sending the parsed log data and the identifier of the log topic to the Kafka message system through the log proxy module.

[0078] The log proxy module can be implemented through the Fluentd program. That is, the log proxy module is built through the Fluentd program. The Fluentd program uses the Kafka message system as the log output terminal, that is, the log proxy module uses the Kafka message system as the log output terminal to output the parsed log data and the identifier of the log topic.

[0079] In a possible implementation, the PaaS platform is implemented through the Kubernetes engine, but is not limited to the Kubernetes engine. A replica service operation unit Pod corresponding to the DemeonSet resource is deployed on each server node of the PaaS platform. The replica Pod serves as a log agent module on the server node where it is located, and is used to monitor the logs of all service operation units on the server node.

[0080] Since the Kafka message system receives the log data parsed by the PaaS platform and the identifier of the log topic corresponding to the target service, it can store, query, analyze or control the log data based on the log topic indicated by the identifier.

[0081] For example, log data is classified according to the log subject identifiers A, B, and C, which are log data set A, log data set B, and log data set C, respectively. Log data set A, log data set B, and log data set C are stored and collected separately without interfering with each other. When querying, log data can be queried based on identifiers A, B, and C. When log data needs to be analyzed, it can be analyzed based on the subject classification of the log data to be analyzed. When a user only has the authority to obtain log data with subject identifier A, log data set B and log data set C are not open to the user.

[0082] Figure 3 The flow of another log collection method is illustrated, which can be understood by referring to the description in the above embodiment.

[0083] Based on the log collection method provided in the above embodiment, the present application also provides a log collection system. The implementation of the system is described below in conjunction with the accompanying drawings and embodiments.

[0084] System Example

[0085] Figure 4 A structural diagram of a log collection system provided in an embodiment of the present application. Figure 4 As shown, the log collection system includes: PaaS platform and Kafka message system, and the PaaS platform and Kafka message system are connected in communication;

[0086] The PaaS platform is used to receive a start request for a target service, start the target service with a log topic corresponding to the target service; output the logs generated during the operation of the target service to a preset log storage directory corresponding to the target service; each log includes service operation unit information, and the service operation unit information includes an identifier of the log topic corresponding to the target service; parse the identifier of the log topic and log data of the newly added log according to the service operation unit information of the newly added log in the preset log storage directory; and send the parsed log data and the identifier of the log topic to the Kafka message system;

[0087] The Kafka message system is used to receive log data and log topic identifiers sent by the PaaS platform.

[0088] In the present application, since the target service is started with its corresponding log topic, the logs generated during the operation of the target service can carry the identification of the log topic corresponding to the target service, and the identification of the log data and the log topic can be obtained by parsing the service operation unit information of the log. The log data is sent to the Kafka message system for collection together with the representation of the log topic, so that Kafka can collect the topic identification of the log generated by the target service and build the association relationship between the log of the target service and the log topic identification. Collecting logs using the identification of the log topic is conducive to reducing the complexity of the subsequent use and processing of the log and reducing the difficulty of applying the log. In addition, the PaaS platform can directly connect to the Kafka message system, and the log collection system has high practicality.

[0089] Optionally, the PaaS platform includes at least one server node; a log agent module is deployed on each of the at least one server node; at least one server node includes a target server node; the target service is started on the target server node;

[0090] The log agent module deployed on the target server node is specifically used to monitor the preset log storage directory. When it is monitored that there is a new log in the preset log storage directory, the log topic identifier and log data of the new log are parsed according to the service operation unit information of the new log, and the parsed log data and log topic identifier are sent to the Kafka message system.

[0091] Optionally, the log agent module is implemented by a Fluentd program; the Fluentd program uses the Kafka message system as a log output terminal.

[0092] Optionally, the PaaS platform is further used to receive configuration information of a log topic corresponding to a target service; the configuration information of the log topic includes an identifier of the log topic; the identifier indicates a target log topic; and the target log topic is configured as a log topic corresponding to the target service according to the configuration information;

[0093] The PaaS platform is specifically used to start the target service with the target log topic.

[0094] Optionally, the PaaS platform is in communication connection with the user terminal device; the PaaS platform is also used to receive a business request for a target service from the user terminal device.

[0095] Optionally, the log topic received by the Kafka messaging system is used as the basis for performing any of the following operations on the log data:

[0096] Storage, query, analysis or permission control.

[0097] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A log collection method, characterized in that: The method is applied to the PaaS platform, including: Receive a request to start a target service, and start the target service with a log topic corresponding to the target service; Outputting the logs generated during the operation of the target service to a preset log storage directory corresponding to the target service; each log includes service operation unit information, and the service operation unit information includes an identifier of a log subject corresponding to the target service; According to the service operation unit information of the newly added log in the preset log storage directory, parsing out the identifier of the log subject and log data of the newly added log; Send the parsed log data and log topic identifier to the Kafka messaging system; The target service is started on the target server node; a log agent module is deployed on the target server node; The step of parsing the log subject identifier and log data of the newly added log according to the service operation unit information of the newly added log in the preset log storage directory specifically includes: The preset log storage directory is monitored by the log proxy module. When a new log is detected in the preset log storage directory, the log proxy module parses the log subject identifier and log data of the new log according to the service operation unit information of the new log. The sending of the parsed log data and the identifier of the log topic to the Kafka message system specifically includes: Send the parsed log data and the identification of the log topic to the Kafka message system through the log proxy module; The log proxy module is implemented by the Fluentd program; the Fluentd program uses the Kafka message system as the log output terminal; Before receiving the start request for the target service, the method further includes: receiving configuration information of a log topic corresponding to the target service; the configuration information of the log topic includes an identifier of the log topic; the identifier indicates a target log topic; Configuring the target log topic as the log topic corresponding to the target service according to the configuration information; Before receiving the start request for the target service, the method further includes: Receiving deployment parameters for the target service; the deployment parameters include: first identification information of the artificial intelligence model; Pre-storing an image of the artificial intelligence model; the image of the artificial intelligence model corresponds one-to-one to the first identification information; The start request includes: second identification information; after receiving the start request for the target service, further including: Determine whether the second identification information is consistent with the first identification information, and if so, run the image of the artificial intelligence model; the image is used to provide various services of the PaaS platform during operation; The starting the target service with the log topic corresponding to the target service specifically includes: The target service is started with the target log topic.

2. The log collection method according to claim 1, characterized in that: The PaaS platform is implemented through the Kubernetes engine; the PaaS platform includes at least one server node, and the at least one server node includes the target server node; a replica service running unit corresponding to the DemeonSet resource is deployed on each of the at least one server node; the replica service running unit serves as a log agent module on the server node, and is used to monitor the logs of all service running units on the server node.

3. The log collection method according to claim 1, characterized in that: The method further comprises: A service request for the target service is received.

4. The log collection method according to any one of claims 1 to 3, characterized in that: The log topic received by the Kafka messaging system is used as a basis for performing any of the following operations on the log data: Storage, query, analysis, and permission control.

5. A log collection system, characterized in that: include: PaaS platform and Kafka message system, the PaaS platform and the Kafka message system are in communication connection; The PaaS platform is used to receive a start request for a target service and start the target service with a log topic corresponding to the target service; Outputting the logs generated during the operation of the target service to a preset log storage directory corresponding to the target service; Each log includes service operation unit information, and the service operation unit information includes an identifier of a log subject corresponding to the target service; According to the service operation unit information of the newly added log in the preset log storage directory, the identifier of the log topic and the log data of the newly added log are parsed; and the parsed log data and the identifier of the log topic are sent to the Kafka message system; The Kafka message system is used to receive the log data and log topic identifier sent by the PaaS platform; The target service is started on the target server node; A log agent module is deployed on the target server node; The step of parsing the log subject identifier and log data of the newly added log according to the service operation unit information of the newly added log in the preset log storage directory specifically includes: The preset log storage directory is monitored by the log proxy module. When a new log is detected in the preset log storage directory, the log proxy module parses the log subject identifier and log data of the new log according to the service operation unit information of the new log. The sending of the parsed log data and the identifier of the log topic to the Kafka message system specifically includes: Send the parsed log data and the identification of the log topic to the Kafka message system through the log proxy module; The log proxy module is implemented by the Fluentd program; the Fluentd program uses the Kafka message system as the log output terminal; Before receiving the start request for the target service, the method further includes: receiving configuration information of a log topic corresponding to the target service; the configuration information of the log topic includes an identifier of the log topic; the identifier indicates a target log topic; Configuring the target log topic as the log topic corresponding to the target service according to the configuration information; Before receiving the start request for the target service, the method further includes: Receiving deployment parameters for the target service; the deployment parameters include: first identification information of the artificial intelligence model; Pre-storing an image of the artificial intelligence model; the image of the artificial intelligence model corresponds one-to-one to the first identification information; The start request includes: second identification information; after receiving the start request for the target service, further including: Determine whether the second identification information is consistent with the first identification information, and if so, run the image of the artificial intelligence model; the image is used to provide various services of the PaaS platform during operation; The starting the target service with the log topic corresponding to the target service specifically includes: The target service is started with the target log topic.

6. The log collection system according to claim 5, characterized in that: The PaaS platform includes at least one server node; each of the at least one server node is respectively deployed with a log agent module; the at least one server node includes a target server node; the target service is started on the target server node; The log agent module deployed on the target server node is specifically used to monitor the preset log storage directory. When a new log is detected in the preset log storage directory, the log subject identifier and log data of the new log are parsed according to the service operation unit information of the new log, and the parsed log data and log subject identifier are sent to the Kafka message system.

Images