Log processing method and system and computer readable storage medium

Abstract

The invention provides a log processing method and system and a computer readable storage medium, and the method applied to an agent module comprises the steps: calling a first target service instance, and sending a log generated by the agent module to the first target service instance, the first target service instance being at least one called service instance in a plurality of service instances, the log comprises an application service identifier, the application service identifier is used for distinguishing different applications, and at least one first target service instance is called when the applications are operated; and associating the application service identifier with the first target service instance, and storing the association relationship, so that the log is subsequently sent to the first target service instance associated with the application service identifier in the log according to the association relationship. According to the invention, the occupation of the whole log processing service on CPU and memory resources can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and in particular to a log processing method, system and computer-readable storage medium. Background Art

[0002] In the process of network communication application management, especially cloud-native application management, the application process from construction and deployment to operation and maintenance management is completed in the cloud-native application management platform. In the application management process, the application running status needs to be analyzed and processed through the application logs. Therefore, the application logs need to be collected and aggregated for reporting.

[0003] At present, the mainstream log collection and storage solutions in the cloud-native environment are ELK (Elasticsearch / Logstash / Kibana, Elasticsearch for log storage, Logstash for log collection, Kibana for interface display) and EFK (Elasticsearch / Fluentd / Kibana, Fluentd for log collection) solutions. However, whether using Logstash components or Fluentd components for log collection, it will consume a lot of memory and CPU (Central Processing Unit, central processing unit). At the same time, in the Kubernetes (an open source container orchestration system) cluster, a Pod (instance) needs to be deployed on each node, so that the overall log processing service will occupy a lot of CPU and memory resources. Summary of the invention

[0004] To solve the above problems, the log processing method, system and computer-readable storage medium provided by the present invention can effectively reduce the overall CPU and memory resource occupancy of the log processing service by setting an application service identifier for distinguishing different applications in the log and managing the service instance based on the application service identifier.

[0005] In a first aspect, the present invention provides a log processing method, which is applied to an agent module, and the method comprises:

[0006] Calling a first target service instance, and sending a log generated by the proxy module to the first target service instance, wherein the first target service instance is at least one of the service instances called among the multiple service instances, and the log includes an application service identifier, wherein the application service identifier is used to distinguish different applications, and at least one of the first target service instances is called when the application is operated;

[0007] The application service identifier is associated with the first target service instance and the association relationship is stored, so that the log is subsequently sent to the first target service instance associated with the application service identifier in the log according to the association relationship.

[0008] Optionally, the step of calling the first target service instance includes:

[0009] Read the instance information corresponding to all active service instances;

[0010] According to the instance information, a service instance with the least number of calls is selected as the first target service instance, and the first target service instance is called.

[0011] Optionally, before the step of calling the first target service instance, the method further includes:

[0012] Receive a first page request sent by a console module; the first page request includes an operation parameter;

[0013] An application is operated according to the operation parameters in the first page request, and a log of the operation of the application is generated.

[0014] In a second aspect, the present invention provides a log processing method, which is applied to a service instance, and the method includes:

[0015] receiving a log sent by the proxy module through the first target service instance; the log includes an application service identifier corresponding to the first target service instance, the application service identifier is used to distinguish different applications, at least one of the first target service instances is called when the application is operated, and the first target service instance is at least one of the service instances called by the proxy module among the multiple service instances;

[0016] The log is stored in a shared storage module with the application service identifier as a dimension, so that all the service instances can read the log in the shared storage module.

[0017] Optionally, the method further comprises:

[0018] A broadcast message is sent, where the broadcast message is used to instruct all the service instances to update their own storage relationship lists, where the storage relationship list stores the corresponding relationship between the application service identifier and the log.

[0019] Optionally, the method further comprises:

[0020] receiving a second page request sent by the console module through a second target service instance; the second target service instance is the service instance called by the console module among the multiple service instances, and the second page request includes an application service identifier corresponding to the log requested to be viewed;

[0021] According to the application service identifier in the second page request, the log requested to be viewed is obtained, and the log requested to be viewed is forwarded to the console module.

[0022] Optionally, the second target service instance is the service instance with the least number of calls among multiple service instances.

[0023] In a third aspect, the present invention provides a log processing system, the system comprising a console, an agent module and a service instance;

[0024] The console is used to receive a page request, the page request includes a first page request and a second page request, the first page request is used to operate an application, and the second page request is used to request to view a log; the console is also used to send the first page request to the proxy module, and send the second page request to the service instance;

[0025] The proxy module is used to execute the log processing method as described in any one of the first aspects;

[0026] The service instance is used to execute the log processing method as described in any one of the second aspects.

[0027] Optionally, the system further includes an application management module;

[0028] The application management module is used to configure one of the service instances before the console module receives the page request, and to adjust the number of the service instances according to the usage rate of the load device;

[0029] The proxy module is also used to register the instance information of the proxy module in the form of key-value pairs to a distributed key-value storage system when it is started, and the distributed key-value storage system is configured in the application management module;

[0030] The service instance is also used to register the instance information of the service instance in the form of a key-value pair to the distributed key-value storage system when starting;

[0031] The application management module is also used to start monitoring the key-value pairs corresponding to the instance information of the proxy module and the key-value pairs corresponding to the instance information of the service instance when the console module is started, and to notify the console module to update the service list when the key-value pairs corresponding to the instance information of the proxy module and / or the key-value pairs corresponding to the instance information of the service instance change. The service list stores the instance information pairs of the proxy module and the instance information of the service instance and is located in the memory of the console module.

[0032] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method as described in any one of the first aspect and the second aspect is implemented.

[0033] The log processing method, system and computer-readable storage medium provided by the embodiments of the present invention effectively reduce the overall occupation of CPU and memory resources by the log processing service by setting the application service identifier used to distinguish different applications in the log and managing the service instance according to the application service identifier. Among them, for the proxy module, the number of deployed service instances is effectively reduced; for the service instance, not only the number can be reduced, but also the effect of single-node storage and multi-node readability is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 A schematic flow chart of a log processing method according to an embodiment of the present application;

[0036] Figure 2 This is a schematic workflow diagram of a cloud native application management platform according to an embodiment of the present application;

[0037] Figure 3 A schematic flow chart of a log processing method according to an embodiment of the present application;

[0038] Figure 4 A flowchart for processing from user triggering a request to determining the request type;

[0039] Figure 5 This is the log processing flow chart when the request type is the Console service receiving a request to deploy an application;

[0040] Figure 6This is the log processing flow chart when the request type is the Console service receiving a request to operate an application;

[0041] Figure 7 This is the log processing flow chart when the request type is the Console service receiving a request to view application logs. DETAILED DESCRIPTION

[0042] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0044] When used herein, the singular forms "a", "an", and "said / the" may also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" etc. specify the presence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof.

[0045] First, the main terms involved in the present invention and their functions in the present invention are explained as follows:

[0046] Console service (Console Server): used to receive operation instructions and log requests. The Console service obtains the instance information of the Agent service and EventLog service (event viewer) from Etcd, forwards the operation instruction request to the Agent service for execution, and forwards the log request to the service instance of the EventLog service for processing.

[0047] Etcd is a distributed, highly available, consistent key-value storage system written in Go, which is used to provide reliable distributed key-value storage, configuration sharing, service discovery and other functions.

[0048] Agent service: used to execute various operation instructions issued by the cloud native application management platform. These instructions will generate logs (including user asynchronous operation logs, application build logs, and application operation logs). The logs contain an application service identifier (service_id). When the Agent service and each EventLog service instance are started, their own information (input service address, port, etc.) is registered in Etcd. The Agent service queries Etcd, reads all active instance information of the EventLog service, selects an instance with the least number of calls, and sends the log to the EventLog service through the interface. At the same time, the relationship between the service_id and the called instance is recorded. The log generated by the same service_id next time is sent to the same EventLog service instance.

[0049] ZeroMQ service: used for EventLog services to achieve inter-node communication through ZeroMQ's publish-subscribe model.

[0050] EventLog service: All EventLog services are connected to ZeroMQ in publish-subscribe mode. After receiving the log, the Eventlog service aggregates, buffers and stores the log (stores it to disk or Elasticsearch) based on the service_id dimension, and broadcasts it synchronously in the cluster. All EventLog service instances update the relationship between service_id and the corresponding log file according to the broadcast message. The Console service forwards the log request to any EventLog service to query the log, thus providing distributed log subscription capabilities and achieving the effect of single-node storage and multi-node readability.

[0051] In a first aspect, an embodiment of the present invention provides a log processing method, which is applied to an agent module, see Figure 1 and Figure 2 The method includes steps S101 to S102.

[0052] Step S101: calling a first target service instance, and sending a log generated by an agent module to the first target service instance.

[0053] The first target service instance is at least one service instance called among the multiple service instances; the log includes an application service identifier, which is used to distinguish different applications. When the application is operated, at least one first target service instance is called.

[0054] In this embodiment, the service instance is in the Kubernetes cluster, that is, each service instance is a Pod, and the proxy module is an Agent service deployed on the cloud native application management platform.

[0055] In a further optional embodiment of the present embodiment, the step of calling the first target service instance includes: reading instance information corresponding to all active service instances; selecting the service instance with the least number of calls as the first target service instance based on the instance information, and calling the first target service instance.

[0056] By selecting the service instance with the least number of calls as the first target service instance to generate logs, the load of the entire platform is more balanced, which is conducive to improving the platform's ability to process logs.

[0057] In a further optional embodiment of this embodiment, before the step of calling the first target service instance, the method further includes: receiving a first page request sent by a console module, i.e., a Console service; the first page request includes an operation parameter; operating the application according to the operation parameter in the first page request, and generating a log of the operation application. This embodiment does not specifically limit the process of the proxy module operating the application and generating the corresponding log.

[0058] Step S102: Associating the application service identifier with the first target service instance and storing the association relationship, so that the log is subsequently sent to the first target service instance associated with the application service identifier in the log according to the association relationship.

[0059] The log processing method provided by the embodiment of the present invention effectively reduces the number of deployed service instances by setting an application service identifier for distinguishing different applications in the log and managing the service instance based on the application service identifier, thereby effectively reducing the overall CPU and memory resource occupancy of the log processing service.

[0060] In a second aspect, an embodiment of the present invention provides a log processing method. Based on the method in the first aspect, the method in this embodiment is applied to a service instance. Figure 2 and Figure 3 The method includes steps S201 to S202.

[0061] Step S201: receiving a log sent by an agent module through a first target service instance.

[0062] Among them, the log includes an application service identifier corresponding to the first target service instance, the application service identifier is used to distinguish different applications, at least one first target service instance is called when the application is operated, and the first target service instance is at least one service instance called by the proxy module among multiple service instances.

[0063] Step S202: The logs are stored in a shared storage module with the application service identifier as a dimension, so that all service instances can read the logs in the shared storage module.

[0064] It is understandable that the logs are stored in the shared storage module with the application service identifier as the dimension, that is, the logs are differentiated according to the different application service identifiers and stored in different areas of the shared storage module respectively, which facilitates the management of the logs by the platform.

[0065] The log processing method and computer-readable storage medium provided by the embodiments of the present invention not only effectively reduce the number of deployed service instances, but also achieve the effect of single-node storage and multi-node readability by setting an application service identifier for distinguishing different applications in the log and managing the service instance based on the application service identifier.

[0066] In a further optional embodiment of this embodiment, the method also includes: sending a broadcast message.

[0067] The broadcast message is used to instruct all service instances to update their own storage relationship lists, and the storage relationship list stores the corresponding relationship between the application service identifier and the log.

[0068] It should be noted that sending a broadcast message can be performed simultaneously with storing the log in the shared storage module, or can be performed after storing the log in the shared storage module. In the present embodiment, sending a broadcast message is performed simultaneously with storing the log in the shared storage module. By sending a broadcast message, any service instance can manage the log specified by the user, thereby improving the flexibility of the platform in managing the log and further improving the platform's ability to process the log.

[0069] In a further optional embodiment of the present embodiment, the method further includes: receiving a second page request sent by the console module through a second target service instance; obtaining the log requested to be viewed according to the application service identifier in the second page request, and forwarding the log requested to be viewed to the console module.

[0070] The second target service instance is a service instance called by the console module among the multiple service instances, and the second page request includes an application service identifier corresponding to the log requested to be viewed.

[0071] In a further optional embodiment of this embodiment, the second target service instance is the service instance with the least number of calls among the multiple service instances. By selecting the service instance with the least number of calls as the second target service instance, the load of the entire platform is further balanced, which is conducive to improving the platform's ability to process logs.

[0072] In a third aspect, an embodiment of the present invention provides a log processing method, see Figure 2 ,This method is applied to the cloud native application management platform.

[0073] The method includes: deploying and starting the Console service, Agent service, Evenlog service, ZeroMQ service, and Elasticsearch storage service on a cloud native application management platform.

[0074] When the Agent service and Evenlog service are started, they register the instance information of their service instances in the form of key-value pairs in Etcd. Among them, the format of the key is as follows: / service / {service-name} / {pod-id}; the instance information of the service instance includes the IP address of the service instance and the port name of the service instance. Taking the Eventlog service as an example, if the Eventlog service has two service instances, the registration information of the Eventlog service is / service / eventlog-svc / pod-1 and / service / eventlog-svc / pod-2. The value content corresponding to the key, for example: {"address":"192.168.1.100","port":"8080"}. The key-value pairs registered in Etcd have an expiration time, and expired key-value pairs will be automatically deleted. After registration is completed, each service instance regularly updates the corresponding key-value pairs to ensure that the key-value pairs will not be deleted by Etcd due to expiration.

[0075] When the Console service is started, it monitors the services under the prefixes / service / eventlog-svc and / service / agent-svc, obtains the instance information of the active service instance of the Eventlog service based on the prefix / service / eventlog-svc, and obtains the instance information of the Agent service based on the prefix / service / agent-svc, and forwards the user's page request to the corresponding service instance. When Etcd monitors the changes in the key-value pairs of the Console service, Agent service, and Eventlog service through its own monitoring mechanism, it will immediately notify the monitored party. Among them, the Console service updates its own service list according to the changes in the service instances of the Eventlog service and / or Agent service, and forwards the user's page request to the corresponding service instance based on the updated service list.

[0076] Among them, the instance information of the Eventlog service instance and the Agent service instance stored in the platform is obtained by the Console service from Etcd and stored in the service list maintained in the memory. When forwarding the page request, the Console service selects the instance information from the service list for forwarding; the page request generation process is mainly when the user accesses the cloud native application management platform, the user clicks the button on the display page of the platform to deploy (operation type) an application or view the log of an application, the page triggers a request to the backend Console service, and the Console service receives the page request.

[0077] It should be noted that the Console service may not have its own service list. Each time the Console service queries the service list, it queries from Etcd, which is equivalent to treating Etcd as the service list of the Console service. This can reduce the occupancy of the platform memory. In an optional embodiment, the Console service maintains a service list by itself, and the Console service updates the service list when Etcd changes. Since this service list is in the memory of the Console service, the Console service can query the service list more efficiently and better manage it. This embodiment does not specifically limit the service list of the Console service.

[0078] When the Agent service performs an operation, it generates a log and calls the interface of the EventLog service to send the log to the corresponding service instance. At the same time, the relationship between the corresponding application service identifier (service_id) and the service instance is saved in the memory, so that logs with the same application service identifier (service_id) will be sent to the same service instance.

[0079] After receiving the log, the service instance in the EventLog service aggregates, buffers and stores the log (stores it to disk or Elasticsearch) based on the service_id dimension, and broadcasts it synchronously in the cluster of the EventLog service. All EventLog services update the relationship between the service_id and the corresponding log according to the broadcast message. In this way, when the Console service forwards the page request to any instance in the EventLog service, it can find the corresponding log. The publish-subscribe mode of the ZeroMQ service can realize the communication between multiple service instances of the EventLog service, and the log generation and storage process is completed.

[0080] When a user views or operates a log, the page request initiated by the user is forwarded from the Console service to the Evenlog service. During this process, the Console service selects a service instance with the least number of calls from the service list for forwarding, so that the request is load balanced. The Eventlog service returns the corresponding log information, and the log viewing or operation process ends.

[0081] When the platform is used for the first time, the minimum number of EventLog service instances can be configured is 1. When the load of the EventLog service increases, the number of service instances in the EventLog service can be increased to improve the platform's ability to process logs. Each service instance will be registered with the Etcd service, so that the Console service and the Agent service can both identify the service instance. This method allows service instances to be configured on demand, thereby saving platform CPU and memory.

[0082] In a further optional embodiment of this embodiment, in combination with Figures 4 to 7 The method also includes: determining the type of page request.

[0083] When it is determined that the Console service receives a request for deploying an application, the Console service calls the Agent service interface and transfers the page request to the Agent service, which processes the page request. After that, the Agent service starts executing the application deployment task according to the parameters of the page request. Specifically, the Agent service creates a pod to compile and deploy the application according to the application information in the page request. At the same time, the Agent service obtains the compilation and deployment logs generated by the pod that processes the application, and temporarily saves the log content and the application id (service_id) in the memory; then, the Agent service selects the instance address of a service instance in the Eventlog service from the service list, calls the interface of the service instance, sends the log content in the form of a request, and stores the relationship between the service_id and the service instance in the memory, where the same service_id is only sent to the same service instance; then, the service instance receives the log request, and according to the service_id parameter in the request, the log is uniformly summarized with the service_id as the identifier, and the log content is stored in the form of a file in the shared storage module, that is, Elasticsearch; finally, the Eventlog service is broadcast synchronously between the Eventlog service clusters through Zeromq, and all service instances update the relationship between the service_id and the corresponding log file according to the broadcast message, thus completing the log production and storage process of application deployment.

[0084] When it is determined that the Console service receives a request to operate an application, such as starting, stopping, or restarting an application, the Console service also calls the Agent service interface and transfers the page request to the Agent service, which processes the page request. After that, the Agent service starts to operate the corresponding application according to the parameters of the page request, and generates a log record of the operation application (the Agent service prints the operation log at the key part of the operation), and saves the recorded log content and the application id (service_id) in the memory. Next, the Agent service selects an instance address of a service instance in the Eventlog service from the service list, calls the interface of the service instance, sends the log content in the form of a request, and stores the relationship between the service_id and the service instance in the memory, where the same service_id is only sent to the same service instance. Finally, the service instance receives the log request, and according to the service_id parameter in the request, the log is uniformly summarized with the service_id as the identifier, and the log content is stored in the form of a file in the shared storage module, that is, Elasticsearch, thus completing the log production and storage process of the application operation.

[0085] When it is determined that the Console service receives a request to view the application log, the Console service calls the Eventlog service interface, selects a service instance with the least number of calls and forwards the request to the service instance, and transfers the request to the service instance for processing; after receiving the request, the service instance determines the type of log requested to be viewed according to the parameters of the request. When it is determined that the log is a log for viewing the deployment and construction of the application, the service instance obtains the corresponding log file according to the service_id, reads the file content and returns it to the Console service to display it to the user. When it is determined that the log is an operation log for viewing the application, the service instance queries the log from Elasticsearch according to the service_id and returns it to the Console service to display it to the user. When it is determined that the log is a log for viewing the real-time output of the application container, the service instance creates a websocket communication connection with the user according to the application container id in the request, and the service instance calls the Kubernetes interface according to the container id to obtain the log content output by the container in real time, and pushes the log content to the browser used by the user through the websocket communication connection, so that the browser can display the log content in real time.

[0086] In the method provided in this embodiment, at the beginning, a cluster only needs to deploy one service instance for log processing. Later, the number of service instances of the log processing component can be dynamically increased as needed according to the number of application services, thereby reducing the platform resources occupied by the log collection and processing service, improving the log processing capability, and also facilitating the internal integration of the platform and the implementation of customization.

[0087] In a fourth aspect, an embodiment of the present invention provides a log processing system, the system including a console, an agent module and a service instance.

[0088] The console is used to receive a page request, the page request includes a first page request and a second page request, the first page request is used to operate the application, and the second page request is used to request to view the log; the console is also used to send the first page request to the proxy module, and send the second page request to the service instance. The proxy module is used to execute the log processing method as any one of the first aspects. The service instance is used to execute the log processing method as any one of the second aspects.

[0089] In a further optional embodiment of this embodiment, an application management module, namely Kubernetes, is also included.

[0090] The application management module is used to configure a service instance in the cloud native application management platform before the console module receives the page request, and to adjust the number of service instances according to the utilization rate of the load device.

[0091] The proxy module is also used to register the instance information of the proxy module in the form of key-value pairs to the distributed key-value storage system when starting. The distributed key-value storage system is configured in the application management module.

[0092] The service instance is also used to register the instance information of the service instance in the form of a key-value pair to the distributed key-value storage system at startup.

[0093] The application management module is also used to start listening to the key-value pairs corresponding to the instance information of the proxy module and the key-value pairs corresponding to the instance information of the service instance when the console module is started. When the key-value pairs corresponding to the instance information of the proxy module and / or the key-value pairs corresponding to the instance information of the service instance change, the console module is notified to update the service list. The service list stores the instance information pairs of the proxy module and the instance information of the service instance and is located in the memory of the console module.

[0094] In a fifth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the methods in the first aspect and the second aspect are implemented.

[0095] In the description of this specification, the description with reference to the terms "some embodiments", "other embodiments", "ideal embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example.

[0096] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A log processing method, characterized in that: The method is applied to an agent module, and the method comprises: Calling a first target service instance, and sending a log generated by the proxy module to the first target service instance, wherein the first target service instance is at least one of the service instances called among the multiple service instances, and the log includes an application service identifier, wherein the application service identifier is used to distinguish different applications, and at least one of the first target service instances is called when the application is operated; The application service identifier is associated with the first target service instance and the association relationship is stored, so that the log is subsequently sent to the first target service instance associated with the application service identifier in the log according to the association relationship.

2. The method according to claim 1, characterized in that The step of calling the first target service instance includes: Read the instance information corresponding to all active service instances; According to the instance information, a service instance with the least number of calls is selected as the first target service instance, and the first target service instance is called.

3. The method according to claim 1, characterized in that Before the step of calling the first target service instance, the method further includes: Receive a first page request sent by a console module; the first page request includes an operation parameter; The application is operated according to the operation parameters in the first page request, and a log of the operation of the application is generated.

4. A log processing method, characterized in that: The method is applied to a service instance, and the method comprises: receiving a log sent by the proxy module through the first target service instance; the log includes an application service identifier corresponding to the first target service instance, the application service identifier is used to distinguish different applications, at least one of the first target service instances is called when the application is operated, and the first target service instance is at least one of the service instances called by the proxy module among the multiple service instances; The log is stored in a shared storage module with the application service identifier as a dimension, so that all the service instances can read the log in the shared storage module.

5. The method according to claim 4, characterized in that The method further comprises: A broadcast message is sent, where the broadcast message is used to instruct all the service instances to update their own storage relationship lists, where the storage relationship list stores the corresponding relationship between the application service identifier and the log.

6. The method according to claim 4, characterized in that The method further comprises: receiving a second page request sent by the console module through a second target service instance; the second target service instance is the service instance called by the console module among the multiple service instances, and the second page request includes an application service identifier corresponding to the log requested to be viewed; According to the application service identifier in the second page request, the log requested to be viewed is obtained, and the log requested to be viewed is forwarded to the console module.

7. The method according to claim 6, characterized in that The second target service instance is the service instance with the least number of calls among the multiple service instances.

8. A log processing system, characterized in that: The system includes a console, an agent module and a service instance; The console is used to receive a page request, the page request includes a first page request and a second page request, the first page request is used to operate an application, and the second page request is used to request to view a log; the console is also used to send the first page request to the proxy module, and send the second page request to the service instance; The proxy module is used to execute the log processing method according to any one of claims 1 to 3; The service instance is used to execute the log processing method as described in any one of claims 4 to 7.

9. The system according to claim 8, characterized in that The system also includes an application management module; The application management module is used to configure one of the service instances before the console module receives the page request, and to adjust the number of the service instances according to the usage rate of the load device; The proxy module is also used to register the instance information of the proxy module in the form of key-value pairs to a distributed key-value storage system when it is started, and the distributed key-value storage system is configured in the application management module; The service instance is also used to register the instance information of the service instance in the form of a key-value pair to the distributed key-value storage system when starting; The application management module is also used to start monitoring the key-value pairs corresponding to the instance information of the proxy module and the key-value pairs corresponding to the instance information of the service instance when the console module is started, and when the key-value pairs corresponding to the instance information of the proxy module and / or the key-value pairs corresponding to the instance information of the service instance change, notify the console module to update the service list, and the service list stores the instance information pairs of the proxy module and the instance information of the service instance and is located in the memory of the console module.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images