Data processing method, device and equipment and readable storage medium

Abstract

The invention discloses a data processing method, device and equipment and a readable storage medium, and the method comprises the steps: obtaining a target resource instantiation event enabling a user-defined alarm resource to generate resource change according to a resource change signal sent by a resource controller; the target resource instantiation event carries an alarm rule configured for the target service application; creating a load controller based on an alarm rule configured for the target service application; deploying a load controller into a container arrangement system; and a load controller deployed in the container arrangement system and used for carrying out alarm detection on the log data in the log storage library corresponding to the target business application based on the alarm rule configured for the target business application. The method can be applied to various scenes such as the map field, the traffic field, the automatic driving field, the vehicle-mounted scene, the cloud technology, artificial intelligence, intelligent traffic and auxiliary driving, and the log alarm performance is improved.

Description

Technical Field

[0001] This application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, device, and readable storage medium. Background Art

[0002] With the development of cloud native, container orchestration systems (such as Kubernetes, abbreviated as the K8s system) are used by more and more applications. Different from the previous single-machine single-use scenarios, many different applications may be deployed on the same K8s system. As the number of deployed applications increases, the system log data will also increase significantly. Among them, log data is an important data source for data monitoring and alerting, and data analysis of applications. Through log data analysis and alerting, various situations such as access anomalies, access volume growth, and access latency can be discovered in a timely manner.

[0003] In traditional technologies, a unified log alerting platform is usually used to monitor and alert the logs of all applications. However, there are differences in the alerting requirements of different applications (for example, Application A may need to issue an alert when the access latency reaches 100 milliseconds, while Application B only needs to issue an alert when it reaches 10 milliseconds; another example is that Application A needs to alert for status code 1001, while Application B has no such alerting requirement). Using a unified log alerting platform to monitor and alert the logs of all applications, on the one hand, the matching degree with the alerting requirements of the applications is poor, and it is impossible to accurately monitor the log data that meets the application alerting conditions, which will reduce the log alerting performance; on the other hand, the unified log alerting platform is deployed based on physical machines. If the log data volume is large and the alerting data is numerous, and the single-machine computing power is insufficient, the log alerting performance will also be reduced. It can be seen that there is an urgent need for a log alerting solution in the container orchestration system to improve the log alerting performance of the system. Summary of the Invention

[0004] Embodiments of this application provide a data processing method, apparatus, device, and readable storage medium, which can improve the log alerting performance of a container orchestration system.

[0005] On the one hand, embodiments of this application provide a data processing method. The method is applied to a container orchestration system. The container orchestration system is pre-registered with custom alerting resources. The method is executed by a target node in the container orchestration system. The target node refers to a container in the container orchestration system where a resource controller is deployed. The resource controller is used to monitor resource changes of the custom alerting resources, including:

[0006] According to a resource change signal sent by the resource controller, obtain a target resource instantiation event that causes a resource change in the custom alerting resource; the target resource instantiation event carries an alerting rule configured for a target business application; the target business application refers to any business application running in the container orchestration system;

[0007] Create a load controller based on the alarm rules configured for the target business application;

[0008] Deploy the load controller to the container orchestration system; the load controller deployed to the container orchestration system is used to perform alarm detection on the log data in the log repository corresponding to the target business application based on the alarm rules configured for the target business application.

[0009] One aspect of the embodiments of the present application provides a data processing device, including:

[0010] An event acquisition module, configured to acquire a target resource instantiation event that causes a resource change of a custom alarm resource according to a resource change signal sent by a resource controller; the target resource instantiation event carries the alarm rules configured for the target business application; the target business application refers to any business application running in the container orchestration system;

[0011] A controller creation module, configured to create a load controller based on the alarm rules configured for the target business application;

[0012] A controller deployment module, configured to deploy the load controller to the container orchestration system; the load controller deployed to the container orchestration system is used to perform alarm detection on the log data in the log repository corresponding to the target business application based on the alarm rules configured for the target business application.

[0013] In one embodiment, before the event acquisition module acquires a target resource instantiation event that causes a resource change of a custom alarm resource according to a resource change signal sent by a resource controller, the data processing device further includes:

[0014] A resource controller deployment module, configured to acquire a program skeleton including a program entry function;

[0015] The resource controller deployment module is further configured to fill an event processing function for the custom alarm resource in the program skeleton; the event processing function includes configured event processing logic, and the event processing function is used to respond to different resource instantiation events of the custom alarm resource based on the event processing logic;

[0016] The resource controller deployment module is further configured to determine the program skeleton including the event processing function as a resource controller and deploy the resource controller to the target node.

[0017] In one embodiment, before the event acquisition module acquires a target resource instantiation event that causes a resource change of a custom alarm resource according to a resource change signal sent by a resource controller, the data processing device further includes:

[0018] A custom resource deployment module for generating a basic structure of a custom alarm resource through a custom controller development framework; the basic structure contains basic resource requirement fields configured by the custom controller development framework;

[0019] The custom resource deployment module is also used to obtain specific resource requirement fields defined for the custom alarm resource, and based on the basic resource requirement fields and the specific resource requirement fields, write a declaration style file for the custom alarm resource;

[0020] The custom resource deployment module is also used to register the custom alarm resource into the container orchestration system through the declaration style file.

[0021] In one embodiment, the target resource instantiation event also carries a target resource instance identifier;

[0022] The data processing device further includes:

[0023] An event attribute determination module for determining the event attribute of the target resource instantiation event according to the target resource instance identifier in the target resource instantiation event; the event attribute includes a creation attribute and a modification attribute;

[0024] A parameter detection module for obtaining the basic resource requirement fields and specific resource requirement fields of the custom alarm resource if the event attribute is a creation attribute;

[0025] The parameter detection module is also used to perform parameter detection on the input parameters of the target resource instantiation event according to the basic resource requirements and specific resource requirement fields to obtain a parameter detection result; the input parameters include the alarm rules configured for the target business application and the target resource instance identifier;

[0026] A step execution module for executing the step of creating a load controller based on the alarm rules configured for the target business application if the parameter detection result indicates that the input parameters conform to the field rules.

[0027] In one embodiment, the specific implementation manner for the event attribute determination module to determine the event attribute of the target resource instantiation event according to the target resource instance identifier in the target resource instantiation event includes:

[0028] Obtain the existing resource instance set of the custom alarm resource in the container orchestration system;

[0029] Traverse the existing resource instance set according to the target resource instance identifier in the resource instance event;

[0030] If there is an existing resource instance in the existing resource instance set with the same resource instance identifier as the target resource instance identifier, determine the event attribute of the target resource instantiation event as the modification attribute;

[0031] If there is no existing resource instance in the existing resource instance set whose resource instance identifier is the same as that of the target resource instance, the event attribute of the target resource instantiation event is determined as the creation attribute.

[0032] In one embodiment, the specific implementation manner of the parameter detection module for detecting the input parameters of the target resource instantiation event according to the basic resource requirements and the specific resource requirement fields includes:

[0033] Traverse the input parameters according to the basic resource requirement field and the specific resource requirement field;

[0034] If it is determined that there is a first field parameter required by the basic resource requirement field and a second field parameter required by the specific resource requirement field in the input parameters, the parameter legality of the first field parameter and the second field parameter is detected to obtain the parameter detection result of the input parameters;

[0035] If it is determined that there is no first field parameter required by the basic resource requirement field or no second field parameter required by the specific resource requirement field in the input parameters, the parameter detection result of the input parameters is determined as a detection failure result.

[0036] In one embodiment, the specific implementation manner of the parameter detection module for detecting the parameter legality of the first field parameter and the second field parameter to obtain the parameter detection result of the input parameters includes:

[0037] Obtain the first standard field format indicated by the basic resource requirement field and the second standard field format indicated by the specific resource requirement field;

[0038] Obtain the first parameter format of the first field parameter and the second parameter format of the second field parameter;

[0039] Match the first parameter format with the first standard field format and match the second parameter format with the second standard field format;

[0040] If the first parameter format matches the first standard field format successfully and the second parameter format matches the second standard field format successfully, the parameter detection result of the input parameters is determined as a detection pass result;

[0041] If the first parameter format fails to match the first standard field format or the second parameter format fails to match the second standard field format, the parameter detection result of the input parameters is determined as a detection failure result.

[0042] In one embodiment, the modification attribute includes deleting sub-attributes and updating sub-attributes;

[0043] After the event attribute determination module determines the event attributes of the target resource instantiation event based on the target resource identifier in the target resource instantiation event, the data processing device further includes:

[0044] A sub-attribute determination module, configured to, if the event attribute is a modification attribute, obtain the input parameters of the target resource instantiation event; the input parameters include the alarm rules configured for the target business application;

[0045] The sub-attribute determination module is further configured to determine the event sub-attribute to which the target resource instantiation event belongs between the deletion sub-attribute and the update sub-attribute based on the input parameters;

[0046] An instance clearing module, configured to, if it is determined that the event sub-attribute to which the target resource instantiation event belongs is a deletion sub-attribute, obtain the target existing resource instance with the resource instance identifier being the target resource identifier in the container orchestration system, and perform a clearing process on the target existing resource instance.

[0047] In one embodiment, after the sub-attribute determination module determines the event sub-attribute to which the target resource instantiation event belongs between the deletion sub-attribute and the update sub-attribute based on the input parameters, the data processing device further includes:

[0048] An instance update module, configured to, if it is determined that the event sub-attribute to which the target resource instantiation event belongs is an update sub-attribute, obtain the target existing resource instance with the resource instance identifier being the target resource identifier in the container orchestration system, and obtain the existing load controller corresponding to the target existing resource instance;

[0049] The instance update module is further configured to obtain the existing configuration file mounted by the existing load controller, and determine the existing configuration file mounted by the existing load controller as the target existing configuration file;

[0050] The instance update module is further configured to update the file content included in the target existing configuration file according to the input parameters.

[0051] In one embodiment, the specific implementation manner of the sub-attribute determination module to determine the event sub-attribute to which the target resource instantiation event belongs between the deletion sub-attribute and the update sub-attribute based on the input parameters includes:

[0052] Perform a non-null check on each parameter item included in the input parameters;

[0053] If it is determined that each parameter item has non-nullity, determine that the event sub-attribute to which the target resource instantiation event belongs is a deletion sub-attribute;

[0054] If it is determined that any one parameter item does not have non-nullity, determine that the event sub-attribute to which the target resource instantiation event belongs is an update sub-attribute.

[0055] In one embodiment, the specific implementation manner in which the controller deployment module deploys the load controller to the container orchestration system includes:

[0056] Obtain the input parameters of the target resource instantiation event; the input parameters include the alarm rules configured for the target business application;

[0057] Create a configuration file for the input parameters, and fill the input parameters into the configuration file to obtain a filled configuration file;

[0058] Mount the filled configuration file onto the load controller to obtain a mounted load controller;

[0059] Deploy the mounted load controller to the container orchestration system.

[0060] In one embodiment, the specific implementation manner in which the controller deployment module deploys the mounted load controller to the container orchestration system includes:

[0061] Perform container packaging processing on the mounted load controller according to the container packaging rules to obtain a packaged container corresponding to the mounted load controller;

[0062] Deploy the packaged container to the container orchestration system.

[0063] One aspect of the embodiments of the present application provides a computer device, including: a processor and a memory;

[0064] The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the methods in the embodiments of the present application.

[0065] One aspect of the embodiments of the present application provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and the computer program includes program instructions, and when the program instructions are executed by the processor, the methods in the embodiments of the present application are executed.

[0066] One aspect of the present application provides a computer program product, and the computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the methods provided in one aspect of the embodiments of the present application.

[0067] In an embodiment of the present application, a log alerting solution for a container orchestration system is provided, which can improve the log alerting performance of the container orchestration system. Specifically, in the present application, by adding a way to register custom alerting resources in the container orchestration system, different business applications in the container orchestration system can utilize the custom alerting resources to input alerting rules suitable for their own alerting requirements of the business applications, so as to implement customized alerting resource instances based on the custom alerting resources. At the same time, to timely detect the usage of the custom alerting resources, the present application will write and deploy a resource controller into the container orchestration system to monitor the resource changes of the custom alerting resources. In this way, whenever there is a business application that utilizes the custom alerting resources to expect to add a new resource instance or modify an already created resource instance, the resource controller will send out a resource change signal. Taking the target business application using the custom alerting resources and generating a target resource instantiation event as an example, in this case, the resource controller will send out a resource change signal. After the target node where the resource controller is deployed determines that the target resource instantiation event is an event of adding a new resource instance, it will obtain the alerting rules passed in by the target business application based on the custom alerting resources. According to the alerting rules, the target node can automatically create a load controller and deploy the load controller into the container orchestration system. In this way, the container orchestration system can monitor the log data in the log database corresponding to the target business application according to the alerting rules indicated by the load controller and perform log alerting. It should be understood that by registering or deploying custom alerting resources and resource controllers in the container orchestration system, different business applications can efficiently customize different resource instances based on different alerting rules, obtain different log monitoring and alerting solutions for different business applications, have a very high matching degree with the alerting requirements of the business applications, can accurately monitor the log data that meets the alerting requirements of the business applications themselves, and can improve the log alerting performance. In addition, since the log alerting solution of this solution is a solution proposed based on the containers in the container orchestration system, different business applications can utilize the custom alerting resources to deploy multiple sets of resource instances in the container orchestration system, which is different from the way of performing log monitoring and alerting on a single machine. The way of synchronously performing log monitoring and alerting for multiple sets of resource instances can adapt to alerting scenarios with a large amount of log data and a large number of alerting data, and can also improve the log alerting performance. In summary, the log alerting solution provided by the present application can improve the log alerting performance of the container orchestration system. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0069] Figure 1 It is a schematic diagram of a container orchestration system provided by an embodiment of the present application;

[0070] Figure 2 It is a schematic diagram of another container orchestration system provided by an embodiment of the present application;

[0071] Figure 3 It is a schematic architecture diagram of a log monitoring and alarm system provided by an exemplary embodiment of the present application;

[0072] Figure 4 It is a schematic flowchart of a data processing method provided by an exemplary embodiment of the present application;

[0073] Figure 5 It is a schematic flowchart for creating a load controller;

[0074] Figure 6 It is a schematic logical architecture diagram of a log alarm scheme provided by an embodiment of the present application;

[0075] Figure 7 It is a schematic structural diagram of a data processing device provided by an embodiment of the present application;

[0076] Figure 8 It is a schematic structural diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0077] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0078] The present application relates to related technologies of a container orchestration system. For ease of understanding, the following will first elaborate on related concepts such as the container orchestration system.

[0079] Container Orchestration System: It can schedule, scale, and manage containers. Specifically, a container orchestration system is an open-source system for managing containerized applications on multiple hosts in a cloud platform. Its goal is to make the deployment of containerized applications simple and efficient. The container orchestration system provides a mechanism for application deployment, planning, updating, and maintenance. Among them, the container orchestration system can include a node cluster composed of one or more (usually at least two) computing nodes. Each computing node in the node cluster can refer to a node computer used to run containerized applications, that is, each computing node can provide services for containerized applications. When the container orchestration system is running, it can be understood that the node cluster is running.

[0080] Container: A container is a method to achieve operating system virtualization, which can enable a process to run in an isolated environment.

[0081] Minimum Scheduling Unit: In a container orchestration system, the POD object can refer to the smallest unit of resource scheduling in the container orchestration system, which can have the same meaning as pod. A pod can contain one or more containers, which can share storage resources and network resources. A pod can contain the Internet Protocol (IP) of the container, the IP of the physical machine where it is located, the running process, and the current status, etc. In a container orchestration system, each container will have a corresponding POD object.

[0082] Load Controller (deployment): In a container orchestration system, a pod is the smallest control unit, but the container orchestration system rarely directly controls pods. Generally, it is completed through pod controllers. A pod controller is used for pod management to ensure that the pod resources are in the expected state. When a pod resource fails, it will try to restart or rebuild the pod. The load controller deployment is one type of pod controller, and deployment is used to manage releases in a container orchestration system.

[0083] Namespace (namespace, ns): A namespace provides a scope for resource names. It can divide resources that do not belong to a group into non-overlapping groups. If multiple users or user groups use the same K8s cluster and each manages its own unique set of resources, they should use their respective namespaces, which can avoid modifying or deleting the resources of other users and there is no need to worry about name conflicts.

[0084] Search Server: It can be used for log storage. The search server can be understood as a log database and can be used to store the log data of multiple business applications.

[0085] For easy understanding, please also refer to Figure 1, Figure 1 is a schematic diagram of a container orchestration system provided by an embodiment of the present application. As Figure 1 shown, the container orchestration system may include one or more containers, for example: Container 1, Container M, Container N, etc., where M and N are both integers greater than 1, and M is less than N. In this container orchestration system, each container has a corresponding POD object. Among them, the POD object of Container 1 is POD Object 1, the POD object of Container M is POD Object M, and the POD object of Container N is POD Object N.

[0086] Among them, it can be understood that the containers in the container orchestration system can be located on one physical machine or on multiple physical machines. For ease of understanding, please also refer to Figure 2 , Figure 2 is a schematic diagram of another container orchestration system provided by an embodiment of the present application. As Figure 2 shown, the container orchestration system shown takes the containers being located on multiple physical machines as an example. As Figure 2 shown, Container 1 can be located on Physical Machine 1; Container M and Container N can be located on Physical Machine X, where X can be any integer greater than 1.

[0087] In actual application scenarios, different business applications (business applications may include, but are not limited to: social applications, multimedia applications (such as short video applications), entertainment applications (such as game applications), educational applications, live broadcast applications) will be deployed to a container orchestration system, and a certain container in the container orchestration system will provide corresponding services for the business application. For these business applications running in the container orchestration system, relevant log data will be generated during operation. The container orchestration system can monitor and analyze whether there are abnormal or non-compliant operation data (such as abnormal access, operation data with access delays, etc.) based on the log data of the business application. It can be seen that the log data of the container orchestration system is important data for application detection and application optimization of business applications. Therefore, it is very important to perform log monitoring and alerting on the log data of the container orchestration system. By performing log monitoring and alerting on the log data, abnormal or non-compliant operation data can be discovered in a timely manner. In traditional technologies, for log monitoring and alerting of container orchestration systems, a unified log alerting scheme is generally used to perform log monitoring and alerting on all applications. For example, first deploy a set of log alerting schemes in the container orchestration system based on relevant available plugins, and then use this log alerting scheme to monitor and alert the log data of each application. Among them, in the process of deploying the log alerting scheme in traditional technologies, when performing relevant file configurations, relatively complex and obscure file configuration syntax is used, which will make file configuration take more time and effort and increase the file configuration burden. In addition, due to the problem of inconsistent log formats and fields of different business applications, a unified log alerting platform cannot cover the alerting requirements of all business applications in the log database, that is, a unified log alerting scheme is difficult to match different log databases. Moreover, the log alerting scheme deployed in traditional technologies is deployed based on physical machines and does not face the container cluster of the container orchestration system. In the case of a large amount of log data and a large number of alert data, it is easy to have a problem that the single-machine computing power cannot be supported, resulting in a decrease in alert performance.

[0088] In summary, in order to reduce the configuration and deployment cost of the log alarm solution of the container orchestration system and improve the log alarm performance of the container orchestration system, the present application provides a log monitoring and alarm solution for the container orchestration system, which can combine the capabilities of containers and open source tools for creating load controllers to achieve a fully free log monitoring and alarm solution process, and in the process of deploying the log monitoring and alarm solution, when performing file configuration of the alarm solution, it is also easy to operate and understand, and does not take up much time for file configuration; in addition, the present solution can facilitate different business applications to deploy log alarm solutions that meet their own alarm needs in a targeted manner, without the need to use the same log alarm platform, thereby solving the problem of insufficient computing power of a single machine. Among them, the log monitoring and alarm scheme for the container orchestration system involved in this scheme can include at least the following five consecutive steps: 1. First, a custom resource (Custom Resource Definition, CRD) can be created and registered in the container orchestration system based on the log alarm scenario. This custom resource is used for different business applications, so that different business applications can use the custom resource to specifically define an alarm resource instance that meets their own alarm needs in the container orchestration system. This alarm resource instance can be used as a log alarm scheme for the business application. It should be understood that since the custom resource is specially defined and created based on the log alarm scenario, the present application may refer to the custom resource as a custom alarm resource. For the custom alarm resource, the present application may use a custom controller development framework (for example, a framework such as operator-sdk) to define and design its necessary requirement fields (necessary requirement fields may include but are not limited to: certificate fields, configuration fields, alarm rule fields, pod template fields in the container orchestration system, status fields, etc. Among them, the status field may include version fields, phrase fields, message fields, etc., and the necessary requirement fields may be designed and defined based on specific business scenarios). Based on these designed and defined necessary requirement fields, if a business application wants to create an alarm resource instance in the container orchestration system through the custom alarm resource, it is necessary to pass in the corresponding parameters according to these necessary requirement fields (that is, the parameters required by each necessary requirement field, for example, the certificate field requires certificate parameters, and the alarm rule field requires alarm rule parameters);2. To facilitate the timely discovery of the usage of this custom alert resource by different business applications (it can be considered that the custom alert resource is used when a business application creates, deletes, or updates an alert resource instance through this custom alert resource, or creates, deletes, or updates the alert resource instance of this custom alert resource), this application can write a logic code for monitoring the resource change situation of this custom alert resource and handling different resource change events of the custom alert resource. Based on this logic code, a resource controller for the custom alert resource can be generated. This resource controller can control the current custom alert resource operation management actions based on this logic code. Then, by deploying this resource controller to a certain computing node in the container orchestration system, the computing node can use this resource controller to monitor the custom alert resource in real time, timely discover the resource change situation of this custom alert resource, and respond and handle different resource change events accordingly; 3. Different business application managers can use this custom alert resource to pass in corresponding parameters (i.e., the parameters required by the above-mentioned necessary requirement fields) to customize an alert resource instance (Custom Resource, CR) for this business application, and use the alert rules indicated by this alert resource instance to monitor the log data of this business application; 4. For a single use of the custom alert resource by a business application (such as the creation of a resource instance), it can be understood that a resource change has occurred to the custom alert resource. The above-mentioned resource controller can monitor the resource change of the custom alert resource in real time and send out a resource change signal. Then, the computing node deployed with the resource controller can, based on the corresponding logic code, obtain this resource instance event (such as a resource instance creation event) that causes the resource change of the custom alert resource, and obtain the above-mentioned passed-in parameters;5. According to the alarm rules in the input parameters, the computing node can automatically create a load controller for this resource instance event of the service application (i.e., a deployment object, which is essentially a resource object in the container orchestration system and is mainly used to store the relevant configuration content of the service application. In this application, the relevant configuration content of the service application is mainly included in the parameters passed in by the user). After creating a load controller, it can be packaged into a container and deployed into the container orchestration system. Subsequently, this packaged container can provide corresponding services for the service application (such as providing an alarm service based on the alarm rules). Thus, an alarm resource instance for this service application is deployed into the container orchestration system, that is, a log alarm solution adapted to the alarm requirements of the service application itself has been deployed in the container orchestration system. Subsequently, the container obtained by packaging based on the resource controller can be used to monitor the log data of the service application in real time. Once data that conforms to the alarm rules is found, an alarm notification for this data can be output, and the maintenance personnel of the service application can conduct troubleshooting and optimization on the corresponding log data based on this alarm notification.;

[0089] It should be understood that the prerequisite for this application to implement log monitoring and alarming in the container orchestration system is to pre-register a custom alarm resource in the container orchestration system and deploy a resource controller for monitoring this custom alarm resource in a certain container of the container orchestration system. In this way, different service applications can customize alarm resource instances of different custom alarm resources in the container. It is also convenient to bundle and configure different service applications and different alarm resource instances to achieve one-key startup. Different service applications do not need to share the same set of log alarm platforms, and the log alarm capabilities of different service applications can be effectively split to improve the log alarm performance of different service applications; and through the resource controller, the usage situation of the custom alarm resource can also be detected in a timely manner, so as to be able to respond to the resource instance events of the service application in a timely manner, which helps to improve the processing efficiency of resource instances.

[0090] Specifically, the log monitoring and alarm scheme for the container orchestration system provided in the embodiment of the present application may include the following general process: first, a computing node (which may be referred to as a target node) deployed with a resource controller may perform real-time monitoring of the custom alarm resources pre-registered in the container orchestration system. If a business application (such as a target business application) wants to customize an alarm resource instance in the container orchestration system through a custom alarm resource, then the manager of the target business application may use the custom alarm resource to customize the alarm resource instance. The manager of the target business application may pass corresponding parameters to the target node of the container orchestration system based on the necessary requirement fields of the custom alarm resource (the parameters passed in need to include the parameters required by the necessary requirement fields, for example, the alarm rules required by the alarm rule fields need to be included, and the alarm rules passed in by the manager of the target business application may refer to alarm rules specially configured for the target business application). Then, the resource controller may detect that the target business application has passed in the parameters corresponding to the custom alarm resource, and then the resource controller may deem that there is a resource change event for the custom alarm resource, and this resource change event is real. The nature of the event is that the target business application wants to create an alarm resource instance, that is, the resource change event can be identified as a resource instantiation event (specifically, a resource instance creation event); further, the resource controller can issue a resource change signal about the custom alarm resource based on the resource instantiation event, and the target node can obtain the resource instantiation event based on the resource change signal, and obtain the parameters passed in by this resource instantiation event (including the alarm rules configured for the target business application); after obtaining the parameters passed in by this event, the target node can create a load controller for this resource instantiation event according to the alarm rules, and the target node can deploy the load controller to the container orchestration system. At this point, an alarm resource instance about the target business application is successfully deployed to the container orchestration system, and then the log data of the target business application can be monitored according to the alarm rules in the load controller, and when the log data that meets the alarm rules is found, an alarm notification is output, and the maintenance personnel of the target business application can check and optimize the access status or business functions of the target business application based on the alarm notification.

[0091] For example, assume that both business application A and business application B are running in a container orchestration system, and the relevant log data of business application A and business application B are stored in the same set of log databases. For business application A, its alert requirement is that the access latency of the business application reaches or exceeds 100 ms. For business application B, its alert requirement is that the login retry frequency of the business application reaches or exceeds 5 times. When there is a custom alert resource pre-registered in the container orchestration system, the manager a1 of business application A (such as an application maintenance user) can write a corresponding alert rule (such as the access latency is greater than or equal to 100 ms) based on the alert requirement of business application A. The manager a1 can use the custom alert resource in the container orchestration system to create an alert resource instance. Specifically, the manager a1 can pass the alert rule and other necessary parameters required by the custom alert resource into the container orchestration system according to the necessary requirement fields of the custom alert resource. Then, the resource controller deployed in the container orchestration system can detect this event generated by the manager a1 for the custom alert resource. The resource controller can send a resource change signal to indicate that there is a resource instantiation event for the current custom alert resource. The target node with the resource controller deployed can obtain the alert rule passed in by the manager a1 based on this resource change signal and automatically create a load controller for this event. Then, the target node can deploy this load controller into the container orchestration system. In subsequent log monitoring, the log data in the log database where business application A is located can be monitored according to the alert rule of business application A indicated by the load controller. Once it is found that a certain log data meets this alert rule, an alert notification for this log data can be output. Similarly, the manager a2 of business application B (such as an application maintenance user) can also write a corresponding alert rule (such as the login retry frequency reaches or exceeds 5 times) based on the alert requirement of business application B and deploy this alert rule into the container orchestration system through the load controller using the custom alert resource in the container orchestration system. In subsequent log monitoring, the log data in the log database where business application B is located can be monitored according to the alert rule of business application B indicated by the load controller. Once it is found that a certain log data meets this alert rule, an alert notification for this log data can be output.

[0092] It should be noted that since business application A and business application B share the same set of log databases, the alarm rules deployed for business application A can also apply to the log data of business application B (that is, the alarm rules of business application A can also monitor the log data of business application B). Similarly, the alarm rules deployed for business application B also apply to the log data of business application A (that is, the alarm rules of business application B can also monitor the log data of business application A). However, for a certain business application, the other business application may not have the same alarm requirements. Then, for the business application without the same alarm requirements, the alarm notifications sent may be useless. Based on this, for a business application without alarm requirements, when sending an alarm notification for the log data of this business application, this application can additionally output an alarm prompt message to indicate that this alarm is sent based on the alarm rules of another business application. For example, taking the alarm rules of business application A as an example, if business application B does not need to search for abnormal application data based on the alarm rules of business application A in actual use, then after finding that some log data of business application B meets the alarm rules of business application A, an alarm prompt message can be added when sending the alarm notification. This alarm prompt message can be used to prompt the manager a2 of business application B that this alarm is sent based on the alarm rules of business application A. In this way, the manager a2 of business application B can actively choose not to conduct any troubleshooting or optimization of the log data of business application B based on this alarm prompt message.

[0093] It should be noted that since business application A and business application B share the same set of log databases, in this set of log databases, the log data of business application A and business application B can be stored in partitions. In this way, for the alarm rules deployed for business application A, they can only apply to the log data in the partition where business application A is located in the log database; for the alarm rules deployed for business application B, they can only apply to the log data in the partition where business application B is located in the log database. This way of storing log data in partitions can also reduce the situation of ineffective monitoring and alarming of the log data of business applications without corresponding alarm requirements, thereby reducing the computing volume and helping to improve the log alarm performance.

[0094] It can be seen that through the method of adding a custom new resource type to the container orchestration system by customizing the CRD (that is, creating and registering a custom alarm resource in the container orchestration system), the present application can implement a customizable log alarm scheme for different alarm requirements scenarios of different business applications. For each alarm resource instance created based on the custom alarm resource, a load controller will be created simultaneously. These alarm resource instances do not interfere with each other and are independent based on container capabilities, and can specifically monitor and alarm the log data of business applications, accurately detect the log data that meets the alarm conditions of the business application itself, and improve the performance of log alarm.

[0095] The log monitoring and alarm scheme for the container orchestration system provided by the embodiments of the present application can be applied to any business application running in the container orchestration system and having log alarm requirements. These business applications can include, but are not limited to: short video push applications, game applications, browser applications, education applications, network disk applications, etc. Here, the business applications will not be exemplified one by one. That is to say, the log monitoring and alarm scheme for the container orchestration system provided by the embodiments of the present application can be applied to any application scenario with log alarm requirements, which can specifically include, but are not limited to: short video push scenarios, game scenarios, browser search scenarios, education scenarios, network disk storage scenarios, and so on. Among them, taking the application scenarios of game scenarios and network disk storage scenarios as examples:

[0096] The network disk application can provide users with different functions (for example, create folder function, upload file function, preview file function, share group function, move file function, electronic signature function, favorite function, unfavorite function, rename function, access management center function, empty expired file function, etc.). The network disk storage scenario can refer to the scenario where users perform cloud storage of relevant data based on various functions provided by the network disk application. Users can store data in a certain folder, share the data in a certain folder to a certain group, preview a certain data in the file, etc. by performing various operation behaviors.

[0097] The game scenario can refer to the scenario where users run game applications on terminal devices to play games.

[0098] In summary, the log monitoring and alarm scheme provided by the embodiments of the present application can implement a customizable log alarm scheme for different business applications and different scenarios by adding a custom CRD, and the log alarm resource instances of each application are independent of each other and do not interfere with each other, effectively improving the business coverage to a certain extent (such as expanding the applicable scenarios).

[0099] It should be noted that the above several application scenarios are only examples and will not limit the application scenarios applicable to the log monitoring and alarm scheme provided by the embodiments of the present application.

[0100] Furthermore, the log monitoring and alerting solution provided in the embodiments of the present application can be executed by a computer device, which can refer to any computing node in the container orchestration system where the above resource controller is deployed. The computing node can specifically include a terminal or a server, and the computer device can also include a terminal and a server. To facilitate understanding of the log monitoring and alerting solution provided in the embodiments of the present application, the following combines Figure 3 the log monitoring and alerting system shown in Figure 3 to introduce the application scenarios involved in the embodiments of the present application; among them, Figure 3 is a schematic architecture diagram of a log monitoring and alerting system provided in an exemplary embodiment of the present application. As shown in

[0101] 1) The terminal 101 can include the terminal device used by the user. Of course, according to the different application scenarios and fields to which the present solution is applied, the terminals providing the solutions in the embodiments of the present application are different. The terminal device can include but is not limited to: smart phones (such as smart phones deployed with the Android system or smart phones deployed with the Internetworking Operating System (IOS)), tablet computers, portable personal computers, Mobile Internet Devices (MID), in-vehicle devices, head-mounted devices, smart homes, and smart voice interaction devices, etc. The embodiments of the present application do not limit the type of the terminal device, and this is hereby explained.

[0102] 2) The server 102 can be a computing node in the container orchestration system where a resource controller (i.e., the logic code for monitoring custom alert resources) is deployed, and can be used to interact with the terminal to receive the resource instantiation event of the custom alert resource from the terminal, and create a container for a certain business application running on the terminal in the container orchestration system based on the alert rule in the event (generate a load controller based on the alert rule and package it as a container) so that the container can provide corresponding alert services for the business application. Among them, the server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0103] 3) The container orchestration system 103 can be the aboveFigure 1 The system shown, in which the container orchestration system may include one or more containers, each created by a computing node and used to provide corresponding services for a certain business application.

[0104] Among them, the terminal 101 and the server 102 can be directly or indirectly connected through wired or wireless communication methods, and this application does not limit this here. In addition, the number of terminals and servers in the embodiments of this application is not limited; in Figure 3 it is only an example that the numbers of both the terminal 101 and the server 102 are single, and in actual applications, it may include multiple servers distributed in a distributed manner, and this is specifically stated here.

[0105] Based on the above-described solutions and system architectures, the following points need to be further explained:

[0106] ① The system shown mentioned in the embodiments of this application Figure 3 is to more clearly illustrate the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided by the embodiments of this application. Those of ordinary skill in the art know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of this application are equally applicable to similar technical problems. For example, the above takes the execution subjects of the embodiments of this application, "computer devices", including terminals and servers as an example, that is, the solution provided by the embodiments of this application is jointly executed by terminals and servers as an example to introduce an application scenario of this solution; it should be understood that in actual applications, the computer device can also be a terminal or a server, that is to say, it supports the solution provided by the embodiments of this application to be executed alone by a terminal or a server.

[0107] ② In the embodiments of this application, the collection and processing of relevant data should be strictly in accordance with the requirements of relevant laws and regulations. Obtaining personal information requires the informed consent of the personal subject (or having a legal basis for information acquisition), and subsequent data use and processing behaviors should be carried out within the scope authorized by laws and regulations and the personal information subject. For example, when the embodiments of this application are applied to specific products or technologies, such as obtaining the current media data played by a user, the permission or consent of the user needs to be obtained, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards in the relevant regions.

[0108] Based on the above-described solutions and application scenarios, the embodiments of this application propose a more detailed log monitoring and alarming method, and the log monitoring and alarming method proposed by the embodiments of this application will be introduced in detail below with reference to the accompanying drawings.

[0109] Please refer to Figure 4 , Figure 4It is a schematic flowchart of a data processing method provided by an exemplary embodiment of the present application. This process may refer to the process of the log monitoring and alerting method for the container orchestration system provided by the embodiments of the present application. This data processing method can be executed by a computer device in the aforementioned system. For example, the computer device is a target node deployed with a resource controller (for monitoring resource changes of custom alert resources, where the custom alert resources need to be pre-registered in the container orchestration system). This data processing method may at least include the following steps S201 - S203:

[0110] Step S201: According to the resource change signal sent by the resource controller, obtain a target resource instantiation event that causes a resource change in the custom alert resource. The target resource instantiation event carries an alert rule configured for the target business application. The target business application refers to any business application running in the container orchestration system.

[0111] In the present application, the container orchestration system can schedule, scale, and manage containers. Specifically, the container orchestration system is used to manage containerized applications on multiple hosts in the cloud platform. The goal is to make the deployment of containerized applications simple and efficient, and it provides a mechanism for application deployment, planning, update, and maintenance. Among them, the container orchestration system may include one or more (usually at least two) computing nodes, and each computing node can be used to run containerized applications. The node cluster in the embodiments of the present application may refer to a cluster composed of one or more computing nodes in the container orchestration system. When the container orchestration system is running, it can be understood that the node cluster is running. It should be understood that an application (such as the target business application shown in the above embodiment) can correspond to multiple containers. When an object (such as an enterprise) uses the corresponding application (such as the network disk application shown in the above embodiment), the corresponding container in the container orchestration system provides services for the object. And the containers in the container orchestration system can be created by the node cluster.

[0112] For a certain business application running in a container orchestration system, operation log data will be generated as the application runs. In the container orchestration system, a log database can be used to store the log data of different business applications. For example, a search server can be used to store the log data of business applications. It should be understood that the container orchestration system can monitor and analyze the log data of business applications to analyze whether there are abnormalities or operation data that does not conform to the regulations of the business application (such as abnormal access, operation data with access latency, etc.). Once abnormalities or non-compliant operation data are detected, the application maintenance user corresponding to the business application can be notified, and the application maintenance user can then perform corresponding processing on the business application: for example, perform maintenance, optimization, etc. on the business application based on this operation data. It can be seen that the log data of business applications stored in the container orchestration system is important data for application maintenance and optimization of business applications. Therefore, how to effectively monitor and alarm the log data of the container orchestration system is very important.

[0113] In order to improve the log monitoring and alarm performance of the container orchestration system and reduce the deployment cost of the log alarm solution, the embodiments of the present application provide a log alarm solution for the container orchestration system. Specifically, the present application can add a new resource type of custom alarm resources to the application programming interface (API) of the container orchestration system through a custom CRD, so as to enable different business applications in different scenarios to customize a log alarm solution that adapts to their own alarm requirements through this custom alarm resource. That is to say, the custom alarm resource in the present application is a custom CRD that is pre-created and registered in the container orchestration system. By pre-creating and registering this custom alarm resource, it can be transformed into an existing resource of the container orchestration system. Then, for each business application running in the container orchestration system, its corresponding application manager can customize different log alarm solutions for the business application through this existing resource (for example, create an alarm resource instance of this custom alarm resource based on the alarm rule to deploy a customized log alarm solution in the container orchestration system).

[0114] In a specific implementation, the method for pre-registering custom alarm resources in a container orchestration system may include at least the following three steps: 1. First, it is necessary to customize a new resource type in the container orchestration system based on the alarm scenario. This newly customized resource type can be used as the custom alarm resource specially created in this application. Among them, CRD itself is a resource in the container orchestration system, which defines specifications and metadata and allows users to customize new resource types based on these specifications and metadata. Then, users can use this CRD to create a new resource type of custom alarm resources based on the alarm scenario (the created custom alarm resource is essentially a customized CRD); 2. Then, for the created custom alarm resources, it is necessary to design necessary requirement fields for them. These necessary requirement fields can indicate which parameters need to be passed in when creating a resource instance of the custom alarm resource. In this way, when each user creates a resource instance of the custom alarm resource, they can input corresponding parameters through these necessary requirement fields. In this application, the necessary requirement fields for the custom alarm resource can be jointly defined in combination with the custom controller development framework. Specifically, the custom controller development framework can refer to operator-sdk. Through this custom controller development framework, a scaffold for creating custom alarm resources can be created. Some basic requirement fields for custom resource creation in the custom controller development framework will be included in this scaffold (which can be called basic resource requirement fields, such as the encapsulation field of the API). On top of the basic resource requirement fields included in the scaffold, users creating custom alarm resources can configure additional necessary requirement fields for the custom alarm resources. The necessary requirement fields designed and configured by users can be called specific resource requirement fields. Specific resource requirement fields may include, but are not limited to: alarm rule fields, status fields. Through the basic resource requirement fields and the specific resource requirement fields, all the necessary requirement fields of the custom alarm resource can be determined. In this application, the necessary requirement fields for the custom alarm resource may at least include: certificate fields, configuration fields, alarm rule fields, pod template fields, status fields, etc. Among them, the status field may further include a version field, a phrase field, a message field, etc.; 3. After designing and configuring the necessary requirement fields of the custom alarm resource, these necessary requirement fields can be used as the specifications and metadata of the custom alarm resource. Then, the custom alarm resource containing the specifications and metadata can be registered in the container orchestration system. In a specific implementation, the custom alarm resource can be injected into the type registration component of the container orchestration system to complete the registration of the custom alarm resource.Among them, it should be understood that the type registration component in this application is mainly used to manage and register the information and types of API objects. Through the type registration component, a custom resource type can be registered as the type of an API object in the container orchestration system. Then, injecting the custom alarm resource into the type registration component means injecting the custom alarm resource as an existing resource in the container orchestration system. Taking the type registration component as the scheme component as an example, by injecting the custom alarm resource into the scheme, it can be considered that the custom alarm resource is registered in the container orchestration system.

[0115] In summary, this application needs to pre-register a custom alarm resource in the container orchestration system in advance. The specific implementation process can include but is not limited to: First, generate the basic structure of the custom alarm resource through the custom controller development framework (the basic structure is, for example, a structure definition composed of programming languages). This basic structure can be understood as the scaffolding of the above-mentioned custom alarm resource. In essence, it is an initial project framework of the custom alarm resource. The basic structure will contain the basic resource requirement fields configured by the custom controller development framework. These basic resource requirement fields will specify some basic requirement parameters of the custom alarm resource; on this basis, the user can design and configure additional requirement fields for the custom alarm resource. These requirement fields can be used as specific resource requirement fields defined by the user configuration. The specific resource requirement fields can specify some specific requirement parameters of the custom alarm resource, and the target node can obtain the specific resource requirement fields defined for the custom alarm resource and write a declaration style file for the custom alarm resource based on the basic resource requirement fields and the specific resource requirement fields; among them, the declaration sample file here can refer to the Deployment YAML file. Since the declaration style file is written based on the basic resource requirement fields and the specific resource requirement fields, the declaration style file will specify what the necessary requirement fields of the custom alarm resource are. Through the declaration style file, the custom alarm resource can be injected into the type registration component of the container orchestration system.

[0116] Furthermore, for the custom alarm resources defined by the user, a controller can be created and deployed to monitor the resource changes of the custom alarm resources in real time to implement the corresponding logic. Specifically, the present application can develop the controller code for real-time monitoring of the custom alarm resources and deploy it to a certain computing node (such as the target node) of the container orchestration system. Thus, the target node can run the controller code to achieve real-time monitoring of the custom alarm resources. The controller indicated by the controller code can be called a resource controller. In the specific implementation, the process of developing and deploying the resource controller to the target node can at least include the following three steps: 1. First, a program skeleton containing the program entry function can be obtained. This program entry function can refer to the main.go function, and the program skeleton can refer to a complete program skeleton containing main.go. 2. After obtaining the program skeleton, an event handling function for the custom alarm resources can be filled in the program skeleton. The event handling function should contain the corresponding event handling logic. Then, whenever there is a change in the CR (resource instance) of the custom alarm resources (i.e., there is a resource instantiation event), the event handling function can be called back and make corresponding responses to different resource instantiation events through the event handling logic. 3. Determine the program skeleton containing the event handling function as the resource controller. Then, according to the program skeleton containing the event handling function, a declaration style file for the resource controller can be written and generated. Through this declaration style file, the resource controller can be deployed to the target node of the container orchestration system.

[0117] As described above, in the present application, by adding a custom alarm resource to the container orchestration system through CRD, users of different business applications can pass in parameters to create specific resource instances of the custom alarm resource. Of course, for the resource instances of the custom alarm resource that have been created, users can also delete them or change the configured parameters of the resource instances. Whether it is the creation and addition of resource instances, the deletion of resource instances, or the update of resource instances, it can be considered a resource instantiation event of the custom alarm resource, and this resource instantiation event will cause a change in the custom alarm resource. That is to say, whenever a resource change is detected in the custom alarm resource, it can be considered that there is a resource instantiation event in the custom alarm resource. This resource instantiation event may be a resource instance creation event, a resource instance deletion event, or a resource instance update event. The target resource instantiation event in the present application refers to a resource instantiation event initiated for the target business application, which may be a resource instance creation event, a resource instance deletion event, or a resource instance update event. Based on the resource instance creation event, the resource instance deletion event, and the resource instance update event, the present application can define event attributes for a certain resource instantiation event (such as the target resource instantiation event). For example, a creation attribute can be defined based on the resource instance creation event, a deletion attribute can be defined based on the resource instance deletion event, and an update attribute can be defined based on the resource instance update event. Since both the resource instance deletion event and the resource instance update event are to delete or update an already created resource instance, they can be combined into a resource instance modification event. Similarly, the deletion attribute and the update attribute can be combined into a modification attribute. For the sake of distinction, the deletion attribute belonging to the modification attribute is called the modification sub-attribute, and the update attribute of the attribute modification attribute is called the update sub-attribute.

[0118] Based on this, if the resource controller detects a resource change in the custom alarm resource, the resource controller can send a resource change signal, and the target node can, based on this resource change signal, obtain the current target resource instantiation event that causes the resource change in the custom alarm resource and determine whether the event attribute of the target resource instantiation event is a creation attribute or a modification attribute (that is, determine whether the target resource instantiation event is a resource instance creation event or a resource instance modification event). When it is determined that the target resource instantiation event is a resource instance creation event, the subsequent steps S202 - S203 can be executed; if it is determined that the target resource instantiation event is a resource instance modification event, a certain resource instance deployed in the container orchestration system can be deleted, or the relevant configuration parameters of the resource instance deployed in the container orchestration system can be updated based on the parameters passed in by the user. For the specific method of determining whether the event attribute of the target resource instantiation event is a creation attribute or a modification attribute, please refer to the followingFigure 5 the description in the corresponding embodiment.

[0119] Step S202: Create a load controller based on the alarm rules configured for the target business application.

[0120] In this application, the load controller may refer to a deployment object. When it is determined that the target resource instantiation event is a resource instance creation event, the deployment object of the resource instance indicated by the target resource instantiation event can be created. It should be noted that for the creation of the load controller, it is necessary to first determine the event attributes of the target resource instantiation event. After determining that the target resource instantiation event is a resource instance creation event, the parameters passed in for this event (that is, the parameters passed in by the user based on the necessary requirement fields of the custom alarm resource, and these passed-in parameters can be understood as the configuration content defined by the user for this resource instance) should also be detected to determine whether the parameters passed in by the user comply with the rules. That is to say, for the step of creating a load controller based on the alarm rules configured for the target business application, it needs to be executed on the premise that the event attribute of the target resource instantiation event is the creation attribute and the input parameters of the target resource instantiation event pass the detection. For the determination method of the event attributes of the target resource instantiation event and the parameter detection method of the input parameters of the target resource instantiation event, reference can be made to the description in the subsequent Figure 5 the description in the corresponding embodiment.

[0121] It should be noted that since the load controller corresponding to the resource instance in this application will be packaged into a container and deployed to the container orchestration system, each resource instance is essentially a container that runs the resource instance based on a pod. Based on this, the load controller created in this application should have the attributes that a pod should have, and the writing method of the load controller attributes should be the same as the pod object fields of the container orchestration system. Therefore, when creating the load controller in this application, the "code integration" method can be adopted to set the attributes set by the user and the default values of the pod attributes to determine the attributes of the load controller. In addition, for the name of each load controller in this application, it can be the same as the name of the resource instance configured by the user. In this way, it can be explained that this load controller is created by the resource controller of the custom alarm resource. During the process of managing the load controller of this custom alarm resource, other irrelevant controllers in the container orchestration system can be quickly filtered out through the name of the load controller.

[0122] Step S203: Deploy the load controller to the container orchestration system; the load controller deployed to the container orchestration system is used to perform alarm detection on the log data in the log repository corresponding to the target business application based on the alarm rules configured for the target business application.

[0123] In this application, after the load controller is created, the input parameters of the target resource instantiation event can be obtained (i.e., the parameters passed in by the user based on the necessary requirement fields of the custom alarm resource, and these parameters include the alarm rules configured by the user for the target business application), and a configuration file is created for the input parameters. Here, the configuration file can include a first configuration file (such as a configmap object) and a second configuration file (such as a secret object). The first configuration file can be used to record some configuration contents in the input parameters (such as alarm rules), and the second configuration file can be used to record the private information in the input parameters (such as authentication information, for example: user password, certificate information, address of the log database, etc.). Then, for some parameters in the input parameters including alarm rules, they can be written into the first configuration file, and for the private parameters in the input parameters, they can be written into the second configuration file; that is to say, the input parameters can be filled into the corresponding configuration files. After filling the input parameters into the configuration files, for the configuration files containing the input parameters, this application can call them filled configuration files. Regarding the parameters written in the filled configuration files, their data structures can be re-integrated to adjust the data formats of these parameters to the standard template format of a unified resource instance; then, the filled configuration files with the standard template format can be mounted on the above-mentioned load controller (specifically, the configmap object and the secret object can be respectively mounted to the specified paths). For the load controller with the filled configuration files mounted, this application can call it a mounted load controller. Finally, deploying the mounted load controller to the container orchestration system can complete the creation and deployment of the current resource instance of the target business application.

[0124] In specific implementation, the specific implementation process of deploying the mounted load controller to the container orchestration system can include but is not limited to: according to the container packaging rules, the mounted load controller can be packaged into a container to obtain the container corresponding to the mounted load controller. For easy distinction, the container corresponding to the mounted load controller can be called a packaged container; then, the packaged container can be deployed to the container orchestration system. After the packaged container is deployed to the container orchestration system, it can be considered that the alarm middleware of the resource instance is deployed to the container orchestration system. The packaged container can provide corresponding services for the target business application. For example, it can provide a log monitoring and alarm service for the target business application. Through the unique alarm rules in the packaged container, the log data in the log database where the target business application is located can be monitored in real time. Once log data that conforms to the alarm rules is found, an alarm notification can be sent.

[0125] It should be understood that through the custom controller, different resource instances can be divided based on the definition of the configuration file, and the alarm rules of different resource instances (i.e., different business applications) can be split at the smallest granularity. In the scenario of multiple sets of log databases and multiple business applications, it is also possible to effectively isolate the alarm rules of different business applications, effectively preventing different business applications or multiple business teams from sharing the same log alarm platform for log monitoring and alarming. It can also effectively prevent the situation that the log monitoring and alarming platform is restarted when a new alarm rule is released for a certain business application, which affects the operation of other business applications. At the same time, this application abstracts the resource objects built into the container orchestration system and extends the application program interface of the system. Developers or application maintenance users of business applications only need to be familiar with the configuration rules of custom alarm resources to perform various operations on resource instances. The entire process of creating, deleting, and updating resource instances is free of charge, reducing the resource usage cost and also reducing the deployment cost of the log alarm solution.

[0126] In the embodiment of this application, by registering or deploying custom alarm resources and resource controllers in the container orchestration system, different business applications can efficiently customize different resource instances based on different alarm rules, obtain different log monitoring and alarming solutions for different business applications, and have a very high matching degree with the alarm requirements of business applications. It can accurately monitor the log data that meets the alarm requirements of the business application itself, and can improve the log alarm performance. In addition, since the log alarm solution of this scheme is a solution proposed in combination with container capabilities, different business applications can use custom alarm resources to deploy multiple sets of resource instances in the container orchestration system. Different from the way of performing log monitoring and alarming on a single machine, the way of synchronously performing log monitoring and alarming on multiple sets of resource instances can adapt to alarm scenarios with a large amount of log data and a large number of alarm data, and can also improve the log alarm performance.

[0127] Furthermore, the following will elaborate in detail on the method of determining the event attributes of the target resource instantiation event and detecting the input parameters of the target resource instantiation event in combination with the accompanying drawings. Please refer to Figure 5 , Figure 5 is a schematic flow diagram of creating a load controller, and its process can correspond to the process of creating a load controller after obtaining the target resource instantiation event in the corresponding embodiment above. As Figure 4 shown, this process can at least include the following steps S501 - step S504: Figure 5 shown, this process can at least include the following steps S501 - step S504:

[0128] Step S501, determine the event attributes of the target resource instantiation event according to the target resource instance identifier in the target resource instantiation event; the event attributes include creation attributes and modification attributes.

[0129] Specifically, the target resource instance identifier in the target resource instantiation event can refer to the name configured by the user for this resource instance (which can be called the resource instance name), and this resource instance name can be input as a parameter when inputting parameters. That is to say, the target resource instance identifier refers to a resource instance name carried in the target resource instantiation event. The event attributes of the target resource instantiation event can be determined through this target resource instance identifier. The specific implementation process can include, but is not limited to: First, the existing resource instance set of the custom alarm resource can be obtained in the container orchestration system. Each existing resource instance refers to each resource instance of the custom alarm resource that has been created in the container orchestration system; then, the existing resource instance set is traversed according to the target resource instance identifier in the resource instance event; if there is an existing resource instance in the existing resource instance set whose resource instance identifier is the same as the target resource instance identifier, it can be considered that the resource instance specified by the user has been deployed in the container orchestration system, and the purpose of the user's resource instantiation event this time is to delete or update the existing resource instance. At this time, the event attribute of the target resource instance event can be determined as the modification attribute; correspondingly, if there is no existing resource instance in the existing resource instance set whose resource instance identifier is the same as the target resource instance identifier, it can be considered that the resource instance specified by the user does not exist in the container orchestration system yet, and the purpose of the user's resource instantiation event this time is to create a resource instance. At this time, the event attribute of the target resource instance event can be determined as the creation attribute.

[0130] It should be noted that since different business applications have corresponding namespaces in the container orchestration system, the user can also input the namespace of the target business application when inputting parameters. In this way, the target node can accurately and quickly determine whether a specified resource instance exists in the container orchestration system based on the resource instance name and the namespace of the target business application in the input parameters.

[0131] It should be understood that the event attributes of the target resource instantiation event may include creation attributes and modification attributes. When the event attribute of the target resource instantiation event is a creation attribute, the subsequent step S502 can be executed; while when the event attribute of the target resource instantiation event is a modification attribute, it is possible to further determine whether the event attribute of the target resource instantiation event is a deletion sub-attribute or an update sub-attribute. If it is a deletion sub-attribute, the resource instance specified by the user can be deleted (for example, the corresponding packaging container and configuration file can be cleared); if it is an update sub-attribute, the resource instance specified by the user can be updated (for example, the parameters in the configuration file can be updated). That is to say, when it is determined that the event attribute is a modification attribute, the input parameters of the target resource instance event can be obtained; then, based on the input parameters, the event sub-attribute to which the target resource instance event belongs can be determined between the deletion sub-attribute and the update sub-attribute. The specific process of determining the event sub-attribute may include but is not limited to: First, each parameter item included in the input parameters (that is, each parameter item required by each necessary requirement field) is respectively subjected to a non-nullity check to determine whether each parameter input by the user is a null value. If it is determined that each parameter item has non-nullity, it can be determined that the user did not input any parameters this time, and the user's intention is to delete this resource instance, then it can be determined that the event sub-attribute to which the target resource instance event belongs is a deletion sub-attribute; while if it is determined that there is any parameter item that does not have non-nullity, it can be considered that the user hopes to update an existing parameter based on the currently input parameters. At this time, it can be determined that the event sub-attribute to which the target resource instance event belongs is an update sub-attribute.

[0132] Further, if it is determined that the event sub-property to which the target resource instance event belongs is the deletion sub-property, the target existing resource instance with the resource instance identifier being the target resource instance identifier can be obtained in the container orchestration system, and the target existing resource instance can be cleared. If it is determined that the event sub-property to which the target resource instance event belongs is the update sub-property, the target existing resource instance with the resource instance identifier being the target resource instance identifier can be obtained in the container orchestration system, and the existing load controller corresponding to the target existing resource instance can be obtained; then, the existing configuration file mounted by the existing load controller can be obtained, and the existing configuration file mounted by the existing load controller can be determined as the target existing configuration file; the file content included in the target existing configuration file can be updated according to the input parameters (i.e., some parameters are updated). It should be noted that after the target existing resource instance is updated, the field used to control the restart of the load controller in the existing load controller can be updated accordingly, so that after the target existing configuration file corresponding to the existing load controller is updated, the existing load controller can be restarted accordingly to reload the updated target existing configuration file. For example, assume that the field used to control the restart of the load controller is the annotations field, and the field value specifically used to control the restart in this field is the "restart_at" value. Then, the "restart_at" value in this field can be updated. By updating the "restart_at" value of this field, the existing load controller can be triggered to restart and then reload the updated target existing configuration file, and based on the updated configuration file, the log data of the target business application can be monitored and alerted.

[0133] Step S502, if the event property is the creation property, obtain the basic resource requirement field and the specific resource requirement field of the custom alarm resource.

[0134] Specifically, after the above judgment, if it is determined that the event property of the target resource instantiation event is the creation property, the basic resource requirement field and the specific resource requirement field of the custom alarm resource can be obtained. The basic resource requirement field and the specific resource requirement field here need to be configured and defined together when registering the custom alarm resource. The specific definition method can refer to the description content in the corresponding embodiment above. Figure 4 The description content in the corresponding embodiment.

[0135] Step S503, perform parameter detection on the input parameters of the target resource instantiation event according to the basic resource requirements and the specific resource requirement field to obtain a parameter detection result; the input parameters include the alarm rule configured for the target business application and the target resource instance identifier.

[0136] Specifically, after obtaining the basic resource requirement field and the specific resource requirement field, the input parameters of the target resource instantiation event can be parameter-checked according to the basic resource requirement and the specific resource requirement field, so as to check whether the parameters input by the user meet the requirements of these fields. In a specific implementation, for the specific implementation process of parameter-checking the input parameters of the target resource instance event according to the basic resource requirement and the specific resource requirement field to obtain the parameter-checking result, it may include but is not limited to: First, according to the basic resource requirement field and the specific resource requirement field, the input parameters can be traversed to check whether the input parameters contain the parameters required by each requirement field; if it is determined that the first field parameter required by the basic resource requirement field exists in the input parameters, and the second field parameter required by the specific resource requirement field exists, then the parameter legality of the first field parameter and the second field parameter can be further checked to obtain the final parameter-checking result of the input parameters; if it is determined that the first field parameter required by the basic resource requirement field does not exist in the input parameters, or the second field parameter required by the specific resource requirement field does not exist, then the parameter-checking result of the input parameters can be directly determined as a non-passing result.

[0137] In a specific implementation, for the specific implementation process of checking the parameter legality of the first field parameter and the second field parameter to obtain the parameter-checking result of the input parameters, it may include but is not limited to: The first standard field format indicated by the basic resource requirement field and the second standard field format indicated by the specific resource requirement field can be obtained; then, the first parameter format of the first field parameter and the second parameter format of the second field parameter can be obtained; the first parameter format is matched with the first standard field format, so as to determine whether the parameter format of the first field parameter meets the requirements, and the second parameter format is matched with the second standard field format, so as to determine whether the parameter format of the second field parameter meets the requirements; if the first parameter format matches successfully with the first standard field format, and the second parameter format matches successfully with the second standard field format, then it can be determined that the parameter formats of all parameters meet the field format requirements, and the parameter-checking result of the input parameters can be determined as a passing result; if the first parameter format fails to match with the first standard field format, or the second parameter format fails to match with the second standard field format, then it can be considered that the parameter formats of some parameters do not meet the field format requirements, and the parameter-checking result of the input parameters can be determined as a non-passing result.

[0138] It should be understood that when performing parameter detection on the input parameters, the input parameters can be traversed first to determine whether the parameters required by each necessary requirement field of the custom alarm resource are included in the input parameters. If the user has input the parameters required by each necessary requirement field, then the parameter format and other contents of each input parameter can be further detected. Only after the parameter formats of all input parameters pass the detection can the parameter detection result of the input parameters be determined as the detection pass result. Of course, this application only describes the method of parameter detection for input parameters by way of example. For different services with different service requirements, the detection methods of input parameters will also be different. In other words, the parameter detection method of input parameters can be set accordingly according to the service requirements, and this application does not limit it. It is hereby specifically stated.

[0139] Step S504, if the parameter detection result indicates that the input parameters conform to the field rules, then execute the step of creating a load controller based on the alarm rules configured for the target business application.

[0140] Specifically, if the parameter result of the input parameters is the detection pass result, then it can be considered that the input parameters conform to the field rules. At this time, the step of creating a load controller based on the alarm rules configured for the target business application can be executed. If the parameter result of the input parameters is the detection failure result, then a prompt indicating that the parameters are unqualified can be returned to the user, and this prompt is used to instruct the user to re-enter the qualified parameters.

[0141] It should be noted that if a resource instance (such as the resource instance indicated by the target resource instantiation event) is successfully created and deployed to the container orchestration system, the field value of the status field of the resource instance can be modified to "success" to indicate that the creation of the resource instance is completed; otherwise, the field value of the status field of the resource instance can be modified to "failed" to indicate that the creation of the resource instance is not completed. When the creation of the resource instance is not completed, additional instructions can be added to the message field. This additional instruction can be used to explain the reason for the failure of the resource instance creation, which can facilitate the maintenance personnel of the container orchestration system to troubleshoot the faults of the container orchestration system based on past usage habits. It should also be noted that for the above-mentioned resource controller for monitoring custom alarm resources, a background thread can be started separately to detect the health status of the load controller of the custom alarm resources. If a pod resource under a certain load controller fails, the resource controller can promptly modify the health status of the corresponding resource instance: for example, set the numerical value of the status field of the corresponding resource instance to "failed". Optionally, for the load controller, certain measures can also be taken to monitor the running status of the load controller. By monitoring the running status of the load controller, the health status of the load controller can also be monitored. In this way, when there is an abnormal situation in the load controller, the status data of the corresponding resource instance can be promptly modified.

[0142] In the embodiments of the present application, by registering or deploying custom alarm resources and resource controllers in the container orchestration system, different business applications can conveniently customize different resource instances based on different alarm rules, obtain different log monitoring and alarm schemes for different business applications, have a very high matching degree with the alarm requirements of business applications, can accurately monitor the log data that meets the alarm requirements of business applications themselves, and can improve the log alarm performance; in addition, since the log alarm scheme of this solution is a scheme proposed in combination with container capabilities, different business applications can use custom alarm resources to deploy multiple sets of resource instances in the container orchestration system. Different from the way of single-machine log monitoring and alarm, the way of multiple sets of resource instances synchronously performing log monitoring and alarm can adapt to alarm scenarios with a large amount of log data and a large number of alarm data, and can also improve the log alarm performance.

[0143] Furthermore, for the sake of understanding, the following will elaborate on the process logic framework of this solution in conjunction with the accompanying drawings. Please refer to Figure 6 , Figure 6 which is a schematic diagram of the logical architecture of a log alarm scheme provided by the embodiments of the present application. In combination with the accompanying Figure 6, the logical process for the present application to implement log monitoring and alerting of the container orchestration system may at least include the following steps: 1. First, a code repository needs to be deployed and a corresponding rule configuration repository needs to be created; 2. After the repositories are configured, a container cluster can be deployed, that is, a container orchestration system can be deployed; 3. Deploy an efficient operation and maintenance tool for container applications (such as the fluxcd tool) to the container orchestration system, and configure the address of the code repository that needs to be synchronized and related authentication content for it; 4. Deploy a log database cluster, and write the log data of different business applications into different log databases in the log database cluster; 5. Through the above steps 1 - 4, the deployment of the container orchestration system and related authentication processes can be completed. After that, an open-source tool (such as elastalert) for creating a load controller can be used to package the open-source tool process into an image of a container creation tool (such as docker) according to a specified syntax (such as the dockerfile syntax), and push this image to the container tool library. This step can specify that when creating a load controller later, an image should be specified to package the load controller into a container; 6. Package the resource controller of the developed custom alert resource into an image of a container creation tool according to a specified syntax and name it (such as naming it elastalert-operator); 7. Deploy the packaged resource controller to the container orchestration system (that is, deploy it to the target node) through the deployment resource. Through steps 6 - 7, the development, creation, packaging, and deployment of the resource controller can be completed; 8. Further, create a custom alert resource and generate the basic structure of this resource. This step is also to write a declaration style file (such as a YAML file) for the custom alert resource, with the purpose of being able to register the custom alert resource in the container orchestration system. Regarding the sequence of creating and registering the custom alert resource and creating and deploying the resource controller, the present application does not limit it and hereby explains it; 9. After the resource controller is deployed in the container orchestration system, the user can write the declaration style file of the corresponding resource instance according to the rules of the custom alert resource and their own log alerting requirements for business applications. The user can push this declaration style file to the code repository, then the target node with the deployed resource controller can obtain this declaration style file from the code repository, and further obtain the resource instantiation event of the custom alert resource by the user. After determining that it is a resource instance creation event, the target node can generate a load controller and a configuration file for this alert rule (the number of configuration files can be multiple. Among them, configuration file 1 can be a configmap object, and configuration file 2 can be a secret object). The resource controller can mount this configuration file to the load controller, and package and generate a corresponding container and deploy it to the container orchestration system;10. Through the alarm rules in the deployed containers, the log data in the log database where the business application is located can be monitored and alarmed.

[0144] It should be understood that the overall essence of this application is to develop a controller for the container orchestration system. By adding custom alarm resources, creating resource controllers, creating a deployment object of the resource instance, and packaging it into a container, the control ability of the container orchestration system can be realized, a resource instance can be quickly launched in the container orchestration system, and any custom configuration file can be read and mounted into the application of the resource instance, enabling the business application to obtain corresponding configuration information and alarm rules from the configuration file. This solution is applicable to any business application running on the container orchestration system and having log alarm requirements, facilitating the bundling and configuration of the business application and the resource instance. Different business applications can deploy multiple sets of alarm resource instances in their own namespaces to obtain multiple sets of log alarm solutions. The multiple sets of log alarm solutions work together to solve the problem of insufficient single-machine performance, and thus the log alarm performance can also be improved.

[0145] Further, please refer to Figure 7 , Figure 7 is a schematic structural diagram of a data processing device provided by an embodiment of the present application. The data processing device can be a computer program (including program code) running in a computer device. For example, the data processing device is an application software; the data processing device can be used to execute Figure 4 the method shown. As Figure 7 shown, the data processing device 1 may include: an event acquisition module 11, a controller creation module 12, and a controller deployment module 13.

[0146] The event acquisition module 11 is used to obtain a target resource instantiation event that causes a resource change of the custom alarm resource according to a resource change signal sent by the resource controller; the target resource instantiation event carries alarm rules configured for the target business application; the target business application refers to any business application running on the container orchestration system;

[0147] The controller creation module 12 is used to create a load controller based on the alarm rules configured for the target business application;

[0148] The controller deployment module 13 is used to deploy the load controller to the container orchestration system; the load controller deployed to the container orchestration system is used to perform alarm detection on the log data in the log repository corresponding to the target business application based on the alarm rules configured for the target business application.

[0149] Among them, for the specific implementation manners of the event acquisition module 11, the controller creation module 12, and the controller deployment module 13, reference can be made to the aboveFigure 4 The descriptions of steps S201 - S203 in the corresponding embodiments will not be elaborated here.

[0150] In one embodiment, before the event acquisition module 11 acquires the target resource instantiation event that causes a resource change in the custom alarm resource according to the resource change signal sent by the resource controller, the data processing device 1 further includes: a resource controller deployment module 14.

[0151] The resource controller deployment module 14 is used to acquire a program skeleton containing an entry function of the program;

[0152] The resource controller deployment module 14 is further used to fill an event handling function for the custom alarm resource in the program skeleton; the event handling function contains configured event handling logic, and the event handling function is used to respond to different resource instantiation events of the custom alarm resource based on the event handling logic;

[0153] The resource controller deployment module 14 is further used to determine the program skeleton containing the event handling function as the resource controller and deploy the resource controller to the target node.

[0154] Among them, for the specific implementation manner of the resource controller deployment module 14, reference can be made to the relevant description in step S201 of the above Figure 4 The corresponding embodiments will not be elaborated here.

[0155] In one embodiment, before the event acquisition module 11 acquires the target resource instantiation event that causes a resource change in the custom alarm resource according to the resource change signal sent by the resource controller, the data processing device 1 further includes: a custom resource deployment module 15.

[0156] The custom resource deployment module 15 is used to generate a basic structure of the custom alarm resource through a custom controller development framework; the basic structure contains basic resource requirement fields configured by the custom controller development framework;

[0157] The custom resource deployment module 15 is further used to acquire specific resource requirement fields defined for the custom alarm resource and write a declaration style file for the custom alarm resource based on the basic resource requirement fields and the specific resource requirement fields;

[0158] The custom resource deployment module 15 is further used to register the custom alarm resource into the container orchestration system through the declaration style file.

[0159] Among them, for the specific implementation manner of the custom resource deployment module 15, reference can be made to the relevant description in step S201 of the above Figure 4 The corresponding embodiments will not be elaborated here.

[0160] In one embodiment, the target resource instantiation event also carries a target resource instance identifier;

[0161] The data processing device 1 further includes: an event attribute determination module 16, a parameter detection module 17, and a step execution module 18.

[0162] The event attribute determination module 16 is configured to determine the event attribute of the target resource instantiation event according to the target resource instance identifier in the target resource instantiation event; the event attribute includes a creation attribute and a modification attribute;

[0163] The parameter detection module 17 is configured to, if the event attribute is a creation attribute, obtain the basic resource requirement field and the specific resource requirement field of the custom alarm resource;

[0164] The parameter detection module 17 is further configured to perform parameter detection on the input parameters of the target resource instantiation event according to the basic resource requirement and the specific resource requirement field to obtain a parameter detection result; the input parameters include the alarm rule configured for the target business application and the target resource instance identifier;

[0165] The step execution module 18 is configured to, if the parameter detection result indicates that the input parameters conform to the field rules, execute the step of creating a load controller based on the alarm rule configured for the target business application.

[0166] Among them, for the specific implementation manners of the event attribute determination module 16, the parameter detection module 17, and the step execution module 18, reference may be made to the descriptions of steps S501 - S504 in the corresponding embodiment above, which will not be elaborated here. Figure 5 In one embodiment, the specific implementation manner of the event attribute determination module 16 determining the event attribute of the target resource instantiation event according to the target resource instance identifier in the target resource instantiation event includes:

[0167] Obtaining the existing resource instance set of the custom alarm resource in the container orchestration system;

[0168] Traversing the existing resource instance set according to the target resource instance identifier in the resource instance event;

[0169] If there is an existing resource instance in the existing resource instance set with a resource instance identifier identical to the target resource instance identifier, determining the event attribute of the target resource instantiation event as a modification attribute;

[0170] If there is no existing resource instance in the existing resource instance set with a resource instance identifier identical to the target resource instance identifier, determining the event attribute of the target resource instantiation event as a creation attribute.

[0171] ​

[0172] In one embodiment, the specific implementation manner in which the parameter detection module 17 performs parameter detection on the input parameters of the target resource instantiation event according to the basic resource requirements and the specific resource requirements fields includes:

[0173] Traverse the input parameters according to the basic resource requirements field and the specific resource requirements field;

[0174] If it is determined that there is a first field parameter required by the basic resource requirements field and a second field parameter required by the specific resource requirements field in the input parameters, then detect the parameter legality of the first field parameter and the second field parameter to obtain the parameter detection result of the input parameters;

[0175] If it is determined that there is no first field parameter required by the basic resource requirements field or no second field parameter required by the specific resource requirements field in the input parameters, then determine the parameter detection result of the input parameters as a detection failure result.

[0176] In one embodiment, the specific implementation manner in which the parameter detection module 17 detects the parameter legality of the first field parameter and the second field parameter to obtain the parameter detection result of the input parameters includes:

[0177] Obtain the first standard field format indicated by the basic resource requirements field and the second standard field format indicated by the specific resource requirements field;

[0178] Obtain the first parameter format of the first field parameter and the second parameter format of the second field parameter;

[0179] Match the first parameter format with the first standard field format, and match the second parameter format with the second standard field format;

[0180] If the first parameter format matches the first standard field format successfully and the second parameter format matches the second standard field format successfully, then determine the parameter detection result of the input parameters as a detection pass result;

[0181] If the first parameter format fails to match the first standard field format or the second parameter format fails to match the second standard field format, then determine the parameter detection result of the input parameters as a detection failure result.

[0182] In one embodiment, modifying an attribute includes deleting a sub-attribute and updating a sub-attribute;

[0183] After the event attribute determination module 16 determines the event attribute of the target resource instantiation event according to the target resource instance identifier in the target resource instantiation event, the data processing device 1 further includes: a sub-attribute determination module 19 and an instance clearing module 20.

[0184] The sub - attribute determination module 19 is configured to, if the event attribute is a modification attribute, obtain the input parameters of the target resource instantiation event; the input parameters include the alarm rules configured for the target business application.

[0185] The sub - attribute determination module 19 is further configured to determine the event sub - attribute to which the target resource instantiation event belongs between the deletion sub - attribute and the update sub - attribute based on the input parameters.

[0186] The instance clearing module 20 is configured to, if it is determined that the event sub - attribute to which the target resource instantiation event belongs is the deletion sub - attribute, obtain the target existing resource instance with the resource instance identifier being the target resource instance identifier in the container orchestration system, and perform a clearing process on the target existing resource instance.

[0187] Among them, for the specific implementation manners of the sub - attribute determination module 19 and the instance clearing module 20, reference can be made to the relevant descriptions in step S501 of the corresponding embodiment above, which will not be elaborated here. Figure 5 The relevant descriptions in step S501 of the corresponding embodiment above will not be elaborated here.

[0188] In one embodiment, after the sub - attribute determination module 19 determines the event sub - attribute to which the target resource instantiation event belongs between the deletion sub - attribute and the update sub - attribute based on the input parameters, the data processing device 1 further includes: an instance update module 21.

[0189] The instance update module 21 is configured to, if it is determined that the event sub - attribute to which the target resource instantiation event belongs is the update sub - attribute, obtain the target existing resource instance with the resource instance identifier being the target resource instance identifier in the container orchestration system, and obtain the existing load controller corresponding to the target existing resource instance.

[0190] The instance update module 21 is further configured to obtain the existing configuration file mounted by the existing load controller, and determine the existing configuration file mounted by the existing load controller as the target existing configuration file.

[0191] The instance update module 21 is further configured to update the file content included in the target existing configuration file according to the input parameters.

[0192] Among them, for the specific implementation manner of the instance update module 21, reference can be made to the relevant descriptions in step S501 of the corresponding embodiment above, which will not be elaborated here. Figure 5 The relevant descriptions in step S501 of the corresponding embodiment above will not be elaborated here.

[0193] In one embodiment, the specific implementation manner of the sub - attribute determination module 19 to determine the event sub - attribute to which the target resource instantiation event belongs between the deletion sub - attribute and the update sub - attribute based on the input parameters includes:

[0194] Perform a non - emptiness check on each parameter item included in the input parameters respectively;

[0195] If it is determined that each parameter item has non - emptiness, determine that the event sub - attribute to which the target resource instantiation event belongs is the deletion sub - attribute;

[0196] If it is determined that any one parameter item does not have non - emptiness, determine that the event sub - attribute to which the target resource instantiation event belongs is the update sub - attribute.

[0197] In one embodiment, the specific implementation manner in which the controller deployment module 13 deploys the load controller to the container orchestration system includes:

[0198] Obtain the input parameters of the target resource instantiation event; the input parameters include the alarm rules configured for the target business application;

[0199] Create a configuration file for the input parameters, and fill the input parameters into the configuration file to obtain a filled configuration file;

[0200] Mount the filled configuration file to the load controller to obtain a mounted load controller;

[0201] Deploy the mounted load controller to the container orchestration system.

[0202] In one embodiment, the specific implementation manner in which the controller deployment module 13 deploys the mounted load controller to the container orchestration system includes:

[0203] Perform container packaging processing on the mounted load controller according to the container packaging rules to obtain a packaged container corresponding to the mounted load controller;

[0204] Deploy the packaged container to the container orchestration system.

[0205] In the embodiments of the present application, by registering or deploying custom alarm resources and resource controllers in the container orchestration system, different business applications can conveniently customize different resource instances based on different alarm rules, obtain different log monitoring and alarm schemes for different business applications, which have a very high matching degree with the alarm requirements of business applications, can accurately monitor log data that meets the alarm requirements of the business application itself, and can improve the log alarm performance; in addition, since the log alarm scheme of this solution is proposed in combination with container capabilities, different business applications can use custom alarm resources to deploy multiple sets of resource instances in the container orchestration system. Different from the way of performing log monitoring and alarm on a single machine, the way of synchronously performing log monitoring and alarm on multiple sets of resource instances can adapt to alarm scenarios with a large amount of log data and a large number of alarm data, and can also improve the log alarm performance.

[0206] Further, please refer to Figure 8 ,Figure 8 is a schematic structural diagram of a computer device provided by an embodiment of the present application. As Figure 8 shown, the computer device 8000 may include: a processor 8001, a network interface 8004, and a memory 8005. In addition, the computer device 8000 further includes: a user interface 8003 and at least one communication bus 8002. Among them, the communication bus 8002 is used to realize the connection and communication between these components. Among them, the user interface 8003 may include a display screen (Display) and a keyboard (Keyboard). Optionally, the user interface 8003 may further include a standard wired interface and a wireless interface. The network interface 8004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 8005 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. Optionally, the memory 8005 may further be at least one storage device located far from the aforementioned processor 8001. As Figure 8 shown, in the memory 8005 as a computer-readable storage medium, there may be included an operating system, a network communication module, a user interface module, and a device control application program.

[0207] In Figure 8 the computer device 8000 shown, the network interface 8004 can provide network communication functions; while the user interface 8003 is mainly used to provide an input interface for users; and the processor 8001 can be used to call the device control application program stored in the memory 8005 to implement:

[0208] According to the resource change signal sent by the resource controller, obtain the target resource instantiation event that causes the custom alarm resource to have a resource change; the target resource instantiation event carries the alarm rule configured for the target service application; the target service application refers to any service application running in the container orchestration system;

[0209] Create a load controller based on the alarm rule configured for the target service application;

[0210] Deploy the load controller to the container orchestration system; the load controller deployed to the container orchestration system is used to perform alarm detection on the log data in the log repository corresponding to the target service application based on the alarm rule configured for the target service application.

[0211] It should be understood that the computer device 8000 described in the embodiments of the present application can execute the description of the data processing method in the corresponding embodiments described above, and can also execute the above Figures 4 to 6 and can also execute the above Figure 7The description of the data processing device 1 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated either.

[0212] In addition, it should be noted here that: The embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the aforementioned computer device 8000 for data processing. The computer program includes program instructions. When the aforementioned processor executes the program instructions, it can execute the Figures 4 to 6 description of the aforementioned data processing method in the corresponding embodiment. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated either. For the technical details not disclosed in the embodiment of the computer-readable storage medium involved in the present application, please refer to the description of the method embodiment of the present application.

[0213] The aforementioned computer-readable storage medium may be the internal storage unit of the data processing device or the aforementioned computer device provided in any of the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit and the external storage device of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store the data that has been output or will be output.

[0214] In one aspect of the present application, a computer program product is provided. The computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in one aspect of the embodiment of the present application.

[0215] The terms "first", "second", etc. in the description, claims and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally further includes steps or modules not listed, or optionally further includes other step units inherent to these processes, methods, devices, products or equipment.

[0216] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the function of the module or unit.

[0217] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0218] The methods and related devices provided in the embodiments of the present application are described with reference to the method flowcharts and / or structural schematic diagrams provided in the embodiments of the present application. Specifically, each process and / or block of the method flowchart and / or structural schematic diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 a process or multiple processes and / or structural schematic Figure 1 a block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 a process or multiple processes and / or structural schematic Figure 1 a block or multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide for implementing the functions specified in Figure 1One process or multiple processes and / or structures indicate steps of functions specified in one box or multiple boxes.

[0219] The foregoing disclosure is only for the preferred embodiments of the present application, and of course cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. A data processing method, characterized in that, The method is applied to a container orchestration system which has pre-registered custom alarm resources. The method is executed by a target node in the container orchestration system. The target node refers to a computing node in the container orchestration system where a resource controller is deployed. The resource controller is used to monitor resource changes of the custom alarm resources. The method includes: According to the resource change signal sent by the resource controller, obtain a target resource instantiation event that causes the custom alarm resource to have a resource change. The target resource instantiation event carries an alarm rule configured for a target business application. The target business application refers to any business application running in the container orchestration system; Create a load controller based on the alarm rule configured for the target business application; Deploy the load controller into the container orchestration system. The load controller deployed into the container orchestration system is used to perform alarm detection on the log data in the log repository corresponding to the target business application based on the alarm rule configured for the target business application.

2. The method according to claim 1, wherein Before obtaining, according to the resource change signal sent by the resource controller, a target resource instantiation event that causes the custom alarm resource to have a resource change, the method further includes: Obtain a program skeleton containing an entry function; Fill an event processing function for the custom alarm resource in the program skeleton. The event processing function contains configured event processing logic and is used to respond to different resource instantiation events of the custom alarm resource based on the event processing logic; Determine the program skeleton containing the event processing function as a resource controller and deploy the resource controller into the target node.

3. The method according to claim 1, wherein Before obtaining, according to the resource change signal sent by the resource controller, a target resource instantiation event that causes the custom alarm resource to have a resource change, the method further includes: Generate a basic structure of the custom alarm resource through a custom controller development framework. The basic structure contains basic resource requirement fields configured by the custom controller development framework; Obtain specific resource requirement fields defined for the custom alarm resource and write a declaration style file for the custom alarm resource based on the basic resource requirement fields and the specific resource requirement fields; Register the custom alarm resource into the container orchestration system through the declaration style file.

4. The method according to claim 1, wherein The target resource instantiation event also carries a target resource instance identifier; The method further includes: Determine the event attribute of the target resource instantiation event according to the target resource instance identifier in the target resource instantiation event. The event attribute includes a creation attribute and a modification attribute; If the event attribute is a creation attribute, obtain the basic resource requirement fields and specific resource requirement fields of the custom alarm resource; Perform parameter detection on the input parameters of the target resource instantiation event according to the described basic resource requirements and the specific resource requirement fields, and obtain a parameter detection result; the input parameters include the alarm rules configured for the target business application and the target resource instance identifier. If the parameter detection result indicates that the input parameters conform to the field rules, then perform the step of creating a load controller based on the alarm rules configured for the target business application.

5. The method according to claim 4, characterized in that, Determine the event attributes of the target resource instantiation event according to the target resource instance identifier in the target resource instantiation event, including: Obtain the existing resource instance set of the custom alarm resource in the container orchestration system; Traverse the existing resource instance set according to the target resource instance identifier in the resource instance event; If there is an existing resource instance in the existing resource instance set with a resource instance identifier identical to the target resource instance identifier, then determine the event attribute of the target resource instantiation event as a modification attribute; If there is no existing resource instance in the existing resource instance set with a resource instance identifier identical to the target resource instance identifier, then determine the event attribute of the target resource instantiation event as a creation attribute.

6. The method according to claim 4, wherein Perform parameter detection on the input parameters of the target resource instantiation event according to the described basic resource requirements and the specific resource requirement fields, and obtain a parameter detection result, including: Traverse the input parameters according to the basic resource requirement fields and the specific resource requirement fields; If it is determined that the first field parameter required by the basic resource requirement fields exists in the input parameters and the second field parameter required by the specific resource requirement fields exists, then perform parameter legality detection on the first field parameter and the second field parameter to obtain the parameter detection result of the input parameters; If it is determined that the first field parameter required by the basic resource requirement fields does not exist in the input parameters or the second field parameter required by the specific resource requirement fields does not exist, then determine the parameter detection result of the input parameters as a detection failure result.

7. The method according to claim 6, wherein Perform parameter legality detection on the first field parameter and the second field parameter to obtain the parameter detection result of the input parameters, including: Obtain the first standard field format indicated by the basic resource requirement fields and the second standard field format indicated by the specific resource requirement fields; Obtain the first parameter format of the first field parameter and the second parameter format of the second field parameter; Match the first parameter format with the first standard field format and match the second parameter format with the second standard field format; If the first parameter format matches the first standard field format successfully and the second parameter format matches the second standard field format successfully, then determine the parameter detection result of the input parameters as a detection pass result; If the matching between the first parameter format and the first standard field format fails, or the matching between the second parameter format and the second standard field format fails, then determine the parameter detection result of the input parameter as a detection failure result.

8. The method according to claim 4, wherein The modification attributes include deleting sub-attributes and updating sub-attributes; After determining the event attributes of the target resource instantiation event according to the target resource instance identifier in the target resource instantiation event, the method further includes: If the event attribute is a modification attribute, obtain the input parameter of the target resource instantiation event; the input parameter includes the alarm rules configured for the target business application; Based on the input parameter, determine the event sub-attribute to which the target resource instantiation event belongs among the deleting sub-attributes and the updating sub-attributes; If it is determined that the event sub-attribute to which the target resource instantiation event belongs is the deleting sub-attribute, obtain the target existing resource instance with the resource instance identifier being the target resource instance identifier in the container orchestration system, and perform a clearing process on the target existing resource instance.

9. The method according to claim 8, characterized in that, After determining the event sub-attribute to which the target resource instantiation event belongs among the deleting sub-attributes and the updating sub-attributes based on the input parameter, the method further includes: If it is determined that the event sub-attribute to which the target resource instantiation event belongs is the updating sub-attribute, obtain the target existing resource instance with the resource instance identifier being the target resource instance identifier in the container orchestration system, and obtain the existing load controller corresponding to the target existing resource instance; Obtain the existing configuration file mounted by the existing load controller, and determine the existing configuration file mounted by the existing load controller as the target existing configuration file; Update the file content included in the target existing configuration file according to the input parameter.

10. The method according to claim 8, wherein The determining the event sub-attribute to which the target resource instantiation event belongs among the deleting sub-attributes and the updating sub-attributes based on the input parameter includes: Perform a non-emptiness check on each parameter item included in the input parameter; If it is determined that each parameter item has non-emptiness, determine that the event sub-attribute to which the target resource instantiation event belongs is the deleting sub-attribute; If it is determined that any one parameter item does not have non-emptiness, determine that the event sub-attribute to which the target resource instantiation event belongs is the updating sub-attribute.

11. The method according to claim 1, characterized in that The deploying the load controller to the container orchestration system includes: Obtain the input parameter of the target resource instantiation event; the input parameter includes the alarm rules configured for the target business application; Create a configuration file for the input parameter, and fill the input parameter into the configuration file to obtain a filled configuration file; Mount the filled configuration file to the load controller to obtain a mounted load controller; Deploy the mounted load controller to the container orchestration system.

12. The method according to claim 11, wherein The deploying the mounted load controller to the container orchestration system includes: Perform container packaging processing on the mounted load controller according to the container packaging rules to obtain the packaging container corresponding to the mounted load controller; Deploy the packaged container to the container orchestration system.

13. A data processing device, characterized in that, The device includes: An event acquisition module, configured to acquire a target resource instantiation event that causes a resource change in the custom alarm resource according to a resource change signal sent by the resource controller; the target resource instantiation event carries an alarm rule configured for a target business application; the target business application refers to any business application running in the container orchestration system; A controller creation module, configured to create a load controller based on the alarm rule configured for the target business application; A controller deployment module, configured to deploy the load controller to the container orchestration system; the load controller deployed to the container orchestration system is configured to perform alarm detection on log data in a log repository corresponding to the target business application based on the alarm rule configured for the target business application.

14. A computer device, characterized in that, It includes: A processor, a memory, and a network interface; The processor is connected to the memory and the network interface. Among them, the network interface is used to provide network communication functions, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device executes the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, and the computer program is adapted to be loaded and executed by a processor to execute the method according to any one of claims 1-12.

16. A computer program product, characterized in that, The computer program product includes a computer program, the computer program is stored in a computer-readable storage medium, and the computer program is adapted to be read and executed by a processor so that a computer device having the processor executes the method according to any one of claims 1-12.

Images