Index data collection method, device, server and storage medium

Abstract

The present application provides an indicator data collection method, device, server and storage medium, which relate to the field of computer technology. The method includes: obtaining an indicator string through a virtual interface, the indicator string including: multiple indicator data, the indicator data being an indicator in the process of the server executing at least one distributed task; according to the task identifier of the distributed task to which each indicator data belongs, a preset mapping method is used to determine the mapping relationship between multiple indicator collection components and each indicator data; according to the mapping relationship, each indicator data is sent to the corresponding indicator collection component, so that the data monitoring system collects the corresponding indicator data through each indicator collection component and monitors the indicator data. The present application can enable the data monitoring system to obtain indicator data through multiple indicator collection components, thereby improving the performance of the data monitoring system in monitoring and analyzing indicator data.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an indicator data collection method, device, server, and storage medium. Background Art

[0002] With the rise of computer communication technologies such as cloud computing and microservices, the real-time distributed processing engine Flink is used to perform stateful computing on unbounded and bounded data streams. Flink tasks built on the Flink engine are run by servers in the service cluster to perform computing at memory speed and at any scale.

[0003] In order to collect and monitor the indicator data when the servers in the service cluster execute Flink tasks, an open source data monitoring system is used to pull the indicator data obtained by the indicator collection component from the servers in the service cluster from the indicator collection component, and analyze, calculate and store the indicator data.

[0004] However, according to Flink's current configuration, the data monitoring system can only use one indicator collection component to pull indicator data. When there are a lot of Flink tasks, the corresponding indicator data will also be very large. The indicator data will accumulate in the indicator collection component and cannot be pulled by the data monitoring system in a timely manner, seriously affecting the data monitoring system's performance in monitoring and analyzing indicator data. Summary of the Invention

[0005] The purpose of this application is to provide an indicator data collection method, device, server and storage medium to address the deficiencies in the above-mentioned existing technologies, so that a data monitoring system can obtain indicator data through multiple indicator collection components and improve the performance of the data monitoring system in monitoring and analyzing indicator data.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0007] In a first aspect, an embodiment of the present application provides an indicator data collection method, which is applied to a server in a service cluster, wherein the server is deployed with a virtual interface, and the method includes:

[0008] Acquire an indicator string through the virtual interface, wherein the indicator string includes: a plurality of indicator data, wherein the indicator data is an indicator in a process in which the server executes at least one distributed task;

[0009] According to the task identifier of the distributed task to which each indicator data belongs, a preset mapping method is used to determine the mapping relationship between the multiple indicator collection components and each indicator data;

[0010] According to the mapping relationship, each indicator data is sent to a corresponding indicator collection component, so that the data monitoring system collects the corresponding indicator data through each indicator collection component and monitors the indicator data.

[0011] In a second aspect, an embodiment of the present application further provides an indicator data collection device, which is applied to a server in a service cluster, wherein the server is deployed with a virtual interface, and the device includes:

[0012] An indicator data acquisition module, configured to acquire an indicator string through the virtual interface, wherein the indicator string includes: a plurality of indicator data, wherein the indicator data is an indicator in a process in which the server executes at least one distributed task;

[0013] A mapping relationship determination module is used to determine the mapping relationship between the plurality of indicator collection components and each indicator data according to the task identifier of the distributed task to which each indicator data belongs, using a preset mapping method;

[0014] The indicator data sending module is used to send each indicator data to the corresponding indicator collection component according to the mapping relationship, so that the data monitoring system collects the corresponding indicator data through each indicator collection component and monitors the indicator data.

[0015] The beneficial effects of this application are:

[0016] The present application provides an indicator data collection method, device, server and storage medium, which set a virtual interface and multiple indicator collection components. After obtaining indicator data through the virtual interface, the indicator data is distributed to different indicator collection components, so that the data monitoring system pulls the indicator data through each indicator collection component, avoiding the problem that a large amount of indicator data can only be sent to the data monitoring system through one indicator collection component, causing the indicator data to accumulate in the indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, which can share the pressure of a single indicator collection component, thereby improving the performance of the data monitoring system in monitoring and analyzing indicator data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A schematic diagram of the structure of the indicator data collection system provided in an embodiment of the present application;

[0019] Figure 2 Schematic diagram of the process of the indicator data collection method provided in the embodiment of this application Figure 1 ;

[0020] Figure 3 Schematic diagram of the process of the indicator data collection method provided in the embodiment of this application Figure 2 ;

[0021] Figure 4 A schematic diagram of a hash ring mapping provided in an embodiment of the present application;

[0022] Figure 5 Schematic diagram of the process of the indicator data collection method provided in the embodiment of this application Figure 3 ;

[0023] Figure 6 Schematic diagram of the process of the indicator data collection method provided in the embodiment of this application Figure 4 ;

[0024] Figure 7 Schematic diagram of the process of the indicator data collection method provided in the embodiment of this application Figure 5 ;

[0025] Figure 8 A schematic diagram of the structure of the indicator data collection device provided in an embodiment of the present application;

[0026] Figure 9 A schematic diagram of the server provided for this application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0028] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0029] In addition, the terms "comprises" and "having" and any variations thereof are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product or apparatus.

[0030] First, the relevant contents involved in the embodiments of this application are introduced:

[0031] After submitting a Flink task to a server cluster for execution, developers often struggle to understand the actual status of the task, such as its execution speed and any abnormalities. They also lack the ability to view all task logs in real time, making it difficult to manage large or numerous Flink tasks. Flink Metrics can help developers understand task status and server operation status by collecting Flink content indicators.

[0032] Metrics has two major functions. First, it serves as a measurement system that collects Flink task metrics in real time. Users can view the status, latency, and monitoring information of their submitted tasks on the Flink UI. Second, it can be exposed to external systems. By configuring a reporter as an HTTP interface in the Flink configuration file flink-conf.yaml, the metrics data of the entire cluster can be reported to third-party monitoring systems such as Prometheus and Grafana through the interface.

[0033] Currently, Flink's UI only displays the instantaneous indicator data of the task, but cannot view historical indicator data. Therefore, you can use Flink's HTTP interface to build a visual monitoring system through a data monitoring system, indicator collection components, and visualization components. When a user submits a Flink task to a server in a service cluster using the Flink framework, Flink pushes the indicator data to the indicator collection component through the HTTP interface. The data monitoring system then pulls the collected indicators from the indicator collection component and displays them through visualization tools.

[0034] According to Flink's current configuration, the data monitoring system can only use one indicator collection component to pull indicator data. When there are many Flink tasks, the corresponding indicator data will also be very large. The indicator data will accumulate in the indicator collection component and cannot be pulled by the data monitoring system in a timely manner, seriously affecting the data monitoring system's performance in monitoring and analyzing indicator data.

[0035] In order to solve the problems existing in the above-mentioned prior art, in an embodiment of the present application, a virtual interface and multiple indicator collection components are set up. After the indicator data is sent to the virtual interface, the indicator data is distributed to different indicator collection components through the virtual interface, so that the data monitoring system pulls the indicator data through each indicator collection component, avoiding the problem that a large amount of indicator data can only be sent to the data monitoring system through one indicator collection component, causing the indicator data to accumulate in the indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, which can share the pressure of a single indicator collection component, thereby improving the performance of the data monitoring system in monitoring and analyzing indicator data.

[0036] Before describing in detail the indicator data collection method, device, server, and storage medium provided in this application, in order to facilitate understanding of the solution of this application, the indicator data collection system targeted by this application is first introduced.

[0037] Please refer to Figure 1 , is a structural diagram of the indicator data collection system provided in an embodiment of the present application, such as Figure 1 As shown, the indicator data collection system may include a service cluster 10 , multiple indicator collection components 20 , a data monitoring system 30 and a visualization component 40 , wherein the service cluster includes: multiple task servers 11 and an interface server 12 .

[0038] Multiple task servers 11 are used to receive and execute Flink tasks sent by users through the task configuration end. The task configuration end can be set on an independent terminal device, or it can also be set on the terminal device to which the task server belongs. When multiple task servers 11 send indicator data through the HTTP interface, the indicator data will be sent to the interface server 12 in a unified manner according to the address of the interface server 12.

[0039] A virtual interface is created on the interface server 12, and the interface server 12 is respectively communicated with the task server 11 and at least one data collection component 20 to receive indicator data from the task server 11 through the virtual interface, and push the indicator data to different indicator collection components 20 according to the Flink task to which the indicator data belongs.

[0040] The indicator collection component 20 is respectively connected to the interface server 12 and the data monitoring system 30 for communication, and is used to receive the indicator data pushed by the interface server and pull the indicator data as a target by the data monitoring system 30.

[0041] The data monitoring system 30 is respectively connected to the at least one indicator collection component 20 and the visualization component 40 for communication, and is used to pull indicator data from the at least one indicator collection component and visualize the indicator data through the visualization component 40 .

[0042] In some embodiments, an indicator data visualization monitoring system is composed of multiple indicator collection components Pushgateway, a data monitoring system Prometheus and a visualization component Grafana. The HTTP interface of the indicator data is configured as PrometheusPushGatewayReporter in the FLink task configuration file flink-conf.yaml, and a function package is sent to the task server 11. The function in the function package is used to instruct the task server 11 to send the indicator data to the interface server 12 corresponding to the preset interface address defined in the function package when sending the indicator data to Pushgateway through PrometheusPushGatewayReporter.

[0043] Based on the above-mentioned indicator data collection system, an embodiment of the present application provides an indicator data collection method.

[0044] Please refer to Figure 2 , which is a flow chart of the indicator data collection method provided in the embodiment of the present application Figure 1 , the indicator data collection method is executed by the virtual interface in the interface server in the above indicator data collection system, such as Figure 2 As shown, the method may include:

[0045] S10: Obtain an indicator string through a virtual interface, where the indicator string includes: multiple indicator data, where the indicator data is an indicator in a process in which the server executes at least one distributed task.

[0046] In this embodiment, the virtual interface is used to receive indicator calls sent from at least one task server, wherein at least one task server executes at least one distributed task, and the indicator data is data indicating the running status of the task server and the execution status of the distributed task during the execution of the distributed task by the task server. During the execution of the distributed task, the task server sends the indicator data in the format of an indicator string to the virtual interface of the interface server according to the address of the virtual interface pre-created on the interface server.

[0047] S20: According to the task identifier of the distributed task to which the indicator data belongs, a preset mapping method is used to determine a mapping relationship between multiple indicator collection components and each indicator data.

[0048] In this embodiment, in order to evenly send indicator data generated by at least one distributed task to different indicator collection components Pushgateways, it is necessary to determine a mapping relationship between the indicator data and the indicator collection component Pushgateway.

[0049] Specifically, the indicator data includes the numerical value of the indicator and the task identifier of the distributed task to which the indicator belongs. Taking the distributed task as the unit, the indicator data obtained from the indicator string is divided into different indicator data groups according to the task identifier of the distributed task. Each indicator data group corresponds to a distributed task, and the task identifier of the distributed task is used as the identifier of each indicator data group. According to the task identifier of the distributed task and the identifiers of multiple indicator collection components, a preset mapping method is used to determine the mapping relationship between each indicator data group and each indicator collection component.

[0050] For example, the preset mapping method can be to group M indicator data groups according to the number N of indicator collection components, the number of groups is the same as the number of indicator collection components, and the number of indicator data groups in each group can be or For example, if the number of indicator collection components N is 4 and the number of indicator data groups M is 12, then the number of indicator data groups in the four groups is 3; if the number of indicator collection components N is 4 and the number of indicator data groups M is 14, then the number of indicator data groups in two groups is 4, and the number of indicator data groups in the remaining two groups is 3.

[0051] S30: sending each indicator data to a corresponding indicator collection component according to the mapping relationship, so that the data monitoring system collects the corresponding indicator data through each indicator collection component and monitors the indicator data.

[0052] In this embodiment, according to the mapping relationship between the indicator data group and the indicator collection component, the indicator data included in each indicator data group is pushed to the corresponding indicator collection component Pushgateway. The data monitoring system Prometheus pulls the indicator data from each indicator collection component Pushgateway, monitors and analyzes the indicator data, and can also send the indicator data to the visualization component Granafa for display.

[0053] The indicator data collection method provided in the above embodiment sets up a virtual interface and multiple indicator collection components. After obtaining the indicator data through the virtual interface, the indicator data is distributed to different indicator collection components, so that the data monitoring system pulls the indicator data through each indicator collection component, avoiding the problem that a large amount of indicator data can only be sent to the data monitoring system through one indicator collection component, causing the indicator data to accumulate in the indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, which can share the pressure of a single indicator collection component, thereby improving the performance of the data monitoring system in monitoring and analyzing indicator data.

[0054] In a possible implementation, the above S20, based on the task identifier of the distributed task to which the indicator data belongs, uses a preset mapping method to determine the mapping relationship between the multiple indicator collection components and the indicator data, which may include:

[0055] According to the task identifier of each distributed task and the identifier of each indicator collection component, a hash algorithm is used to determine the mapping relationship between each indicator collection component and each indicator data.

[0056] In this embodiment, a correspondence between multiple key values ​​and the identifier of each indicator collection component is created in advance, and a hash function (Hash Function) is calculated on the task identifier of each distributed task using a hash algorithm to determine the key value corresponding to the task identifier of each distributed task. Based on the correspondence between the multiple key values ​​and the identifier of each indicator collection component, the indicator collection component corresponding to each distributed task is determined, and a mapping relationship between the indicator data group of each distributed task and each indicator collection component is established.

[0057] Furthermore, before the Flink task is completed, the indicator data during the Flink task execution process needs to be pushed to each indicator collection component Pushgateway in real time. To ensure the consistency of the indicator data, it is necessary to ensure that the indicator data of each Flink task is pushed to the same indicator collection component Pushgateway as much as possible.

[0058] The following describes the mapping relationship between indicator data and the indicator collection component Pushgateway when the indicator collection component Pushgateway changes.

[0059] In some embodiments, if a new indicator collection component is detected, or the offline indicator collection component is not the indicator collection component of the target indicator data mapping, it is determined that the indicator collection component of the target indicator data mapping remains unchanged.

[0060] In this embodiment, after the virtual interface determines the mapping relationship between each indicator data and each indicator collection component, if the virtual interface detects a newly added indicator collection component, when the virtual interface receives the target indicator data, it determines whether the target indicator data has already established a mapping relationship with the indicator collection component. There is no need to use the preset mapping method to re-map the target indicator data to a new indicator collection component. The mapping relationship between the target indicator data and the original indicator collection component is maintained, and the target indicator data can continue to be sent to the corresponding indicator collection component. If the target indicator data has not established a mapping relationship with the indicator collection component, the preset mapping method is used to determine the indicator collection component corresponding to the target indicator data from the original indicator collection component and the newly added indicator collection component.

[0061] If the virtual interface detects that an indicator collection component is offline or down, it is necessary to determine whether the offline or down indicator collection component is the indicator collection component previously mapped to the target indicator data. If not, continue to send the target indicator data to the original corresponding indicator collection component.

[0062] In other embodiments, if it is detected that the indicator collection component mapped to the target indicator data is offline or crashes, a new indicator data collection component is selected from the remaining indicator collection components based on the task identifier of the distributed task to which the target indicator data belongs, and the mapping relationship between the target indicator data and the new indicator collection component is determined.

[0063] In this embodiment, after the virtual interface receives the target indicator data, if it is detected that the indicator collection component previously mapped to the target indicator data is offline or crashes, it is necessary to select a new indicator data collection component from the remaining indicator collection components based on the task identifier of the distributed task to which the target indicator data belongs, so that all indicator data of the distributed task are mapped to the new indicator collection component, and all indicator data in the distributed task, including the target indicator data, are sent to the new indicator collection component.

[0064] In a possible implementation, the virtual interface may select the indicator collection component with the least amount of data as the new indicator collection component based on the amount of indicator data sent to the remaining indicator collection components.

[0065] Please refer to Figure 3 , which is a flow chart of the indicator data collection method provided in the embodiment of the present application Figure 2 ,like Figure 3 As shown, the specific implementation method of determining the mapping relationship between each indicator collection component and each indicator data using a hash algorithm based on the task identifier of each distributed task and the identifier of each indicator collection component may include:

[0066] S21: Perform hash calculation according to the address of each indicator collection component to determine the position of each indicator collection component on the preset hash ring.

[0067] In this embodiment, a consistent hashing algorithm may be used to determine the mapping relationship between the task identifier of each distributed task and the address of each indicator collection component, thereby determining the mapping relationship between the indicator data of each distributed task and each indicator collection component.

[0068] Specifically, the consistent hashing algorithm is to form the entire hash value space into a virtual ring, that is, a hash ring, which consists of 2 ^23 nodes, nodes 0 to (2 ^23 -1) Connect end to end. Use the address of each indicator collection component Pushgateway as the key value to perform hash calculation to determine the position of each indicator collection component Pushgateway on the hash ring.

[0069] For examples, please refer to Figure 4 , is a schematic diagram of a hash ring mapping provided in an embodiment of the present application, such as Figure 4 As shown in the figure, the four indicator collection components Pushgateway are located on the hash ring at NodeA, NodeB, NodeC, and NodeD.

[0070] S22: Perform hash calculation according to the task identifier of each distributed task to determine the position of each indicator data on the preset hash ring.

[0071] In this embodiment, the corresponding key value is calculated through a hash function according to the task identifier of each distributed task, and the indicator data corresponding to each distributed task is hashed on the hash ring according to the key value to determine the position of each indicator data on the hash ring.

[0072] For example, Figure 4 As shown, for the indicator data objectA, objectB, objectC and objectD corresponding to the four distributed tasks, the position of each indicator data on the hash ring is determined according to the task identifier of the distributed task corresponding to each indicator data.

[0073] S23: Determine the indicator collection component that is closest to each indicator data in the preset clockwise direction of the preset hash ring, and obtain a mapping relationship between each indicator data and each indicator collection component.

[0074] In this embodiment, since the hash ring is a closed ring structure, after determining the position of each indicator data and each indicator collection component on the hash ring, the preset clockwise direction can be used as the mapping direction, and the position of each indicator data can be used as the starting point to search along the mapping direction. The first indicator collection component encountered in the mapping direction is the indicator collection component corresponding to the indicator data. Through this mapping method, the mapping relationship between each indicator data and each indicator collection component can be determined.

[0075] For example, Figure 4 As shown, the preset clockwise direction is defined as clockwise, the indicator collection component corresponding to the indicator data objectA is NodeA, the indicator collection component corresponding to the indicator data objectB is NodeB, the indicator collection component corresponding to the indicator data objectC is NodeC, and the indicator collection component corresponding to the indicator data objectD is NodeD.

[0076] In some embodiments, if the indicator collection component NodeC crashes, the indicator collection components mapped by indicator data A, indicator data B and indicator data D will not be affected. You only need to search for indicator data C again in a clockwise direction and map indicator data C to indicator collection component D.

[0077] The indicator data collection method provided in the above embodiment determines the position of each indicator data and each indicator collection component on the hash ring through hash calculation, and determines the indicator collection component corresponding to each indicator data along the preset clockwise direction on the hash ring, so as to allocate multiple indicator data to different indicator collection components, so that the data monitoring system pulls the indicator data through each indicator collection component, shares the pressure of a single indicator collection component, and thus improves the performance of the data monitoring system in monitoring and analyzing the indicator data.

[0078] Based on the above embodiment, in order to ensure that the indicator collection component can identify the indicator data, the indicator data needs to be registered in the indicator collection component first. The following describes the process of registering the indicator data in the indicator collection component.

[0079] Please refer to Figure 5 , which is a flow chart of the indicator data collection method provided in the embodiment of the present application Figure 3 ,like Figure 5 As shown, after obtaining the indicator string through the virtual interface, the method may include:

[0080] S41: Parse the indicator character string to obtain multiple indicator data.

[0081] S42: Initiate a registration request to at least one indicator collection component to register multiple indicator data in the indicator collection component.

[0082] In this embodiment, a Flink task sends an HTTP request to a virtual interface, carrying a metric string consisting of multiple metric data. After receiving the metric string, the virtual interface parses the metric string to obtain multiple metric data. For example, the multiple metric data may include CPU, memory, and thread data of the Flink task.

[0083] After parsing is complete, each type of indicator data needs to be registered in the indicator collection component based on its type, so that the indicator collection component can identify the indicator data sent by the virtual interface. For example, for the indicator collection component Pushgateway, the collection registration component CollectorRegistry contained therein is specifically used to register indicator data. The virtual interface sends a registration request to the collection registration component CollectorRegistry, which carries the type identifiers of multiple types of indicator data. After receiving the registration request, the collection registration component CollectorRegistry completes the registration of multiple types of indicator data based on the type identifiers.

[0084] It should be noted that each indicator data can be represented from three dimensions, namely the total amount of indicator data (Count), the instantaneous value of the indicator data (Gauge), and the average value of the indicator data within a preset time period (Meter).

[0085] The indicator data collection method provided by the above embodiment ensures that the indicator collection component can identify the indicator data sent by the virtual interface by registering the indicator data in the indicator collection component.

[0086] Because in the existing official Flink configuration, when the server in the service cluster uses PrometheusPushGatewayReporter to send indicator data, it can only set the address of one indicator collection component Pushgateway. In order to ensure that the indicator data can be evenly distributed to the multiple indicator collection components Pushgateway set in the embodiment of the present application, it is necessary to create a virtual interface so that when the server in the service cluster uses PrometheusPushGatewayReporter to send indicator data, the indicator data can be sent to the virtual interface, and the virtual interface will distribute the indicator data to different indicator collection components Pushgateway.

[0087] The following describes in detail the process of sending metric data to a virtual interface.

[0088] Please refer to Figure 6, which is a flow chart of the indicator data collection method provided in the embodiment of the present application Figure 4 , the indicator data collection method is executed by at least one task server in the above indicator data collection system, such as Figure 6 As shown, the method may include:

[0089] S50: Receive a distributed task request sent by the task configuration end, where the distributed task request includes: at least one distributed task and a function package, where the function in the function package is used to instruct to send the indicator string to a virtual interface of a preset interface address.

[0090] S60: Execute the distributed task and obtain indicator data during the task execution process.

[0091] S70: Execute the functions in the function package and send the indicator data of each distributed task to the virtual interface in the form of an indicator string.

[0092] In this embodiment, a distributed task request is sent to at least one task server in the service cluster through the task configuration end, and the distributed task request carries the distributed task to be executed by the task server and a function package, wherein the function package includes a predefined function, which is used to execute the task server. When calling the HTTP interface to send the indicator string, the indicator data is sent to the virtual address through the preset interface address.

[0093] After receiving a distributed task request, the task server executes the distributed task in the distributed task request and obtains the indicator data during the execution of the distributed task. When the task server calls the HTTP interface to send the indicator data, it executes the function in the function package and replaces the address of the single indicator collection component originally defined at the HTTP interface with the preset interface address defined in the function to send the indicator data to the virtual interface in the format of an indicator string.

[0094] It should be noted that in specific scenarios, Flink generally provides two running modes: Jar package and SQL. In both running modes, the function is sent to the task server in the form of a Jar (Java ARchive) package.

[0095] In some embodiments, when sending each distributed task to each task server, it is necessary to determine the sending form of the function package based on the submission mode of the distributed task. If the submission mode of the distributed task is Session mode, that is, the task servers in the service cluster use the same resources to perform distributed tasks, then it is only necessary to send the function package to one task server in the service cluster, and other task servers can share the function package from the task server; if the submission mode of the distributed task is per-job mode, that is, each task server in the service cluster is independent of each other, each time a distributed task is sent, a function package needs to be submitted synchronously.

[0096] The indicator data collection method provided by the above embodiment carries a function package in a distributed task request, and indicates the preset interface address of the virtual interface through the function of the function package, so that the task server sends the indicator data to the virtual interface after executing the distributed task, and the virtual interface uniformly distributes the indicator data to multiple indicator collection components, thereby avoiding the task server directly sending a large amount of indicator data to the data monitoring system through one indicator collection component, causing the indicator data to accumulate in a single indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, shares the pressure of a single indicator collection component, and thus improves the performance of the data monitoring system in monitoring and analyzing indicator data.

[0097] When sending metrics data for Flink tasks, existing task servers can only send the metrics data to the address of a single metrics collection component, the Pushgateway. When creating multiple metrics collection components, to ensure that Flink tasks can identify the function package that changes the metrics data sending address, it is necessary to create the function package using aspect-oriented programming (AOP). The following describes the process of creating a function package using AOP.

[0098] Please refer to Figure 7 , which is a flow chart of the indicator data collection method provided in the embodiment of the present application Figure 5 ,like Figure 7 As shown, before executing the distributed task request sent by the receiving task configuration terminal, the method may further include:

[0099] S81: Define aspects using aspect-oriented programming.

[0100] S82: Create functions for aspects and package the functions into function packages.

[0101] In this example, a Java class is created and the @Aspect annotation in Spring AOP is used to define the Java class as an aspect for the container to read. An aspect extracts the common behavior in a program into a reusable module so that the aspect is called when the program executes the common behavior, reducing the complexity of the program.

[0102] After the aspect is successfully created, you need to create a function in the aspect. This function is used to indicate that the address of the original single indicator collection component is reassigned to the preset interface address. For this function, use the @After annotation of the aspect to define a post notification. This post notification is used to define the execution logic of the function.

[0103] For example, the post-notification defined by the @After annotation can be expressed as: @After(value="execution(io.prometheus.client.exporter.PushGateway.new(..))&&args(serverB aseURL)"), where the cut point in the aspect is the new method of the PushGateway class, and the parameter of the new method is serverBaseURL, which represents the address of the data collection component Pushgateway. The @After definition notification is used to instruct the task server to call the function logic after executing the new method of the PushGateway class, and reassign serverBaseURL in the function logic to change the address of the data collection component Pushgateway to the preset interface address of the virtual interface, so that the task server sends all indicator data of the Flink task to the virtual interface.

[0104] Specifically, the program for sending indicator data by the task server is extracted into a section, and a function is created in the section to change the address of the data collection component Pushgateway to the preset interface address of the virtual interface. The post-notification defined by @After is used to indicate that when the task server executes the new method of the PushGateway class to send indicator data, the function of changing the address of the data collection component Pushgateway to the preset interface address of the virtual interface is executed, so that the task server sends the indicator data to the virtual interface.

[0105] The indicator data collection method provided by the above embodiment creates a function in the aspect defined for the programming method so that when the task server calls the aspect to send indicator data, the indicator data is sent to the virtual interface, thereby avoiding the task server directly sending a large amount of indicator data to the data monitoring system through an indicator collection component, causing the indicator data to accumulate in a single indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, sharing the pressure of a single indicator collection component, thereby improving the performance of the data monitoring system in monitoring and analyzing indicator data.

[0106] Based on the above embodiment, the embodiment of the present application also provides an indicator data collection device, which is applied to a server in a service cluster, and the server is deployed with a virtual interface. Figure 8 , is a structural diagram of the indicator data collection device provided in an embodiment of the present application, such as Figure 8 As shown, the device includes:

[0107] The indicator data acquisition module 101 is used to acquire an indicator string through a virtual interface, wherein the indicator string includes: a plurality of indicator data, where the indicator data is an indicator of a process in which a server executes at least one distributed task;

[0108] A mapping relationship determination module 102 is configured to determine a mapping relationship between a plurality of indicator collection components and each indicator data using a preset mapping method according to a task identifier of a distributed task to which each indicator data belongs;

[0109] The indicator data sending module 103 is used to send each indicator data to the corresponding indicator collection component according to the mapping relationship, so that the data monitoring system collects the corresponding indicator data through each indicator collection component and monitors the indicator data.

[0110] The indicator data collection device provided by the above embodiment is equipped with a virtual interface and multiple indicator collection components. After obtaining the indicator data through the virtual interface, the indicator data is distributed to different indicator collection components, so that the data monitoring system pulls the indicator data through each indicator collection component, avoiding the problem that a large amount of indicator data can only be sent to the data monitoring system through one indicator collection component, causing the indicator data to accumulate in the indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, which can share the pressure of a single indicator collection component, thereby improving the performance of the data monitoring system in monitoring and analyzing the indicator data.

[0111] In a possible implementation, the mapping relationship determination module 102 is specifically configured to determine the mapping relationship between each indicator collection component and each indicator data using a hash algorithm according to the task identifier of each distributed task and the identifier of each indicator collection component.

[0112] In some embodiments, the mapping relationship determination module 102 is further configured to determine that the indicator collection component of the target indicator data mapping remains unchanged if a new indicator collection component is detected or the offline indicator collection component is not the indicator collection component of the target indicator data mapping.

[0113] In other embodiments, the mapping relationship determination module 102 is also used to select a new indicator data collection component from the remaining indicator collection components based on the task identifier of the distributed task to which the target indicator data belongs, and determine the mapping relationship between the target indicator data and the new indicator collection component if it is detected that the indicator collection component mapped by the target indicator data is offline or crashes.

[0114] Optionally, the mapping relationship determination module 102 includes:

[0115] A component position determination unit, configured to perform hash calculations based on the addresses of the indicator collection components to determine the positions of the indicator collection components on a preset hash ring;

[0116] A data location determination unit, configured to perform hash calculations based on the task identifiers of each distributed task to determine the location of each indicator data on a preset hash ring;

[0117] The mapping relationship determination unit is used to determine the indicator collection component that is closest to each indicator data in the preset clockwise direction of the preset hash ring, and obtain the mapping relationship between each indicator data and each indicator collection component.

[0118] The indicator data collection device provided in the above embodiment determines the position of each indicator data and each indicator collection component on the hash ring through hash calculation, and determines the indicator collection component corresponding to each indicator data along the preset clockwise direction on the hash ring, so as to allocate multiple indicator data to different indicator collection components, so that the data monitoring system pulls the indicator data through each indicator collection component, shares the pressure of a single indicator collection component, and thus improves the performance of the data monitoring system in monitoring and analyzing the indicator data.

[0119] Optionally, the device further includes:

[0120] The string parsing module is used to parse the indicator string to obtain multiple indicator data;

[0121] The registration request initiating module is used to initiate a registration request to at least one indicator collection component to register multiple indicator data in the indicator collection component.

[0122] The indicator data collection device provided by the above embodiment registers the indicator data in the indicator collection component, thereby ensuring that the indicator collection component can identify the indicator data sent by the virtual interface.

[0123] Optionally, the device further includes:

[0124] A task request receiving module is used to receive a distributed task request sent by a task configuration terminal, wherein the distributed task request includes: at least one distributed task and a function package, wherein the function in the function package is used to instruct to send an indicator string to a preset interface address of a virtual interface;

[0125] The task execution module is used to execute distributed tasks and obtain indicator data during the task execution process;

[0126] The function execution module is used to execute the functions in the function package and send the indicator data of each distributed task to the virtual interface in the form of an indicator string.

[0127] The indicator data collection device provided by the above embodiment carries a function package in a distributed task request, and indicates the preset interface address of the virtual interface through the function of the function package, so that the task server sends the indicator data to the virtual interface after executing the distributed task, and the virtual interface uniformly distributes the indicator data to multiple indicator collection components, thereby avoiding the task server directly sending a large amount of indicator data to the data monitoring system through one indicator collection component, causing the indicator data to accumulate in a single indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, shares the pressure of a single indicator collection component, and thus improves the performance of the data monitoring system in monitoring and analyzing indicator data.

[0128] Optionally, the device further includes:

[0129] Aspect definition module, used to define aspects using aspect-oriented programming method;

[0130] The function creation module is used to create functions for aspects and package the functions into function packages.

[0131] The indicator data collection device provided by the above embodiment creates a function in the aspect defined for the programming method so that when the task server calls the aspect to send indicator data, the indicator data is sent to the virtual interface, thereby avoiding the task server directly sending a large amount of indicator data to the data monitoring system through an indicator collection component, causing the indicator data to accumulate in a single indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, sharing the pressure of a single indicator collection component, thereby improving the performance of the data monitoring system in monitoring and analyzing indicator data.

[0132] The above-mentioned device is used to execute the method provided in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0133] The above modules can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0134] Please refer to Figure 9 , which is a schematic diagram of the server provided by this application, such as Figure 9 As shown, server 200 includes: a processor 201, a storage medium 202, and a bus. Storage medium 202 stores program instructions executable by processor 201. When server 200 is running, processor 201 and storage medium 202 communicate via the bus, and processor 201 executes the program instructions to perform the above-mentioned indicator data collection method. It should be noted that server 200 is the task server 11 or interface server 12 in the indicator data collection system.

[0135] Specifically, the processor executes the steps of the above-mentioned indicator data collection method including:

[0136] An indicator string is obtained through a virtual interface, where the indicator string includes: multiple indicator data, where the indicator data is an indicator in the process of the server executing at least one distributed task; based on the task identifier of the distributed task to which the indicator data belongs, a preset mapping method is used to determine the mapping relationship between multiple indicator collection components and each indicator data; based on the mapping relationship, each indicator data is sent to a corresponding indicator collection component, so that the data monitoring system collects the corresponding indicator data through each indicator collection component and monitors the indicator data.

[0137] The processor executes the indicator data collection method provided by the above embodiment, sets a virtual interface and multiple indicator collection components, and after obtaining indicator data through the virtual interface, distributes the indicator data to different indicator collection components, so that the data monitoring system pulls the indicator data through each indicator collection component, avoiding the problem that a large amount of indicator data can only be sent to the data monitoring system through one indicator collection component, causing the indicator data to accumulate in the indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, which can share the pressure of a single indicator collection component, thereby improving the performance of the data monitoring system in monitoring and analyzing indicator data.

[0138] In one possible implementation, the process of the processor executing the above-mentioned task identifier of the distributed task to which the indicator data belongs and using a preset mapping method to determine the mapping relationship between the multiple indicator collection components and the respective indicator data may include:

[0139] According to the task identifier of each distributed task and the identifier of each indicator collection component, a hash algorithm is used to determine the mapping relationship between each indicator collection component and each indicator data.

[0140] In some embodiments, if the processor detects a newly added indicator collection component, or the offline indicator collection component is not the indicator collection component of the target indicator data mapping, it determines that the indicator collection component of the target indicator data mapping remains unchanged.

[0141] In other embodiments, if the processor detects that the indicator collection component mapped to the target indicator data is offline or crashes, it selects a new indicator data collection component from the remaining indicator collection components based on the task identifier of the distributed task to which the target indicator data belongs, and determines the mapping relationship between the target indicator data and the new indicator collection component.

[0142] Optionally, a specific implementation method of the processor performing the above-mentioned determination of the mapping relationship between each indicator collection component and each indicator data using a hash algorithm based on the task identifier of each distributed task and the identifier of each indicator collection component may include:

[0143] Perform hash calculation based on the address of each indicator collection component to determine the position of each indicator collection component on the preset hash ring; perform hash calculation based on the task identifier of each distributed task to determine the position of each indicator data on the preset hash ring; determine the indicator collection component that is closest to each indicator data in the preset clockwise direction of the preset hash ring, and obtain the mapping relationship between each indicator data and each indicator collection component.

[0144] The processor executes the indicator data collection method provided by the above embodiment, determines the position of each indicator data and each indicator collection component on the hash ring through hash calculation, and determines the indicator collection component corresponding to each indicator data along the preset clockwise direction on the hash ring, so as to allocate multiple indicator data to different indicator collection components, so that the data monitoring system pulls the indicator data through each indicator collection component, shares the pressure of a single indicator collection component, and thus improves the performance of the data monitoring system in monitoring and analyzing the indicator data.

[0145] Based on the above embodiment, in order to ensure that the indicator collection component can identify the indicator data, the indicator data needs to be registered in the indicator collection component first. The following describes the process of registering the indicator data in the indicator collection component.

[0146] Optionally, after the processor obtains the indicator string through the virtual interface, the steps executed may further include:

[0147] The indicator character string is parsed to obtain a plurality of indicator data; and a registration request is initiated to at least one indicator collection component to register the plurality of indicator data in the indicator collection component.

[0148] The indicator data collection method provided by the above embodiment of the processor ensures that the indicator collection component can identify the indicator data sent by the virtual interface by registering the indicator data in the indicator collection component.

[0149] In a possible implementation, the processor may further execute the steps of the above-mentioned indicator data collection method:

[0150] Receive a distributed task request sent by the task configuration end, which includes: at least one distributed task and a function package, where the function in the function package is used to indicate that the indicator string is sent to the virtual interface of the preset interface address; execute the distributed task and obtain the indicator data during the task execution process; execute the function in the function package and send the indicator data of each distributed task to the virtual interface in the form of an indicator string.

[0151] The processor executes the indicator data collection method provided by the above embodiment, by carrying a function package in a distributed task request, and indicating the preset interface address of the virtual interface through the function of the function package, so that the task server sends the indicator data to the virtual interface after executing the distributed task, and the virtual interface uniformly distributes the indicator data to multiple indicator collection components, thereby avoiding the task server directly sending a large amount of indicator data to the data monitoring system through one indicator collection component, causing the indicator data to accumulate in a single indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, shares the pressure of a single indicator collection component, and thus improves the performance of the data monitoring system in monitoring and analyzing indicator data.

[0152] In a possible implementation, the processor may further execute the steps of the above-mentioned indicator data collection method:

[0153] Use aspect-oriented programming to define aspects; create functions for aspects and package the functions into function packages.

[0154] The processor executes the indicator data collection method provided by the above embodiment, by creating a function in the aspect defined for the programming method, so that when the task server calls the aspect to send indicator data, the indicator data is sent to the virtual interface, thereby avoiding the task server directly sending a large amount of indicator data to the data monitoring system through an indicator collection component, causing the indicator data to accumulate in a single indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, sharing the pressure of a single indicator collection component, thereby improving the performance of the data monitoring system in monitoring and analyzing indicator data.

[0155] Optionally, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the above-mentioned indicator data collection method are executed.

[0156] Specifically, the steps of the computer program executing the above-mentioned indicator data collection method include:

[0157] An indicator string is obtained through a virtual interface, where the indicator string includes: multiple indicator data, where the indicator data is an indicator in the process of the server executing at least one distributed task; based on the task identifier of the distributed task to which the indicator data belongs, a preset mapping method is used to determine the mapping relationship between multiple indicator collection components and each indicator data; based on the mapping relationship, each indicator data is sent to a corresponding indicator collection component, so that the data monitoring system collects the corresponding indicator data through each indicator collection component and monitors the indicator data.

[0158] The computer program executes the indicator data collection method provided by the above embodiment, sets a virtual interface and multiple indicator collection components, and after obtaining indicator data through the virtual interface, distributes the indicator data to different indicator collection components, so that the data monitoring system pulls the indicator data through each indicator collection component, avoiding the problem that a large amount of indicator data can only be sent to the data monitoring system through one indicator collection component, causing the indicator data to accumulate in the indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, which can share the pressure of a single indicator collection component, thereby improving the performance of the data monitoring system in monitoring and analyzing indicator data.

[0159] In one possible implementation, the process of a computer program executing the above-mentioned task identifier of the distributed task to which the indicator data belongs, using a preset mapping method to determine the mapping relationship between the multiple indicator collection components and the respective indicator data, may include:

[0160] According to the task identifier of each distributed task and the identifier of each indicator collection component, a hash algorithm is used to determine the mapping relationship between each indicator collection component and each indicator data.

[0161] In some embodiments, if the computer program detects a newly added indicator collection component, or the offline indicator collection component is not the indicator collection component of the target indicator data mapping, it determines that the indicator collection component of the target indicator data mapping remains unchanged.

[0162] In other embodiments, if the computer program detects that the indicator collection component mapped to the target indicator data is offline or crashes, it selects a new indicator data collection component from the remaining indicator collection components based on the task identifier of the distributed task to which the target indicator data belongs, and determines the mapping relationship between the target indicator data and the new indicator collection component.

[0163] Optionally, a specific implementation method of the computer program executing the above-mentioned determination of the mapping relationship between each indicator collection component and each indicator data using a hash algorithm based on the task identifier of each distributed task and the identifier of each indicator collection component may include:

[0164] Perform hash calculation based on the address of each indicator collection component to determine the position of each indicator collection component on the preset hash ring; perform hash calculation based on the task identifier of each distributed task to determine the position of each indicator data on the preset hash ring; determine the indicator collection component that is closest to each indicator data in the preset clockwise direction of the preset hash ring, and obtain the mapping relationship between each indicator data and each indicator collection component.

[0165] The computer program executes the indicator data collection method provided by the above embodiment, determines the position of each indicator data and each indicator collection component on the hash ring through hash calculation, and determines the indicator collection component corresponding to each indicator data along the preset clockwise direction on the hash ring, so as to allocate multiple indicator data to different indicator collection components, so that the data monitoring system pulls the indicator data through each indicator collection component, shares the pressure of a single indicator collection component, and thus improves the performance of the data monitoring system in monitoring and analyzing the indicator data.

[0166] Based on the above embodiment, in order to ensure that the indicator collection component can identify the indicator data, the indicator data needs to be registered in the indicator collection component first. The following describes the process of registering the indicator data in the indicator collection component.

[0167] Optionally, after the computer program obtains the indicator string through the virtual interface, the steps executed may further include:

[0168] The indicator character string is parsed to obtain a plurality of indicator data; and a registration request is initiated to at least one indicator collection component to register the plurality of indicator data in the indicator collection component.

[0169] The indicator data collection method provided by the above embodiment of the computer program ensures that the indicator collection component can identify the indicator data sent by the virtual interface by registering the indicator data in the indicator collection component.

[0170] In a possible implementation, the steps of the computer program executing the above-mentioned indicator data collection method may further include:

[0171] Receive a distributed task request sent by the task configuration end, which includes: at least one distributed task and a function package, where the function in the function package is used to indicate that the indicator string is sent to the virtual interface of the preset interface address; execute the distributed task and obtain the indicator data during the task execution process; execute the function in the function package and send the indicator data of each distributed task to the virtual interface in the form of an indicator string.

[0172] The computer program executes the indicator data collection method provided by the above embodiment, by carrying a function package in a distributed task request, and indicating the preset interface address of the virtual interface through the function of the function package, so that the task server sends the indicator data to the virtual interface after executing the distributed task, and the virtual interface uniformly distributes the indicator data to multiple indicator collection components, thereby avoiding the task server directly sending a large amount of indicator data to the data monitoring system through one indicator collection component, causing the indicator data to accumulate in a single indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, shares the pressure of a single indicator collection component, and thus improves the performance of the data monitoring system in monitoring and analyzing indicator data.

[0173] In a possible implementation, the steps of the computer program executing the above-mentioned indicator data collection method may further include:

[0174] Use aspect-oriented programming to define aspects; create functions for aspects and package the functions into function packages.

[0175] The computer program executes the indicator data collection method provided by the above embodiment, by creating a function in the aspect defined for the programming method, so that when the task server calls the aspect to send indicator data, the indicator data is sent to the virtual interface, thereby avoiding the task server directly sending a large amount of indicator data to the data monitoring system through an indicator collection component, causing the indicator data to accumulate in a single indicator collection component and affecting the performance of the data monitoring system. The data monitoring system obtains indicator data through multiple indicator collection components, sharing the pressure of a single indicator collection component, thereby improving the performance of the data monitoring system in monitoring and analyzing indicator data.

[0176] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0177] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0178] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0179] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (English: Read-Only Memory, abbreviated: ROM), a random access memory (English: Random Access Memory, abbreviated: RAM), a disk or an optical disk, and other media that can store program code.

[0180] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions that can be easily conceived by any person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for collecting indicator data, characterized in that: Applied to a server in a service cluster, the server being deployed with a virtual interface, the method comprising: Acquire an indicator string through the virtual interface, wherein the indicator string includes: a plurality of indicator data, wherein the indicator data is an indicator in a process in which the server executes at least one distributed task; Dividing the indicator character string into a plurality of indicator data groups according to the task identifier of the distributed task to which each indicator data belongs, and determining a mapping relationship between a plurality of indicator collection components and each indicator data group using a preset mapping method; According to the mapping relationship, each indicator data group is sent to a corresponding indicator collection component, so that the data monitoring system collects the corresponding indicator data through each indicator collection component and monitors the indicator data; The step of dividing the indicator character string into a plurality of indicator data groups according to the task identifier of the distributed task to which each indicator data belongs, and determining a mapping relationship between a plurality of indicator collection components and each indicator data group using a preset mapping method includes: Performing hash calculations based on the addresses of the indicator collection components to determine the positions of the indicator collection components on the preset hash ring; Performing hash calculations based on the task identifiers of the distributed tasks to determine the positions of the indicator data groups on the preset hash ring; Determine the indicator collection component that is closest to each indicator data group in the preset clockwise direction of the preset hash ring, and obtain a mapping relationship between each indicator data group and each indicator collection component.

2. The method according to claim 1, wherein The method further comprises: Receive a distributed task request sent by a task configuration end, wherein the distributed task request includes: at least one distributed task and a function package, wherein the function in the function package is used to instruct to send the indicator string to a preset interface address of the virtual interface; Execute the distributed task and obtain indicator data during the task execution; The functions in the function package are executed, and the indicator data of each of the distributed tasks is sent to the virtual interface in the form of the indicator character string.

3. The method according to claim 2, wherein Before receiving the distributed task request sent by the task configuration end, the method further includes: Define aspects using aspect-oriented programming; The function is created for the aspect, and the function is packaged into the function package.

4. The method according to claim 1, wherein After determining the mapping relationship between the plurality of indicator collection components and each indicator data using a preset mapping method according to the task identifier of the distributed task to which each indicator data belongs, the method further includes: If a new indicator collection component is detected, or the offline indicator collection component is not the indicator collection component mapped to the target indicator data, it is determined that the indicator collection component mapped to the target indicator data remains unchanged.

5. The method according to claim 4, wherein The method further comprises: If it is detected that the indicator collection component mapped by the target indicator data is offline or crashes, a new indicator collection component is selected from the remaining indicator collection components according to the task identifier of the distributed task to which the target indicator data belongs, and the mapping relationship between the target indicator data and the new indicator collection component is determined.

6. The method according to any one of claims 1 to 5, wherein: After obtaining the indicator character string through the virtual interface, the method further includes: Parsing the indicator character string to obtain multiple indicator data; A registration request is initiated to the at least one indicator collection component to register the plurality of indicator data in the indicator collection component.

7. An indicator data collection device, characterized in that: Applied to a server in a service cluster, the server being deployed with a virtual interface, the apparatus comprising: An indicator data acquisition module, configured to acquire an indicator string through the virtual interface, wherein the indicator string includes: a plurality of indicator data, wherein the indicator data is an indicator in a process in which the server executes at least one distributed task; a mapping relationship determination module, configured to divide the indicator character string into a plurality of indicator data groups according to the task identifier of the distributed task to which each indicator data belongs, and determine the mapping relationship between the plurality of indicator collection components and each indicator data group using a preset mapping method; An indicator data sending module, configured to send each of the indicator data groups to a corresponding indicator collection component according to the mapping relationship, so that the data monitoring system collects the corresponding indicator data through each of the indicator collection components and monitors the indicator data; The mapping relationship determination module is specifically used to perform hash calculation based on the address of each indicator collection component to determine the position of each indicator collection component on the preset hash ring; perform hash calculation based on the task identifier of each distributed task to determine the position of each indicator data group on the preset hash ring; determine the indicator collection component that is closest to each indicator data group in the preset clockwise direction of the preset hash ring, and obtain the mapping relationship between each indicator data group and each indicator collection component.

8. A server, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor. When the server is running, the processor and the storage medium communicate via the bus, and the processor executes the program instructions to perform the steps of the indicator data collection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which, when executed by a processor, executes the steps of the indicator data collection method according to any one of claims 1 to 6.

Images