A method and system for accessing a target server

Abstract

The present application provides a method and system for accessing a target server, wherein the access to the target server is achieved by developing a server to call a framework including a request processor and a response processor, wherein the code with common logic constructs a request entity by developing a server and uses the request processor to parse the request entity to obtain an expression, and adopts a functional programming method to encapsulate the code with common logic into a common method, and the operation and maintenance personnel or users can implement the programming of business logic by calling with different parameters, and, when the target server is a Prometheus server, based on this method, the query and operation of Prometheus monitoring data can be realized without maintaining the underlying PromQL, and the problems of confusing logic and poor readability of the underlying PromQL code caused by different technologies and logical methods of the operation and maintenance personnel or users are avoided.

Description

Technical Field

[0001] The present application relates to fields related to operation and maintenance monitoring, and specifically, to a method and system for accessing a target server. Background Art

[0002] Prometheus is an open source system monitoring and alarm system. It has now joined the Cloud Native Computing Foundation and has become the second project hosted by the Cloud Native Computing Foundation after k8s. It supports multiple exporters (broadly speaking, all programs that can provide monitoring sample data to Prometheus can be called an exporter) to collect data, and also supports pushgateway (which is an important tool in the Prometheus ecosystem) for data reporting. Prometheus performance is sufficient to support server clusters of tens of thousands of units.

[0003] Prometheus is used in the project to collect the operating status of underlying resources, including indicators such as CPU usage, memory usage, network transmission rate, and hard disk read and write rate. Users view resource monitoring on different pages and call different Java service interfaces. The interfaces contain different PromQL expressions to query Prometheus, format and encapsulate the return data of Prometheus, and display it to users in the form of line charts or Top sorting.

[0004] Therefore, the application of Prometheus is very important for resource monitoring, and users and operation and maintenance personnel often query and operate Prometheus monitoring data. However, it is difficult to unify the style of PromQL writing and management due to the different technical and logical methods of users or operation and maintenance personnel, resulting in confusing logic of the underlying PromQL code, poor readability, and difficulty in maintenance. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a method and system for accessing a target server, so as to solve the problem that in the prior art, it is difficult to unify the style of PromQL writing and management due to different technical and logical methods of operation and maintenance personnel or users, resulting in confusing code logic, poor readability and difficulty in maintenance of the underlying PromQL.

[0006] A method for accessing a target server in an embodiment of the present application is applied to accessing the target server by developing a server to call a framework including a request processor and a response processor; the method includes:

[0007] Use the development server to construct one or more request entities and send the request entities to the request processor;

[0008] The request processor parses the request entity, constructs a request message containing the expression, and requests the target server; and

[0009] The response processor receives the target data returned by the target server, parses the target data, and returns the parsed data to the development server.

[0010] In the above technical solution, access to the target server is achieved by developing a server to call a framework including a request processor and a response processor, wherein the code with common logic constructs a request entity through the development server and uses the request processor to parse the request entity to obtain an expression. It adopts a functional programming approach to encapsulate the code with common logic into a common method, and the operation and maintenance personnel or users can implement the programming of business logic by calling with different parameters. Moreover, when the target server is a Prometheus server, based on this approach, the query and operation of Prometheus monitoring data can be realized without maintaining the underlying PromQL, and the problems of confusing logic and poor readability of the underlying PromQL code caused by different technical and logical methods of the operation and maintenance personnel or users are avoided.

[0011] In some optional implementations, when the request processor parses one or more request entities, a processing thread is allocated to each request entity.

[0012] In the above technical solution, in the case where there may be multiple dimensions of information query in a page request, for example, a query may query various indicators of CPU, memory, network, and hard disk, a multi-threaded method is used for parallel query. Compared with the serial request method commonly used by programmers in the prior art, the time span of serial query is longer, and the multi-threaded parallel query method can reduce the time spent on query and thus improve the overall user experience.

[0013] In some optional implementations, the request processor parses the request entity and constructs a request message containing an expression, including:

[0014] By processing the thread, the dopost method in the request entity is called to construct the expression of the request; wherein the dopost method, when called, constructs an expression of one or more parameters.

[0015] When the target server is the Prometheus server, since Prometheus only accepts supported string expressions, the request entity needs to convert the programmer's specific settings for the entity into a string expression. Therefore, the request entity also includes a dopost method for constructing an expression that concatenates one or more parameters.

[0016] In some optional implementations, the request processor parses the request entity, further comprising:

[0017] Receive the configuration file sent by the development server through the request processor; wherein the configuration file includes the IP port number of the target server; and

[0018] Through the request processor, the request address is constructed according to the target server IP port number.

[0019] In the above technical solution, in order to obtain the target data of a specific target server, in the process of constructing a request message using the request entity, the request address is also constructed by developing a configuration file including the IP port number of the target server sent by the server.

[0020] In some optional implementations, the framework further includes a logging module, and the logging module is used to record the request entity sent to the request processor and the data returned by the response processor.

[0021] In the above technical solution, the information of each actual request and the returned information are recorded through the log recording module, and the response time and the reason for the query abnormality are recorded to facilitate later analysis and troubleshooting.

[0022] In some optional implementations, the request message is an http request message;

[0023] The method further includes: the response processor also receives the return code of the http request returned by the target server and performs parsing processing, the parsing processing including:

[0024] Determine whether the return code is a return code indicating an HTTP native error;

[0025] If so, log it and return the default return content to the main business program.

[0026] In the above technical solution, the target server returns a return code for the http request after receiving the http request. The return codes include: 200, OK, successful request; 400, Bad Request, incorrect syntax request; 401, Unauthorized, unauthenticated; 403, Forbidden, request denied; 404, Not Found, server location not found; 500, InternalServer Error, internal service error. These common return codes are classified and processed. If the return code is 200, it is a normal return, then the next step of the response processor is entered to parse the returned content. For other error information, the log is recorded first, and then the default return content is returned to the main business program to prevent error messages from popping up on the user display page due to the lack of normal content return, which reduces the user experience or causes subsequent business to fail to proceed normally.

[0027] In some optional implementations, the target server includes a Prometheus server, and the target data is monitoring data in json format;

[0028] The response processor receives the target data returned by the target server and parses the target data, including:

[0029] When the return code indicates that the request is successful, the returned monitoring data is used to determine whether the status of the monitoring data is incorrect:

[0030] If yes, log it and return the default data content to the main business program;

[0031] If not, the point data of the monitoring data is parsed.

[0032] In the above technical solution, the target server is the Prometheus server. When the return code is a return code indicating a successful request, the query status is first determined for the returned monitoring data in json format. When the status status is error error, there is no data in the following data. In order to enable subsequent business processing to proceed normally, the response processor will first log the message body and then return the subsequent json message.

[0033] In some optional implementations, after the point data of the monitoring data is parsed, the character string data in the point data is processed as null value data to obtain digital data without a character string.

[0034] In the above technical solution, for the monitoring data in the returned json format, when the status status is success, the point data in the monitoring data is processed, and the string data in the point data is processed as null value data, so that the point data is uniformly processed into digital data. Among them, the point data includes three cases: the first case is for example [1650244850.224,"3844224"], the second case is for example [1650244850.224,"NaN"], and the third case is for example [1650244850.224,null]. In this embodiment, the point data in the monitoring data is processed to process the data belonging to the second case into the data of the third case.

[0035] A system for accessing a target server according to an embodiment of the present application includes a development server, a request processor, and a response processor;

[0036] The development server is used to construct one or more request entities and send the request entities to the request processor;

[0037] The request processor is used to parse the request entity, construct the request message and request the target server;

[0038] The response processor is used to receive the target data returned by the target server, parse the target data, and return the parsed data to the development server.

[0039] In the above technical solution, access to the target server is achieved by developing a server to call a framework including a request processor and a response processor, wherein the code with common logic constructs a request entity through the development server and uses the request processor to parse the request entity to obtain an expression. It adopts a functional programming approach to encapsulate the code with common logic into a common method, and the operation and maintenance personnel or users can implement the programming of business logic by calling with different parameters. Moreover, when the target server is a Prometheus server, based on this approach, the query and operation of Prometheus monitoring data can be realized without maintaining the underlying PromQL, and the problems of confusing logic and poor readability of the underlying PromQL code caused by different technical and logical methods of the operation and maintenance personnel or users are avoided.

[0040] In some optional implementations, when the request processor parses one or more request entities, a processing thread is allocated to each request entity.

[0041] In some optional implementations, the request processor is further configured to:

[0042] By processing the thread, the dopost method in the request entity is called to construct the expression of the request; wherein the dopost method, when called, constructs an expression of one or more parameters.

[0043] In some optional embodiments,

[0044] The request message includes a request address;

[0045] The development server is also used to send a configuration file to the request processor. The configuration file includes the target server IP port number.

[0046] The request processor is also used to construct a request address based on the target server IP port number.

[0047] In some optional implementations, the framework further includes a logging module, which is used to record the request entity sent to the request processor and the data returned by the response processor.

[0048] In some optional embodiments,

[0049] The request message is an HTTP request message;

[0050] The response processor is also used to receive the return code of the http request returned by the target server and perform parsing processing, which includes:

[0051] Determine whether the return code is a return code indicating an HTTP native error;

[0052] If so, log it and return the default return content to the main business program.

[0053] In some optional embodiments,

[0054] The target server includes the Prometheus server, and the target data is monitoring data in json format;

[0055] The response processor is also used to receive the target data returned by the target server and parse and process the target data, including:

[0056] When the return code indicates that the request is successful, the returned monitoring data is used to determine whether the status of the monitoring data is incorrect:

[0057] If yes, log it and return the default data content to the main business program;

[0058] If not, the point data of the monitoring data is parsed.

[0059] In some optional implementations, the response processor is further configured to:

[0060] After parsing the point data of the monitoring data, the character string data in the point data is processed as null value data to obtain digital data without character string. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 A step diagram of a method for accessing a target server according to an embodiment of the present application;

[0063] Figure 2 A flowchart of a request processor in an embodiment of the present application;

[0064] Figure 3 A schematic diagram of the workflow of the response processor provided in the embodiment of the present application;

[0065] Figure 4 A schematic diagram of a system for accessing a Prometheus server according to an embodiment of the present application;

[0066] Figure 5 This is a workflow diagram of a system for accessing a Prometheus server according to an embodiment of the present application.

[0067] Icons: 1-Java server, 2-request processor, 3-response processor, 4-logging module, 5-Prometheus server. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0069] Please refer to Figure 1 , Figure 1 The present invention is a flowchart of steps of a method for accessing a target server. The method is applied to accessing the target server by developing a server to call a framework including a request processor and a response processor. The method includes:

[0070] Step S100: construct one or more request entities using the development server, and send the request entities to the request processor;

[0071] Step S200: The request processor parses the request entity, constructs a request message including an expression, and requests the target server; and

[0072] Step S300: Utilize the response processor to receive the target data returned by the target server, parse the target data, and return the parsed data to the development server.

[0073] In an embodiment of the present application, access to the target server is achieved by developing a server-side call that includes a request processor and a response processor, wherein the code with common logic constructs a request entity through the development server and uses the request processor to parse the request entity to obtain an expression. This is a process that uses functional programming to encapsulate the code with common logic into a common method, and the operation and maintenance personnel or users can implement the programming of business logic by calling with different parameters. Moreover, when the target server is a Prometheus server, based on this method, the query and operation of Prometheus monitoring data can be implemented without maintaining the underlying PromQL, and the problems of confusing logic and poor readability of the underlying PromQL code caused by different technologies and logical methods of the operation and maintenance personnel or users are avoided.

[0074] In one or more of the following embodiments, a target server is described in detail by taking the Prometheus server as an example.

[0075] The development server can be a Java server, and the request entity can be a CPU request entity, a memory request entity, a network request entity, etc., which includes the actual resource indicators of the request (for example, disk read rate, network card upstream rate, etc.), the operator of the indicator (for example, averaging, taking the maximum value), the time span, the resource screening conditions, etc., and assembles these attributes into a request entity through functional calls.

[0076] The Prometheus server is accessed by calling a framework containing request processors and response processors through the Java server. The code with common logic is encapsulated into a common method, and the business logic is programmed externally through different parameter calls. This is a functional programming method. Based on this method, the query and operation of Prometheus monitoring data can be realized without maintaining the underlying PromQL, and the problems of confusing logic and poor readability of the underlying PromQL code caused by different technologies and logical methods of operation and maintenance personnel or users are avoided.

[0077] Please refer to Figure 2 , Figure 2 This is a flowchart of the workflow of the request processor in the embodiment of the present application.

[0078] In some optional implementations, when the request processor parses one or more request entities, a processing thread is allocated to each request entity.

[0079] In the embodiment of the present application, in view of the situation that there may be multiple dimensions of information query in one page request, for example, one query may query various indicators of CPU, memory, network, and hard disk, a multi-threaded method is used for parallel query. Compared with the serial request method commonly used by programmers in the prior art, the time span of serial query is longer, and the multi-threaded parallel query method can reduce the time spent on query and thus improve the overall user experience.

[0080] In some optional implementations, the request processor parses the request entity, including:

[0081] By processing the thread, the dopost method in the request entity is called to construct the expression of the request; wherein the dopost method, when called, constructs an expression of one or more parameters.

[0082] In the embodiment of the present application, since Prometheus only accepts supported string expressions, the work of the request entity is to convert the programmer's specific settings for the entity into a string expression. Therefore, the request entity also includes a dopost method for constructing an expression that concatenates one or more parameters.

[0083] In some optional implementations, the request processor parses the request entity, further comprising:

[0084] Receive the configuration file sent by the Java server through the request processor; the configuration file includes the IP port number of the Prometheus server; and

[0085] Through the request processor, the request address is constructed according to the Prometheus server IP port number.

[0086] In an embodiment of the present application, in order to obtain monitoring data of a specific Prometheus server, in the process of constructing an http request using a request entity, a request address is also constructed through a configuration file including the Prometheus server ip port number sent by a Java server.

[0087] In some optional implementations, the request processor is also used to construct parameters such as request headers and request types required in the http request.

[0088] In some optional implementations, the request processor also makes a request by calling Apache httpclient (which is used to provide an efficient, up-to-date, and feature-rich client programming toolkit that supports the http protocol).

[0089] In some optional implementations, the framework further includes a logging module, which is used to record the request entity sent to the request processor and the monitoring data returned by the response processor.

[0090] In an embodiment of the present application, a log recording module is used to record the information of each actual request (including the expression of the request, the time of the request, etc.) and the return information, and the response time and the reason for the query exception are recorded to facilitate subsequent analysis and troubleshooting.

[0091] Please refer to Figure 3 , Figure 3 A workflow diagram of the response processor provided for the embodiment of the present application. After requesting Prometheus via http, you need to consider how to process the returned data. Some resources may not have collected data yet, resulting in the returned data being null; some data is returned, but some data is not in a digital format and needs to be replaced; the data is correct but a lot of it is unnecessary and needs to be screened and extracted.

[0092] In some optional implementations, the response processor also receives the return code of the http request returned by the Prometheus server and performs parsing processing, including:

[0093] Determine whether the return code is a return code indicating an HTTP native error;

[0094] If so, log it and return the default return content to the main business program.

[0095] In the embodiment of the present application, the Prometheus server returns a return code to the http request after receiving the http request, and the return code includes: 200, OK, successful request; 400, Bad Request, incorrect syntax request; 401, Unauthorized, unauthenticated; 403, Forbidden, request denied; 404, Not Found, server location not found; 500, Internal Server Error, internal service error. These common Prometheus return codes are classified and processed. If the return code is 200, it is a normal return, and then the next step of the response processor is entered to parse the returned content. For other error messages, the log is recorded first, and then the default return content is returned to the main business program to prevent error messages from popping up on the user display page due to the lack of normal content return, which reduces the user experience or causes subsequent business to fail to proceed normally.

[0096] In some optional implementations, a response processor is used to receive monitoring data returned by the Prometheus server and perform parsing processing, including:

[0097] When the return code indicates that the request is successful, the monitoring data in the returned JSON format is checked to determine whether the status of the monitoring data is incorrect:

[0098] If so, log it and return the default data content to the main business program.

[0099] If not, the point data of the monitoring data is parsed.

[0100] In the embodiment of the present application, when the return code is a return code indicating a successful request, the query status of the returned monitoring data in json format is first determined. When the status status is error, there is no data in the following data. In order to enable subsequent business processing to proceed normally, the response processor will first log the message body and then return the subsequent json message.

[0101] In some optional implementations, after the point data of the monitoring data is parsed, the character string data in the point data is processed as null value data to obtain digital data without a character string.

[0102] In the embodiment of the present application, for the monitoring data in the returned json format, when the status status is success, the point data in the monitoring data is processed, and the string data in the point data is processed as null value data, so that the point data is uniformly processed into digital data. Among them, the point data includes three cases: the first case is for example [1650244850.224,"3844224"], the second case is for example [1650244850.224,"NaN"], and the third case is for example [1650244850.224,null]. In this embodiment, the point data in the monitoring data is processed to process the data belonging to the second case into the data of the third case.

[0103] Please refer to Figure 4 , Figure 4 This is a schematic diagram of a system for accessing a Prometheus server 5 according to an embodiment of the present application, including a Java server 1, a request processor 2, and a response processor 3. For details, please refer to Figure 5 , Figure 5This is a workflow diagram of the access system to the Prometheus server 5 of an embodiment of the present application, wherein the Java server 1 is used to construct one or more request entities and send the request entities to the request processor 2; the request processor 2 is used to parse the request entities, construct an http request and request the Prometheus server 5; the response processor 3 is used to receive the monitoring data returned by the Prometheus server 5, parse and process it, and return the parsed data to the Java server 1.

[0104] In the embodiment of the present application, the Java server 1 calls the framework including the request processor 2 and the response processor 3 to access the Prometheus server 5. The code with common logic is encapsulated into a common method, and the business logic programming is realized by external calls with different parameters. It is a functional programming method. Based on this method, the query and operation of Prometheus monitoring data can be realized without maintaining the underlying PromQL, and the problems of confusing logic and poor readability of the underlying PromQL code caused by different technologies and logical methods of operation and maintenance personnel or users are avoided.

[0105] In some optional implementations, the framework further includes a logging module 4 , which is used to record the request entity sent to the request processor 2 and the monitoring data returned by the response processor 3 .

[0106] In some optional implementations, when the request processor 2 parses one or more request entities, a processing thread is allocated to each request entity.

[0107] In some optional implementations, the request processor 2 is further configured to:

[0108] By processing the thread, the dopost method in the request entity is called to construct the expression of the request; wherein the dopost method, when called, constructs an expression of one or more parameters.

[0109] In some optional implementations, the http request includes a request address;

[0110] Java server 1 is also used to send a configuration file to request processor 2, and the configuration file includes the IP port number of the Prometheus server;

[0111] Request processor 2 is also used to construct a request address based on the Prometheus server IP port number.

[0112] In some optional implementations, the response processor 3 is further configured to:

[0113] Receive the return code of the http request returned by the Prometheus server 5 and parse it, including:

[0114] Determine whether the return code is a return code indicating an HTTP native error;

[0115] If so, log it and return the default return content to the main business program.

[0116] In some optional implementations, the response processor 3 is further configured to:

[0117] When the return code indicates that the request is successful, the monitoring data in the returned JSON format is checked to determine whether the status of the monitoring data is incorrect:

[0118] If so, log it and return the default data content to the main business program.

[0119] If not, the point data of the monitoring data is parsed.

[0120] In some optional implementations, the response processor 3 is further configured to:

[0121] After parsing the point data of the monitoring data, the character string data in the point data is processed as null value data to obtain digital data without character string.

[0122] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0123] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0125] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0126] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for accessing a target server, characterized in that: The method is applied to access the target server by developing a server to call a framework including a request processor and a response processor; The target server is a Prometheus server, the development server is a Java server, and the method includes: The development server is used to assemble and construct one or more request entities by means of functional calls, and the request entity is sent to the request processor; the request entity includes the actual resource index of the request, the operator for the index, the time span, the resource screening condition, and the dopost method for constructing an expression spliced ​​with one or more parameters; The request processor parses the request entity, constructs a request message containing an expression, and requests the target server. When the request processor parses one or more request entities, it allocates a processing thread for each request entity. The request processor parses the request entity and constructs a request message containing an expression, including: by processing a thread, calling the dopost method in the request entity to construct the request expression, that is, using a functional programming method to encapsulate the code with common logic into a common method; constructing the request address by developing a configuration file including the IP port number of the Prometheus server sent by the server, and constructing the request header and request type required in the http request; The target data returned by the target server is received by the response processor, the target data is parsed and processed, and the parsed data is returned to the development server.

2. The method according to claim 1, characterized in that The request processor parses the request entity, and further comprises: The configuration file sent by the development server is received through the request processor; wherein the configuration file includes the IP port number of the target server.

3. The method according to claim 1, characterized in that in, The framework further comprises a logging module, which is used to record the request entity sent to the request processor and the data returned by the response processor.

4. The method according to claim 1, characterized in that The response processor also receives the return code of the http request returned by the target server and performs parsing processing, which includes: Determine whether the return code is a return code indicating an http native error; If so, log it and return the default return content to the main business program.

5. The method according to claim 4, characterized in that The target data is monitoring data in json format; The using a response processor to receive target data returned by the target server and performing parsing processing on the target data includes: When the return code is a return code indicating a successful request, the returned monitoring data is used to determine whether the status of the monitoring data is incorrect: If yes, log it and return the default data content to the main business program; If not, the point data of the monitoring data is parsed.

6. The method according to claim 5, characterized in that After parsing the point data of the monitoring data, the character string data in the point data is processed as null value data to obtain digital data without character string.

7. A system for accessing a target server, characterized in that: It includes a development server, a request processor and a response processor; the target server is a Prometheus server, and the development server is a Java server; The development server assembles and constructs one or more request entities by means of functional calls, and sends the request entity to the request processor; the request entity includes the actual resource indicator of the request, the operator for the indicator, the time span, the resource screening condition, and the dopost method for constructing an expression spliced ​​with one or more parameters; The request processor parses the request entity, constructs a request message including an expression, and requests the target server; When the request processor parses one or more request entities, it allocates a processing thread for each request entity; The request processor parses the request entity and constructs a request message containing an expression, including: by processing a thread, calling a dopost method in the request entity to construct a request expression, that is, using a functional programming method to encapsulate codes with common logic into a common method; Construct the request address by developing a configuration file including the Prometheus server IP and port number sent by the server, and construct the request header and request type required in the http request; The response processor is used to receive the target data returned by the target server, parse the target data, and return the parsed data to the development server.

Images