Index time sequence data access method, system and device and storage medium

Abstract

The invention relates to the field of data access, and discloses an index time sequence data access method, system and device and a storage medium, and the method comprises the steps: sending an http request through business software, and pushing index time sequence data to an identity authentication component; performing identity authentication and standard verification on the http request through an identity authentication component, and forwarding the http request to a shunting component after the verification is passed; the load condition of each push gateway is checked regularly through a check notification assembly, and a shunting strategy is generated according to the load condition and sent to a shunting assembly; forwarding the http request to a target push gateway through the shunting component based on a shunting strategy; and pulling the index time sequence data of the target push gateway at regular time through the Prometheus server. According to the method, the Pushgateway is combined with identity authentication and load shunting, so that the security, the reliability and the flexibility are high, and various complex monitoring requirements can be met.

Description

Technical Field

[0001] The present disclosure relates to the field of data access technology, and in particular to an indicator time series data access method, system, device and storage medium. Background Art

[0002] In the civil aviation industry, indicator time series data is a key component of real-time data such as flight status, equipment monitoring, and meteorological information. With the rapid development of civil aviation business, the processing requirements for indicator time series data are becoming higher and higher, and it is necessary to achieve efficient access, accurate storage, and fast query of data. Traditional data processing methods often seem to be unable to cope with large amounts of high-frequency indicator time series data. As a database system specifically used to process time series data, the indicator time series database (TSDB) has attracted much attention for its ability to efficiently process time series data. Among them, Prometheus, as an open source indicator time series database and monitoring and alarm tool, has been widely used in the civil aviation industry because of its simplicity, ease of use, powerful functions, and active community.

[0003] Prometheus provides two main data access methods: pull and push. In pull mode, the Prometheus server will actively pull data from the target service; while in push mode, the target service will actively push data to the Prometheus server. For civil aviation business, the push access method has significant advantages. First, the push access method can realize real-time data push. In civil aviation business, indicator time series data often needs to be transmitted to the monitoring system in a timely manner for analysis and processing. Through the push method, the data source can actively push data to the Prometheus server without waiting for Prometheus's pull request, thereby realizing real-time data update. Secondly, the push access method has higher flexibility. Different civil aviation business systems may have different data formats and transmission protocols. Through the push method, the data source can customize the data push method and content according to its own needs, so as to better adapt to different business scenarios. In addition, the push access method can also reduce the load on the Prometheus server. In pull mode, Prometheus needs to regularly pull data from a large number of target services, which may cause excessive server load. When the push method is adopted, the data source can actively push data to the Prometheus server, reducing the number of pull requests on the server and thus reducing the load on the server.

[0004] However, despite the many advantages of push access, it still faces some challenges in practical applications. How to ensure the accuracy and security of data push and how to ensure the efficiency of data processing need to be considered when designing the indicator time series data access method. Summary of the invention

[0005] The embodiments of the present disclosure provide an indicator time series data access method, system, device and storage medium to solve or alleviate one or more of the above technical problems in the prior art.

[0006] According to one aspect of the present disclosure, a method for accessing indicator time series data is provided, comprising:

[0007] Send HTTP requests through business software to push indicator time series data to the identity authentication component;

[0008] The http request is authenticated and verified by the identity authentication component, and after the verification passes, the http request is forwarded to the diversion component;

[0009] The load condition of each push gateway is checked regularly by the checking notification component, and a diversion strategy is generated according to the load condition and sent to the diversion component;

[0010] Forwarding the http request to the target push gateway through the diversion component based on the diversion strategy;

[0011] The Prometheus server is used to periodically pull the indicator time series data of the target push gateway.

[0012] In a possible implementation, the load of each push gateway is periodically checked by the checking notification component, and a diversion strategy is generated according to the load and sent to the diversion component, which includes:

[0013] The diversion component diverts and forwards the http request to the specified push gateway based on the preset diversion strategy.

[0014] In a possible implementation, the load of each push gateway is periodically checked by the checking notification component, and a diversion strategy is generated according to the load and sent to the diversion component, which specifically includes:

[0015] Set the saturation rate threshold;

[0016] Regularly check the saturation rate of the push gateway;

[0017] When the saturation rate of the push gateway is less than the saturation rate threshold, the http request is forwarded to the designated push gateway based on the preset diversion strategy;

[0018] When the saturation rate of the push gateway is not less than the saturation rate threshold, the statistical data items are counted by business software name and sorted in ascending order;

[0019] Switch the push gateways corresponding to the top n business software.

[0020] In a possible implementation, the push gateways corresponding to the top n business software are switched, specifically including:

[0021] Determine whether there is a push gateway whose saturation rate is less than the saturation rate threshold;

[0022] If it exists, the push gateway corresponding to the top n business software is switched to the push gateway whose saturation rate is less than the saturation rate threshold;

[0023] Otherwise, a backup push gateway is deployed, and the push gateways corresponding to the top n business software are switched to the backup push gateway.

[0024] According to one aspect of the present disclosure, there is provided an indicator time series data access system, comprising:

[0025] Business software, used to push indicator time series data to the identity authentication component by sending HTTP requests;

[0026] The identity authentication component is used to perform identity authentication and standard verification on the http request, and after passing the verification, forward the http request to the diversion component;

[0027] The inspection notification component is used to regularly check the load of each push gateway, and generate a diversion strategy based on the load and send it to the diversion component;

[0028] A diversion component, used to forward the http request to a target push gateway based on a diversion strategy;

[0029] The Prometheus server is used to periodically pull the indicator time series data of the target push gateway.

[0030] In a possible implementation, the diversion component is further used to divert and forward the http request to the designated push gateway based on a preset diversion strategy before periodically checking the load status of each push gateway.

[0031] In a possible implementation, the checking notification component specifically includes:

[0032] A setting unit, used for setting a saturation rate threshold;

[0033] The checking unit is used to periodically check the saturation rate of the push gateway;

[0034] A forwarding unit, configured to forward the http request to a designated push gateway based on a preset diversion strategy when the saturation rate of the push gateway is less than a saturation rate threshold;

[0035] A statistical unit is used to collect the number of statistical data items according to the business software name and sort them in ascending order when the saturation rate of the push gateway is not less than the saturation rate threshold;

[0036] The switching unit is used to switch the push gateways corresponding to the top n business software in the ranking.

[0037] In a possible implementation, the switching unit specifically includes:

[0038] A judgment module, used to judge whether there is a push gateway whose saturation rate is less than a saturation rate threshold;

[0039] A switching module, for switching the push gateways corresponding to the top n business software to the push gateways having a saturation rate less than the saturation rate threshold when there is a push gateway having a saturation rate less than the saturation rate threshold;

[0040] A deployment module, used for deploying a backup push gateway when there is no push gateway with a saturation rate less than a saturation rate threshold;

[0041] The second switching module is used to switch the push gateway corresponding to the top n business software to the standby push gateway.

[0042] According to one aspect of the present disclosure, there is provided a device, comprising:

[0043] Processor and memory;

[0044] The memory is used to store a computer program, and the processor calls the computer program stored in the memory to execute any of the above-mentioned indicator time series data access methods.

[0045] According to one aspect of the present disclosure, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the processor is enabled to execute any one of the above-mentioned indicator time series data access methods.

[0046] The present disclosure has the following beneficial effects: First, through the introduction of identity authentication, Pushgateway ensures the security and privacy of data. Only users who have been authenticated and obtained corresponding permissions can push access to monitoring data. This effectively prevents malicious access or tampering of data by unauthorized users, ensuring the integrity and confidentiality of the data; secondly, combined with load balancing technology, Pushgateway can automatically adjust resource allocation according to the load of the system. When the amount of monitoring data increases, the system can automatically expand resources and improve data processing capabilities, which not only improves the availability and reliability of the system, but also significantly reduces operating costs; in summary, Pushgateway's solution combining identity authentication and load balancing has a high degree of security, reliability and flexibility, can better meet various complex monitoring needs, and provide powerful and efficient protection for monitoring systems of various sizes.

[0047] The details of one or more embodiments of the present application are presented in the following drawings and descriptions. Other features and advantages of the present application will become apparent from the accompanying drawings. It should be understood that the above general description and the detailed descriptions below are only exemplary and explanatory and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0049] Figure 1 is a flow chart of an indicator time series data access method of this exemplary embodiment;

[0050] Figure 2 is a system component diagram of the present exemplary embodiment;

[0051] Figure 3 is a block diagram of an indicator time series data access system of the present exemplary embodiment;

[0052] Figure 4 It is a schematic structural diagram of a device of this exemplary embodiment. DETAILED DESCRIPTION

[0053] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0054] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware units or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0055] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0056] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example.

[0057] In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or sub-modules is not necessarily limited to those steps or sub-modules explicitly listed, but may include other steps or sub-modules not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0058] Terminology explanation:

[0059] Time series database: A specially designed and optimized database management system specifically for efficient storage, indexing, querying, and analysis of time series data.

[0060] Prometheus: An open source monitoring and alarm system based on a time series database.

[0061] Prometheus server: Prometheus server, used for data collection and storage.

[0062] Pushgateway: An important component in the Prometheus ecosystem, which allows temporary or batch jobs to actively push indicator data to Prometheus instead of being obtained by the Prometheus server through Pull.

[0063] Load sharing: A computer network technology whose core goal is to evenly distribute a large number of network requests or computing tasks to multiple servers or other resources to improve the overall responsiveness, availability, and reliability of the system.

[0064] Figure 1 is a flow chart of an indicator time series data access method of this exemplary embodiment. Figure 1 As shown, an exemplary embodiment of the present disclosure provides an indicator time series data access method, including:

[0065] Send HTTP requests through business software to push indicator time series data to the identity authentication component;

[0066] The http request is authenticated and verified by the identity authentication component, and after the verification passes, the http request is forwarded to the diversion component;

[0067] The load condition of each push gateway is checked regularly by the checking notification component, and a diversion strategy is generated according to the load condition and sent to the diversion component;

[0068] Forwarding the http request to the target push gateway through the diversion component based on the diversion strategy;

[0069] The Prometheus server is used to periodically pull the indicator time series data of the target push gateway.

[0070] The time series data access method proposed in this embodiment realizes unified access and efficient management of time series data in different business fields by classifying and standardizing civil aviation business. This embodiment optimizes the data push mechanism and improves data processing efficiency and accuracy.

[0071] This embodiment proposes a practical method for accessing civil aviation business monitoring data to a time series database. This method is based on an identity authentication component, a (real-time) check notification component, a (switching) diversion component, a Pushgateway service, and a Prometheus server. It can implement identity authentication and load diversion according to civil aviation business software, thereby ensuring data security and high system availability.

[0072] like Figure 2 As shown, this embodiment consists of an identity authentication component, a diversion component, a check notification component, a Pushgateway service, and a Prometheus server.

[0073] Identity authentication component: Civil aviation business software accesses by sending http requests, verifies the account and password of the request, and implements identity authentication; after the identity authentication is passed, the parameters in the request string are verified to see if they meet the software classification standards to prevent malicious access or dirty data from being stored.

[0074] Check notification component: Regularly check the usage of each Pushgateway, calculate and generate a new diversion strategy and notify when the load is too large.

[0075] (Switch) Diversion component: Divert data to different Pushgateways based on the amount of data requested by each software to ensure access performance and high availability of the system. Receive notifications from the "check notification component" and switch the diversion according to the new diversion strategy.

[0076] Pushgateway service and Prometheus server: Each software will be sent to Pushgateway and stored by Prometheus server for indicator data monitoring and alarm.

[0077] Specifically, the load of each push gateway is checked regularly by checking the notification component, and a diversion strategy is generated according to the load and sent to the diversion component, which includes:

[0078] The diversion component diverts and forwards the http request to the specified push gateway based on the preset diversion strategy.

[0079] This embodiment proposes a practical method for accessing a time series database for civil aviation business monitoring data. The specific principle is: for civil aviation software that needs to access the Prometheus time series database, an SDK is provided for the software, and accounts and access standards are divided according to the software. Each software pushes the indicator time series data (a series of data sets formed by arranging indicator data in chronological order) to the identity authentication component by sending an http request (Hypertext Transfer Protocol Request), and accepts the request after passing the check; sends the request to the diversion component for diversion and forwarding (according to the default policy set in advance) to the specified Pushgateway; configures scheduled pull in the Prometheus server, and pulls the indicator data in the Pushgateway to Prometheus for storage. In order to ensure the high availability of the service, the load of the Pushgateway will be monitored and checked regularly, a new diversion strategy will be generated according to the load, and the diversion component will be notified to switch.

[0080] This embodiment mainly has four main processing flows: identity authentication flow, inspection and diversion strategy generation flow, load diversion flow, backup Pushgateway service inspection flow and data storage flow, which are described in detail below.

[0081] Identity authentication process:

[0082] 1. Each software sends an http request;

[0083] 2. Intercept the request and perform identity authentication and standard verification;

[0084] 3. After verification, the request is forwarded to the diversion component.

[0085] Inspection and diversion strategy generation process:

[0086] 1. Regularly check the load of each Pushgateway;

[0087] 2. Calculate the load, hereinafter referred to as saturation rate: current data volume / 5w*100% (tested and verified, the maximum load of Pushgateway is set to 5w);

[0088] 3. Determine whether a new diversion strategy needs to be generated based on the load situation;

[0089] 3.1 If the saturation rate is <90%, there is no need to switch and wait for the next inspection;

[0090] 3.2 If the saturation rate is > 90%, collect the statistical data items by the name of the business software, sort them in ascending order, and obtain the first n software names (the data volume of the first n software accounts for no less than 5%). These n software need to switch Pushgateway;

[0091] 3.3 Select a Pushgateway node with a saturation rate < 90%. If there is no optional node, notify the deployment of a backup Pushgateway service;

[0092] 3.4 Generate a diversion strategy based on the above information, such as (software name 1, original Pushgateway, new Pushgateway);

[0093] 4. Notify the diversion component of the inspection results and new diversion strategy (if any).

[0094] Load sharing process:

[0095] 1. Accept the request and receive the inspection results and new diversion strategy of the "Inspection and diversion strategy generation process";

[0096] 2. According to the inspection results, if there is no change, forward the message according to the initialization strategy (initialization configuration);

[0097] If changes are required, read the new diversion strategy and perform the following operations (for example, if the new diversion strategy is <Software A, Pushgateway1, Pushgateway3>, it means that the request of Software A was originally distributed to Pushgateway1, and now needs to be changed to Pushgateway3):

[0098] (1) Request interception, storing the received request data of software A into the cache;

[0099] (2) Get data from Pushgateway1, store it in the cache, and merge it with the cache in the previous step;

[0100] (3) Wait for confirmation that the data in Pushgateway1 has been pulled by Prometheus (at least 2 / 3 of the data has been pulled);

[0101] (4) Clean up Pushgateway1;

[0102] (5) Perform switching according to the new diversion strategy.

[0103] Backup Pushgateway service deployment check process:

[0104] 1. Check whether there is a notification for deploying a backup Pushgateway service (notification issued during the inspection and diversion strategy generation process);

[0105] 2. If yes, execute the Pushgateway deployment script.

[0106] Data entry process:

[0107] 1. Configure the pull configuration of Pushgateway on each Prometheus Server, including the pull frequency, etc.;

[0108] 2. Regularly pull Pushgateway's indicator data to Prometheus.

[0109] Specifically, the load of each push gateway is checked regularly by the checking notification component, and a diversion strategy is generated according to the load and sent to the diversion component, which specifically includes:

[0110] Set the saturation rate threshold;

[0111] Regularly check the saturation rate of the push gateway;

[0112] When the saturation rate of the push gateway is less than the saturation rate threshold, the http request is forwarded to the designated push gateway based on the preset diversion strategy;

[0113] When the saturation rate of the push gateway is not less than the saturation rate threshold, the statistical data items are counted by business software name and sorted in ascending order;

[0114] Switch the push gateways corresponding to the top n business software.

[0115] Specifically, the push gateways corresponding to the top n business software are switched, including:

[0116] Determine whether there is a push gateway whose saturation rate is less than the saturation rate threshold;

[0117] If it exists, the push gateway corresponding to the top n business software is switched to the push gateway whose saturation rate is less than the saturation rate threshold;

[0118] Otherwise, a backup push gateway is deployed, and the push gateways corresponding to the top n business software are switched to the backup push gateway.

[0119] When accessing time series data, in order to ensure data security and high availability of access services, this proposal mainly involves five specific implementation links: identity authentication, inspection and diversion strategy generation, diversion forwarding, backup Pushgateway service deployment, and data storage. The details are as follows:

[0120] Identity Authentication:

[0121] 1. Before pushing time series data, business personnel store the account password corresponding to the business software and the data push URI standard in the identity information table. An example of the identity information table data is shown in Table 1:

[0122] Table 1 Identity information table:

[0123]

[0124] 2. The HTTP request format of the time series data pushed by the business software is as shown in Table 2:

[0125] Table 2: HTTP request format for time series data pushed by business software:

[0126]

[0127] When receiving a time series data request pushed by business software, the following authentication process needs to be completed:

[0128] (1) According to the software name information contained in the headers of this request, the data push URI standard is obtained from the identity information table and a regular expression match is performed with the URI of this request. If no match is found, a 401 error code is directly returned. If a match is found, step (2) is entered.

[0129] (2) Based on the software name information contained in the headers of this request, the account and password information is obtained from the identity information table and compared with the account and password information of this request. If the comparison fails, a 401 error code is directly returned. If the comparison succeeds, the flow enters the diversion and forwarding stage.

[0130] Inspection and diversion strategy generation:

[0131] The business personnel store the information of all current Pushgateway services in the Pushgateway service information table, set the status of the deployed and available service addresses of the Pushgateway services to enabled, and set the status of the others to disabled. The data of the Pushgateway service information table is shown in Table 3:

[0132] Table 3 Pushgateway service information table:

[0133]

[0134] The following logic is executed every 1 minute:

[0135] (1) Obtain the addresses of all enabled Pushgateway services from the Pushgateway service information table.

[0136] (2) According to the addresses of all Pushgateway services obtained in step (1), query the time series data pushed to the Pushgateway service in turn, and parse the response data line by line. If the line has a job tag, use regular matching to obtain the software name in the job tag value, and count the number of data entries for the software name in the line; if the line does not have a job tag, skip the statistics of the line. The http request format for querying the time series data pushed to the Pushgateway service is shown in Table 4:

[0137] Table 4: HTTP request format of time series data pushed to Pushgateway service:

[0138]

[0139]

[0140] (3) The saturation of the T time series data pushed to each Pushgateway service is counted based on the address of the Pushgateway service. The calculation rules are as follows:

[0141] Saturation of Pushgateway service = sum (number of data items of each software name pushed by the Pushgateway service) / 50000 * 100%.

[0142] (4) Sort the saturations of the Pushgateway services calculated in step (3) from large to small, and determine whether there is a Pushgateway service with a saturation of less than or equal to 90%. If so, record the address, saturation, and other information of the Pushgateway service as divertible Pushgateway service information for diversion strategy generation in step (6); if not, skip the diversion strategy generation and send a notification to the backup Pushgateway service deployment link to notify it to deploy a new backup Pushgateway service.

[0143] (5) According to the saturation of each Pushgateway service calculated in step (3), determine whether there is a Pushgateway service with a saturation greater than 90%. If the saturation of the Pushgateway service is greater than 90%, calculate the saturation corresponding to each software name under the Pushgateway service (the saturation corresponding to each software name = the number of data items corresponding to the software name / 50000*100), and then add up the saturations corresponding to each software name in order from small to large. When the added saturation is greater than 5%, stop adding up, and record the software names involved in the addition, the corresponding saturation, the original Pushgateway service address and other information as software diversion strategy information for diversion strategy generation in step (6); if the saturation of the Pushgateway service is less than or equal to 90%, skip the diversion strategy generation for the Pushgateway service.

[0144] (6) Obtain the divertible Pushgateway service information recorded in step (4) and the software diversion strategy information recorded in (5), and add the saturation corresponding to the software name to the saturation corresponding to the Pushgateway service in turn. If the sum of the saturation of the software name and the saturation corresponding to all Pushgateway services does not exist <= 90%, then record the software name as the software information to be diverted; if it exists, terminate the current addition logic, add the Pushgateway service address added this time as the new Pushgateway service address to the diversion strategy information record for diversion forwarding, and update the saturation corresponding to the Pushgateway service to the sum of the saturation corresponding to the software name and the saturation corresponding to the Pushgateway service. Finally, if the software to be diverted is not empty, send a notification to the backup Pushgateway service deployment link to notify it to deploy a new backup Pushgateway service.

[0145] Traffic forwarding:

[0146] All received time series data requests pushed by business software are intercepted to complete the following diversion and forwarding logic.

[0147] (1) According to the software name information contained in the headers of this request, check whether there is a diversion policy corresponding to the software name in the diversion policy information generated in the previous link. If not, randomly obtain an address of an enabled Pushgateway service from the Pushgateway service information table and proceed to step (3); if it exists, proceed to step (2).

[0148] (2) The business personnel record the addresses of all currently deployed Prometheus services as Prometheus service information, use the current time as the base time, and query the status of all Prometheus services in the Prometheus service information to capture the data pushed by the Pushgateway service according to the software name in the diversion strategy information. The query request format is shown in Table 5:

[0149] Table 5 Query request format:

[0150]

[0151]

[0152] Parse the response data. Determine whether the scrapeUrl field value is consistent with the original Pushgateway service address in the diversion strategy information. If they are consistent, determine whether the lastScrape field value is later than the reference time. If it is later, count the Prometheus service as having captured the push data of the software name. After querying all Prometheus services that have captured the push data of the software name, determine whether the number of Prometheus services that have captured the push data of the software name / the number of all Prometheus services>2 / 3. If not, continue to step (2); if yes, use the original Pushgateway service address of the software name in the diversion strategy information to query the pushed data. The query method refers to the http request format for querying the time series data pushed to the Pushgateway service in the inspection and diversion strategy generation links. Filter the response data by software name, retain only the pushed data of the software name, and then delete the pushed data on the Pushgateway service according to the job field value and instance field value in the pushed data. The Pushgateway delete data http request format is shown in Table 6:

[0153] Table 6 Pushgateway delete data http request format table:

[0154]

[0155]

[0156] Then, the original Pushgateway service address in the diversion strategy information is updated to the new Pushgateway service address. Finally, the time series data pushed by the business software requested this time is grouped by software name as cache data information to be deduplicated and recorded for message forwarding in (3).

[0157] (3) Check whether there is corresponding cache data in the cache data information according to the software name information contained in the headers of this request. If not, directly forward this request to the Pushgateway service address; if so, deduplicate this request and the cache data and forward it to the original Pushgateway service address corresponding to the software name in the diversion strategy information.

[0158] Backup Pushgateway service deployment:

[0159] When receiving a notification to deploy a new backup Pushgateway service, a Pushgateway service address with a status of not enabled is randomly obtained from the Pushgateway service information table, and the Pushgateway deployment script is executed; after successful deployment, the Pushgateway service information table is updated, and the status of the Pushgateway service address is updated to enabled. If no Pushgateway service address with a status of not enabled is obtained from the Pushgateway service information table or the deployment fails, a notification of the failure of the backup Pushgateway service deployment is issued.

[0160] Data storage:

[0161] Configure the Pushgateway service's access data interface information to Prometheus.yaml to enable Prometheus to periodically crawl and store data. The configuration information is as follows:

[0162] {job_name: Pushgateway / / The monitoring task name is Pushgateway

[0163] scrape_interval:10s / / The time interval for scraping indicator data is 10s

[0164] honor_labels:true / / Configure basic authentication related information, used to authenticate the user name and password for access

[0165] basic_auth:

[0166] Username:****** / / Specific username

[0167] password:****** / / Specific password

[0168] static_configs:

[0169] targets:****** / / Specific targets for capturing indicators, which can be IP addresses, domain names, etc.}

[0170] Access Prometheus through the page to check whether the configured Pushgateway is effective.

[0171] Access Prometheus through the page and query the data to view the stored data.

[0172] In summary, the target service of this embodiment actively pushes data to the Prometheus server in a push manner, realizes real-time data push, reduces the number of pull requests of the server, and thus reduces the load of the server; at the same time, through pre-authentication, ensures the security and privacy of the data; through redundant deployment and load balancing switching of nodes, ensures the high availability of the service.

[0173] Figure 3 is a block diagram of an indicator time series data access system of this exemplary embodiment, such as Figure 3 As shown, an exemplary embodiment of the present disclosure provides an indicator time series data access system, including:

[0174] Business software, used to push indicator time series data to the identity authentication component by sending HTTP requests;

[0175] The identity authentication component is used to perform identity authentication and standard verification on the http request, and after passing the verification, forward the http request to the diversion component;

[0176] The inspection notification component is used to regularly check the load of each push gateway, and generate a diversion strategy based on the load and send it to the diversion component;

[0177] A diversion component, used to forward the http request to a target push gateway based on a diversion strategy;

[0178] The Prometheus server is used to periodically pull the indicator time series data of the target push gateway.

[0179] Specifically, the diversion component is also used to divert and forward the http request to the designated push gateway based on a preset diversion strategy before regularly checking the load status of each push gateway.

[0180] Specifically, the inspection notification component includes:

[0181] A setting unit, used for setting a saturation rate threshold;

[0182] The checking unit is used to periodically check the saturation rate of the push gateway;

[0183] A forwarding unit, configured to forward the http request to a designated push gateway based on a preset diversion strategy when the saturation rate of the push gateway is less than a saturation rate threshold;

[0184] A statistical unit is used to collect the number of statistical data items according to the business software name and sort them in ascending order when the saturation rate of the push gateway is not less than the saturation rate threshold;

[0185] The switching unit is used to switch the push gateways corresponding to the top n business software in the ranking.

[0186] Specifically, the switching unit includes:

[0187] A judgment module, used to judge whether there is a push gateway whose saturation rate is less than a saturation rate threshold;

[0188] A switching module, for switching the push gateways corresponding to the top n business software to the push gateways having a saturation rate less than the saturation rate threshold when there is a push gateway having a saturation rate less than the saturation rate threshold;

[0189] A deployment module, used for deploying a backup push gateway when there is no push gateway with a saturation rate less than a saturation rate threshold;

[0190] The second switching module is used to switch the push gateway corresponding to the top n business software to the standby push gateway.

[0191] Figure 4 Schematic diagram of the structure of a device of this exemplary embodiment. Figure 4 As shown, corresponding to the indicator time series data access method provided above, the present disclosure also provides a device. Since the embodiment of the device is similar to the above method embodiment, the description is relatively simple. Please refer to the description of the above method embodiment for relevant details. The device described below is only schematic. The device may include: a processor 1, a memory 2, a communication bus (i.e., the above device bus) and a search engine, wherein the processor 1 and the memory 2 communicate with each other through the communication bus and communicate with the outside through the communication interface. The processor 1 can call the logic instructions in the memory 2 to execute the indicator time series data access method.

[0192] In addition, the logic instructions in the above-mentioned memory 2 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: a storage chip, a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes.

[0193] On the other hand, an embodiment of the present disclosure further provides a processor-readable storage medium, on which a computer program 3 is stored. When the computer program 3 is executed by the processor 1, the indicator timing data access method provided in the above embodiments is implemented.

[0194] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor 1, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0195] The above are only preferred implementations of the present disclosure, and the protection scope of the present disclosure is not limited to the above embodiments. All technical solutions under the concept of the present disclosure belong to the protection scope of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present disclosure should be regarded as the protection scope of the present disclosure.

Claims

1. A method for accessing indicator time series data, characterized in that: include: Send HTTP requests through business software to push indicator time series data to the identity authentication component; The http request is authenticated and verified by the identity authentication component, and after the verification passes, the http request is forwarded to the diversion component; The load condition of each push gateway is checked regularly by the checking notification component, and a diversion strategy is generated according to the load condition and sent to the diversion component; Forwarding the http request to the target push gateway through the diversion component based on the diversion strategy; The Prometheus server is used to periodically pull the indicator time series data of the target push gateway.

2. The indicator time series data access method according to claim 1, characterized in that: The load of each push gateway is checked regularly by the check notification component, and a diversion strategy is generated according to the load and sent to the diversion component, which includes: The diversion component diverts and forwards the http request to the specified push gateway based on the preset diversion strategy.

3. The index time series data access method according to claim 1, characterized in that: The load of each push gateway is checked regularly by the check notification component, and a diversion strategy is generated according to the load and sent to the diversion component, which specifically includes: Set the saturation rate threshold; Regularly check the saturation rate of the push gateway; When the saturation rate of the push gateway is less than the saturation rate threshold, the http request is forwarded to the designated push gateway based on the preset diversion strategy; When the saturation rate of the push gateway is not less than the saturation rate threshold, the statistical data items are counted by business software name and sorted in ascending order; Switch the push gateways corresponding to the top n business software.

4. The indicator time series data access method according to claim 3, characterized in that: Switch the push gateways corresponding to the top n business software, including: Determine whether there is a push gateway whose saturation rate is less than the saturation rate threshold; If it exists, the push gateway corresponding to the top n business software is switched to the push gateway whose saturation rate is less than the saturation rate threshold; Otherwise, a backup push gateway is deployed, and the push gateways corresponding to the top n business software are switched to the backup push gateway.

5. An indicator time series data access system, characterized in that: include: Business software, used to push indicator time series data to the identity authentication component by sending HTTP requests; The identity authentication component is used to perform identity authentication and standard verification on the http request, and after passing the verification, forward the http request to the diversion component; The inspection notification component is used to regularly check the load of each push gateway, and generate a diversion strategy based on the load and send it to the diversion component; A diversion component, used to forward the http request to a target push gateway based on a diversion strategy; The Prometheus server is used to periodically pull the indicator time series data of the target push gateway.

6. The indicator time series data access system according to claim 5, characterized in that: The diversion component is also used to divert and forward the http request to the designated push gateway based on the preset diversion strategy before regularly checking the load status of each push gateway.

7. The indicator time series data access system according to claim 5, characterized in that: The inspection notification components specifically include: A setting unit, used for setting a saturation rate threshold; The checking unit is used to periodically check the saturation rate of the push gateway; A forwarding unit, configured to forward the http request to a designated push gateway based on a preset diversion strategy when the saturation rate of the push gateway is less than a saturation rate threshold; A statistical unit is used to collect the number of statistical data items according to the business software name and sort them in ascending order when the saturation rate of the push gateway is not less than the saturation rate threshold; The switching unit is used to switch the push gateways corresponding to the top n business software in the ranking.

8. The indicator time series data access system according to claim 7, characterized in that: The switching unit specifically includes: A judgment module, used to judge whether there is a push gateway whose saturation rate is less than a saturation rate threshold; A switching module, for switching the push gateways corresponding to the top n business software to the push gateways having a saturation rate less than the saturation rate threshold when there is a push gateway having a saturation rate less than the saturation rate threshold; A deployment module, used for deploying a backup push gateway when there is no push gateway with a saturation rate less than a saturation rate threshold; The second switching module is used to switch the push gateway corresponding to the top n business software to the standby push gateway.

9. A device, characterized in that: include: Processor and memory; The memory is used to store a computer program, and the processor calls the computer program stored in the memory to execute the indicator time series data access method described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is enabled to execute the indicator time series data access method described in any one of claims 1 to 4.

Images