An information collection method, device, equipment and storage medium

Abstract

This application discloses an information collection method, apparatus, device, and storage medium applied to Nginx servers. Multiple Nginx servers are distributed and deployed in an Nginx cluster, with at least one load balancer (LB) Nginx namespace deployed on each Nginx server. The method includes: when etcd is updated, obtaining all LB configuration information from etcd and creating a task queue for all LBs to perform collection tasks; based on all LB configuration information, determining when the current Nginx server deploys an LBi Nginx namespace, and obtaining the first collection time of LBi from the task queue; based on the first collection time, collecting traffic forwarding volume from the LBi Nginx namespace. This distributed and decentralized approach to collecting LB traffic forwarding volume avoids the problems of remote login failures, disconnections, and timeouts that occur when a centralized monitoring node cannot remotely log in to an unspecified number of Nginx servers during centralized information collection, ensuring the integrity and accuracy of the collected traffic forwarding volume information (i.e., outbound traffic information).

Description

Technical Field

[0001] This application relates to cloud computing technology, and more particularly to an information acquisition method, apparatus, device, and storage medium. Background Technology

[0002] With the development of 5G technology, network applications such as short videos, live video streaming, video chat, and video conferencing are experiencing explosive growth, leading to a surge in network traffic. Elastic load balancers (LB), as crucial network forwarders in the cloud computing field, play a key role in resolving network congestion and distributing traffic.

[0003] Currently, the load balancer (LB) in the cloud platform supports Layer 4 and Layer 7 traffic distribution. One existing load balancer deployment implementation is as follows: Layer 4 load balancing traffic forwarding is achieved through a Linux Virtual Server (LVS) + Keepalived approach. LVS is the main entry point for network traffic, and the Keepalived component is responsible for configuring and managing LVS and providing high availability services. Layer 7 load balancing egress traffic forwarding is implemented through an Nginx server, which acts as a backend server for LVS, further parsing and forwarding Layer 7 egress traffic. In LVS Direct Routing (DR) mode, egress traffic is obtained from the backend RS (Remote Server) of the LVS server, i.e., the Nginx server. To ensure high service availability, the Nginx server is deployed in a distributed manner, i.e., deployed on different nodes. Egress traffic information can be obtained by separately acquiring the forwarded traffic of each Nginx server. Specifically, a monitoring information collection center node is used. This center node contains remote connection information for all distributed Nginx servers. When it is necessary to collect the egress traffic forwarding volume of a specific LB at a certain moment, this center node needs to remotely log into all distributed Nginx namespaces to obtain the egress traffic information at the same time.

[0004] However, when there are a large number of load balancers (LBs) to monitor in the cloud platform, each LB requires remote login to a variable number of distributed Nginx servers. This can lead to problems such as remote login failures, disconnections, and timeouts, resulting in incomplete or failed outbound traffic information obtained by the distributed Nginx servers. Summary of the Invention

[0005] To address the aforementioned technical problems, this application aims to provide an information acquisition method, apparatus, device, and storage medium.

[0006] The technical solution of this application is implemented as follows:

[0007] Firstly, an information collection method is provided, applied to Nginx servers. Multiple Nginx servers are deployed in a distributed manner within an Nginx cluster, and each Nginx server deploys at least one Nginx namespace of a load balancer (LB). The method includes:

[0008] When the monitor detects that there is a data update in the etcd database, it retrieves all LB configuration information in the Nginx cluster from the etcd database, as well as the task queue created by the periodic collection task for all LB forwarding volume.

[0009] Based on all LB configuration information, when determining the Nginx namespace on the current Nginx server where the LBi in the task queue is deployed, the first collection time of the LBi in the current period is obtained from the task queue; i is a positive integer;

[0010] Based on the first collection time, the corresponding traffic forwarding volume is collected from the Nginx namespace of the LBi.

[0011] In the above scheme, the step of collecting the corresponding traffic forwarding volume from the Nginx namespace of the LBi based on the first collection time includes: obtaining a first preset duration for delaying the execution of the information collection command; determining a second collection time based on the first collection time and the first preset duration; and collecting the traffic forwarding volume of the LBi during the second collection time from the Nginx namespace of the LBi.

[0012] In the above scheme, the method further includes: uploading the identification information of the LBi, the second collection time, and the traffic forwarding volume to the etcd database for storage, so that the controller can obtain the traffic forwarding volume from the etcd database and display it.

[0013] In the above scheme, the information acquisition time of the controller is the time after a second preset delay on the second acquisition time.

[0014] In the above scheme, the LB configuration information includes at least one Nginx namespace and the identification information of the Nginx server to which each Nginx namespace belongs.

[0015] In the above scheme, the method further includes: obtaining the configuration information of the LBi from all LB configuration information based on the identification information of the LBi in the task queue; determining whether the current Nginx server is included among all Nginx servers to which the Nginx namespaces of the LBi belong in the configuration information of the LBi; if yes, determining that the Nginx namespace of the LBi in the task queue is deployed on the current Nginx server; if no, determining that the Nginx namespace of the LBi in the task queue is not deployed on the current Nginx server.

[0016] In the above scheme, the method further includes: after the current period collection task ends, if the total traffic forwarding volume of the LBi is greater than a first forwarding volume threshold, determining to start a new Nginx server and deploy the Nginx namespace of the LB, and updating the LB configuration information and the task queue in the etcd database; or, after the current period collection task ends, if the total traffic forwarding volume of the LBi is less than a second forwarding volume threshold, determining to delete redundant Nginx servers and delete the Nginx namespace of the LB, and updating the LB configuration information and the task queue in the etcd database; wherein, the first forwarding volume threshold is greater than the second forwarding volume threshold.

[0017] Secondly, an information collection device is provided, applied to an Nginx server. Multiple Nginx servers are distributed and deployed in an Nginx cluster, and each Nginx server deploys at least one load balancer (LB) Nginx namespace. The device includes:

[0018] The acquisition unit is used to acquire all LB configuration information in the Nginx cluster from the etcd database when it detects that there is a data update in the etcd database, and to create a task queue for periodically collecting data on all LB forwarding volumes.

[0019] The acquisition unit is used to determine the Nginx namespace on the current Nginx server where the task queue LBi is deployed based on all LB configuration information, and then obtain the first collection time of LBi in the current period from the task queue; i is a positive integer;

[0020] The collection unit is used to collect the corresponding traffic forwarding volume from the Nginx namespace of the LBi based on the first collection time.

[0021] Thirdly, an information acquisition device is provided for use on an Nginx server, comprising: a processor and a memory configured to store a computer program capable of running on the processor, wherein the processor is configured to execute the steps of the aforementioned method when running the computer program.

[0022] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the aforementioned method.

[0023] This application provides an information collection method applied to Nginx servers. Multiple Nginx servers are distributed and deployed in an Nginx cluster, and each Nginx server deploys at least one load balancer (LB) namespace. The method includes: when data updates are detected in the etcd database, retrieving all LB configuration information within the Nginx cluster from the etcd database, and creating a task queue for periodically collecting traffic from all LB forwarding volumes; based on all LB configuration information, determining when the Nginx namespace of LBi in the task queue is deployed on the current Nginx server, and retrieving the first collection time of LBi within the current period from the task queue; where i is a positive integer; and based on the first collection time, collecting the corresponding traffic forwarding volume from the Nginx namespace of LBi. This distributed and decentralized approach to collecting LB traffic forwarding volume avoids the problems of remote login failures, disconnections, and timeouts that occur when a centralized monitoring node cannot remotely log in to an unspecified number of Nginx servers, ensuring the integrity and accuracy of the collected traffic forwarding volume information. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of traffic forwarding in the existing LVS DR mode;

[0025] Figure 2 This is a schematic diagram of an existing namespace-based distributed elastic load balancer deployment.

[0026] Figure 3 This is a schematic diagram of the first process of the information collection method in the embodiments of this application;

[0027] Figure 4 This is a first schematic diagram of a namespace-based distributed elastic load balancing deployment in an embodiment of this application.

[0028] Figure 5 This is a schematic diagram of the second process of the information collection method in the embodiments of this application;

[0029] Figure 6 This is a schematic diagram illustrating the relationship between the task queue LBi and the first acquisition time in an embodiment of this application;

[0030] Figure 7 This is a second schematic diagram of a namespace-based distributed elastic load balancing deployment in an embodiment of this application.

[0031] Figure 8 This is a timing diagram of information collection in an embodiment of this application;

[0032] Figure 9 This is a schematic diagram of the structure of the information acquisition device in the embodiments of this application;

[0033] Figure 10 This is a schematic diagram of the structure of the information acquisition device in the embodiments of this application. Detailed Implementation

[0034] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0035] Figure 1 This is a diagram illustrating traffic forwarding in the existing LVSDR mode, such as... Figure 1 As shown, in DR mode, when the LVS server receives a request from the client, it rewrites the MAC address in the request packet and sends the request to the real server (RS). The RS then directly returns the response to the client. Therefore, in DR mode, the entry point for Layer 4 traffic is LVS, and the exit point is directly returned to the client by its backend RS. When obtaining LB traffic forwarding information, inbound traffic can be obtained from the LVS server using the `ipvsadm` command, and outbound traffic can be obtained from the LVS server's backend RS, i.e., the Nginx server.

[0036] To achieve high service availability, a namespace-based distributed elastic load balancing deployment scheme is presented. Figure 2 This is a schematic diagram of an existing namespace-based distributed elastic load balancing deployment, such as... Figure 2 As shown, the LVS cluster includes two servers, namely LVS1 and LVS2. Here, a pair of primary and standby Keepalived servers contained in the same load balancer are deployed on LVS1 and LVS2 respectively. Keepalived is used to configure and manage the LVS servers and implement Layer 4 ingress traffic forwarding. Specifically, ingress traffic is obtained from the namespace deployed on the LVS server using the ipvsadm command.

[0037] The LVS cluster backend deploys a distributed Nginx cluster, consisting of Nginx1, Nginx2, ..., NginxN servers. Each Nginx server deploys at least one load balancer (LB) Nginx namespace. Specifically, Nginx1 deploys Nginx namespaces -LB1, -LB2, and -LBi; Nginx2 deploys Nginx namespaces -LB1, -LB2, and -LBi; and NginxN deploys Nginx namespaces -LB2 and -LBi. The number of 'i' and 'N' is determined based on actual user traffic. The Nginx servers are used to implement Layer 7 traffic forwarding. When acquiring outbound traffic, a monitoring information collection center node is used. This center node contains remote connection information for all distributed Nginx servers. When it's necessary to collect the outbound traffic forwarding volume of a specific LB at a given time, this center node needs to remotely log into all distributed Nginx namespaces to obtain the outbound traffic information at that specific moment. However, when there are a large number of load balancers (LBs) to monitor in the cloud platform, each LB requires remote login to a variable number of distributed Nginx servers. This can lead to problems such as remote login failures, disconnections, and timeouts, resulting in incomplete or failed outbound traffic information obtained by the distributed Nginx servers.

[0038] Therefore, this application proposes an export flow information collection method, which solves the problem of incomplete or failed export flow information acquisition in existing methods.

[0039] Figure 3 This is a schematic diagram of the first process of the information collection method in this application embodiment. This information collection method is applied to Nginx servers. Multiple Nginx servers are deployed in a distributed manner within the Nginx cluster, and each Nginx server deploys at least one Nginx namespace for load balancing. For example, this information collection method can be applied to a monitor on the Nginx server. To intuitively illustrate the relationship between the various components involved in the information collection method, this application embodiment provides a schematic diagram of a namespace-based distributed elastic load balancing deployment. Figure 4 ,like Figure 4As shown, the Nginx cluster is deployed in a distributed manner with servers Nginx1, Nginx2, ..., NginxN. Each Nginx server deploys one monitor and at least one load balancer (LB) Nginx namespace. Specifically, server Nginx1 deploys monitor 1, Nginx namespace-LB1, Nginx namespace-LB2, and Nginx namespace-LBi; server Nginx2 deploys monitor 2, Nginx namespace-LB1, Nginx namespace-LB2, and Nginx namespace-LBi; and server NginxN deploys monitor N, Nginx namespace-LB2, and Nginx namespace-LBi. The number of i and N is set according to the actual user traffic.

[0040] The monitors mentioned above are mainly used to collect LB (Load Balancer) outbound traffic information. The following section details the information collection method for any one monitor in the Nginx cluster; the information collection methods for other monitors can be referenced. Figure 3 As shown, this information collection method may specifically include:

[0041] Step 301: When data updates are detected in the etcd database, obtain all LB configuration information in the Nginx cluster from the etcd database, and create a task queue for periodically collecting all LB forwarding volumes.

[0042] In this application, the LB forwarding volume is collected according to a preset period, and the data update of the etcd database refers to the fact that the etcd database stores all LB configuration information and task queues in the Nginx cluster corresponding to the new period.

[0043] Here, the monitor is used to monitor whether there are updates to the data in the etcd database and to collect the LB forwarding volume. When the monitor detects that there is a data update in the etcd database, it indicates that the task of collecting all LB forwarding volume in the previous period has been completed. At this time, the monitor retrieves all LB configuration information and task queues in the Nginx cluster corresponding to the new period from the etcd database to execute the collection task for all LB forwarding volume in the new period.

[0044] It's important to note that etcd is a highly available, strongly consistent distributed key-value database. Here, etcd stores all load balancer (LB) configuration information for the current period, with each LB identifier serving as the key and each LB configuration information as the value. etcd also stores a task queue created for periodically collecting data on all LB forwarding volumes. This task queue includes each LB identifier as the key and the collection time for each LB within the current period (i.e., the first collection time mentioned below) as the value.

[0045] In some embodiments, the LB configuration information includes at least one Nginx namespace and the identification information of the Nginx server to which each Nginx namespace belongs.

[0046] For example, Figure 4 If N equals 3, then the LB1 configuration information includes two Nginx namespaces, which are deployed on the Nginx1 server and the Nginx2 server respectively.

[0047] Step 302: Based on all LB configuration information, when determining the Nginx namespace of LBi in the task queue deployed on the current Nginx server, obtain the first collection time of LBi in the current period from the task queue; i is a positive integer.

[0048] It should be noted that since not every load balancer's Nginx namespace is deployed on the Nginx server, when the monitor collects LBi forwarding volume based on the load balancer order in the task queue, it needs to first determine whether the current Nginx server has deployed the LBi Nginx namespace. If deployment is confirmed, the first collection time of LBi is obtained from the task queue, so that the corresponding traffic forwarding volume can be collected from the LBi Nginx namespace based on the first collection time.

[0049] For example, in some embodiments, the method further includes: obtaining the configuration information of the LBi from all LB configuration information based on the identification information of the LBi in the task queue; determining whether the current Nginx server is included among all Nginx servers to which the Nginx namespaces of the LBi belong in the configuration information of the LBi; if yes, determining that the Nginx namespace of the LBi in the task queue is deployed on the current Nginx server; if no, determining that the Nginx namespace of the LBi in the task queue is not deployed on the current Nginx server.

[0050] The LB configuration information includes at least one Nginx namespace and the identification information of the Nginx server to which each Nginx namespace belongs. In this embodiment, when determining whether an Nginx server has an LBi Nginx namespace deployed, the system identifies all Nginx servers with deployed LBi Nginx namespaces based on the LBi configuration information. It then determines whether the current Nginx server is among these servers. If it is, it indicates that the current Nginx server has an LBi Nginx namespace deployed; otherwise, it indicates that the current Nginx server has not deployed an LBi Nginx namespace.

[0051] For example, Figure 4 If N equals 3, meaning the Nginx cluster deploys 3 Nginx servers: Nginx1, Nginx2, and Nginx3, and LB1 configuration includes 2 Nginx namespaces deployed on Nginx1 and Nginx2 respectively. When Monitor 1 determines whether Nginx1 deploys LB1's Nginx namespace, it determines, based on LB1 configuration, that all Nginx servers deployed with LB1's Nginx namespace include Nginx1; that is, Nginx1 deploys LB1's Nginx namespace. When Monitor 2 determines whether Nginx2 deploys LB1's Nginx namespace, it determines, based on LB1 configuration, that all Nginx servers deployed with LB1's Nginx namespace include Nginx2; that is, Nginx2 deploys LB1's Nginx namespace. When monitor 3 determines whether the Nginx3 server has deployed the Nginx namespace of LB1, monitor 3 determines, based on the LB1 configuration information, that the Nginx3 server is not among all the Nginx servers deployed in the Nginx namespace of LB1, that is, the Nginx3 server has not deployed the Nginx namespace of LB1.

[0052] Step 303: Based on the first collection time, collect the corresponding traffic forwarding volume from the Nginx namespace of the LBi.

[0053] This can be understood as collecting the traffic forwarding volume of LBi at the first collection time from LBi's Nginx namespace.

[0054] Here, the execution entity for steps 301 to 303 can be the processor of the information acquisition device.

[0055] By adopting the above technical solution, a distributed and decentralized approach is used to collect LB traffic forwarding data. This avoids situations where the central node cannot remotely log in to an unspecified number of Nginx servers during centralized monitoring data collection, resulting in remote login failures, disconnections, timeouts, etc., thus ensuring the integrity and accuracy of the collected traffic forwarding data.

[0056] Based on the above embodiments, this application also proposes another information collection method. Figure 5 This is a schematic diagram of the second process of the information collection method in this application embodiment. This information collection method is applied to a monitor on an Nginx server. Multiple Nginx servers are deployed in a distributed manner within the Nginx cluster, and each Nginx server deploys at least one load balancer (LB) Nginx namespace. For example... Figure 5 As shown, the steps of this information collection method may include:

[0057] Step 501: When data updates are detected in the etcd database, obtain all LB configuration information in the Nginx cluster from the etcd database, and create a task queue for periodically collecting all LB forwarding traffic.

[0058] In some embodiments, the LB configuration information includes at least one Nginx namespace and the identification information of the Nginx server to which each Nginx namespace belongs.

[0059] Step 502: Based on all LB configuration information, when determining the Nginx namespace of LBi in the task queue deployed on the current Nginx server, obtain the first collection time of LBi in the current period from the task queue; i is a positive integer.

[0060] For example, in some embodiments, the method further includes: obtaining the configuration information of the LBi from all LB configuration information based on the identification information of the LBi in the task queue; determining whether the current Nginx server is included among all Nginx servers to which the Nginx namespaces of the LBi belong in the configuration information of the LBi; if yes, determining that the Nginx namespace of the LBi in the task queue is deployed on the current Nginx server; if no, determining that the Nginx namespace of the LBi in the task queue is not deployed on the current Nginx server.

[0061] Step 503: Obtain the first preset duration for delaying the execution of the information collection command.

[0062] Step 504: Determine the second acquisition time based on the first acquisition time and the first preset duration.

[0063] Step 505: Collect the traffic forwarding volume of LBi at the second collection time from the Nginx namespace of LBi.

[0064] Steps 503 to 505 should be noted that multiple Nginx servers are deployed in a distributed manner in the Nginx cluster, and a monitor is deployed on each Nginx server, such as... Figure 3 In this application, monitoring devices 1, 2, ..., N are configured. To eliminate the time delay caused by each monitoring device updating all load balancer (LB) configuration information and task queues from the etcd database to its local machine when it detects updates in the etcd database, resulting in inconsistent information collection times, a first preset duration is set to delay the execution of information collection commands by each monitoring device, based on the first collection time in the task queue. This ensures that each monitoring device collects the corresponding traffic forwarding volume from the Nginx namespace of the LBi at the second collection time. This ensures that all monitoring devices execute a unified information collection command at a consistent time, achieving absolute consistency in the time of information collection.

[0065] In some embodiments, the method further includes: uploading the identification information of the LBi, the second collection time, and the traffic forwarding volume to the etcd database for storage, so that the controller can obtain the traffic forwarding volume from the etcd database and display it.

[0066] In other words, the monitor uploads the LBi traffic forwarding volume collected during the second collection time to the etcd database for storage, so that the controller can retrieve the LBi traffic forwarding volume from the etcd database and display it for users to view.

[0067] It should be further explained that multiple Nginx servers are deployed in a distributed manner in the Nginx cluster, and a monitor is deployed on each Nginx server. When multiple monitors collect the traffic forwarding volume of LBi at a certain moment, the controller needs to retrieve all the traffic forwarding volume of LBi from the etcd database and aggregate it, and display the aggregated total traffic forwarding volume for users to view.

[0068] In some embodiments, the information acquisition time of the controller is the time after a second preset delay on the second acquisition time.

[0069] Here, the second preset duration is a time delay for executing the information acquisition command, which is set in advance based on the second acquisition time to ensure strong consistency in the time when the controller performs the information aggregation operation.

[0070] Using the above technical solution, each monitor on each Nginx server collects the corresponding traffic forwarding volume from the Nginx namespace of LBi according to the second collection time, rather than according to the first collection time in the task queue. The second collection time is the first collection time delayed by a first preset duration. This is to eliminate the time delay caused by each monitor updating all LB configuration information and task queues obtained from the etcd database to its local machine when it detects updates in the etcd database, which could lead to inconsistencies in the time of information collected by different monitors. This method ensures that all monitors execute a unified information collection command at an absolute time, achieving the effect of absolutely consistent information collection time.

[0071] Based on the above embodiments, this application provides an exemplary description of how the Nginx cluster's LB configuration information stored in the etcd database, and the task queue created by periodically collecting all LB forwarding volumes, are obtained.

[0072] For example, the device involved in the information collection method of this application also includes a coordinator. The coordinator can be understood as a load balancer (LB) information configuration platform. That is, the coordinator configures the LB information according to the actual traffic used by the user, generates all LB configuration information after configuration, and uploads it to the etcd database. The LB configuration information includes at least one Nginx namespace and the identification information of the Nginx server to which each Nginx namespace belongs.

[0073] In addition, the coordinator also maintains a task queue. When the coordinator starts, the initial time is set to T0. Based on the number of buffers (LBs) cached at this time and the information collection time interval T, the specific first collection time ti (ti = T0 + i * T / P, where P is the current number of LBs, i is the identifier of the LB creation order, 1 <= i <= P) is determined for each LB in the current period. The LB identifier information and the first collection time ti are stored in the task queue in the order of LB creation and then uploaded to the etcd database.

[0074] Furthermore, after an information collection time interval T, i.e., when T0+T is reached, the latest LB count and the corresponding first collection time are calculated again and placed at the end of the task queue. This process is repeated to ensure that the task queue can execute the latest LB information collection task. Figure 6 This is a schematic diagram illustrating the relationship between the task queue LBi and the first acquisition time in an embodiment of this application, as shown below. Figure 6As shown, for example, at time T0, LBi = 3, indicating that there are currently 3 LBs. The information collection time interval T is divided into 3 equal parts. At this time, the information collection time for LB1 is set to t1 = T0 + 1*T / 3. At time T0+T, LBi = 5, indicating that there are currently 5 LBs. The information collection time interval is divided into 5 equal parts. At this time, the information collection time for the second LB is set to t2 = T0 + T + 2*T / 5. And so on. Within each information collection time interval T, a fixed information collection time is assigned to each LB.

[0075] The issue of varying load balancer (LB) capacity across different periods arises because actual user traffic changes at different times. When the total traffic forwarding volume of the LB increases to a certain threshold, new Nginx servers need to be started to expand capacity; conversely, redundant Nginx servers need to be deleted to reduce capacity.

[0076] For example, in some embodiments, the method further includes: after the current periodic collection task ends, if the total traffic forwarding volume of the LBi is greater than a first forwarding volume threshold, determining to start a new Nginx server and deploy the Nginx namespace of the LB, and updating the LB configuration information and the task queue in the etcd database; or,

[0077] After the current periodic data collection task is completed, if the total traffic forwarding volume of the LBi is less than the second forwarding volume threshold, it is determined to delete the redundant Nginx servers and delete the Nginx namespace of the LB, and update the LB configuration information and the task queue in the etcd database; wherein, the first forwarding volume threshold is greater than the second forwarding volume threshold.

[0078] Specifically, after the current period's data collection task concludes, if the coordinator detects that the total traffic forwarding volume of the LBi exceeds the first forwarding threshold, it will expand the Nginx cluster by adding new Nginx servers and deploying new Nginx namespaces on these servers. If the coordinator detects that the total traffic forwarding volume of the LBi is less than the second forwarding threshold, it will select the Nginx server with the lowest traffic in the cluster and, through dynamic rate limiting commands, gradually reduce the number of connections to that Nginx server belonging to that LBi's Nginx namespace until the number of connections is reduced to 0. Then, it will delete that Nginx namespace and reclaim its resources. In this way, the coordinator will upload all new LBi configuration information and task queues to the etcd database, allowing the monitor to obtain the new LBi configuration information and task queues from the etcd database and complete the LBi traffic forwarding volume collection again.

[0079] Based on the above embodiments, this application provides a schematic diagram of a namespace-based distributed elastic load balancing deployment, and also provides an information collection timing diagram. Figure 7 This is a second schematic diagram of a namespace-based distributed elastic load balancing deployment in an embodiment of this application. Figure 8 This is a timing diagram of information collection in the embodiments of this application, combined with Figure 7 and Figure 8 The information collection process is explained in detail.

[0080] Step 1: Configure Network Time Protocol (NTP) synchronization to synchronize the initial clocks of the monitoring center, coordinator, aggregator (same as the controller mentioned earlier), etcd database, and each monitor.

[0081] Here, the monitor is primarily responsible for collecting traffic forwarding information, specifically relying on data from the etcd database. This data is periodically updated by the coordinator. The aggregator combines the traffic forwarding data collected simultaneously by multiple monitors from the LBi to determine the total traffic forwarding volume. For example, the monitor, coordinator, etcd database, and aggregator can be deployed on the information collection device; that is, in practical applications, a single information collection device can simultaneously perform the functions of a monitor, coordinator, etcd database, and aggregator. The monitoring center is used for information display and can be deployed on this information collection device or on other display devices.

[0082] Step 2: The coordinator pushes updated LB configuration information to the etcd database. The LB configuration information includes at least one Nginx namespace, and each Nginx namespace is distributed with the identification information of the Nginx server.

[0083] Step 3: At the initial time T0, the coordinator obtains the total number of LBs currently created and further subdivides the information collection time interval to determine the information collection time ti (i.e., the first collection time) for each LB.

[0084] Step 4: Format each LB identifier and its information collection execution time ti into a list in key-value format to form an information collection task queue.

[0085] Step 5: Update the task queue to the etcd database.

[0086] Step 6: Each monitor and aggregator monitors the changes in the task queue in the etcd database and caches the latest task queue locally.

[0087] Step 7: Based on the task queue LB order, each monitor determines whether the Nginx namespace of LBi is on the current Nginx server; if yes, proceed to step 8; otherwise, continue to step 7.

[0088] Step 8: At time ti+tj (i.e., the second collection time), each monitor obtains the corresponding traffic forwarding volume from the Nginx namespace of LBi.

[0089] Specifically, to ensure that all monitors execute monitoring information collection commands at the same time, each monitor extends the collection time ti in the task queue by a time tj (which is configurable). This eliminates inconsistencies in the time spent by all monitors on monitoring changes in the task queue in the etcd database and caching the task queue information from the etcd database to their local machines. This ensures that each monitor obtains the corresponding traffic forwarding volume from the Nginx namespace of LBi at time ti+tj (i.e., the second collection time), guaranteeing strong consistency in the time of information collection.

[0090] Step 9: After the LBi information collection is completed, each controller updates the LBi identification information, collection time ti+tj, and the collected traffic forwarding volume to the etcd database.

[0091] Step 10: At time ti+tj+tk (i.e., information acquisition time), the aggregator retrieves the traffic forwarding volume of all Nginx namespaces belonging to LBi from the etcd database and aggregates it.

[0092] Specifically, in order to accurately obtain the traffic forwarding volume reported by all monitors corresponding to LBi, the aggregator extends the time by tk (which is configurable) after time ti+tj to eliminate the time consumed by each monitor in collecting information and the time consumed in reporting information to the etcd database. This ensures that all monitors have reported the collected information. That is, at time ti+tj+tk, the aggregator obtains the traffic forwarding volume of all Nginx namespaces belonging to LBi from the etcd database and aggregates it to obtain the accurate traffic forwarding volume of LBi at time ti+tj.

[0093] Step 11: The aggregator updates the aggregated results of LBi and uploads them to the monitoring center for users to query and view.

[0094] At this point, the monitoring information collection for LBi is complete, and the task of collecting monitoring information for the next LBi in the task queue will continue.

[0095] To implement the method of the embodiments of this application, based on the same inventive concept, an information collection device is also provided in the embodiments of this application. Figure 9This is a schematic diagram of the structure of the information collection device, which is applied to an Nginx server. Multiple Nginx servers are deployed in a distributed manner in the Nginx cluster, and at least one load balancer (LB) Nginx namespace is deployed on each Nginx server.

[0096] like Figure 9 As shown, the information acquisition device 90 includes:

[0097] The acquisition unit 901 is used to acquire all LB configuration information in the Nginx cluster from the etcd database when it detects that there is data update in the etcd database, and to create a task queue for periodically collecting all LB forwarding volume.

[0098] The acquisition unit 901 is used to, based on all LB configuration information, determine the Nginx namespace on the current Nginx server where the LBi in the task queue is deployed, and then obtain the first collection time of the LBi in the current period from the task queue; i is a positive integer;

[0099] The acquisition unit 902 is used to acquire the corresponding traffic forwarding volume from the Nginx namespace of the LBi based on the first acquisition time.

[0100] By adopting the above technical solution, a distributed and decentralized approach is used to collect LB traffic forwarding data. This avoids situations where the central node cannot remotely log in to an unspecified number of Nginx servers during centralized monitoring data collection, resulting in remote login failures, disconnections, timeouts, etc., thus ensuring the integrity and accuracy of the collected traffic forwarding data.

[0101] In some embodiments, the acquisition unit 902 is specifically used to acquire a first preset duration for delaying the execution of an information acquisition command; determine a second acquisition time based on the first acquisition time and the first preset duration; and acquire the traffic forwarding volume of the LBi during the second acquisition time from the Nginx namespace of the LBi.

[0102] In some embodiments, the identification information of the LBi, the second collection time, and the traffic forwarding volume are uploaded to the etcd database for storage, so that the controller can obtain the traffic forwarding volume from the etcd database and display it.

[0103] In some embodiments, the information acquisition time of the controller is the time after a second preset delay on the second acquisition time.

[0104] In some embodiments, the LB configuration information includes at least one Nginx namespace and the identification information of the Nginx server to which each Nginx namespace belongs.

[0105] In some embodiments, based on the identification information of the LBi in the task queue, the configuration information of the LBi is obtained from all LBi configuration information; based on the identification information of the Nginx server to which each Nginx namespace belongs in the configuration information of the LBi, it is determined whether the current Nginx server is included among all the Nginx servers to which the Nginx namespaces of the LBi belong; if yes, it is determined that the Nginx namespace of the LBi in the task queue is deployed on the current Nginx server; if no, it is determined that the Nginx namespace of the LBi in the task queue is not deployed on the current Nginx server.

[0106] In some embodiments, after the current periodic data collection task ends, if the total traffic forwarding volume of the LBi is greater than a first forwarding volume threshold, it is determined to start a new Nginx server and deploy the Nginx namespace of the LB, and update the LB configuration information and the task queue in the etcd database; or, after the current periodic data collection task ends, if the total traffic forwarding volume of the LBi is less than a second forwarding volume threshold, it is determined to delete redundant Nginx servers and delete the Nginx namespace of the LB, and update the LB configuration information and the task queue in the etcd database; wherein, the first forwarding volume threshold is greater than the second forwarding volume threshold.

[0107] This application also provides another information acquisition device. Figure 10 This is a schematic diagram of the structure of the information collection device in the embodiments of this application, such as... Figure 10 As shown, the information acquisition device includes: a processor 1001 and a memory 1002 configured to store computer programs capable of running on the processor;

[0108] When the processor 1001 is configured to run a computer program, it executes the method steps described in the foregoing embodiments.

[0109] Of course, in practical applications, such as Figure 10 As shown, the various components in this information acquisition device are coupled together via a bus system 1003. It can be understood that the bus system 1003 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 1003 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 10 The general labeled all buses as Bus System 1003.

[0110] In practical applications, the aforementioned processor can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), controller, microcontroller, and microprocessor. It is understood that, for different devices, the electronic devices used to implement the functions of the aforementioned processor can also be other types, and the embodiments of this application do not specifically limit this.

[0111] The aforementioned memory can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provides instructions and data to the processor.

[0112] In an exemplary embodiment, this application also provides a computer-readable storage medium for storing a computer program.

[0113] Optionally, the computer-readable storage medium can be applied to any of the methods in the embodiments of this application, and the computer program causes the computer to execute the corresponding processes implemented by the processor in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0115] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0116] Furthermore, in the various embodiments of the present invention, all functional units can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0118] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0119] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0120] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An information collection method, applied to an Nginx server, characterized in that, In an Nginx cluster, multiple Nginx servers are deployed in a distributed manner, and each Nginx server deploys at least one Nginx namespace of a load balancer (LB); the method includes: When data updates are detected in the etcd database, the configuration information of all load balancers (LBs) in the Nginx cluster is obtained from the etcd database, and a task queue is created to perform periodic collection tasks on all LB forwarding volumes. Based on all LB configuration information, when determining the Nginx namespace on the current Nginx server where the LBi in the task queue is deployed, the first collection time of the LBi in the current period is obtained from the task queue; i is a positive integer; Based on the first collection time, the corresponding traffic forwarding volume is collected from the Nginx namespace of the LBi.

2. The method according to claim 1, characterized in that, The step of collecting the corresponding traffic forwarding volume from the Nginx namespace of the LBi based on the first collection time includes: Obtain the first preset duration for delaying the execution of the information collection command; The second collection time is determined based on the first collection time and the first preset duration; Collect the traffic forwarding volume of LBi at the second collection time from the Nginx namespace of LBi.

3. The method according to claim 2, characterized in that, The method further includes: The LBi identification information, the second collection time, and the traffic forwarding volume are uploaded to the etcd database for storage, so that the controller can obtain the traffic forwarding volume from the etcd database and display it.

4. The method according to claim 3, characterized in that, The information acquisition time of the controller is the time after a second preset delay from the second acquisition time.

5. The method according to claim 1, characterized in that, The LB configuration information includes at least one Nginx namespace of the LB, and the identification information of the Nginx server to which each Nginx namespace belongs.

6. The method according to claim 5, characterized in that, The method further includes: Based on the identification information of the LBi in the task queue, the configuration information of the LBi is obtained from all LBi configuration information; Based on the identification information of the Nginx server to which each Nginx namespace belongs in the configuration information of the LBi, determine whether the current Nginx server is included among all the Nginx servers to which all Nginx namespaces belong in the LBi. If so, determine the Nginx namespace of the LBi in the task queue deployed on the current Nginx server; If not, it is determined that the Nginx namespace of LBi in the task queue is not deployed on the current Nginx server.

7. The method according to claim 1, characterized in that, The method further includes: After the current periodic collection task is completed, if the total traffic forwarding volume of the LBi is greater than the first forwarding volume threshold, it is determined to start a new Nginx server and deploy the Nginx namespace of the LB, and update the LB configuration information and the task queue in the etcd database. or, After the current periodic data collection task is completed, if the total traffic forwarding volume of the LBi is less than the second forwarding volume threshold, it is determined to delete the redundant Nginx servers and delete the Nginx namespace of the LB, and update the LB configuration information and the task queue in the etcd database; wherein, the first forwarding volume threshold is greater than the second forwarding volume threshold.

8. An information collection device, applied to an Nginx server, characterized in that, The Nginx cluster comprises multiple distributed Nginx servers, each with at least one load balancer (LB) deployed in its Nginx namespace; the device includes: The acquisition unit is used to acquire all LB configuration information in the Nginx cluster from the etcd database when it detects that there is a data update in the etcd database, and to create a task queue for periodically collecting data on all LB forwarding volumes. The acquisition unit is used to determine the Nginx namespace on the current Nginx server where the task queue LBi is deployed based on all LB configuration information, and then obtain the first collection time of LBi in the current period from the task queue; i is a positive integer; The collection unit is used to collect the corresponding traffic forwarding volume from the Nginx namespace of the LBi based on the first collection time.

9. An information acquisition device, applied to an Nginx server, characterized in that, The information acquisition device includes: a processor and a memory configured to store computer programs capable of running on the processor. Wherein, when the processor is configured to run the computer program, it performs the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 7.

Images