Cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping

Abstract

The application belongs to the technical field of cloud computing resource monitoring, and discloses a cloud resource distribution and scale monitoring system and method based on dynamic sampling and mixed mapping, which comprises a lightweight HTTP response analysis stage, a dynamic resource scale estimation stage, an active mapping stage, a passive mapping stage, and a system for fusing and calibrating the data collected in each stage; the system combines dynamic data acquisition technology and fine mapping methods, can monitor the state of cloud resources in real time, and dynamically analyze and optimize according to multi-dimensional data. The system not only improves the accuracy and real-time performance of resource scale estimation, but also effectively responds to the dynamic changes of cloud resources, ensuring the efficiency and sustainability of resource management. This technical solution has wide application prospects in the field of cloud computing resource management and dynamic monitoring.

Description

Technical Field

[0001] This invention belongs to the field of cloud computing resource monitoring technology, and in particular relates to a cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping. Background Technology

[0002] With the rapid development of cloud computing and big data technologies, efficient management and dynamic monitoring of cloud resources have become particularly important. In modern cloud computing environments, resources are typically distributed in a decentralized and dynamically changing manner, which places higher demands on resource monitoring and optimization. Especially in cloud infrastructures that span multiple regions and data centers, how to obtain real-time information such as the resource status, load, traffic data, and geographical distribution of cloud nodes has become a major challenge in cloud resource management.

[0003] Currently, most traditional cloud resource monitoring systems rely on static data collection and analysis, which cannot effectively cope with the dynamic changes in cloud resources. As cloud services continue to expand, existing monitoring technologies have also revealed shortcomings in accuracy, real-time performance, and adaptability. Furthermore, existing network topology analysis and traffic estimation methods mostly focus on static topology analysis, lacking real-time awareness of the global distribution of cloud resources and service changes, and thus failing to comprehensively reflect the true usage of resources. In the current cloud computing environment, the dynamic changes and large-scale distribution of cloud resources make it difficult for traditional monitoring methods to achieve efficient and accurate resource management, especially in complex network environments with multiple regions and data centers, where the dynamic distribution and changes of resources are often difficult to grasp in real time. Summary of the Invention

[0004] The purpose of this invention is to provide a cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping to solve the above-mentioned technical problems.

[0005] To address the aforementioned technical problems, the specific technical solution of the cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping of the present invention is as follows:

[0006] A cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping includes a lightweight HTTP response parsing stage, a dynamic resource scale estimation stage, an active mapping stage, a passive mapping stage, and a system for fusing and calibrating the data collected in each stage.

[0007] The lightweight HTTP response parsing stage involves quickly obtaining key information about cloud resource nodes by sending lightweight HTTP requests, parsing the HTTP response headers, and extracting basic node data to provide foundational information for subsequent analysis.

[0008] The dynamic resource scale estimation stage: Based on the node information obtained in the first stage, the system evaluates each node through a multi-dimensional key indicator quantification method;

[0009] The active mapping phase involves: actively acquiring key performance data about the target nodes by directly measuring their network performance and topology; and actively mapping and collecting network performance data between nodes by initiating probes from multiple distributed measurement points.

[0010] The passive mapping phase: The passive mapping phase uses external data sources to supplement and verify the results of active mapping. This phase obtains the geographical location, network affiliation, and connection relationship information between nodes by parsing public databases, domain name information, and registration information.

[0011] The system integrates and calibrates the data collected at each stage by combining active and passive mapping data, eliminating redundancy and errors, and ensuring the accuracy and consistency of the data.

[0012] This invention also discloses a monitoring method for a cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping, comprising the following steps:

[0013] Step 1: Lightweight HTTP Response Parsing:

[0014] Step 2: Dynamic resource size estimation:

[0015] Step 3: Active surveying;

[0016] Step 4: Passive surveying.

[0017] Furthermore, step 1 includes the following specific steps:

[0018] Step 1.1: Receive the list of domain names provided by the user, construct the test set, and ensure that the data input covers a wide range of areas;

[0019] Step 1.2: Use lightweight HEAD requests to obtain response header information, reduce network resource consumption, and support client or server operation;

[0020] Step 1.3: Send HEAD requests to each target domain one by one, and record the response header information and latency performance data;

[0021] Step 1.4: Parse the response header fields, extract node identifiers and attributes, map information, and verify the database results to ensure data integrity;

[0022] Step 1.5: Clean and standardize the data. Call the Clean function to remove failed or timed-out nodes, deduplicate and merge multiple request messages to ensure field consistency.

[0023] The Clean function is used to clean and standardize node data. By traversing the response status and latency of each request, the function checks whether the request is successful and whether the response time is within the normal range. When the request meets the conditions, the system adds the valid nodes to the cleaned data list and removes failed, timed out, or duplicate nodes. Finally, the system returns a cleaned dataset containing all valid nodes, ensuring the integrity and consistency of the data. Step 1.6: Store the standardized data locally or in the cloud for subsequent calls, and output node identifiers, geographical locations, and service attribute information.

[0024] Furthermore, step 2 includes the following specific steps:

[0025] Step 2.1: Receive and verify the initial set of nodes to ensure data integrity;

[0026] Step 2.2: Calculate multi-dimensional metrics for each node, including topology importance T(v), traffic and load L(v), dynamic changes D(v), and abnormal behavior A(v);

[0027] Step 2.3: Calculate a comprehensive score based on the indicators to quantify the importance of each node.

[0028] k(v)=w1·T(v)+w2·L(v)+w3·D(v)+w4·A(v);

[0029] This function combines multiple indicators into a comprehensive score k(v) through a weighted summation method. Finally, the function calculates and returns the comprehensive score of the node based on each indicator and its weight.

[0030] Step 2.4: Analyze the dynamic change trend of the nodes;

[0031] Step 2.5: Output the estimation report, generating a list of key nodes and regional distribution statistics.

[0032] Furthermore, step 3 includes the following specific steps:

[0033] Step 3.1: When selecting nodes, prioritize nodes with a score k(v) > 0.6, high dynamism, high traffic share, or core topology locations;

[0034] Step 3.2: Measure the path delay of the target node through multiple measurement points, collect the delay data from the target node to nodes with different hop counts, thereby obtaining the network path delay, measuring the response time and packet loss rate of the target node, and evaluating its performance stability.

[0035] Step 3.3: Perform performance testing on each target node, measuring its response time and packet loss rate. Through multiple tests, analyze the stability, latency, and packet loss of the nodes, merge path and performance data, verify the consistency of node information, perform logical calibration, and verify the accuracy of hop count and latency data.

[0036] Step 3.4: By conducting parallel probing at multiple measurement points distributed in different geographical locations, obtain latency and packet loss data of the target node under different network environments;

[0037] The system measures path delay and performance of the target node. It also evaluates the network performance of the target node at different measurement points using measure_path_delay(node,point) and measure_performance(node,point). When the evaluation results meet the preset conditions, the system will automatically record the measurement data and save it to the database, thereby ensuring the accuracy of the node's performance data.

[0038] Step 3.5: Save all measurement data to the database and perform data cleaning to remove outliers and invalid data;

[0039] After completing the active mapping, the system enters the subsequent data analysis phase. By analyzing the performance data of each node, the system can identify potential problems or bottleneck nodes, and further optimize the network topology and resource allocation.

[0040] Furthermore, step 4 includes the following specific steps:

[0041] Step 4.1: Use DNS reverse lookup technology to query the domain name of the target node and obtain the node's basic geographic information and service type;

[0042] Step 4.2: Obtain the registration information of the target node, including Autonomous System (ASN), network service provider, and IP range, and verify whether the node is registered at a network exchange point or data center to further understand the node's position and role in the global network topology.

[0043] Step 4.3: Verify whether the node is registered at the network exchange point or data center to further understand the node's position and role in the global network topology. The system will merge the data from active and passive mapping to eliminate redundancy and errors, ensuring the accuracy and consistency of the final measurement results.

[0044] Step 4.4: Query the network connection information recorded in the database, supplement and verify the network topology relationship between the target nodes, and draw a more complete network topology diagram.

[0045] Passive mapping of target nodes is performed by querying external data sources to obtain relevant information about the nodes, including reverse DNS resolution, WHOIS query, and PeeringDB verification. The function first performs reverse DNS resolution, WHOIS query, and PeeringDB query for each target node to obtain the node's domain name, registration information, and network interconnection relationship from these external sources. The system integrates this information into a dictionary node_measurements to ensure that complete network topology and attribute data are collected for each node. Finally, the passive mapping results of all nodes are stored in the passive_measurement_results dictionary for subsequent analysis and network topology optimization.

[0046] Step 4.5: The system merges the data from active and passive surveying to eliminate redundancy and errors, ensuring the accuracy and consistency of the final measurement results.

[0047] The cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping of the present invention has the following advantages: This system and method combines dynamic data acquisition technology with refined mapping methods, enabling real-time monitoring of cloud resource status and dynamic analysis and optimization based on multi-dimensional data. This system not only improves the accuracy and real-time performance of resource scale estimation but also effectively addresses dynamic changes in cloud resources, ensuring efficient and sustainable resource management. This technical solution has broad application prospects in the field of cloud computing resource management and dynamic monitoring. Attached Figure Description

[0048] Figure 1 This is a model diagram of the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping according to the present invention;

[0049] Figure 2 This is a schematic diagram of the system operation process of the present invention;

[0050] Figure 3 This is a schematic diagram of the Clean pseudocode in the cleaning and standardization data of this invention;

[0051] Figure 4 This is a schematic diagram of the pseudocode for calculating the comprehensive score in this invention;

[0052] Figure 5 This is a pseudocode diagram illustrating the path delay and performance testing of the present invention;

[0053] Figure 6 This is a schematic diagram of the reverse parsing pseudocode of the present invention. Detailed Implementation

[0054] To better understand the purpose, structure, and function of this invention, the cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping will be described in further detail below with reference to the accompanying drawings.

[0055] Explanation of relevant knowledge:

[0056] HTTP Request and Response Parsing

[0057] This lightweight HTTP parser is designed to efficiently extract response header information from target servers using the HEAD request method, avoiding the transmission of the complete response body and thus significantly reducing network resource consumption. The HEAD method returns only the response header fields, enabling rapid acquisition of core server metadata, making it suitable for monitoring and analyzing large-scale network nodes. The parser supports high concurrency and asynchronous I / O technologies, allowing it to process a large number of domain name requests in a short time, improving data collection efficiency. Response header parsing is one of the core functions of this lightweight HTTP parser. By extracting key fields from the response header, the parser can identify server node identification information, geographical distribution, and service attributes. This information includes node identifiers, network paths, and node service types, providing a precise data foundation for network performance analysis. Furthermore, the parser can record performance metrics such as request response time and network latency for dynamic performance optimization and trend analysis. By integrating request scheduling and parsing functions, this lightweight HTTP parser demonstrates excellent performance and reliability in resource-constrained environments.

[0058] Data cleaning and standardization

[0059] Data cleaning and standardization are indispensable steps in network data processing, directly impacting the accuracy and efficiency of subsequent analysis. Data cleaning aims to remove invalid or abnormal data records, such as nodes that have timed out or failed to respond, ensuring the reliability and integrity of the final data. The parser filters captured node data using predefined rules, eliminating redundant and duplicate records and retaining only uniquely identified nodes and their related information, thus avoiding redundant storage and processing. Data standardization aims to unify the structure and fields of the data, ensuring consistency during storage and retrieval. By uniformly encoding and formatting fields such as node identifiers, geographic locations, and service attributes, the parser generates standardized, high-quality datasets. These datasets not only support rapid storage and retrieval but also provide standardized input for subsequent dynamic analysis, trend monitoring, and model training. With the dual guarantees of data cleaning and standardization, the parser can efficiently process complex network data and provide reliable data support for various application scenarios.

[0060] Analysis of CDN Node Geographic Distribution

[0061] The geographical distribution of CDN nodes is a crucial foundation for optimizing content delivery network performance. By deploying distributed edge servers globally, CDNs can route user requests to the nearest nodes, reducing network latency and improving data delivery efficiency. The geographical location of CDN nodes is typically represented by identifiers in response headers. These identifiers, combined with geodatabase mappings, provide precise support for regional network performance optimization. The geographical distribution of nodes directly impacts content delivery efficiency and user experience. By extracting node distribution information, the resolver can identify critical areas and high-load nodes in the network and dynamically adjust them. Node attribute analysis further reveals the role of nodes in the overall network topology, providing data support for resource scheduling and traffic optimization. Geographic distribution resolution not only supports performance monitoring but can also be used for topology modeling, building a more efficient and stable network architecture.

[0062] Network topology and graph theory

[0063] Network topology, described by a graph G(V,E), depicts the relationship between nodes V and connections E, and is an important tool for studying computer networks and distributed systems. In graph theory, degree centrality (DC(v)) measures the number of direct connections to a node, reflecting its local importance; betweenness centrality (BC(v)) measures a node's role as a bridge, reflecting its global influence. Furthermore, dynamic topology focuses on the addition, loss, and migration of nodes, which significantly impact network performance and resource distribution. In dynamic resource estimation, network topology, combined with attributes such as latency and bandwidth, is modeled using weighted graphs, providing a mathematical foundation for multi-dimensional index quantification and comprehensive scoring calculations, and serving as crucial theoretical support for system analysis.

[0064] Time series models

[0065] Time series models are a method for analyzing and predicting data that changes over time, playing a crucial role in dynamic resource sizing. These models predict future patterns of change by mining trends, periodicity, and randomness in historical data. Commonly used models include autoregressive models (AR), moving average models (MA), and autoregressive integral moving average models (ARIMA), with ARIMA being suitable for handling non-stationary time series. Furthermore, deep learning models such as Long Short-Term Memory (LSTM) networks excel at capturing complex dependencies and long-term trends. In dynamic resource sizing, time series models are used to monitor the dynamic changes in node additions, losses, and migrations, analyze traffic trends, and predict future resource demands, providing accurate support for cloud resource management and network optimization.

[0066] Cloud resource nodes

[0067] Cloud resource nodes are fundamental building blocks in a cloud computing environment, responsible for providing computing, storage, and networking services. Depending on their function, cloud resource nodes can include edge nodes, cache nodes, and database nodes. Each node offers different service types and performance metrics. By monitoring information such as the node's geographical location, load, and bandwidth, administrators can efficiently manage and optimize the use of cloud resources.

[0068] Cloud resource nodes are a key component of a cloud computing environment, representing concrete entities that provide various cloud services. Each cloud resource node can provide different types of computing, storage, networking, and other services. Edge nodes are typically located at the edge of the network, close to users or devices, to reduce latency and provide fast response services, especially suitable for applications such as the Internet of Things (IoT), video streaming, and real-time data processing. Cache nodes are used to store frequently used data to reduce the transmission and access latency of duplicate data, and are typically used in content delivery networks (CDNs) to ensure more efficient data access speeds. Database nodes are mainly used to store and manage data, handling large-scale data requests and operations. Monitoring cloud resource nodes covers multiple metrics, including node geolocation, service type, traffic load, and bandwidth utilization. By monitoring these metrics in real time, administrators can optimize resource configuration and allocation, ensuring system stability, performance, and reliability.

[0069] like Figure 1 As shown, the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping of the present invention includes...

[0070] Lightweight HTTP Response Parsing Phase: In this phase, the system quickly obtains key information about cloud resource nodes by sending lightweight HTTP requests (such as HEAD requests). The HTTP response headers are parsed to extract basic data such as node identifiers, geographical location, and service type, providing foundational information for subsequent analysis. This phase primarily addresses the initial collection and cleaning of cloud resource node information, ensuring data accuracy and completeness.

[0071] Dynamic Resource Scale Estimation Phase: Based on the node information acquired in the first phase, the system evaluates each node using multi-dimensional key indicator quantification methods (such as network topology importance, traffic load, dynamic changes, and abnormal behavior). This phase aims to estimate the scale and load of cloud resource nodes and generate a dynamic trend analysis report. Through this phase, the system can accurately estimate the overall scale of cloud resources and their real-time changes, providing resource scheduling and optimization suggestions.

[0072] Active Mapping Phase: This phase primarily involves directly measuring the network performance and topology of target nodes to proactively acquire key performance data about them. By initiating probes from multiple distributed measurement points, active mapping collects network performance data such as path latency, response time, and packet loss rate between nodes. This phase also involves initiating measurement requests to perform real-time testing of cloud resource nodes to understand their performance stability, reachability, and behavior under different network environments.

[0073] Passive mapping phase: The passive mapping phase supplements and verifies the results of active mapping by using external data sources. This phase does not require directly initiating probe requests to nodes; instead, it obtains information such as node geolocation, network affiliation, and inter-node connectivity by parsing public databases, domain information, and registration information. The purpose of passive mapping is to fill data gaps that active mapping cannot cover, especially for node characteristics that cannot be directly measured (such as service provider information, autonomous systems, etc.).

[0074] The system integrates and calibrates the data collected at each stage. By combining active and passive mapping data, redundancy and errors are eliminated, ensuring data accuracy and consistency.

[0075] like Figure 2 As shown, the cloud resource distribution and scale monitoring method based on dynamic sampling and hybrid mapping of the present invention includes the following steps:

[0076] Step 1: Lightweight HTTP Response Parsing:

[0077] Suppose the system under test requires standardization and cleaning of a set of input data. To ensure the integrity and reliability of the input data, the system will perform the following steps sequentially upon receiving the input:

[0078] Step 1.1: Receive the list of domain names provided by the user, construct the test set, and ensure that the data input is comprehensive.

[0079] Step 1.2: Use a lightweight HEAD request to obtain response header information, reduce network resource consumption, and support operation on the client or server side.

[0080] Step 1.3: Send HEAD requests to each target domain one by one, and record the response header information and performance data such as latency.

[0081] Step 1.4: Parse the response header fields, extract node identifiers and attributes, map information, and verify the database results to ensure data integrity.

[0082] Step 1.5: Clean and standardize the data. Call the Clean function to remove failed or timed-out nodes, deduplicate and merge multiple request messages to ensure field consistency. The pseudocode for the Clean function is as follows: Figure 3 As shown.

[0083] This function is primarily used to clean and standardize node data. By iterating through the response status and latency of each request, the function checks whether the request was successful and whether the response time is within the normal range. When a request meets the criteria, the system adds the valid nodes to the cleaned data list and removes failed, timed-out, or duplicate nodes. Finally, the system returns a cleaned dataset containing all valid nodes, ensuring data integrity and consistency, and providing reliable foundational data for subsequent network analysis and resource management.

[0084] Step 1.6: Store standardized data locally or in the cloud for later use, and output information including node identifier, geographical location, and service attributes.

[0085] Step 2: Dynamic resource size estimation:

[0086] Suppose the system needs to perform dynamic resource size estimation on an initial set of nodes. The node set contains basic information for each node, including an identifier (node_id), IP address (ip_address), geographic location, and service type (service_type, such as edge node or cache node). After receiving the nodes, the system will process them sequentially according to the following steps:

[0087] Step 2.1: Receive and verify the initial set of nodes to ensure data integrity;

[0088] Step 2.2: Calculate multi-dimensional metrics for each node, including topology importance T(v), traffic and load L(v), dynamic changes D(v), and abnormal behavior A(v);

[0089] Step 2.3: Calculate a comprehensive score based on the indicators to quantify the importance of each node.

[0090] k(v)=w1·T(v)+w2·L(v)+w3·D(v)+w4·A(v);

[0091] like Figure 4As shown, this function combines multiple metrics (such as topology importance, traffic and load, dynamic changes, and abnormal behavior) into a comprehensive score k(v) using a weighted summation method. This score helps the system determine the priority and importance of each node in resource scheduling. In the score calculation, the weights parameter represents the importance weight of each metric, while util is the node's resource utilization (such as bandwidth utilization), and T, D, and A are the scores for topology importance, dynamic changes, and abnormal behavior, respectively. Finally, the function calculates and returns the comprehensive score of the node based on each metric and its weight.

[0092] Step 2.4: Analyze the dynamic changes of nodes, including changes in number, additions, and loss of nodes;

[0093] Step 2.5: Output the estimation report, generating a list of key nodes and regional distribution statistics;

[0094] Through the above steps, the system comprehensively analyzed the size and dynamic characteristics of the node set, and the generated estimation report identified key nodes and dynamic regions. The next stage will use these results to further map and optimize the key nodes and regions, providing more accurate data support for network-wide resource management.

[0095] Step 3: Active Surveying

[0096] After completing the initial data collection and scale estimation of cloud resource nodes, further analysis is conducted on the network performance, geographical distribution, and dynamic changes of the nodes. The steps are as follows:

[0097] Step 3.1: When selecting nodes, prioritize nodes with a score k(v) > 0.6, high dynamism, high traffic share, or core topology locations;

[0098] Step 3.2: Measure the path delay of the target node at multiple measurement points, collecting delay data from the target node to nodes with different hop counts to obtain the network path latency. Measure the response time and packet loss rate of the target node to evaluate its performance stability.

[0099] Step 3.3: Perform performance tests on each target node, measuring its response time, packet loss rate, and other metrics. Through multiple tests, analyze the node's stability, latency, and packet loss to provide data support for evaluating network performance. Merge path and performance data to verify the consistency of node information. Perform logical calibration to verify the accuracy of hop count and latency data.

[0100] Step 3.4: By conducting parallel probing at multiple measurement points distributed in different geographical locations, obtain latency and packet loss data of the target node under different network environments;

[0101] like Figure 5As shown, this function is primarily used to measure path delay and performance of a target node. It evaluates the network performance of the target node at different measurement points using `measure_path_delay(node,point)` and `measure_performance(node,point)`. Once the evaluation results meet preset conditions, the system automatically records the measurement data and saves it to the database, ensuring the accuracy of the node's performance data and providing support for subsequent resource optimization and network topology construction.

[0102] Step 3.5: Save all measurement data to the database and perform data cleaning to remove outliers and invalid data, ensuring the accuracy and reliability of the final data.

[0103] After completing the active mapping, the system moves into the subsequent data analysis phase. By analyzing the performance data of each node, the system can identify potential problem or bottleneck nodes, further optimizing the network topology and resource allocation.

[0104] Step 4: Passive Surveying

[0105] The passive mapping phase verifies and supplements the information collected in the active mapping phase by using external data sources, especially for node features that cannot be covered or directly measured in the active mapping phase.

[0106] Step 4.1: Use DNS reverse lookup technology to query the domain name of the target node and obtain the node's basic geographic information and service type.

[0107] Step 4.2: Obtain the target node's registration information, including Autonomous System (ASN), network service provider, IP range, etc. This information helps determine the node's network affiliation and functional characteristics. Verify whether the node is registered at a network exchange point or data center to further understand the node's location and role in the global network topology.

[0108] Step 4.3: Verify whether the node is registered at the network exchange point or data center to further understand the node's location and role in the global network topology. The system will fuse actively and passively mapped data to eliminate redundancy and errors, ensuring the accuracy and consistency of the final measurement results.

[0109] Step 4.4: Query the network connection information recorded in the database, supplement and verify the network topology relationship between the target nodes, and draw a more complete network topology diagram.

[0110] like Figure 6As shown, this function performs passive mapping of target nodes by querying external data sources to obtain relevant information about the nodes, including reverse DNS lookup, WHOIS query, and PeeringDB verification. Specifically, the function first performs reverse DNS lookup, WHOIS query, and PeeringDB query for each target node to obtain the node's domain name, registration information, and network interconnection relationships from these external sources. The system integrates this information into a dictionary called `node_measurements` to ensure that complete network topology and attribute data are collected for each node. Finally, the passive mapping results of all nodes are stored in the `passive_measurement_results` dictionary for subsequent analysis and network topology optimization.

[0111] Step 4.5: The system merges the data from active and passive surveying to eliminate redundancy and errors, ensuring the accuracy and consistency of the final measurement results.

[0112] Combining passive mapping results with active mapping data provides a more comprehensive and accurate network topology map, as well as the relationships and interactions between nodes. Ultimately, by fusing the data from both sources, the system can generate more precise cloud resource monitoring and network topology analysis reports, providing strong support for resource optimization and troubleshooting.

[0113] In summary, the present invention has the following characteristics:

[0114] Through dynamic sampling and real-time data monitoring, the system can accurately acquire and analyze changes and load conditions of cloud resource nodes.

[0115] By combining active and passive mapping methods, we can comprehensively collect the geographical location, performance, and topology information of cloud resource nodes, thereby improving mapping accuracy.

[0116] The status and scale of cloud resource nodes are comprehensively evaluated by quantifying multiple dimensions such as network topology, traffic load, dynamic changes and abnormal behavior.

[0117] Based on multi-dimensional real-time monitoring and analysis, it provides accurate cloud resource scale estimation and dynamic change trends.

[0118] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping, characterized in that, It includes a lightweight HTTP response parsing phase, a dynamic resource size estimation phase, an active mapping phase, a passive mapping phase, and the system's fusion and calibration of data collected in each phase; The lightweight HTTP response parsing stage involves quickly obtaining key information about cloud resource nodes by sending lightweight HTTP requests, parsing the HTTP response headers, and extracting basic node data to provide foundational information for subsequent analysis. The dynamic resource scale estimation stage: Based on the node information obtained in the first stage, the system evaluates each node through a multi-dimensional key indicator quantification method; The active mapping phase involves: actively acquiring key performance data about the target nodes by directly measuring their network performance and topology; and actively mapping and collecting network performance data between nodes by initiating probes from multiple distributed measurement points. The passive mapping phase: The passive mapping phase uses external data sources to supplement and verify the results of active mapping. This phase obtains the geographical location, network affiliation, and connection relationship information between nodes by parsing public databases, domain name information, and registration information. The system integrates and calibrates the data collected at each stage by combining active and passive mapping data, eliminating redundancy and errors, and ensuring the accuracy and consistency of the data.

2. A monitoring method for a cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping as described in claim 1, characterized in that, Includes the following steps: Step 1: Lightweight HTTP Response Parsing: Step 2: Dynamic resource size estimation: Step 3: Active surveying; Step 4: Passive surveying.

3. The monitoring method of the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping according to claim 2, characterized in that, Step 1 includes the following specific steps: Step 1.1: Receive the list of domain names provided by the user, construct the test set, and ensure that the data input covers a wide range of areas; Step 1.2: Use lightweight HEAD requests to obtain response header information, reduce network resource consumption, and support client or server operation; Step 1.3: Send HEAD requests to each target domain one by one, and record the response header information and latency performance data; Step 1.4: Parse the response header fields, extract node identifiers and attributes, map information, and verify the database results to ensure data integrity; Step 1.5: Clean and standardize the data. Call the Clean function to remove failed or timed-out nodes, deduplicate and merge multiple request messages to ensure field consistency. The Clean function is used to clean and standardize node data. By traversing the response status and latency of each request, the function checks whether the request is successful and whether the response time is within the normal range. When the request meets the conditions, the system adds the valid nodes to the cleaned data list and removes failed, timed out, or duplicate nodes. Finally, the system returns a cleaned dataset containing all valid nodes, ensuring the integrity and consistency of the data. Step 1.6: Store the standardized data locally or in the cloud for subsequent calls, and output node identifiers, geographical locations, and service attribute information.

4. The monitoring method of the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping according to claim 2, characterized in that, Step 2 includes the following specific steps: Step 2.1: Receive and verify the initial set of nodes to ensure data integrity; Step 2.2: Calculate multi-dimensional metrics for each node, including topology importance T(v), traffic and load L(v), dynamic changes D(v), and abnormal behavior A(v); Step 2.3: Calculate a comprehensive score based on the indicators to quantify the importance of each node. k(v)=w1·T(v)+w2·L(v)+w3·D(v)+w4·A(v); This function combines multiple indicators into a comprehensive score k(v) through a weighted summation method. Finally, the function calculates and returns the comprehensive score of the node based on each indicator and its weight. Step 2.4: Analyze the dynamic change trend of the nodes; Step 2.5: Output the estimation report, generating a list of key nodes and regional distribution statistics.

5. The monitoring method of the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping according to claim 4, characterized in that, Step 3 includes the following specific steps: Step 3.1: When selecting nodes, prioritize nodes with a score k(v) > 0.6, high dynamism, high traffic share, or core topology locations; Step 3.2: Measure the path delay of the target node through multiple measurement points, collect the delay data from the target node to nodes with different hop counts, thereby obtaining the network path delay, measuring the response time and packet loss rate of the target node, and evaluating its performance stability. Step 3.3: Perform performance testing on each target node, measuring its response time and packet loss rate. Through multiple tests, analyze the stability, latency, and packet loss of the nodes, merge path and performance data, verify the consistency of node information, perform logical calibration, and verify the accuracy of hop count and latency data. Step 3.4: By conducting parallel probing at multiple measurement points distributed in different geographical locations, obtain latency and packet loss data of the target node under different network environments; The system measures path delay and performance of the target node. It also evaluates the network performance of the target node at different measurement points using measure_path_delay(node,point) and measure_performance(node,point). When the evaluation results meet the preset conditions, the system will automatically record the measurement data and save it to the database, thereby ensuring the accuracy of the node's performance data. Step 3.5: Save all measurement data to the database and perform data cleaning to remove outliers and invalid data; After completing the active mapping, the system enters the subsequent data analysis phase. By analyzing the performance data of each node, the system can identify potential problems or bottleneck nodes, and further optimize the network topology and resource allocation.

6. The monitoring method of the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping according to claim 2, characterized in that, Step 4 includes the following specific steps: Step 4.1: Use DNS reverse lookup technology to query the domain name of the target node and obtain the node's basic geographic information and service type; Step 4.2: Obtain the registration information of the target node, including Autonomous System (ASN), network service provider, and IP range, and verify whether the node is registered at a network exchange point or data center to further understand the node's position and role in the global network topology. Step 4.3: Verify whether the node is registered at the network exchange point or data center to further understand the node's position and role in the global network topology. The system will merge the data from active and passive mapping to eliminate redundancy and errors, ensuring the accuracy and consistency of the final measurement results. Step 4.4: Query the network connection information recorded in the database, supplement and verify the network topology relationship between the target nodes, and draw a more complete network topology diagram; Passive mapping of target nodes is performed by querying external data sources to obtain relevant information about the nodes, including reverse DNS resolution, WHOIS query, and PeeringDB verification. The function first performs reverse DNS resolution, WHOIS query, and PeeringDB query for each target node to obtain the node's domain name, registration information, and network interconnection relationship from these external sources. The system integrates this information into a dictionary called node_measurements to ensure that complete network topology and attribute data are collected for each node. Finally, the passive mapping results of all nodes are stored in the passive_measurement_results dictionary for subsequent analysis and network topology optimization. Step 4.5: The system merges the data from active and passive mapping to eliminate redundancy and errors, ensuring the accuracy and consistency of the final measurement results.

Images