Cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid surveying and mapping

Abstract

The invention belongs to the technical field of cloud computing resource monitoring, and discloses a cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid surveying and mapping. Comprising a lightweight HTTP response analysis stage, a dynamic resource scale estimation stage, an active surveying and mapping stage, a passive surveying and mapping stage and a system for fusing and calibrating data collected in each stage. According to the system, a dynamic data acquisition technology and a refined surveying and mapping method are combined, the state of cloud resources can be monitored in real time, and dynamic analysis and optimization can be carried out according to multi-dimensional data. The system not only improves the precision and real-time performance of resource scale estimation, but also can effectively cope with the dynamic change of cloud resources, and ensures the high efficiency and sustainability of resource management. The technical scheme has a wide application prospect in the fields of cloud computing resource management and dynamic monitoring.

Description

Technical Field

[0001] The present invention belongs to the technical field of cloud computing resource monitoring, and in particular relates to a cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping. Background Art

[0002] With the rapid development of cloud computing and big data technologies, efficient management and dynamic monitoring of cloud resources have become particularly important. In modern cloud computing environments, the distribution of resources is usually dispersed and dynamically changing, which puts higher requirements on resource monitoring and optimization. Especially in cross-regional and cross-data center cloud infrastructures, how to obtain cloud node resource status, load conditions, traffic data, and geographical distribution information in real time has become a major challenge in cloud resource management.

[0003] At present, most traditional cloud resource monitoring systems rely on static data collection and analysis, and cannot effectively cope with the dynamic changes of cloud resources. With the continuous expansion of cloud service scale, existing monitoring technologies have also exposed deficiencies in accuracy, real-time and adaptability. In addition, existing network topology analysis and traffic estimation methods mostly focus on static topology analysis, lack the ability to perceive the global distribution of cloud resources and service changes in real time, and cannot fully reflect the actual use of resources. In the current cloud computing environment, the dynamic changes and large-scale distribution of cloud resources make it difficult for traditional monitoring methods to achieve efficient and accurate resource management, especially in complex network environments with multiple regions and multiple data centers, where the dynamic distribution and changes of resources are often difficult to grasp in real time. Summary of the invention

[0004] The purpose of the present invention is to provide a cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping to solve the above-mentioned technical problems.

[0005] In order to solve the above technical problems, the specific technical solutions of the cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping of the present invention are as follows:

[0006] A cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping, including a lightweight HTTP response parsing stage, a dynamic resource scale estimation stage, an active mapping stage, a passive mapping stage, and the system integrates and calibrates the data collected in each stage;

[0007] The lightweight HTTP response parsing stage: quickly obtain key information of cloud resource nodes by sending lightweight HTTP requests, parse HTTP response headers, extract basic data of nodes, and provide basic information for subsequent analysis;

[0008] The dynamic resource scale estimation stage: based on the node information obtained in the first stage, the system evaluates each node through a multi-dimensional key indicator quantification method;

[0009] The active mapping stage: by directly measuring the network performance and topology of the target node, actively obtaining key performance data about the node, and by initiating detection from multiple distributed measurement points, actively mapping and collecting network performance data between nodes;

[0010] Passive mapping stage: The passive mapping stage uses external data sources to supplement and verify the results of active mapping. This stage obtains the geographical location, network affiliation and connection relationship information of nodes by parsing public databases, domain name information and registration information;

[0011] The system fuses and calibrates the data collected at each stage by combining the active and passive surveying data to eliminate redundancy and erroneous information and ensure the accuracy and consistency of the data.

[0012] The present invention also discloses a monitoring method of a cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping, comprising the following steps:

[0013] Step 1: Lightweight HTTP response parsing:

[0014] Step 2: Dynamic resource sizing:

[0015] Step 3: Active mapping;

[0016] Step 4: Passive mapping.

[0017] Furthermore, the step 1 includes the following specific steps:

[0018] Step 1.1: Receive the domain name list provided by the user and build a test set to ensure that a wide range of data inputs are covered;

[0019] Step 1.2: Use lightweight HEAD request to obtain response header information, reduce network resource consumption, and support client or server operation;

[0020] Step 1.3: Initiate HEAD requests to the target domain names one by one, and record the response header information and latency performance data;

[0021] Step 1.4: Parse the response header field, extract the node identifier and attributes, map the information and verify the database results to ensure data integrity;

[0022] Step 1.5: Clean and standardize data, call the Clean function, remove failed or timed-out nodes, merge multiple request information after deduplication, and ensure field consistency;

[0023] The Clean function is used to clean and standardize node data. By traversing the response status and delay of each request, the function checks whether the request is successful and whether the response time is within the normal range. When the request meets the conditions, the system adds the valid node to the cleaned data list and removes the failed, timed out or duplicated nodes. Finally, the system returns a cleaned data set containing all valid nodes to ensure data integrity and consistency; Step 1.6: Store the standardized data locally or in the cloud for subsequent calls, and the output includes node identification, geographic location and service attribute information.

[0024] Furthermore, the step 2 includes the following specific steps:

[0025] Step 2.1: Receive and verify the initial node set nodes to ensure data integrity;

[0026] Step 2.2: Calculate multi-dimensional indicators for each node, including topological importance T(v), traffic and load L(v), dynamic change D(v), and abnormal behavior A(v);

[0027] Step 2.3: Calculate the comprehensive score based on the indicators to quantify the importance of each node.

[0028] k(v)=w1·T(v)+w2·L(v)+w3·D(v)+w4·A(v);

[0029] This function combines multiple indicators into a comprehensive score k(v) by weighted summation. Finally, the function calculates and returns the comprehensive score of the node based on each indicator and its weight.

[0030] Step 2.4: Analyze the dynamic change trend of nodes;

[0031] Step 2.5: Output the estimation report, generate a list of key nodes and regional distribution statistics.

[0032] Furthermore, the step 3 includes the following specific steps:

[0033] Step 3.1: When selecting nodes, give priority to nodes with a score k(v)>0.6, high dynamics, high traffic share, or topological core locations;

[0034] Step 3.2: Measure the path delay of the target node through multiple measurement points, collect the delay data from the target node to nodes with different hop counts, thereby obtaining the network path delay, measuring the response time and packet loss rate of the target node, and evaluating its performance stability;

[0035] Step 3.3: Perform performance tests on each target node, measure its response time and packet loss rate indicators, analyze the stability, delay and packet loss of the node through multiple tests, merge the path and performance data, verify the consistency of node information, perform logic calibration, and verify the accuracy of hop count and delay data;

[0036] Step 3.4: Perform parallel detection at multiple measurement points distributed in different geographical locations to obtain the delay and packet loss data of the target node under different network environments;

[0037] Measure the path delay and performance of the target node, and evaluate the network performance of the target node at different measurement points through measure_path_delay(node,point) and measure_performance(node,point). When the evaluation result meets the preset conditions, the system will automatically record the measurement data and save it to the database to ensure the accuracy of the node performance data;

[0038] Step 3.5: Save all measurement data into the database and perform data cleaning to remove outliers and invalid data;

[0039] After completing active mapping, the subsequent data analysis phase begins. By analyzing the performance data of each node, the system can identify possible problems or bottleneck nodes and further optimize the network topology and resource allocation.

[0040] Furthermore, the step 4 includes the following specific steps:

[0041] Step 4.1: Use DNS reverse resolution technology to query the domain name of the target node and obtain the basic geographic information and service type of the node;

[0042] Step 4.2: Obtain the registration information of the target node, including the autonomous system ASN, network service provider, and IP segment, verify whether the node is registered at the network exchange point or data center, and further understand the location and role of the node in the global network topology;

[0043] Step 4.3: Verify whether the node is registered at the network exchange point or data center to further understand the location and role of the node in the global network topology. The system will fuse the data of active and passive mapping to eliminate redundancy and erroneous information to ensure the accuracy and consistency of the final measurement results.

[0044] Step 4.4: Query the network connection information recorded in the database, supplement and verify the network topology relationship between the target nodes, and draw a more complete network topology map.

[0045] Passively map the target node and obtain relevant information of the node by querying external data sources, including reverse DNS resolution, WHOIS query and PeeringDB verification. The function first performs reverse DNS resolution, WHOIS query and PeeringDB query for each target node, and obtains the node's domain name, registration information and network interconnection relationship from these external sources. The system integrates this information into a dictionary node_measurements to ensure that complete network topology and attribute data are collected for each node. Finally, the passive mapping results of all nodes are saved in the passive_measurement_results dictionary for subsequent analysis and network topology optimization.

[0046] Step 4.5: The system integrates the data from active and passive surveying to eliminate redundant and erroneous information and ensure the accuracy and consistency of the final measurement results.

[0047] The cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping of the present invention have the following advantages: The cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping of the present invention combine dynamic data acquisition technology with refined mapping methods, and can monitor the status of cloud resources in real time and perform dynamic analysis and optimization based on multi-dimensional data. The system not only improves the accuracy and real-time performance of resource scale estimation, but also can effectively respond to dynamic changes in cloud resources and ensure the efficiency and sustainability of resource management. This technical solution has broad application prospects in the field of cloud computing resource management and dynamic monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a model diagram of the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping of the present invention;

[0049] Figure 2 It is a schematic diagram of the system operation flow of the present invention;

[0050] Figure 3 It is a schematic diagram of the Clean pseudo code in the cleaning and standardization data of the present invention;

[0051] Figure 4 It is a pseudo code schematic diagram of calculating comprehensive score of the present invention;

[0052] Figure 5 It is a pseudo code diagram of the path delay and performance test of the present invention;

[0053] Figure 6 It is a schematic diagram of the reverse parsing pseudo code of the present invention. DETAILED DESCRIPTION

[0054] In order to better understand the purpose, structure and function of the present invention, the cloud resource distribution and scale monitoring system and method based on dynamic sampling and hybrid mapping of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0055] Related knowledge explanation:

[0056] HTTP request and response analysis

[0057] The lightweight HTTP parser is designed to efficiently extract the response header information of the target server through the HEAD request method, avoiding the transmission of the complete response body, thereby significantly reducing network resource consumption. The HEAD method only returns the response header field, which can quickly obtain the core meta information of the server and is suitable for monitoring and analysis scenarios of large-scale network nodes. The parser supports high concurrency and asynchronous I / O technology, enabling it to process a large number of domain name requests in a short time and improve data collection efficiency. Response header parsing is one of the core functions of the lightweight HTTP parser. By extracting key fields in the response header, the parser can identify the identification information, geographical distribution, and service attributes of the server node. This information includes node identifiers, network paths, and node service types, which can provide an accurate data basis for network performance analysis. In addition, the parser can also record performance indicators such as request response time and network latency for dynamic performance optimization and trend analysis. By integrating request scheduling and parsing functions, the lightweight HTTP parser demonstrates excellent performance and reliability in resource-constrained environments.

[0058] Data cleaning and standardization

[0059] Data cleaning and standardization are indispensable links in network data processing, which are directly related to the accuracy and efficiency of subsequent analysis. The purpose of data cleaning is to eliminate invalid or abnormal data records, such as nodes that timed out or failed to respond, to ensure the reliability and integrity of the final data. The parser screens the captured node data through predefined rules, eliminates redundant and duplicate records, and retains only uniquely identified nodes and their related information, thereby avoiding redundant storage and processing of data. The goal of data standardization is to unify the structure and fields of data so that they are consistent when stored and called. By uniformly encoding and formatting fields such as node identification, geographic location, and service attributes, the parser can generate standardized high-quality data sets. These data sets not only support fast storage and reading, but also provide standardized input for subsequent dynamic analysis, trend monitoring, and model training. Under the dual guarantee of data cleaning and standardization, the parser can efficiently process complex network data and provide reliable data support for a variety of application scenarios.

[0060] CDN node geographical distribution analysis

[0061] The geographical distribution of CDN nodes is the key foundation for optimizing the performance of content distribution networks. By deploying distributed edge servers around the world, CDN can direct user requests to the nearest node to reduce network latency and improve data delivery efficiency. The geographical location of CDN nodes is usually represented by identifiers in the response header. These identifiers, combined with geographic database mapping, can provide accurate support for regional network performance optimization. The geographical distribution of nodes directly affects the efficiency of content distribution and user experience. By extracting node distribution information, the parser can identify key areas and high-load nodes in the network and dynamically adjust them. Node attribute analysis further reveals the role of nodes in the entire network topology, providing data support for resource scheduling and traffic optimization. Geographic distribution analysis can not only support performance monitoring, but also be used for topology modeling to build a more efficient and stable network system.

[0062] Network Topology and Graph Theory

[0063] Network topology describes the relationship between nodes V and connections E through a graph G(V,E), and is an important tool for studying computer networks and distributed systems. Degree centrality (DC(v)) in graph theory measures the number of direct connections of a node, reflecting its local importance; betweenness centrality (BC(v)) measures the role of a node as a bridge, reflecting its global influence. In addition, dynamic topology focuses on the addition, loss, and migration of nodes. These dynamic characteristics have an important impact on network performance and resource distribution. In dynamic resource scale estimation, network topology combines attributes such as latency and bandwidth, and provides a mathematical basis for multi-dimensional indicator quantification and comprehensive score calculation through weighted graph modeling, which is an important theoretical support for system analysis.

[0064] Time Series Models

[0065] Time series model is a method for analyzing and predicting data that changes over time, and plays an important role in dynamic resource sizing. The model predicts future change patterns by mining the trends, periodicity, and randomness of historical data. Common models include autoregressive model (AR), moving average model (MA), and autoregressive integrated moving average model (ARIMA), among which ARIMA is suitable for processing non-stationary time series. In addition, deep learning models such as long short-term memory network (LSTM) are superior in capturing complex dependencies and long-term trends. In dynamic resource sizing, time series models are used to monitor the dynamic changes of node addition, loss, and migration, analyze traffic trends, and predict future resource requirements, providing precise support for cloud resource management and network optimization.

[0066] Cloud Resource Node

[0067] Cloud resource nodes are the basic components of cloud computing environments, responsible for providing computing, storage, and network services. Depending on their functions, cloud resource nodes can include edge nodes, cache nodes, and database nodes. Each node has different service types and performance indicators. By monitoring the node's geographic location, load, bandwidth, and other information, administrators can efficiently manage and optimize the use of cloud resources.

[0068] Cloud resource nodes are key components in a cloud computing environment, representing specific entities that provide various cloud services. Each cloud resource node can provide different types of computing, storage, network and other services. Edge nodes are usually located at the edge of the network, close to users or devices, and are used to reduce latency and provide fast response services. They are especially suitable for applications such as the Internet of Things (IoT), video streaming, and real-time data processing. Cache nodes are used to store commonly used data to reduce the transmission and access delay of duplicate data. They are usually used in content distribution networks (CDNs) to ensure more efficient data access speeds. Database nodes are mainly used to store and manage data, and are responsible for processing large-scale data requests and operations. The monitoring of cloud resource nodes covers multiple indicators such as the node's geographical location, service type, traffic load, and bandwidth utilization. By monitoring these indicators in real time, administrators can optimize the configuration and allocation of resources to ensure system stability, performance, and reliability.

[0069] like Figure 1 As shown, the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping of the present invention includes

[0070] Lightweight HTTP response parsing stage: In this stage, the system quickly obtains key information of cloud resource nodes by sending lightweight HTTP requests (such as HEAD requests). Parse the HTTP response header and extract basic data such as the node's identifier, geographic location, and service type to provide basic information for subsequent analysis. This stage mainly solves the initial collection and cleaning of cloud resource node information to ensure data accuracy and completeness.

[0071] Dynamic resource scale estimation phase: Based on the node information obtained in the first phase, the system evaluates each node through multi-dimensional key indicator quantification methods (such as network topology importance, traffic load, dynamic changes, and abnormal behavior). This phase aims to estimate the scale and load of cloud resource nodes and generate a dynamic change trend analysis report. Through this phase, the system can accurately estimate the overall scale of cloud resources and their real-time changes, and provide resource scheduling and optimization suggestions.

[0072] Active mapping phase: The active mapping phase mainly measures the network performance and topology of the target node directly to actively obtain key performance data about the node. By initiating detection from multiple distributed measurement points, active mapping collects network performance data such as path delay, response time, and packet loss rate between nodes. In this phase, by initiating measurement requests, cloud resource nodes are tested in real time to understand their performance stability, accessibility, and performance in different network environments.

[0073] Passive mapping phase: The passive mapping phase uses external data sources to supplement and verify the results of active mapping. This phase does not require direct detection requests to nodes, but instead obtains information such as the node's geographic location, network affiliation, and connection relationships between nodes by parsing public databases, domain name information, and registration information. The purpose of passive mapping is to fill in data gaps that cannot be covered by active mapping, especially for node features that cannot be directly measured (such as service provider information, autonomous systems, etc.).

[0074] The system integrates and calibrates the data collected at each stage, combining the data from active and passive surveying to eliminate redundancy and erroneous information, thus ensuring the accuracy and consistency of the data.

[0075] like Figure 2 As shown, the cloud resource distribution and scale monitoring method based on dynamic sampling and hybrid mapping of the present invention includes the following steps:

[0076] Step 1: Lightweight HTTP response parsing:

[0077] Assume that the system under test needs to standardize and clean a set of input data. To ensure that the input data is complete and reliable, the system will perform the following steps after receiving the input:

[0078] Step 1.1: Receive a list of domain names provided by the user and construct a test set to ensure coverage of a wide range of data inputs.

[0079] Step 1.2: Use lightweight HEAD request to obtain response header information, reduce network resource consumption, and support client or server operation.

[0080] Step 1.3: Initiate HEAD requests to the target domain names one by one, and record performance data such as response header information and latency.

[0081] Step 1.4: Parse the response header fields, extract node identifiers and attributes, map the information and verify the database results to ensure data integrity.

[0082] Step 1.5: Clean and standardize data. Call the Clean function to remove failed or timed-out nodes, merge multiple request information after deduplication, and ensure field consistency. The pseudo code of the Clean function is as follows: Figure 3 shown.

[0083] This function is mainly used to clean and standardize node data. By traversing the response status and latency of each request, the function checks whether the request is successful and whether the response time is within the normal range. When the request meets the conditions, the system adds the valid nodes to the cleaned data list and removes failed, timed out, or duplicated nodes. Finally, the system returns a cleaned data set containing all valid nodes to ensure data integrity and consistency, providing reliable basic data for subsequent network analysis and resource management.

[0084] Step 1.6: Store the standardized data locally or in the cloud for subsequent use, and output information including node identification, geographic location, and service attributes.

[0085] Step 2: Dynamic resource sizing:

[0086] Assume that the system needs to perform dynamic resource scale estimation on a set of initial nodes. The node set nodes contains the basic information of each node, including the identifier node_id, IP address ip_address, geographic location information location and service type service_type (such as edge node or cache node). After the system receives nodes, it will process them in the following steps:

[0087] Step 2.1: Receive and verify the initial node set nodes to ensure data integrity;

[0088] Step 2.2: Calculate multi-dimensional indicators for each node, including topological importance T(v), traffic and load L(v), dynamic change D(v), and abnormal behavior A(v);

[0089] Step 2.3: Calculate the comprehensive score based on the indicators to quantify the importance of each node.

[0090] k(v)=w1·T(v)+w2·L(v)+w3·D(v)+w4·A(v);

[0091] like Figure 4As shown in the figure, the function combines multiple indicators (such as topological importance, traffic and load, dynamic changes and abnormal behavior) into a comprehensive score k(v) through weighted summation. This score can help the system determine the priority and importance of each node in resource scheduling. In the score calculation, the weights parameter represents the importance weight of each indicator, while util is the resource utilization of the node (such as bandwidth utilization, etc.), and T, D, and A are the scores of topological importance, dynamic changes, and abnormal behavior, respectively. Finally, the function calculates and returns the comprehensive score of the node based on each indicator and its weight.

[0092] Step 2.4: Analyze the dynamic change trend of nodes, including quantity changes, new and lost nodes, etc.

[0093] Step 2.5: Output the estimation report, generate the key node list and regional distribution statistics;

[0094] Through the above steps, the system comprehensively analyzes the size and dynamic change characteristics of the node set, and the generated estimation report identifies key nodes and dynamic areas. In the next stage, based on these results, further mapping and optimization analysis of key nodes and areas will be carried out to provide more accurate data support for the resource management of the entire network.

[0095] Step 3: Active Mapping

[0096] After completing the preliminary data collection and scale estimation of cloud resource nodes, we further explored the network performance, geographical distribution and dynamic changes of the nodes. The operation is as follows:

[0097] Step 3.1: When selecting nodes, give priority to nodes with a score k(v)>0.6, high dynamics, high traffic share, or topological core locations;

[0098] Step 3.2: Measure the path delay of the target node through multiple measurement points, collect the delay data from the target node to nodes with different hop counts, and thus obtain the delay of the network path. Measure the response time and packet loss rate of the target node to evaluate its performance stability;

[0099] Step 3.3: Perform performance tests on each target node and measure its response time, packet loss rate and other indicators. Through multiple tests, analyze the stability, delay and packet loss of the node to provide data support for evaluating network performance. Combine path and performance data to verify the consistency of node information. Perform logic calibration to verify the accuracy of hop count and delay data;

[0100] Step 3.4: Perform parallel detection at multiple measurement points distributed in different geographical locations to obtain the delay and packet loss data of the target node under different network environments;

[0101] like Figure 5As shown in the figure, this function is mainly used to measure the path delay and performance of the target node, and to evaluate the network performance of the target node at different measurement points through measure_path_delay(node,point) and measure_performance(node,point). When the evaluation result meets the preset conditions, the system will automatically record the measurement data and save it to the database to ensure the accuracy of the node's performance data and provide support for subsequent resource optimization and network topology construction.

[0102] Step 3.5: Save all measurement data into the database and perform data cleaning to remove outliers and invalid data to ensure the accuracy and reliability of the final data.

[0103] After completing active mapping, the subsequent data analysis phase begins. By analyzing the performance data of each node, the system can identify possible problem or bottleneck nodes and further optimize the network topology and resource allocation.

[0104] Step 4: Passive Mapping

[0105] The passive mapping phase verifies and supplements the information collected in the active mapping phase through external data sources, especially for those node features that cannot be covered or directly measured in the active mapping phase.

[0106] Step 4.1: Use DNS reverse resolution technology to query the domain name of the target node and obtain the basic geographic information and service type of the node.

[0107] Step 4.2: Get the registration information of the target node, including the autonomous system (ASN), network service provider, IP segment, etc. This information helps determine the network affiliation of the node and its functional characteristics. Verify whether the node is registered at the network exchange point or data center to further understand the location and role of the node in the global network topology.

[0108] Step 4.3: Verify that the node is registered at the network exchange point or data center to further understand the location and role of the node in the global network topology. The system integrates the data of active and passive mapping to eliminate redundant and erroneous information and ensure the accuracy and consistency of the final measurement results.

[0109] Step 4.4: Query the network connection information recorded in the database, supplement and verify the network topology relationship between the target nodes, and draw a more complete network topology map.

[0110] like Figure 6As shown in the figure, this function is used to passively map the target node and obtain relevant information of the node by querying external data sources, including reverse DNS resolution, WHOIS query and PeeringDB verification. Specifically, the function first performs reverse DNS resolution (reverse_dns_lookup), WHOIS query (query_whois) and PeeringDB query (query_peeringdb) for each target node to obtain the node's domain name, registration information and network interconnection relationship from these external sources. The system integrates this information into a dictionary node_measurements to ensure that complete network topology and attribute data are collected for each node. Finally, the passive mapping results of all nodes will be saved in the passive_measurement_results dictionary for subsequent analysis and network topology optimization.

[0111] Step 4.5: The system integrates the data from active and passive surveying to eliminate redundant and erroneous information and ensure the accuracy and consistency of the final measurement results.

[0112] The results of passive mapping combined with active mapping data can provide a more comprehensive and accurate network topology map, as well as the relationship and interaction between nodes. Ultimately, by integrating the data of both, the system can generate more accurate cloud resource monitoring and network topology analysis reports, providing strong support for resource optimization and troubleshooting.

[0113] In summary, the present invention has the following characteristics:

[0114] Through dynamic sampling and real-time data monitoring, the system can accurately obtain and analyze the changes and load conditions of cloud resource nodes.

[0115] By combining active and passive mapping methods, the geographic location, performance, and topology information of cloud resource nodes are comprehensively collected to improve mapping accuracy.

[0116] By quantifying multiple dimensions such as network topology, traffic load, dynamic changes, and abnormal behavior, the status and scale of cloud resource nodes are comprehensively evaluated.

[0117] Based on multi-dimensional real-time monitoring and analysis, it provides accurate cloud resource scale estimation and dynamic change trends.

[0118] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping, characterized in that: It includes lightweight HTTP response parsing stage, dynamic resource size estimation stage, active mapping stage, passive mapping stage, and the system integrates and calibrates the data collected in each stage; The lightweight HTTP response parsing stage: quickly obtain key information of cloud resource nodes by sending lightweight HTTP requests, parse HTTP response headers, extract basic data of nodes, and provide basic information for subsequent analysis; The dynamic resource scale estimation stage: based on the node information obtained in the first stage, the system evaluates each node through a multi-dimensional key indicator quantification method; The active mapping stage: by directly measuring the network performance and topology of the target node, actively obtaining key performance data about the node, and by initiating detection from multiple distributed measurement points, actively mapping and collecting network performance data between nodes; Passive mapping stage: The passive mapping stage uses external data sources to supplement and verify the results of active mapping. This stage obtains the geographical location, network affiliation and connection relationship information of nodes by parsing public databases, domain name information and registration information; The system fuses and calibrates the data collected at each stage by combining the active and passive surveying data to eliminate redundancy and erroneous information and ensure the accuracy and consistency of the data.

2. A monitoring method for a cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping as claimed in claim 1, characterized in that: The steps include: Step 1: Lightweight HTTP response parsing: Step 2: Dynamic resource sizing: Step 3: Active mapping; Step 4: Passive mapping.

3. The monitoring method of the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping according to claim 2 is characterized in that: The step 1 comprises the following specific steps: Step 1.1: Receive the domain name list provided by the user and build a test set to ensure that a wide range of data inputs are covered; Step 1.2: Use lightweight HEAD request to obtain response header information, reduce network resource consumption, and support client or server operation; Step 1.3: Initiate HEAD requests to the target domain names one by one, and record the response header information and latency performance data; Step 1.4: Parse the response header field, extract the node identifier and attributes, map the information and verify the database results to ensure data integrity; Step 1.5: Clean and standardize data, call the Clean function, remove failed or timed-out nodes, merge multiple request information after deduplication, and ensure field consistency; The Clean function is used to clean and standardize node data. By traversing the response status and delay of each request, the function checks whether the request is successful and whether the response time is within the normal range. When the request meets the conditions, the system adds the valid node to the cleaned data list and removes the failed, timed out or duplicated nodes. Finally, the system returns a cleaned data set containing all valid nodes to ensure data integrity and consistency; Step 1.6: Store the standardized data locally or in the cloud for subsequent calls, and the output includes node identification, geographic location and service attribute information.

4. The monitoring method of the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping according to claim 2 is characterized in that: The step 2 comprises the following specific steps: Step 2.1: Receive and verify the initial node set nodes to ensure data integrity; Step 2.2: Calculate multi-dimensional indicators for each node, including topological importance T(v), traffic and load L(v), dynamic change D(v), and abnormal behavior A(v); Step 2.3: Calculate the comprehensive score based on the indicators to quantify the importance of each node. k(v)=w1·T(v)+w2·L(v)+w3·D(v)+w4·A(v); This function combines multiple indicators into a comprehensive score k(v) by weighted summation. Finally, the function calculates and returns the comprehensive score of the node based on each indicator and its weight. Step 2.4: Analyze the dynamic change trend of nodes; Step 2.5: Output the estimation report, generate a list of key nodes and regional distribution statistics.

5. The monitoring method of the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping according to claim 2 is characterized in that: The step 3 comprises the following specific steps: Step 3.1: When selecting nodes, give priority to nodes with a score k(v)>0.6, high dynamics, high traffic share, or topological core locations; Step 3.2: Measure the path delay of the target node through multiple measurement points, collect the delay data from the target node to nodes with different hop counts, thereby obtaining the network path delay, measuring the response time and packet loss rate of the target node, and evaluating its performance stability; Step 3.3: Perform performance tests on each target node, measure its response time and packet loss rate indicators, analyze the stability, delay and packet loss of the node through multiple tests, merge the path and performance data, verify the consistency of node information, perform logic calibration, and verify the accuracy of hop count and delay data; Step 3.4: Perform parallel detection at multiple measurement points distributed in different geographical locations to obtain the delay and packet loss data of the target node under different network environments; Measure the path delay and performance of the target node, and evaluate the network performance of the target node at different measurement points through measure_path_delay(node,point) and measure_performance(node,point). When the evaluation result meets the preset conditions, the system will automatically record the measurement data and save it to the database to ensure the accuracy of the node performance data; Step 3.5: Save all measurement data into the database and perform data cleaning to remove outliers and invalid data; After completing active mapping, the subsequent data analysis phase begins. By analyzing the performance data of each node, the system can identify possible problems or bottleneck nodes and further optimize the network topology and resource allocation.

6. The monitoring method of the cloud resource distribution and scale monitoring system based on dynamic sampling and hybrid mapping according to claim 2 is characterized in that: The step 4 comprises the following specific steps: Step 4.1: Use DNS reverse resolution technology to query the domain name of the target node and obtain the basic geographic information and service type of the node; Step 4.2: Obtain the registration information of the target node, including the autonomous system ASN, network service provider, and IP segment, verify whether the node is registered at the network exchange point or data center, and further understand the location and role of the node in the global network topology; Step 4.3: Verify whether the node is registered at the network exchange point or data center to further understand the location and role of the node in the global network topology. The system will fuse the data of active and passive mapping to eliminate redundancy and erroneous information to ensure the accuracy and consistency of the final measurement results. Step 4.4: Query the network connection information recorded in the database, supplement and verify the network topology relationship between the target nodes, and draw a more complete network topology map; Passively map the target node and obtain relevant information of the node by querying external data sources, including reverse DNS resolution, WHOIS query and PeeringDB verification. The function first performs reverse DNS resolution, WHOIS query and PeeringDB query for each target node, and obtains the node's domain name, registration information and network interconnection relationship from these external sources. The system integrates this information into a dictionary node_measurements to ensure that complete network topology and attribute data are collected for each node. Finally, the passive mapping results of all nodes are saved in the passive_measurement_results dictionary for subsequent analysis and network topology optimization. Step 4.5: The system merges the data of active and passive mapping to eliminate redundant and erroneous information and ensure the accuracy and consistency of the final measurement results.

Images