Data monitoring and correcting method and system based on cloud network fusion

By acquiring and converting gateway data formats on a cloud platform and using clustering algorithms to predict faults, the complexity and latency issues of gateway management are resolved, enabling efficient data monitoring and fault prediction, and improving the efficiency of gateway performance monitoring and maintenance.

CN118945067APending Publication Date: 2024-11-12CHINA UNION CLOUD COMPUTING SERVICES (SHANGHAI) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411241036.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-05
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

As the number of gateways increases, the configuration and management of monitoring systems become more complex. The diversity of data formats and protocols makes integration difficult, and latency issues are particularly prominent when network conditions are poor or the data volume is large. The centralized monitoring interface of cloud platforms can provide unified data monitoring and management, but existing technologies have not been able to effectively solve these problems.

Method used

By using cloud-network convergence technology, gateway configuration data is acquired on the cloud platform and its format is converted. Clustering algorithms are used to predict faulty gateways, and corrective data is transmitted through backup gateways. This includes the design of modules for configuration conversion, fault prediction, and data monitoring, enabling intelligent data analysis and prediction.

Benefits of technology

It enables intelligent analysis and fault prediction of gateway data, provides multi-dimensional data analysis results, improves the efficiency of gateway performance monitoring and maintenance, and reduces latency issues.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945067A_ABST
    Figure CN118945067A_ABST
Patent Text Reader

Abstract

The invention discloses a data monitoring and correcting method and system based on cloud network fusion, and the method comprises the steps: S1, obtaining the monitoring and operation data of all gateways on a cloud platform based on a cloud network fusion technology, and obtaining the preprocessing data; s2, analyzing the preprocessed data, and predicting a gateway and a fault according to an analysis result; s3, correcting fault gateway transmission data based on a clustering algorithm; and S4, marking a fault gateway, selecting a standby gateway to transmit the correction data, and completing data monitoring and correction. According to the invention, through the data processing capability of the cloud platform, the collected gateway data can be intelligently analyzed, the fault and performance of the gateway can be predicted based on the analysis result, and intervention and extraction are carried out in advance according to the prediction result for maintenance and optimization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of cloud-network integration technology, and in particular to a data monitoring and correction method and system based on cloud-network integration. Background Art

[0002] As the number of gateways grows, the configuration and management of the monitoring system becomes more complex. Data from different gateways may use different formats or protocols. It may be difficult to integrate this data for unified analysis, requiring additional conversion and standardization work. Data is collected from each gateway, transmitted to the monitoring center, and then processed and analyzed. This series of processes may introduce delays. The delay problem is particularly prominent when the network conditions are poor or the amount of data is large. The cloud platform of cloud-network integration can provide a centralized monitoring interface to facilitate unified data monitoring and management of all gateways. By leveraging the elasticity and scalability of cloud services, the operation data of the gateway can be collected in real time, including key indicators such as traffic, latency, and packet loss rate. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides a data monitoring and correction method based on cloud-network integration, the method comprising:

[0004] Step S1: Based on the cloud-network integration technology, the configuration data of all gateways are obtained on the cloud platform, pre-processed data is obtained, and the format of the pre-processed data is converted;

[0005] Step S2: Analyze the preprocessed data and predict gateways and faults based on the analysis results;

[0006] Step S3: correcting the faulty gateway transmission data based on a clustering algorithm;

[0007] Step S4: Mark the faulty gateway, select a standby gateway to transmit the correction data, and complete data monitoring and correction.

[0008] Optionally, in step S1, based on the cloud-network integration technology, the process of acquiring configuration data of all gateways on the cloud platform, obtaining preprocessed data and converting the format of the preprocessed data specifically includes:

[0009] Obtaining a configuration page of a network gateway, and obtaining a first data format based on the configuration page of the network gateway;

[0010] Obtaining a configuration page of a cloud platform gateway, and obtaining a second data format based on the configuration page of the cloud platform gateway;

[0011] Find the protocol conversion configuration from the configuration page, and obtain the target conversion rule between the network gateway and the cloud platform gateway according to the protocol conversion configuration;

[0012] Based on the target conversion rules, data format conversion from the network gateway to the cloud platform gateway can be achieved.

[0013] Optionally, the specific process of implementing data format conversion from the network gateway to the cloud platform gateway based on the target conversion rule includes:

[0014] Determine a source data format and a target data format during conversion based on the first data format and the second data format;

[0015] extracting a first data element from a source data format and extracting a second data element from a target data format based on a corresponding mapping relationship between the first data element and the second data element;

[0016] The data format conversion is implemented based on the mapping relationship.

[0017] Optionally, in step S2, analyzing the preprocessed data, and predicting the gateway and fault content according to the analysis result specifically includes:

[0018] Extracting a transmission data stream of the pre-processed data after format conversion, and when the transmission data stream is abnormally transmitted in the gateway, extracting transmission abnormality features and estimating the size of the communication data packet;

[0019] When the estimated data packet size is greater than or equal to the instantaneous traffic threshold of the gateway fault verification, the data packet is split based on the traffic of the gateway fault verification to obtain a number of data packets whose sizes are smaller than the instantaneous traffic threshold of the gateway fault verification;

[0020] The split data packets are transmitted sequentially through the gateway to obtain the adjusted real-time data streams of different gateway devices;

[0021] Make abnormal judgments on the real-time data stream of the gateway and predict gateway failures based on the judgment results.

[0022] The present invention also discloses a data monitoring and correction system based on cloud-network integration, the system comprising: a configuration conversion module, a fault prediction module, a data correction module and a data monitoring module;

[0023] The configuration conversion module is used to obtain the configuration data of all gateways on the cloud platform based on the cloud-network integration technology, obtain the pre-processed data and perform format conversion on the pre-processed data;

[0024] The fault prediction module is used to analyze the preprocessed data and predict the gateway and fault according to the analysis results;

[0025] The data correction module is used to correct the faulty gateway transmission data based on the clustering algorithm;

[0026] The data monitoring module is used to mark a faulty gateway, select a standby gateway to transmit the correction data, and complete data monitoring and correction.

[0027] Optionally, the workflow of the configuration conversion module specifically includes: a network gateway configuration acquisition submodule, a cloud platform gateway configuration acquisition submodule, a conversion rule submodule and a format conversion submodule;

[0028] The network gateway configuration acquisition submodule is used to acquire a configuration page of the network gateway, and obtain a first data format based on the configuration page of the network gateway;

[0029] The cloud platform gateway configuration acquisition submodule is used to acquire a configuration page of the cloud platform gateway, and obtain a second data format based on the configuration page of the cloud platform gateway;

[0030] The conversion rule submodule finds the protocol conversion configuration from the configuration page, and obtains the target conversion rule between the network gateway and the cloud platform gateway according to the protocol conversion configuration;

[0031] The format conversion submodule can realize data format conversion from the network gateway to the cloud platform gateway based on the target conversion rule.

[0032] Optionally, the workflow of the format conversion submodule specifically includes:

[0033] Determine a source data format and a target data format during conversion based on the first data format and the second data format;

[0034] extracting a first data element from a source data format and extracting a second data element from a target data format based on a corresponding mapping relationship between the first data element and the second data element;

[0035] The data format conversion is implemented based on the mapping relationship.

[0036] Optionally, the fault prediction module includes: a data packet size detection submodule, a data packet splitting submodule, a data flow adjustment submodule and a gateway status detection submodule;

[0037] The data packet size detection submodule is used to extract the transmission data stream of the pre-processed data after format conversion, and when the transmission data stream is abnormally transmitted in the gateway, the transmission abnormality feature is extracted and the size of the communication data packet is estimated;

[0038] The data packet splitting submodule is used to split the data packet based on the traffic of the gateway fault verification when the estimated data packet size is greater than or equal to the traffic instantaneous threshold of the gateway fault verification, to obtain a number of data packets whose sizes are smaller than the traffic instantaneous threshold of the gateway fault verification;

[0039] The data flow adjustment submodule is used to transmit the split data packets through the gateway in sequence to obtain the adjusted real-time data flows of different gateway devices;

[0040] The gateway status detection submodule is used to perform abnormality judgment on the real-time data stream of the gateway and predict faults of the gateway based on the judgment result.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention utilizes the data processing capabilities of the cloud platform to intelligently analyze the collected gateway data, and predict gateway failures and performance based on the analysis results, intervene in advance according to the prediction results, and extract maintenance and optimization. In addition, the cloud-network fusion technology used in the present invention can provide more multi-dimensional data analysis results, which can more deeply reflect the performance of the gateway. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0044] Figure 1 This is a method step diagram of the data monitoring and correction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Embodiment 1

[0047] A data monitoring and correction method based on cloud-network integration, such as Figure 1 As shown, the method includes:

[0048] Step S1: Based on the cloud-network integration technology, the configuration data of all gateways is obtained on the cloud platform, pre-processed data is obtained and format conversion is performed on the pre-processed data, specifically including:

[0049] Obtain the configuration page of the network gateway, and obtain the first data format based on the configuration page of the network gateway; obtain the configuration page of the cloud platform gateway, and obtain the second data format based on the configuration page of the cloud platform gateway; find the protocol conversion configuration from the configuration page, and obtain the target conversion rule between the network gateway and the cloud platform gateway according to the protocol conversion configuration; and implement the data format conversion from the network gateway to the cloud platform gateway based on the target conversion rule. The specific process of data format conversion includes: determining the source data format and target data format during conversion based on the first data format and the second data format; extracting the first data element from the source data format, extracting the second data element from the target data format, based on the corresponding mapping relationship between the first data element and the second data element; and implementing data format conversion based on the mapping relationship.

[0050] Step S2, analyzing the preprocessed data, and predicting the gateway and faults according to the analysis results, specifically comprising: extracting the transmission data stream of the preprocessed data after format conversion, and when the transmission data stream is transmitted abnormally in the gateway, extracting the transmission abnormality characteristics and estimating the size of the communication data packet; when the estimated data packet size is greater than or equal to the instantaneous threshold of the traffic flow of the gateway fault verification, splitting the data packet based on the traffic flow of the gateway fault verification to obtain a number of data packets whose sizes are smaller than the instantaneous threshold of the traffic flow of the gateway fault verification; transmitting the split data packets through the gateway in sequence to obtain the adjusted real-time data streams of different gateway devices; making an abnormal judgment on the real-time data stream of the gateway, and predicting the fault of the gateway based on the judgment result.

[0051] Step S3: correcting the transmission data of the faulty gateway based on the clustering algorithm.

[0052] The BIRCH clustering algorithm is used to describe the data of different gateway node clusters, the gateway sequence is divided into P clusters, the feature tree nodes are defined, the radius critical value of the cluster is calculated, the gateway data sequence is transformed into a spherical cluster unit, and the isolated outliers of the gateway data sequence are selected, and the abnormal number of the gateway data is used as the judgment condition for anomaly detection, thereby obtaining abnormal data. The BIRCH clustering algorithm used in the present invention describes the data of different node clusters through CF, and defines the clustering features within the cluster as triples:

[0053] CF=(F,LC,SST)

[0054] Among them, F is the sum of the data points in the cluster, LC is the linear sum of the data points, and SST is the sum of the squares of the data points. The CF tree saves cluster features at different levels and is a highly balanced tree. The CF tree contains root nodes, branch nodes, and leaf nodes. Each node of the tree is composed of multiple cluster features. The cluster radius critical value is calculated as:

[0055]

[0056] Usually, the result of data clustering is presented as a spherical cluster architecture. A spherical cluster indicates that the s data factors in the data space are evenly distributed and spherical. However, high-dimensional sensor data sequences do not have spherical cluster characteristics, so they need to be transformed into a spherical calculation form, creating the concept of feature vectors in the middle time, and transforming the sensor data sequence into a spherical cluster unit. The conversion process includes:

[0057]

[0058] W lij =[w lij O′ lij a′ ij ]

[0059] Step S4: Mark the faulty gateway, select a standby gateway to transmit the correction data, and complete data monitoring and correction.

[0060] Embodiment 2

[0061] A data monitoring and correction system based on cloud-network integration, the system comprising: a data monitoring and correction system based on cloud-network integration, the system comprising: a configuration conversion module, a fault prediction module, a data correction module and a data monitoring module;

[0062] The configuration conversion module is used to obtain the configuration data of all gateways on the cloud platform based on cloud-network integration technology, obtain pre-processed data and perform format conversion on the pre-processed data.

[0063] The workflow of the configuration conversion module specifically includes: a network gateway configuration acquisition submodule, a cloud platform gateway configuration acquisition submodule, a conversion rule submodule and a format conversion submodule; the network gateway configuration acquisition submodule is used to acquire the configuration page of the network gateway, and obtain a first data format based on the configuration page of the network gateway; the cloud platform gateway configuration acquisition submodule is used to acquire the configuration page of the cloud platform gateway, and obtain a second data format based on the configuration page of the cloud platform gateway; the conversion rule submodule finds the protocol conversion configuration from the configuration page, and obtains the target conversion rule between the network gateway and the cloud platform gateway according to the protocol conversion configuration; the format conversion submodule can realize the data format conversion from the network gateway to the cloud platform gateway based on the target conversion rule.

[0064] The workflow of the format conversion submodule specifically includes: determining the source data format and the target data format during conversion based on the first data format and the second data format; extracting the first data element from the source data format and the second data element from the target data format based on the corresponding mapping relationship between the first data element and the second data element; and realizing data format conversion based on the mapping relationship.

[0065] The fault prediction module is used to analyze the preprocessed data and predict the gateway and fault according to the analysis results. The fault prediction module includes: a data packet size detection submodule, a data packet splitting submodule, a data flow adjustment submodule and a gateway status detection submodule; the data packet size detection submodule is used to extract the transmission data flow of the preprocessed data after format conversion, and when the transmission data flow is abnormally transmitted in the gateway, the transmission abnormality characteristics are extracted and the size of the communication data packet is estimated; the data packet splitting submodule is used to split the data packet based on the traffic of the gateway fault verification when the estimated data packet size is greater than or equal to the instantaneous threshold of the traffic flow of the gateway fault verification, and obtain a number of data packets whose size is less than the instantaneous threshold of the traffic flow of the gateway fault verification; the data flow adjustment submodule is used to transmit the split data packets through the gateway in sequence to obtain the adjusted real-time data flow of different gateway devices; the gateway status detection submodule is used to make an abnormal judgment on the real-time data flow of the gateway, and predict the fault of the gateway based on the judgment result.

[0066] The data correction module is used to find the previous hop gateway of the faulty gateway and obtain the data of the previous hop gateway, and correct the next hop data of the faulty gateway based on the data of the previous hop gateway to obtain corrected data.

[0067] The BIRCH clustering algorithm is used to describe the data of different gateway node clusters, the gateway sequence is divided into P clusters, the feature tree nodes are defined, the radius critical value of the cluster is calculated, the gateway data sequence is transformed into a spherical cluster unit, and the isolated outliers of the gateway data sequence are selected, and the abnormal number of the gateway data is used as the judgment condition for anomaly detection, thereby obtaining abnormal data. The BIRCH clustering algorithm used in the present invention describes the data of different node clusters through CF, and defines the clustering features within the cluster as triples:

[0068] CF=(F,LC,SST)

[0069] Among them, F is the sum of the data points in the cluster, LC is the linear sum of the data points, and SST is the sum of the squares of the data points. The CF tree saves cluster features at different levels and is a highly balanced tree. The CF tree contains root nodes, branch nodes, and leaf nodes. Each node of the tree is composed of multiple cluster features. The cluster radius critical value is calculated as:

[0070]

[0071] Usually, the result of data clustering is presented as a spherical cluster architecture. A spherical cluster indicates that the s data factors in the data space are evenly distributed and spherical. However, high-dimensional sensor data sequences do not have spherical cluster characteristics, so they need to be transformed into a spherical calculation form, creating the concept of feature vectors in the middle time, and transforming the sensor data sequence into a spherical cluster unit. The conversion process includes:

[0072]

[0073] W lij =[w lij O′ lij a′ ij ]

[0074] The data monitoring module is used to mark a faulty gateway, select a standby gateway to transmit the correction data, and complete data monitoring and correction.

[0075] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A data monitoring and correction method based on cloud-network integration, characterized in that: The method comprises: Step S1: Based on the cloud-network integration technology, the configuration data of all gateways are obtained on the cloud platform, pre-processed data is obtained, and the format of the pre-processed data is converted; Step S2: Analyze the preprocessed data and predict gateways and faults based on the analysis results; Step S3: correcting the faulty gateway transmission data based on a clustering algorithm; Step S4: Mark the faulty gateway, select a standby gateway to transmit the correction data, and complete data monitoring and correction.

2. The data monitoring and correction method based on cloud-network integration according to claim 1 is characterized in that: In step S1, based on the cloud-network integration technology, the configuration data of all gateways are obtained on the cloud platform, the pre-processed data is obtained, and the process of format conversion of the pre-processed data specifically includes: Obtaining a configuration page of a network gateway, and obtaining a first data format based on the configuration page of the network gateway; Obtaining a configuration page of a cloud platform gateway, and obtaining a second data format based on the configuration page of the cloud platform gateway; Find the protocol conversion configuration from the configuration page, and obtain the target conversion rule between the network gateway and the cloud platform gateway according to the protocol conversion configuration; Based on the target conversion rules, data format conversion from the network gateway to the cloud platform gateway can be achieved.

3. The data monitoring and correction method based on cloud-network integration according to claim 2 is characterized in that: The specific process of implementing data format conversion from the network gateway to the cloud platform gateway based on the target conversion rule includes: Determine a source data format and a target data format during conversion based on the first data format and the second data format; extracting a first data element from a source data format and extracting a second data element from a target data format based on a corresponding mapping relationship between the first data element and the second data element; The data format conversion is implemented based on the mapping relationship.

4. The data monitoring and correction method based on cloud-network integration according to claim 1 is characterized in that: In step S2, the pre-processed data is analyzed, and the contents of predicting the gateway and the fault according to the analysis result specifically include: Extracting a transmission data stream of the pre-processed data after format conversion, and when the transmission data stream is abnormally transmitted in the gateway, extracting transmission abnormality features and estimating the size of the communication data packet; When the estimated data packet size is greater than or equal to the instantaneous traffic threshold of the gateway fault verification, the data packet is split based on the traffic of the gateway fault verification to obtain a number of data packets whose sizes are smaller than the instantaneous traffic threshold of the gateway fault verification; The split data packets are transmitted sequentially through the gateway to obtain the adjusted real-time data streams of different gateway devices; Make abnormal judgments on the real-time data stream of the gateway and predict gateway failures based on the judgment results.

5. A data monitoring and correction system based on cloud-network integration, the data monitoring and correction system is used to implement the data monitoring and correction method according to any one of claims 1 to 4, characterized in that: The system includes: Configuration conversion module, fault prediction module, data correction module and data monitoring module; The configuration conversion module is used to obtain the configuration data of all gateways on the cloud platform based on the cloud-network integration technology, obtain the pre-processed data and perform format conversion on the pre-processed data; The fault prediction module is used to analyze the preprocessed data and predict the gateway and fault according to the analysis results; The data correction module is used to correct the faulty gateway transmission data based on the clustering algorithm; The data monitoring module is used to mark a faulty gateway, select a standby gateway to transmit the correction data, and complete data monitoring and correction.

6. The data monitoring and correction system based on cloud-network integration according to claim 5 is characterized in that: The workflow of the configuration conversion module specifically includes: a network gateway configuration acquisition submodule, a cloud platform gateway configuration acquisition submodule, a conversion rule submodule and a format conversion submodule; The network gateway configuration acquisition submodule is used to acquire a configuration page of the network gateway, and obtain a first data format based on the configuration page of the network gateway; The cloud platform gateway configuration acquisition submodule is used to acquire a configuration page of the cloud platform gateway, and obtain a second data format based on the configuration page of the cloud platform gateway; The conversion rule submodule finds the protocol conversion configuration from the configuration page, and obtains the target conversion rule between the network gateway and the cloud platform gateway according to the protocol conversion configuration; The format conversion submodule can realize data format conversion from the network gateway to the cloud platform gateway based on the target conversion rule.

7. The data monitoring and correction system based on cloud-network integration according to claim 6 is characterized in that: The workflow of the format conversion submodule specifically includes: Determine a source data format and a target data format during conversion based on the first data format and the second data format; extracting a first data element from a source data format and extracting a second data element from a target data format based on a corresponding mapping relationship between the first data element and the second data element; The data format conversion is implemented based on the mapping relationship.

8. The data monitoring and correction system based on cloud-network integration according to claim 5 is characterized in that: The fault prediction module includes: a data packet size detection submodule, a data packet splitting submodule, a data flow adjustment submodule and a gateway status detection submodule; The data packet size detection submodule is used to extract the transmission data stream of the pre-processed data after format conversion, and when the transmission data stream is abnormally transmitted in the gateway, the transmission abnormality feature is extracted and the size of the communication data packet is estimated; The data packet splitting submodule is used to split the data packet based on the traffic of the gateway fault verification when the estimated data packet size is greater than or equal to the traffic instantaneous threshold of the gateway fault verification, to obtain a number of data packets whose sizes are smaller than the traffic instantaneous threshold of the gateway fault verification; The data flow adjustment submodule is used to transmit the split data packets through the gateway in sequence to obtain the adjusted real-time data flows of different gateway devices; The gateway status detection submodule is used to perform abnormality judgment on the real-time data stream of the gateway and predict faults of the gateway based on the judgment result.