A method for automatically drawing a graph and generating a data table based on CPE data

Abstract

The application provides a kind of method for automatic drawing based on CPE data and generating data table, related to data processing technical field, extract the monitoring running data of user premises equipment, carry out redundancy to monitoring running data, obtain monitoring running redundancy data set;Extract data difference extreme value and data fluctuation degree in monitoring running redundancy data set, determine the data space smoothing factor of monitoring running redundancy data set according to data difference extreme value, determine the feature maintenance factor of monitoring running redundancy data set by data fluctuation degree;Get the data normal vector of monitoring running redundancy data set, convert monitoring running redundancy data set into smooth data sequence based on data normal vector, data space smoothing factor and feature maintenance factor;According to smooth data sequence, generate data trend chart and data table, the application can remove redundant data and anomaly in monitoring running data, to improve the quality of final generated data chart and data table.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a method for automatically drawing graphs and generating data tables based on CPE data. Background Art

[0002] This method automatically plots and generates data tables based on data from Customer Premises Equipment (CPE). This solution collects, processes, and displays equipment operating data in real time, reducing manual operations, improving monitoring efficiency, and providing users with intuitive operational data analysis results. This method collects, stores, and analyzes CPE data in real time, presenting it in charts and tables. This supports application scenarios such as equipment monitoring, fault detection, and performance analysis.

[0003] However, in practice, the monitoring and operation data of customer premises equipment (CPE) is diverse and complex. Furthermore, the data in log databases is large and complex, leading to data redundancy. This complicates data updates and maintenance, and is prone to inconsistencies. Furthermore, CPE data often involves complex, multi-layered relationships, which hinders data maintenance and query efficiency. Therefore, how to remove redundant data and anomalies from monitoring and operation data to improve the quality of the resulting data charts and tables is a challenge facing the industry. Summary of the Invention

[0004] The present application provides a method for automatically drawing graphs and generating data tables based on CPE data, which can remove redundant data and anomalies in monitoring operation data to improve the quality of the data charts and data tables finally generated.

[0005] The present application provides a method for automatically drawing a graph and generating a data table based on CPE data, the method comprising the following steps:

[0006] Extracting monitoring operation data of customer premises equipment, performing redundancy elimination on the monitoring operation data, and obtaining a monitoring operation redundancy elimination data set of the customer premises equipment;

[0007] extracting data difference extreme values ​​and data fluctuations from the monitoring operation redundant dataset, determining a data space smoothing factor for the monitoring operation redundant dataset based on the data difference extreme values, and determining a feature maintenance factor for the monitoring operation redundant dataset based on the data fluctuations;

[0008] Obtaining a data normal vector of the monitoring operation redundant dataset, and converting the monitoring operation redundant dataset into a smoothed data sequence of the customer premises equipment during operation based on the data normal vector, the data space smoothing factor, and the feature preservation factor;

[0009] A data trend graph and a data table of the customer premises equipment during operation are generated according to the smoothed data sequence.

[0010] In this embodiment, performing redundancy elimination on the monitoring operation data to obtain the redundant monitoring operation data set of the customer premises equipment specifically includes:

[0011] Performing anomaly detection on each data point in the monitoring operation data to obtain multiple abnormal data points;

[0012] All abnormal data points are removed from the monitoring operation data, and the monitoring operation data after the abnormal data points are removed is used as the monitoring operation redundant data set of the customer premises equipment.

[0013] In this embodiment, extracting data difference extreme values ​​and data fluctuations from the monitoring operation redundant data set specifically includes:

[0014] Performing data differentiation on the monitoring operation redundant data set to obtain a monitoring operation differential sequence;

[0015] extracting data difference extreme values ​​from the monitoring running difference sequence;

[0016] The data volatility is determined based on the monitoring running difference sequence.

[0017] In this embodiment, the data space smoothing factor is an indicator for controlling the degree of spatial smoothing of the redundant data set of the monitoring operation.

[0018] In this embodiment, the feature preservation factor is an indicator used to maintain the original features of the data when spatial smoothing is performed on the monitoring operation redundant data set.

[0019] In this embodiment, obtaining the data normal vector of the monitoring operation redundant data set specifically includes:

[0020] Standardizing the monitoring and operating redundant data set, and then determining a covariance matrix of the standardized monitoring and operating redundant data set;

[0021] Performing eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvalues ​​and eigenvectors corresponding to each eigenvalue;

[0022] The eigenvector corresponding to the maximum eigenvalue is used as the data normal vector of the monitoring operation redundant data set.

[0023] In this embodiment, converting the monitoring operation redundant data set into a smoothed data sequence of the customer premises equipment during operation based on the data normal vector, the data space smoothing factor, and the feature maintenance factor specifically includes:

[0024] For each data point in the monitoring operation redundant-elimination data set, spatially smoothing the data point is performed using the data normal vector, the data spatial smoothing factor, and the feature maintenance factor to obtain a smoothed data point corresponding to the data point, and further obtain a smoothed data point corresponding to each data point in the monitoring operation redundant-elimination data set, and a sequence consisting of all smoothed data points is used as a smoothed data sequence of the customer premises equipment during operation.

[0025] In this embodiment, the smoothed data sequence is input into a data visualization tool to generate a data trend graph of the CPE during operation.

[0026] In this embodiment, the smoothed data sequence is input into a dynamic table generation tool to generate a data table of the CPE during operation.

[0027] In this embodiment, the monitoring operation data of the CPE is extracted from the log database of the CPE.

[0028] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0029] The invention relates to a method for extracting monitoring operation data of a customer premises equipment and performing redundancy elimination on the monitoring operation data to obtain a redundant monitoring operation data set of the customer premises equipment; extracting data difference extreme values ​​and data fluctuations from the redundant monitoring operation data set, determining a data space smoothing factor for the redundant monitoring operation data set based on the data difference extreme values, and determining a feature maintenance factor for the redundant monitoring operation data set based on the data fluctuations; obtaining a data normal vector of the redundant monitoring operation data set, and converting the redundant monitoring operation data set into a smoothed data sequence of the customer premises equipment during operation based on the data normal vector, the data space smoothing factor, and the feature maintenance factor; and generating a data trend graph and a data table of the customer premises equipment during operation based on the smoothed data sequence.

[0030] It can be seen that in this application, first, redundant data in the monitoring operation data can be removed through the redundant elimination operation; then, by extracting the extreme values ​​of data differences, the most significant fluctuation points in the data set can be identified, and by evaluating the data volatility, unstable areas or abnormal fluctuations in the data can be identified, thereby providing a basis for subsequent smoothing processing. Based on the data space smoothing factor, over-smoothing or over-retention of fluctuations can be avoided, ensuring that the data retains important trends and information while removing redundancy, and based on the feature maintenance factor, it can help retain key data fluctuation characteristics during the smoothing process; secondly, based on the data normal vector, the data space smoothing factor and the feature maintenance factor, the data points in the data space can be effectively smoothed to reduce interference with the overall trend. By constructing a smoothed data sequence, it helps to clearly present the real operating trends and periodic changes of the user's premises equipment, avoiding misleading caused by noise and abnormal data; finally, generating data trend graphs and data tables based on the smoothed data sequence can significantly improve the visualization effect and quality of the data.

[0031] In summary, the technical solution adopted in this application can remove redundant data and anomalies in the monitoring operation data to improve the quality of the data charts and data tables finally generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0033] Figure 1 This is a flowchart of a method for automatically drawing a graph and generating a data table based on CPE data provided by the present application;

[0034] Figure 2 This is an exemplary flow chart for obtaining data normal vectors of a monitoring operation redundant data set provided by the present application. DETAILED DESCRIPTION

[0035] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the examples and accompanying drawings. The exemplary embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention. It should be noted that the present invention is already in the actual development and use stage.

[0036] In order to better understand the above technical solution, the following will be described in detail with reference to the accompanying drawings and specific implementation methods. Figure 1 As shown in FIG, this figure is an exemplary flow chart of a method for automatically drawing a graph based on CPE data and generating a data table according to this embodiment of the present application, the method comprising the following steps:

[0037] In step S1, monitoring operation data of a user premises equipment is extracted, and the monitoring operation data is deduplicated to obtain a monitoring operation deduplicated data set of the user premises equipment.

[0038] In a specific implementation, the monitoring operation data of the user premises equipment can be extracted from a log database of the user premises equipment by using a data query language. It should be noted that, in this application, the monitoring operation data refers to memory usage data of the user premises equipment during operation.

[0039] In this embodiment, the monitoring operation data is deduplicated to obtain the monitoring operation deduplicated data set of the user premises equipment, which can be implemented in the following manner, that is:

[0040] Each data point in the monitoring operation data is detected for anomaly respectively to obtain a plurality of abnormal data points.

[0041] All the abnormal data points in the monitoring operation data are removed, and the monitoring operation data after removing the abnormal data points is taken as the monitoring operation deduplicated data set of the user premises equipment.

[0042] In a specific implementation, first, each data point in the monitoring operation data can be detected for anomaly respectively. The target of anomaly detection is to identify and remove data points that are significantly inconsistent with normal device operation behavior. Generally, abnormal data points are caused by device failure, sensor error, network problem or other factors. A machine learning-based anomaly detection model can be used to detect each data point in the monitoring operation data for anomaly, so that a plurality of abnormal data points can be obtained through anomaly detection. Then, all the abnormal data points in the monitoring operation data can be removed, and the monitoring operation data after removing the abnormal data points is taken as the monitoring operation deduplicated data set of the user premises equipment. It should be noted that, after anomaly detection and removal of abnormal data, the monitoring operation deduplicated data set obtained finally will contain more accurate and noise-free monitoring operation data.

[0043] In step S2, data difference extreme values and data fluctuation degrees are extracted from the monitoring operation deduplicated data set, a data space smoothing factor for the monitoring operation deduplicated data set is determined according to the data difference extreme values, and a feature maintenance factor for the monitoring operation deduplicated data set is determined through the data fluctuation degrees.

[0044] In this embodiment, the data difference extreme values and the data fluctuation degrees can be extracted from the monitoring operation deduplicated data set in the following manner, that is:

[0045] The monitoring operation deduplicated data set is subjected to data difference to obtain a monitoring operation difference sequence.

[0046] The data difference extreme values are extracted from the monitoring operation difference sequence.

[0047] determine the data fluctuation degree according to the monitoring operation difference sequence.

[0048] In the implementation, first, data difference of the monitoring operation redundant data set can be performed, that is, first-order difference of all data points in the monitoring operation redundant data set is performed, so as to obtain a plurality of monitoring operation difference values, that is, difference values between adjacent data points, and a sequence composed of all monitoring operation difference values can be taken as a monitoring operation difference sequence; then, a data difference extreme value can be extracted from the monitoring operation difference sequence, wherein the data difference extreme value is a maximum value in the monitoring operation difference sequence, and the data difference extreme value can be extracted from the monitoring operation difference sequence in a traversal manner; finally, the data fluctuation degree can be determined according to the monitoring operation difference sequence, wherein the data fluctuation degree represents a data fluctuation degree of the monitoring operation difference sequence, and the data fluctuation degree can be determined according to the following formula in the actual implementation:

[0049]

[0050] wherein λ represents the data fluctuation degree, m represents a total number of data in the monitoring operation difference sequence, y j represents a jth monitoring operation difference value in the monitoring operation difference sequence, and μ represents a mean value of all monitoring operation difference values in the monitoring operation difference sequence.

[0051] In the embodiment, a data space smoothing factor for the monitoring operation redundant data set is determined according to the data difference extreme value, wherein the data space smoothing factor is an index for controlling a spatial smoothing degree of the monitoring operation redundant data set, and the data space smoothing factor can be determined by the following formula in the actual implementation:

[0052]

[0053] wherein α represents the data space smoothing factor for the monitoring operation redundant data set, and κ represents the data difference extreme value.

[0054] In the embodiment, a feature maintenance factor for the monitoring operation redundant data set is determined by the data fluctuation degree, wherein the feature maintenance factor is an index for maintaining original features of data when spatial smoothing is performed on the monitoring operation redundant data set, and the feature maintenance factor can be determined by the following formula in the actual implementation:

[0055]

[0056] wherein β represents the feature maintenance factor for the monitoring operation redundant data set, and λ represents the data fluctuation degree.

[0057] It should be noted that by extracting the extreme values ​​of data differences, the most significant fluctuation points in the data set can be identified. By evaluating the data volatility, unstable areas or abnormal fluctuations in the data can be identified, thereby providing a basis for subsequent smoothing processing. Based on the data space smoothing factor, over-smoothing or over-retention of fluctuations can be avoided, ensuring that the data retains important trends and information while removing redundancy. Based on the feature maintenance factor, it can help retain key data fluctuation characteristics during the smoothing process.

[0058] In step S3, a data normal vector of the monitoring operation redundant dataset is obtained, and the monitoring operation redundant dataset is converted into a smoothed data sequence of the CPE during operation based on the data normal vector, the data space smoothing factor, and the feature preservation factor.

[0059] Preferably, in this embodiment, reference Figure 2 As shown in FIG, this figure is an exemplary flow chart of obtaining the data normal vector of the monitoring operation redundant data set in an embodiment of the present application. In this embodiment, obtaining the data normal vector of the monitoring operation redundant data set can be specifically implemented by the following steps:

[0060] First, in step S31, the monitoring operation redundant data set is standardized, and then the covariance matrix of the standardized monitoring operation redundant data set is determined;

[0061] Then, in step S32, the covariance matrix is ​​subjected to eigenvalue decomposition to obtain a plurality of eigenvalues ​​and eigenvectors corresponding to each eigenvalue;

[0062] Finally, in step S33, the eigenvector corresponding to the maximum eigenvalue is used as the data normal vector of the monitoring operation redundant data set.

[0063] In specific implementation, first, the monitoring operation redundant data set can be standardized. Standardization is to eliminate the scale differences of different dimensions in the data so that all features can be compared under the same dimension, so that the covariance matrix of the standardized monitoring operation redundant data set can be calculated; then, the covariance matrix can be decomposed into eigenvalues ​​using existing technology to obtain multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue; finally, the eigenvector corresponding to the maximum eigenvalue can be used as the data normal vector of the monitoring operation redundant data set, where the data normal vector is an important eigenvector that describes the change trend of the data set. Through the data normal vector, the main change direction in the data can be captured, especially in high-dimensional space. The data normal vector can effectively guide subsequent smoothing, noise reduction and feature retention processing; it should be noted that the eigenvector corresponding to the largest eigenvalue represents the main direction of change in the data set.

[0064] In this embodiment, the monitoring operation redundant data set is converted into a smoothed data sequence of the CPE during operation based on the data normal vector, the data space smoothing factor, and the feature maintenance factor in the following manner:

[0065] For each data point in the monitoring operation redundant-elimination data set, spatially smoothing the data point is performed using the data normal vector, the data spatial smoothing factor, and the feature maintenance factor to obtain a smoothed data point corresponding to the data point, and further obtain a smoothed data point corresponding to each data point in the monitoring operation redundant-elimination data set, and a sequence consisting of all smoothed data points is used as a smoothed data sequence of the customer premises equipment during operation.

[0066] In specific implementation, for each data point in the redundant data set of the monitoring operation, the data point is spatially smoothed by the data normal vector, the data space smoothing factor and the feature maintenance factor to obtain the smoothed data point corresponding to the data point. In actual implementation, the smoothed data point corresponding to the data point can be determined by the following formula:

[0067]

[0068] Where x′ i represents the smoothed data point corresponding to the data point, x i Indicates the i-th data point in the monitoring run redundant data set, x j represents the j-th data point in the monitoring operation redundant data set, α represents the data space smoothing factor of the monitoring operation redundant data set, β represents the feature maintenance factor of the monitoring operation redundant data set, a represents the data normal vector, and n represents the total number of data points in the monitoring operation redundant data set. The above method can be used to complete the spatial smoothing of each data point in the monitoring operation redundant data set, thereby obtaining the smoothed data point corresponding to each data point in the monitoring operation redundant data set. The sequence composed of all smoothed data points can be used as the smoothed data sequence of the customer premises equipment during operation.

[0069] It should be noted that based on the data normal vector, data space smoothing factor and feature maintenance factor, the data points in the data space can be effectively smoothed to reduce interference with the overall trend. By constructing a smoothed data sequence, it helps to clearly present the actual operating trends and periodic changes of user premises equipment, avoiding misleading caused by noise and abnormal data.

[0070] In step S4, a data trend graph and a data table of the CPE during operation are generated according to the smoothed data sequence.

[0071] In this embodiment, the smoothed data sequence is input into a data visualization tool to generate a data trend graph of the user premises equipment in runtime; in actual implementation, an existing data visualization tool can be selected to draw the data trend graph of the user premises equipment in runtime, and the data visualization tool selected in this application is Matplotlib, and other data visualization tools can also be selected in actual implementation, which are not limited here, and the data trend graph can be a linear graph, a line graph, and an area graph, etc., which can help the user quickly grasp the overall trend of the equipment and avoid being misled by noise and redundant fluctuations in the data.

[0072] In this embodiment, the smoothed data sequence is input into a dynamic table generation tool to generate a data table of the user premises equipment in runtime; in actual implementation, an existing dynamic table generation tool can be selected to generate the data table of the user premises equipment in runtime, and the dynamic table generation tool selected in this application is Excel, and other dynamic table generation tools can also be selected in actual implementation, which are not limited here, and the data table is mainly used to show the specific values of the user premises equipment at different time points and more detailed performance data.

[0073] It should be noted that generating the data trend graph and the data table based on the smoothed data sequence can significantly improve the visualization effect and quality of the data, and the smoothed data sequence not only has higher accuracy and representativeness, but also can clearly show the real trend of the user premises equipment running, so that the finally generated graph and table are more accurate and stable, and provide strong support for decision-making and optimization.

[0074] As can be seen, in this application, first, the redundant data in the monitored running data can be removed through the redundancy operation; then, the most significant fluctuation point in the data set can be identified through the extraction of the data difference extreme value, and the unstable area or abnormal fluctuation in the data can be identified through the evaluation of the data fluctuation degree, thereby providing a basis for subsequent smoothing processing, the data space smoothing factor can avoid excessive smoothing or excessive retention of fluctuations, ensuring that the data still retains important trends and information while removing redundancy, and the feature maintenance factor can help retain key data fluctuation characteristics in the smoothing process; second, the data points in the data space can be effectively smoothed according to the data normal vector, the data space smoothing factor, and the feature maintenance factor, reducing the interference to the overall trend, and the smoothed data sequence helps to clearly present the real running trend and periodic changes of the user premises equipment, avoiding the misleading caused by noise and abnormal data; finally, the data trend graph and the data table can be generated based on the smoothed data sequence, which can significantly improve the visualization effect and quality of the data.

[0075] In summary, the technical solution adopted in this application can remove redundant data and anomalies in the monitoring operation data to improve the quality of the data charts and data tables finally generated.

[0076] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for automatically drawing a graph and generating a data table based on CPE data, characterized in that: The method comprises the following steps: Extracting monitoring operation data of customer premises equipment, performing redundancy elimination on the monitoring operation data, and obtaining a monitoring operation redundancy elimination data set of the customer premises equipment; Extract data difference extreme values ​​and data volatility from the redundant monitoring data set, determine a data spatial smoothing factor for the redundant monitoring data set based on the data difference extreme values, and determine a feature maintenance factor for the redundant monitoring data set based on the data volatility. The data spatial smoothing factor is an indicator for controlling the degree of spatial smoothing of the redundant monitoring data set, and is determined by the following formula: in, Indicates the data space smoothing factor for the redundant data set of the monitoring operation, Indicates the extreme value of data difference; The feature maintenance factor is an indicator used to maintain the original characteristics of the data when spatially smoothing the redundant monitoring data set. The feature maintenance factor is determined by the following formula: in, represents the feature maintenance factor for the redundant dataset of the monitoring operation, Indicates data volatility; Obtaining a data normal vector of the monitoring operation redundant dataset, and converting the monitoring operation redundant dataset into a smoothed data sequence of the customer premises equipment during operation based on the data normal vector, the data space smoothing factor, and the feature preservation factor; generating a data trend graph and a data table of the customer premises equipment during operation according to the smoothed data sequence; The converting of the monitoring operation redundant data set into a smoothed data sequence of the customer premises equipment during operation based on the data normal vector, the data space smoothing factor, and the feature maintenance factor specifically includes: For each data point in the monitoring operation redundant-elimination data set, spatially smoothing the data point is performed using the data normal vector, the data spatial smoothing factor, and the feature maintenance factor to obtain a smoothed data point corresponding to the data point, and further obtain a smoothed data point corresponding to each data point in the monitoring operation redundant-elimination data set, and a sequence consisting of all smoothed data points is used as a smoothed data sequence of the customer premises equipment during operation.

2. The method for automatically drawing a graph and generating a data table based on CPE data according to claim 1, characterized in that: Deleting the monitoring operation data to obtain a redundant monitoring operation data set of the customer premises equipment specifically includes: Performing anomaly detection on each data point in the monitoring operation data to obtain multiple abnormal data points; All abnormal data points are removed from the monitoring operation data, and the monitoring operation data after the abnormal data points are removed is used as the monitoring operation redundant data set of the customer premises equipment.

3. The method for automatically drawing a graph and generating a data table based on CPE data according to claim 1, wherein: Extracting data difference extreme values ​​and data fluctuations from the redundant data set of the monitoring operation specifically includes: performing data differentiation on the monitoring operation redundant data set to obtain a monitoring operation differential sequence; Extracting data difference extreme values ​​in the monitoring running difference sequence; The data volatility is determined based on the monitoring running difference sequence.

4. The method for automatically drawing a graph and generating a data table based on CPE data according to claim 1, wherein: Obtaining the data normal vector of the monitoring operation redundant data set specifically includes: Standardizing the monitoring and operating redundant data set, and then determining a covariance matrix of the standardized monitoring and operating redundant data set; Performing eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvalues ​​and eigenvectors corresponding to each eigenvalue; The eigenvector corresponding to the maximum eigenvalue is used as the data normal vector of the monitoring operation redundant data set.

5. The method for automatically drawing a graph and generating a data table based on CPE data according to claim 1, wherein: The smoothed data sequence is input into a data visualization tool to generate a data trend graph of the customer premises equipment during operation.

6. The method for automatically drawing a graph and generating a data table based on CPE data according to claim 1, characterized in that: The smoothed data sequence is input into a dynamic table generation tool to generate a data table of the customer premises equipment during operation.

7. The method for automatically drawing a graph and generating a data table based on CPE data according to claim 1, characterized in that: The monitoring operation data of the customer premises equipment is extracted from the log database of the customer premises equipment.

Images