Method for automatically drawing and generating data table based on CPE (Customer Premise Equipment) data

Abstract

The invention provides a method for automatically drawing and generating a data table based on CPE data, and relates to the technical field of data processing, and the method comprises the steps: extracting monitoring operation data of customer premises equipment, carrying out the redundancy of the monitoring operation data, and obtaining a monitoring operation redundancy data set; extracting a data difference extreme value and a data fluctuation degree in the monitoring operation redundancy data set, determining a data space smoothing factor of the monitoring operation redundancy data set according to the data difference extreme value, and determining a feature maintenance factor of the monitoring operation redundancy data set through the data fluctuation degree; acquiring a data normal vector of the monitoring operation redundancy data set, and converting the monitoring operation redundancy data set into a smooth data sequence based on the data normal vector, the data space smoothing factor and the feature maintenance factor; according to the smooth data sequence, the data trend chart and the data table are generated, redundant data and exception in the monitoring operation data can be removed, and the quality of the finally generated data chart and data table is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a method for automatically drawing graphs and generating data tables based on CPE data. Background Art

[0002] The method of automatically drawing graphs and generating data tables based on customer premises equipment (CPE) data is a technical solution that collects, processes and displays equipment operation data in real time, with the aim of reducing manual operations, improving monitoring efficiency, and providing users with intuitive operation data analysis results. This method collects, stores, and analyzes the data generated by CPE devices in real time, and displays the data in the form of charts and tables, providing support for application scenarios such as equipment monitoring, fault detection, and performance analysis.

[0003] However, in actual applications, the monitoring and operation data of user premises equipment is diverse and complex, and the data in the log database is huge and complicated, resulting in data redundancy, making data update and maintenance complicated and prone to inconsistency. In addition, user premises data usually involves multi-level complex relationships, which affects data maintenance and query efficiency. Therefore, how to remove redundant data and anomalies in monitoring and operation data to improve the quality of the final generated data charts and data tables is a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides a method for automatically drawing graphs and generating data tables based on CPE data, which can remove redundant data and anomalies in monitoring operation data to improve the quality of the data charts and data tables finally generated.

[0005] The present application provides a method for automatically drawing a graph based on CPE data and generating a data table, the method comprising the following steps:

[0006] Extracting monitoring operation data of customer premises equipment, deduplicating the monitoring operation data, and obtaining a monitoring operation deduplicated data set of the customer premises equipment;

[0007] Extracting data difference extreme values ​​and data fluctuations from the monitoring operation redundant data set, determining a data space smoothing factor for the monitoring operation redundant data set according to the data difference extreme values, and determining a feature maintenance factor for the monitoring operation redundant data set according to the data fluctuations;

[0008] Acquire a data normal vector of the monitoring operation redundant data set, and convert the monitoring operation redundant data set into a smoothed data sequence of the customer premises equipment when it is running based on the data normal vector, the data space smoothing factor and the feature maintenance factor;

[0009] A data trend graph and a data table of the customer premises equipment during operation are generated according to the smoothed data sequence.

[0010] In this embodiment, the monitoring operation data is deleted redundantly to obtain the monitoring operation redundant data set of the customer premises equipment, specifically including:

[0011] Performing anomaly detection on each data point in the monitoring operation data respectively to obtain multiple abnormal data points;

[0012] All abnormal data points are removed from the monitoring operation data, and the monitoring operation data after the abnormal data points are removed is used as the monitoring operation redundant data set of the customer premises equipment.

[0013] In this embodiment, extracting data difference extreme values ​​and data fluctuations from the monitoring operation redundant data set specifically includes:

[0014] Performing data differentiation on the redundant monitoring operation data set to obtain a monitoring operation differential sequence;

[0015] Extracting data difference extreme values ​​in the monitoring operation difference sequence;

[0016] The data volatility is determined based on the monitoring run difference sequence.

[0017] In this embodiment, the data space smoothing factor is an indicator for controlling the degree of spatial smoothing of the redundant data set of the monitoring operation.

[0018] In this embodiment, the feature preservation factor is an indicator used to maintain the original features of the data when spatial smoothing is performed on the monitoring operation redundant data set.

[0019] In this embodiment, obtaining the data normal vector of the monitoring operation redundant data set specifically includes:

[0020] Standardizing the monitoring operation redundant data set, and then determining a covariance matrix of the standardized monitoring operation redundant data set;

[0021] Performing eigenvalue decomposition on the covariance matrix to obtain multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue;

[0022] The eigenvector corresponding to the maximum eigenvalue is used as the data normal vector of the monitoring operation redundant data set.

[0023] In this embodiment, converting the monitoring operation redundant data set into a smoothed data sequence of the customer premises equipment when it is running based on the data normal vector, the data space smoothing factor and the feature maintenance factor specifically includes:

[0024] For each data point in the monitoring operation redundant data set, spatial smoothing is performed on the data point by using the data normal vector, the data spatial smoothing factor and the feature maintenance factor to obtain a smoothed data point corresponding to the data point, and then a smoothed data point corresponding to each data point in the monitoring operation redundant data set is obtained, and a sequence composed of all smoothed data points is used as a smoothed data sequence of the customer premises equipment during operation.

[0025] In this embodiment, the smoothed data sequence is input into a data visualization tool to generate a data trend graph of the CPE during operation.

[0026] In this embodiment, the smoothed data sequence is input into a dynamic table generation tool to generate a data table of the customer premises equipment during operation.

[0027] In this embodiment, the monitoring operation data of the customer premises equipment is extracted from the log database of the customer premises equipment.

[0028] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects:

[0029] The monitoring operation data of the user premises equipment are extracted and redundantly eliminated to obtain the monitoring operation redundant data set of the user premises equipment; data difference extreme values ​​and data fluctuations are extracted from the monitoring operation redundant data set, a data space smoothing factor for the monitoring operation redundant data set is determined according to the data difference extreme values, and a feature maintenance factor for the monitoring operation redundant data set is determined according to the data fluctuations; a data normal vector of the monitoring operation redundant data set is obtained, and based on the data normal vector, the data space smoothing factor and the feature maintenance factor, the monitoring operation redundant data set is converted into a smoothed data sequence of the user premises equipment when it is in operation; and a data trend graph and a data table of the user premises equipment when it is in operation are generated according to the smoothed data sequence.

[0030] It can be seen that in the present application, firstly, redundant data in the monitoring operation data can be removed by redundant elimination operation; then, by extracting the extreme values ​​of data differences, the most significant fluctuation points in the data set can be identified, and by evaluating the data volatility, unstable areas or abnormal fluctuations in the data can be identified, thereby providing a basis for subsequent smoothing processing, and based on the data space smoothing factor, over-smoothing or over-retention of fluctuations can be avoided, ensuring that the data retains important trends and information while removing redundancy, and based on the feature maintenance factor, it can help retain key data fluctuation characteristics during the smoothing process; secondly, based on the data normal vector, the data space smoothing factor and the feature maintenance factor, the data points in the data space can be effectively smoothed to reduce interference with the overall trend, and by constructing a smoothed data sequence, it helps to clearly present the real operating trend and periodic changes of the user's premises equipment, avoiding misleading caused by noise and abnormal data; finally, generating data trend graphs and data tables based on the smoothed data sequence can significantly improve the visualization effect and quality of the data.

[0031] In summary, the technical solution adopted in this application can remove redundant data and anomalies in the monitoring operation data to improve the quality of the data charts and data tables finally generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0033] Figure 1 It is a flow chart of a method for automatically drawing a graph based on CPE data and generating a data table provided by the present application;

[0034] Figure 2 It is an exemplary flow chart for obtaining data normal vectors of a monitoring operation redundant data set provided by the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and drawings. The schematic implementation modes and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention. It should be noted that the present invention is already in the actual development and use stage.

[0036] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 As shown in the figure, this figure is an exemplary flow chart of a method for automatically drawing a graph based on CPE data and generating a data table according to this embodiment of the present application, and the method includes the following steps:

[0037] In step S1, monitoring operation data of customer premises equipment is extracted, and redundant monitoring operation data is eliminated to obtain a redundant monitoring operation data set of the customer premises equipment.

[0038] In specific implementation, the monitoring operation data of the user premises equipment can be extracted from the log database of the user premises equipment through a data query language; it should be noted that in this application, the monitoring operation data refers to the memory usage data of the user premises equipment during operation.

[0039] In this embodiment, the monitoring operation data is deleted redundantly to obtain the monitoring operation redundant data set of the customer premises equipment in the following manner, namely:

[0040] Performing anomaly detection on each data point in the monitoring operation data respectively to obtain multiple abnormal data points;

[0041] All abnormal data points are removed from the monitoring operation data, and the monitoring operation data after the abnormal data points are removed is used as the monitoring operation redundant data set of the customer premises equipment.

[0042] In specific implementation, first, anomaly detection can be performed on each data point in the monitoring operation data. The goal of anomaly detection is to identify and eliminate data points that are significantly inconsistent with normal equipment operation behavior. Usually, abnormal data points are caused by equipment failure, sensor errors, network problems or other factors. A machine learning-based anomaly detection model can be used to perform anomaly detection on each data point in the monitoring operation data, so that multiple abnormal data points can be obtained through anomaly detection; then, all abnormal data points can be eliminated from the monitoring operation data, so that the monitoring operation data after eliminating the abnormal data points is used as the monitoring operation redundant data set of the user's premises equipment; it should be noted that after anomaly detection and abnormal data elimination, the final monitoring operation redundant data set will contain more accurate and noise-free monitoring operation data.

[0043] In step S2, data difference extreme values ​​and data fluctuations are extracted from the monitoring operation redundant data set, a data space smoothing factor for the monitoring operation redundant data set is determined according to the data difference extreme values, and a feature maintenance factor for the monitoring operation redundant data set is determined according to the data fluctuations.

[0044] In this embodiment, the following method can be used to extract data difference extreme values ​​and data fluctuations from the redundant data set in the monitoring operation, namely:

[0045] Performing data differentiation on the redundant monitoring operation data set to obtain a monitoring operation differential sequence;

[0046] Extracting data difference extreme values ​​in the monitoring operation difference sequence;

[0047] The data volatility is determined based on the monitoring run difference sequence.

[0048] In specific implementation, first, data differentiation can be performed on the monitoring operation redundant data set, that is, all data points in the monitoring operation redundant data set are first-order differentiated to obtain multiple monitoring operation differential values, that is, the difference between adjacent data points, and the sequence composed of all monitoring operation differential values ​​can be used as the monitoring operation differential sequence; then, the data difference extreme value can be extracted from the monitoring operation differential sequence, wherein the data difference extreme value is the maximum value in the monitoring operation differential sequence, and the data difference extreme value can be extracted from the monitoring operation differential sequence by traversal; finally, the data volatility can be determined according to the monitoring operation differential sequence, wherein the data volatility represents the data volatility degree of the monitoring operation differential sequence. In actual implementation, the data volatility can be determined according to the following formula:

[0049]

[0050] Among them, λ represents the data volatility, m represents the total number of data in the monitoring running difference sequence, and y j represents the jth monitoring operation difference value in the monitoring operation difference sequence, and μ represents the mean of all monitoring operation difference values ​​in the monitoring operation difference sequence.

[0051] In this embodiment, a data space smoothing factor for the monitoring operation redundant data set is determined according to the data difference extreme value, wherein the data space smoothing factor is an indicator for controlling the degree of spatial smoothing of the monitoring operation redundant data set. In actual implementation, the data space smoothing factor can be determined by the following formula:

[0052]

[0053] Among them, α represents the data space smoothing factor of the redundant data set of the monitoring operation, and κ represents the extreme value of data difference.

[0054] In this embodiment, the feature maintenance factor of the monitoring operation redundant data set is determined by the data volatility, wherein the feature maintenance factor is an indicator for maintaining the original characteristics of the data when the monitoring operation redundant data set is spatially smoothed. In actual implementation, the feature maintenance factor can be determined by the following formula:

[0055]

[0056] Among them, β represents the feature maintenance factor of the redundant data set for monitoring operation, and λ represents the data volatility.

[0057] It should be noted that by extracting the extreme values ​​of data differences, the most significant fluctuation points in the data set can be identified. By evaluating the data volatility, unstable areas or abnormal fluctuations in the data can be identified, thereby providing a basis for subsequent smoothing processing. Based on the data space smoothing factor, over-smoothing or over-retention of fluctuations can be avoided, ensuring that the data retains important trends and information while removing redundancy. Based on the feature maintenance factor, it can help retain key data fluctuation characteristics during the smoothing process.

[0058] In step S3, a data normal vector of the monitoring operation redundant data set is obtained, and based on the data normal vector, the data space smoothing factor and the feature maintenance factor, the monitoring operation redundant data set is converted into a smoothed data sequence of the customer premises equipment during operation.

[0059] Preferably, in this embodiment, reference Figure 2 As shown, this figure is an exemplary flow chart of obtaining the data normal vector of the monitoring operation redundant data set in an embodiment of the present application. In this embodiment, obtaining the data normal vector of the monitoring operation redundant data set can be specifically implemented by the following steps:

[0060] First, in step S31, the monitoring operation redundant data set is standardized, and then the covariance matrix of the standardized monitoring operation redundant data set is determined;

[0061] Then, in step S32, the covariance matrix is ​​subjected to eigenvalue decomposition to obtain a plurality of eigenvalues ​​and eigenvectors corresponding to each eigenvalue;

[0062] Finally, in step S33, the eigenvector corresponding to the maximum eigenvalue is used as the data normal vector of the monitoring operation redundant data set.

[0063] In the specific implementation, first, the monitoring operation redundant data set can be standardized. The purpose of standardization is to eliminate the scale differences of different dimensions in the data so that all features can be compared under the same dimension, so that the covariance matrix of the standardized monitoring operation redundant data set can be calculated; then, the covariance matrix can be decomposed by eigenvalues ​​using existing technology to obtain multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue; finally, the eigenvector corresponding to the maximum eigenvalue can be used as the data normal vector of the monitoring operation redundant data set, wherein the data normal vector is an important eigenvector that describes the change trend of the data set. Through the data normal vector, the main change direction in the data can be captured, especially in high-dimensional space, the data normal vector can effectively guide subsequent smoothing, denoising and feature retention and other processing; it should be noted that the eigenvector corresponding to the largest eigenvalue represents the main direction of change of the data set.

[0064] In this embodiment, the monitoring operation redundant data set is converted into a smoothed data sequence of the customer premises equipment when it is running based on the data normal vector, the data space smoothing factor and the feature maintenance factor, which can be specifically implemented in the following manner, namely:

[0065] For each data point in the monitoring operation redundant data set, spatial smoothing is performed on the data point by using the data normal vector, the data spatial smoothing factor and the feature maintenance factor to obtain a smoothed data point corresponding to the data point, and then a smoothed data point corresponding to each data point in the monitoring operation redundant data set is obtained, and a sequence composed of all smoothed data points is used as a smoothed data sequence of the customer premises equipment during operation.

[0066] In specific implementation, for each data point in the redundant data set of the monitoring operation, the data point is spatially smoothed by the data normal vector, the data space smoothing factor and the feature maintenance factor to obtain the smoothed data point corresponding to the data point. In actual implementation, the smoothed data point corresponding to the data point can be determined by the following formula:

[0067]

[0068] Among them, x′ i represents the smoothed data point corresponding to the data point, x i represents the i-th data point in the monitoring run redundant data set, x j represents the jth data point in the monitoring operation redundant data set, α represents the data space smoothing factor of the monitoring operation redundant data set, β represents the feature maintenance factor of the monitoring operation redundant data set, a represents the data normal vector, and n represents the total number of data points in the monitoring operation redundant data set. The above method can be used to complete the spatial smoothing of each data point in the monitoring operation redundant data set, so as to obtain the smoothed data point corresponding to each data point in the monitoring operation redundant data set, and the sequence composed of all smoothed data points can be used as the smoothed data sequence of the user premises equipment during operation.

[0069] It should be noted that, based on the data normal vector, data space smoothing factor and feature maintenance factor, the data points in the data space can be effectively smoothed to reduce interference with the overall trend. By constructing a smooth data sequence, it helps to clearly present the real operating trend and periodic changes of the user's premises equipment, avoiding misleading caused by noise and abnormal data.

[0070] In step S4, a data trend graph and a data table of the CPE during operation are generated according to the smoothed data sequence.

[0071] In this embodiment, the smoothed data sequence is input into a data visualization tool to generate a data trend graph of the user premises equipment during operation. In specific implementation, an existing data visualization tool can be selected to draw a data trend graph of the user premises equipment during operation. The data visualization tool selected in this application is Matplotlib. Other data visualization tools can also be selected in actual implementation. There is no limitation here. The data trend graph can be a linear graph, a line graph, an area graph, etc. The data trend graph can help users quickly grasp the overall trend of the equipment and avoid being misled by noise and redundant fluctuations in the data.

[0072] In this embodiment, the smoothed data sequence is input into a dynamic table generation tool to generate a data table of the user premises equipment when it is running; in specific implementation, an existing dynamic table generation tool can be selected to generate a data table of the user premises equipment when it is running. The dynamic table generation tool selected in this application is Excel. Other dynamic table generation tools can also be selected in actual implementation. There is no limitation here. The data table is mainly used to display specific values ​​of the user premises equipment at different time points, as well as more detailed performance data.

[0073] It should be noted that generating data trend graphs and data tables based on smoothed data sequences can significantly improve the visualization and quality of data. Smoothed data sequences not only have higher accuracy and representativeness, but can also clearly show the true trend of user premises equipment operation, making the final generated graphs and tables more accurate and stable, and providing strong support for decision-making and optimization.

[0074] It can be seen that in the present application, firstly, redundant data in the monitoring operation data can be removed by redundant elimination operation; then, by extracting the extreme values ​​of data differences, the most significant fluctuation points in the data set can be identified, and by evaluating the data volatility, unstable areas or abnormal fluctuations in the data can be identified, thereby providing a basis for subsequent smoothing processing, and based on the data space smoothing factor, over-smoothing or over-retention of fluctuations can be avoided, ensuring that the data retains important trends and information while removing redundancy, and based on the feature maintenance factor, it can help retain key data fluctuation characteristics during the smoothing process; secondly, based on the data normal vector, the data space smoothing factor and the feature maintenance factor, the data points in the data space can be effectively smoothed to reduce interference with the overall trend, and by constructing a smoothed data sequence, it helps to clearly present the real operating trend and periodic changes of the user's premises equipment, avoiding misleading caused by noise and abnormal data; finally, generating data trend graphs and data tables based on the smoothed data sequence can significantly improve the visualization effect and quality of the data.

[0075] In summary, the technical solution adopted in this application can remove redundant data and anomalies in the monitoring operation data to improve the quality of the data charts and data tables finally generated.

[0076] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for automatically drawing a graph and generating a data table based on CPE data, characterized in that: The method comprises the following steps: Extracting monitoring operation data of customer premises equipment, deduplicating the monitoring operation data, and obtaining a monitoring operation deduplicated data set of the customer premises equipment; Extracting data difference extreme values ​​and data fluctuations from the monitoring operation redundant data set, determining a data space smoothing factor for the monitoring operation redundant data set according to the data difference extreme values, and determining a feature maintenance factor for the monitoring operation redundant data set according to the data fluctuations; Acquire a data normal vector of the monitoring operation redundant data set, and convert the monitoring operation redundant data set into a smoothed data sequence of the customer premises equipment when it is running based on the data normal vector, the data space smoothing factor and the feature maintenance factor; A data trend graph and a data table of the customer premises equipment during operation are generated according to the smoothed data sequence.

2. A method for automatically drawing a graph and generating a data table based on CPE data as claimed in claim 1, characterized in that: Deleting the monitoring operation data to obtain a monitoring operation deleting data set of the customer premises equipment specifically includes: Performing anomaly detection on each data point in the monitoring operation data respectively to obtain multiple abnormal data points; All abnormal data points are removed from the monitoring operation data, and the monitoring operation data after the abnormal data points are removed is used as the monitoring operation redundant data set of the customer premises equipment.

3. A method for automatically drawing a graph and generating a data table based on CPE data as claimed in claim 1, characterized in that: Extracting data difference extreme values ​​and data fluctuations from the redundant data set of the monitoring operation specifically includes: Performing data differentiation on the redundant monitoring operation data set to obtain a monitoring operation differential sequence; Extracting data difference extreme values ​​in the monitoring operation difference sequence; The data volatility is determined based on the monitoring run difference sequence.

4. A method for automatically drawing a graph and generating a data table based on CPE data as claimed in claim 1, characterized in that: The data space smoothing factor is an indicator used to control the degree of spatial smoothing of the redundant data set of the monitoring operation.

5. A method for automatically drawing a graph and generating a data table based on CPE data as claimed in claim 1, characterized in that: The feature maintenance factor is an indicator used to maintain the original features of the data when spatial smoothing is performed on the monitoring operation redundant data set.

6. A method for automatically drawing a graph and generating a data table based on CPE data as claimed in claim 1, characterized in that: The step of obtaining the data normal vector of the monitoring operation redundant data set specifically includes: Standardizing the monitoring operation redundant data set, and then determining a covariance matrix of the standardized monitoring operation redundant data set; Performing eigenvalue decomposition on the covariance matrix to obtain multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue; The eigenvector corresponding to the maximum eigenvalue is used as the data normal vector of the monitoring operation redundant data set.

7. A method for automatically drawing a graph and generating a data table based on CPE data as claimed in claim 1, characterized in that: Converting the monitoring operation redundant data set into a smoothed data sequence of the customer premises equipment when it is running based on the data normal vector, the data space smoothing factor and the feature maintenance factor specifically includes: For each data point in the monitoring operation redundant data set, spatial smoothing is performed on the data point by using the data normal vector, the data spatial smoothing factor and the feature maintenance factor to obtain a smoothed data point corresponding to the data point, and then a smoothed data point corresponding to each data point in the monitoring operation redundant data set is obtained, and a sequence composed of all smoothed data points is used as a smoothed data sequence of the customer premises equipment during operation.

8. A method for automatically drawing a graph and generating a data table based on CPE data as claimed in claim 1, characterized in that: The smoothed data sequence is input into a data visualization tool to generate a data trend graph of the customer premises equipment during operation.

9. A method for automatically drawing a graph and generating a data table based on CPE data as claimed in claim 1, characterized in that: The smoothed data sequence is input into a dynamic table generation tool to generate a data table of the customer premises equipment during operation.

10. A method for automatically drawing a graph and generating a data table based on CPE data as claimed in claim 1, characterized in that: The monitoring operation data of the customer premises equipment is extracted from the log database of the customer premises equipment.

Images