A method and device for calculating mean service quality index with adaptive noise reduction

Abstract

The embodiment of the present application discloses a method and device for calculating a mean service quality index with adaptive noise reduction, the method comprising: obtaining service quality index data; constructing a full sample set from the service quality index data; extracting second data from the first data based on the extreme value of the full sample set to construct an estimated noise set, the size of the estimated noise set being determined according to the proportion of extreme values; removing ultra-high noise data from the estimated noise set according to the proportion of ultra-high noise to obtain third data, and constructing a noise calculation set from the third data; setting a noise threshold according to the statistical result of the third data, using the noise threshold to filter out fourth data from the second data, and constructing a target noise set from the fourth data; using the target noise set to perform noise reduction processing on the first data to obtain a mean calculation set, and calculating the mean of the service quality index based on the mean calculation set. By constructing an estimated noise set, a noise calculation set, and a target noise set, the present application achieves hierarchical screening and precise identification of noise in data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and device for calculating a mean service quality indicator with adaptive noise reduction. Background Art

[0002] In today's digital and information-based environment, the complexity of business systems and the volume of data are increasing rapidly. Traditional statistical methods for service quality indicators often fail to adequately address noise in the data, potentially reducing the accuracy of the calculation results. For example, extreme values, random fluctuations, and environmental interference can significantly impact metrics such as image clarity in video monitoring, bandwidth usage in traffic monitoring, and system load analysis in intelligent operations and maintenance.

[0003] Furthermore, improving user experience has become a key goal of business development. Quality monitoring, which promptly identifies issues and optimizes user experience, is particularly critical in areas such as video services and network traffic services. However, existing noise reduction methods, such as fixed thresholds, simple quantile statistics, and boxplots, often struggle to achieve satisfactory results in complex and changing data scenarios. This is particularly true when performing real-time computations or processing large amounts of data, where excessive computational complexity becomes a significant issue. Summary of the Invention

[0004] The present application provides a method and device for calculating mean service quality indicators with adaptive noise reduction, which realizes hierarchical screening and accurate identification of noise in data.

[0005] This application provides the following solutions:

[0006] According to a first aspect, a method for calculating a mean service quality indicator with adaptive noise reduction is provided, the method comprising: obtaining service quality indicator data; constructing a full sample set from the service quality indicator data, wherein the data in the full sample set is first data; extracting second data from the first data based on the extreme value of the full sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values; excluding ultra-high noise data from the estimated noise set according to the proportion of ultra-high noise to obtain third data, and constructing a noise calculation set from the third data; setting a noise threshold according to the statistical result of the third data, using the noise threshold to filter out fourth data from the second data, and constructing a target noise set from the fourth data; using the target noise set to perform denoising on the first data to obtain a mean calculation set, and calculating the mean of the service quality indicator based on the mean calculation set.

[0007] According to an achievable method in an embodiment of the present application, the second data is extracted from the first data based on the extreme values ​​of the full sample set to construct an estimated noise set, and the size of the estimated noise set is determined according to the proportion of extreme values, including: low-pass filtering, high-pass filtering or band-pass filtering of the first data to obtain extreme value data; according to the extreme value proportion of the extreme value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed based on the second data.

[0008] According to an implementable method in an embodiment of the present application, when the first data is band-pass filtered to obtain extreme value data, the method includes: band-pass filtering the first data to obtain maximum value data and minimum value data; extracting second data from the first data based on the sum of the extreme value proportion of the maximum value data in the first data and the extreme value proportion of the minimum value data in the first data, and constructing an estimated noise set based on the second data.

[0009] According to an achievable method in an embodiment of the present application, extracting the second data from the first data based on the proportion of the extreme value data in the first data includes: extracting the second data from the first data based on the extraction proportion N; wherein the extraction proportion N is determined by the following formula: N=1.3×M, wherein M is the extreme value proportion of the extreme value data in the first data.

[0010] According to an achievable manner in an embodiment of the present application, the method includes: determining the ultra-high noise ratio based on the extreme value ratio.

[0011] According to an implementable method in an embodiment of the present application, setting a noise threshold based on the statistical results of the third data, using the noise threshold to filter out fourth data from the second data, and constructing a target noise set from the fourth data include: calculating the maximum value, minimum value, mean value and noise fluctuation factor of the third data; setting a noise threshold based on the maximum value, minimum value, mean value and noise fluctuation factor; based on the noise threshold, filtering out data in the second data that meets the noise threshold as fourth data, and the fourth data constitutes the target noise set.

[0012] According to an achievable method in an embodiment of the present application, setting the noise threshold according to the maximum value, minimum value, mean value and noise fluctuation factor includes: the noise threshold T It is defined by the following formula: ;in, is the weight of the noise fluctuation factor, is the noise fluctuation factor, is the mean; and adjusting the weight of the noise fluctuation factor according to the formula so that the noise threshold is greater than the minimum value and less than the maximum value.

[0013] According to a second aspect, a device for calculating a mean service quality indicator with adaptive noise reduction is provided, characterized in that the device includes: an indicator data acquisition unit, configured to acquire service quality indicator data; a full sample set construction unit, configured to construct a full sample set from the service quality indicator data, wherein the data in the full sample set is first data; an estimated noise set construction unit, configured to extract second data from the first data based on the extreme values ​​of the full sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values; a noise calculation set construction unit, configured to remove ultra-high noise data from the estimated noise set according to the proportion of ultra-high noise to obtain third data, and construct a noise calculation set from the third data; a target noise set construction unit, configured to set a noise threshold according to a statistical result of the third data, use the noise threshold to filter out fourth data from the second data, and construct a target noise set from the fourth data; a mean calculation unit, configured to use the target noise set to perform noise reduction processing on the first data to obtain a mean calculation set, and calculate the mean of the service quality indicator based on the mean calculation set.

[0014] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods according to the first aspect are implemented.

[0015] According to a fourth aspect, an electronic device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, execute the steps of the method described in any one of the above-mentioned first aspects.

[0016] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0017] (1) This application realizes hierarchical screening and precise identification of noise in data by constructing an estimated noise set, a noise calculation set, and a target noise set. The size of the estimated noise set is dynamically determined based on the proportion of extreme values, so that the noise range initially screened out is sufficient to cover possible anomalies in the data while avoiding including too much normal data, thereby ensuring the efficiency and accuracy of subsequent processing. On this basis, a noise calculation set is constructed by eliminating ultra-high noise data, further narrowing the noise range and optimizing the statistical analysis results of noise characteristics. Finally, the target noise set is screened out from the second data using a noise threshold. The size and range of the target noise set are strictly calculated and dynamically adjusted to ensure that it accurately covers the actual noise area in the data, providing reliable support for the noise reduction processing of the first data, thereby improving the accuracy and stability of the mean calculation of the service quality indicator.

[0018] (2) This application uses low-pass filtering, high-pass filtering, or band-pass filtering to effectively extract extreme value data from the entire sample set and construct an estimated noise set, thereby achieving preliminary identification and screening of potential noise. This process can dynamically adapt to different data distribution characteristics, making the constructed estimated noise set more accurate and laying the foundation for subsequent noise reduction processing.

[0019] (3) Based on bandpass filtering, this application extracts maximum and minimum data separately and dynamically adjusts the construction process of the estimated noise set based on the proportion of their extreme values ​​in the first data. This method can capture the distribution characteristics of noise in a more detailed manner, especially in data environments with asymmetric extreme value distribution, further improving the accuracy of noise identification.

[0020] (4) This application dynamically calculates the extraction ratio N using the formula N=1.3×M, so that the size of the estimated noise set can be automatically adjusted as the extreme value ratio M changes. This dynamic adjustment method avoids the limitations brought by a fixed noise ratio, making the noise screening range neither too large nor too small, thereby improving the rationality and effectiveness of noise removal.

[0021] (5) This application dynamically determines the proportion of extremely high noise based on the proportion of extreme values, allowing for flexible adjustment of noise processing intensity to ensure that extremely high noise data can be accurately removed under different noise distributions. This approach further reduces the interference of extreme values ​​on data analysis and improves the adaptability and effectiveness of noise removal.

[0022] (6) This application sets a noise threshold based on the statistical results of the third data (maximum value, minimum value, mean value, and noise fluctuation factor), which can more accurately define the criteria for distinguishing normal data from noise data during the noise identification process. By dynamically adjusting the threshold range, the fourth data that meets the conditions is screened from the second data, and the target noise set is constructed, making the noise removal process more accurate and stable.

[0023] (7) This application sets the noise threshold through a formula and flexibly adjusts it using the noise fluctuation factor and its weight to ensure that the threshold range is always between the maximum and minimum values. This process further optimizes the identification and processing methods of noise data, enabling more accurate noise removal while retaining normal data that is important for the calculation of service quality indicators.

[0024] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 Flowchart of the method for calculating the mean service quality indicator for adaptive noise reduction provided in an embodiment of the present application;

[0027] Figure 2 is the statistical result of the estimated noise set;

[0028] Figure 3 This is the comparison result before and after noise reduction processing of the network downlink TCP round-trip delay data;

[0029] Figure 4 The change of noise reduction depth in time series under different noise reduction strength conditions;

[0030] Figure 5 (a) shows the change of data fluctuation range before noise reduction;

[0031] Figure 5(b) shows the change of data fluctuation range after noise reduction;

[0032] Figure 6 A structural block diagram of a mean service quality indicator calculation device for adaptive noise reduction provided in an embodiment of the present application;

[0033] Figure 7 A schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0035] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "an", "the" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0036] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0037] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0038] Several existing technologies often suffer from drawbacks such as excessive computational complexity, poor adaptability to data distributions, or unsuitability for real-time computation. For example, quantile statistics, which only counts the best quantile samples, are difficult to determine in practice, resulting in insufficient noise reduction or a significant amount of legitimate data among the samples excluded from the statistics. Threshold denoising statistics, which excludes data exceeding a threshold, can lead to threshold failure when the overall characteristics of the data set change. The Interquartile Range (IQR) boxplot algorithm is overly simplistic and ineffective when data samples exhibit asymmetric distributions, and its fixed threshold factor exhibits significant bias in its adaptability to datasets. The Three Sigma Rule (3σ) is ineffective when data samples are non-Gaussian. The Mahalanobis distance algorithm can exaggerate the effects of variables with small changes during calculations, leading to the omission of important information and an over-reliance on the stability of the covariance matrix. The ABOD (Angle-Based Outlier Detection) algorithm and the Local Outlier Factor (Local Outlier Detection) algorithm are also notoriously difficult to predict. The Outlier Factor (LOF) algorithm has the problem of high algorithm complexity and is not suitable for real-time calculation.

[0039] In view of this, this application provides a new idea. Figure 1The flowchart of the method for calculating the mean service quality index of the adaptive noise reduction provided in the embodiment of the present application is as follows: Figure 1 As shown in , the method may include the following steps:

[0040] Step 101: Obtain service quality indicator data.

[0041] Step 102: construct a full sample set from the service quality indicator data, where the data in the full sample set is the first data.

[0042] Step 103: extracting second data from the first data based on the extreme values ​​of the entire sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values.

[0043] Step 104: based on the proportion of ultra-high noise, remove ultra-high noise data from the estimated noise set to obtain third data, and construct a noise calculation set based on the third data.

[0044] Step 105: setting a noise threshold according to the statistical result of the third data, using the noise threshold to filter out fourth data from the second data, and constructing a target noise set from the fourth data.

[0045] Step 106: Using the target noise set, perform noise reduction processing on the data sample set to obtain a mean calculation set, and calculate the mean of the service quality indicator based on the mean calculation set.

[0046] It can be seen from the above process that the present application realizes the hierarchical screening and precise identification of noise in the data by constructing an estimated noise set, a noise calculation set and a target noise set. The size of the estimated noise set is dynamically determined based on the proportion of extreme values, so that the noise range initially screened out is sufficient to cover possible anomalies in the data while avoiding including too much normal data, thereby ensuring the efficiency and accuracy of subsequent processing. On this basis, a noise calculation set is constructed by eliminating ultra-high noise data, further narrowing the noise range and optimizing the statistical analysis results of the noise characteristics. Finally, the target noise set is screened out from the second data using the noise threshold. The size and range of the target noise set are strictly calculated and dynamically adjusted to ensure that it accurately covers the actual noise area in the data, providing reliable support for the noise reduction processing of the first data, thereby improving the accuracy and stability of the mean calculation of the service quality indicator.

[0047] First, the above step 101, namely "obtaining service quality indicator data", is described in detail with reference to the embodiment.

[0048] Raw data reflecting service quality is obtained from various monitoring devices, systems, or platforms. This data typically covers various aspects of business operations, service quality, and user experience. In practice, service quality indicator data comes from a variety of types and sources, covering different monitoring areas, such as video surveillance, network traffic monitoring, user feedback, system performance, and device health.

[0049] For example, in the field of video surveillance, service quality indicators may include video clarity, latency, frame rate, packet loss rate, etc. By deploying video surveillance equipment in different locations, real-time video quality data can be obtained and analyzed as service quality indicators.

[0050] The above step 102, i.e., "constructing a full sample set from the service quality indicator data, wherein the data in the full sample set is the first data", is described in detail below in conjunction with an embodiment.

[0051] This technical feature refers to extracting all available business quality indicators from multi-dimensional and multi-source data to form a complete data set (full sample set), and treating all data in the set as preliminary raw data (first data) for subsequent processing and analysis.

[0052] This application performs mean calculation on the business quality indicator data. The business indicator data obtained from various monitoring devices, systems or platforms can be directly used as the first data, or some preprocessing operations can be performed on the business indicator data, such as outlier identification, missing value filling, normalization, etc., and then the preprocessed data can be used as the first data.

[0053] Taking traffic monitoring as an example, assuming the monitoring data includes network response time, by aggregating these indicators in chronological order and performing appropriate normalization or standardization, a set of data that comprehensively reflects the network response time can be obtained. This data is considered the first data.

[0054] The above step 103, i.e., "extracting second data from the first data based on the extreme values ​​of the entire sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values", is described in detail below in conjunction with an embodiment.

[0055] Extreme values ​​refer to extremely high or low values ​​in the data. These values ​​usually represent abnormal fluctuations or errors, and are therefore often considered to be potential sources of noise. In specific operations, it is first necessary to find the maximum and minimum values ​​in the entire sample set through statistical analysis. Then, based on these extreme values, the corresponding data points are extracted from the first data to construct an estimated noise set. The data in the estimated noise set represents possible noise, but not all extreme value data are necessarily noise. In order to ensure the accuracy of the noise, the present invention determines the size of the estimated noise set by the proportion of extreme values. Usually, this ratio is a preset parameter used to limit the number or proportion of noise data to avoid too much data being misjudged as noise. For example, this ratio can be a specific value directly between 5% and 8%, preferably, 5% or 6% can be selected.

[0056] As an implementable method, the present application performs low-pass filtering, high-pass filtering or band-pass filtering on the first data to obtain extreme value data; based on the extreme value proportion of the extreme value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed based on the second data.

[0057] Specifically, low-pass filtering (LPF), high-pass filtering (HPF), and band-pass filtering (BPF) are three commonly used filtering methods, each used to address different types of data anomalies. The specific filtering method chosen can be determined based on the specific type of service quality indicator data. Low-pass filtering is suitable for data containing primarily low-frequency variations, while high-frequency variations are considered noise. For example, in video monitoring, overall changes in video quality often exhibit a smooth trend, while sudden, high-frequency fluctuations (such as short-term freezes or delays) may be considered noise. High-pass filtering is suitable for data with low-frequency trends, while high-frequency fluctuations are considered noise. For example, in network traffic monitoring, short-term network fluctuations or sudden traffic changes may be considered noise, while the long-term trend of overall network traffic is more valuable. Band-pass filtering is suitable for data types containing both low-frequency and high-frequency fluctuations and is typically used when a balanced approach to removing low-frequency and high-frequency noise is required. For example, in traffic monitoring, system traffic data may contain both short-term fluctuations (high-frequency noise) and long-term trends (low-frequency noise).

[0058] After applying these filtering methods, the resulting extreme value data represents the data points that were considered anomalous during processing. These extreme value data reflect extreme variations in the data, which are often associated with noise or errors. Next, based on the proportion of these extreme value data points in the first data set, we determine which data points are noise points and extract a second data set, the estimated noise set.

[0059] For example, consider a video surveillance system that records video stream quality metrics (such as clarity, latency, and frame loss rate) every minute. By low-pass filtering to remove frequent spikes and brief fluctuations, we may discover some extremely high or low quality values, which may be caused by device failures or transient network fluctuations. Based on the proportion of these extreme values ​​in the total data (for example, if the maximum and minimum values ​​account for 10% or more), we can extract these extreme values ​​from the first dataset and construct an estimated noise set. The data in this noise set is considered to be anomalous data and will be further removed or subjected to noise reduction.

[0060] When band-pass filtering is performed on the first data to obtain extreme value data, the first data can be band-pass filtered to obtain maximum value data and minimum value data; based on the sum of the extreme value proportion of the maximum value data in the first data and the extreme value proportion of the minimum value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed based on the second data.

[0061] Specifically, first, the first data is band-pass filtered to obtain two important results: maximum data and minimum data. Maximum data usually represents the peak value of the service quality indicator within a certain period of time. These values ​​often represent abnormal fluctuations, such as quality degradation caused by system overload or emergencies. Minimum data represents the trough value of the service quality indicator, which may be due to quality degradation caused by system failure or unforeseen factors. Through band-pass filtering, these maximum and minimum data will be effectively extracted for further analysis. Next, the proportion of maximum data and minimum data in the entire first data is calculated. Based on the above proportion of maximum data and minimum data, the second data is extracted from the first data. This second data set is the estimated noise set, which contains the extreme data part that is considered to be noise. By removing these extreme maximum and minimum data from the original data set, the estimated noise set helps us further eliminate the impact on the service quality indicators, making the subsequent mean calculation more accurate and stable.

[0062] This application extracts second data from the first data based on the proportion of extreme value data in the first data. This can be achieved in various ways. For example, M is the proportion of extreme value data in the first data, and N is the size of the estimated noise set. As an implementable method, N = M + a; a is an adaptive factor, and the specific value can be set according to the type of service quality indicator data.

[0063] Preferably, the second data is extracted from the first data based on an extraction ratio N; wherein the extraction ratio N is determined by the following formula: N = 1.3 × M, where M is the extreme value ratio of the extreme value data in the first data. M is obtained by analyzing the proportion of these extreme values ​​in the first data set. The extraction ratio N is calculated using the formula N = 1.3 × M. The core of this formula is to expand the proportion of extreme value data by the constant 1.3, the purpose of which is to ensure that when the second data is extracted, sufficient extreme value data is included to effectively identify noise and perform further processing. Through this calculation, the extraction ratio N will be slightly larger than the simple extreme value ratio M, thereby ensuring that potential outliers are not missed during data processing.

[0064] For example, consider a set of video surveillance data. Low-pass filtering detects some extreme values ​​and minimum values ​​within the dataset. It is calculated that these extreme values ​​account for 5% of the total data (i.e., M = 0.05). The extraction percentage is calculated using the formula: N = 1.3 × 0.05 = 0.065. Therefore, the extraction percentage N is 6.5%. This means that from the first dataset, a subset containing 6.5% of the data is selected as the second dataset, where the noisy or extreme data is further identified and processed.

[0065] The above step 104, namely "eliminating ultra-high noise data from the estimated noise set according to the ultra-high noise ratio to obtain third data, and constructing a noise calculation set based on the third data" is described in detail below in conjunction with an embodiment.

[0066] This technical feature describes how to further filter and remove noise data during data processing to ensure data quality and accuracy, thereby constructing a more accurate noise calculation set. This process first requires determining the "excessive noise fraction." This fraction refers to the proportion of noise data in a dataset that deviates significantly from the normal data range and may significantly interfere with the results. This data can fluctuate significantly, often manifesting as abnormal extreme values. This may be caused by hardware failure, data collection errors, or other factors.

[0067] The percentage of excessively high noise can be pre-set to a specific value, or it can be flexibly set based on the data volume and data type of the service quality indicator data. Preferably, the percentage of excessively high noise can be dynamically determined based on the extreme value percentage. For example, the extreme value percentage can be directly used as the percentage of excessively high noise, or the extreme value percentage can be expanded or reduced to a value that is used as the percentage of excessively high noise.

[0068] Assuming the percentage of excessively high noise is Y, we remove the data with excessively high noise from the estimated noise set based on the percentage of excessively high noise. This yields the third set of data. This is done by sorting the second set of data and removing the first Y of them. The remaining second set of data is then used as the third set of data, which together form the noise calculation set. This filtering process eliminates the extreme noise compared to the original estimated noise set, making it more realistic and accurately reflecting the actual characteristics of the data.

[0069] The above step 105, i.e., "setting a noise threshold according to the statistical results of the third data, using the noise threshold to filter the second data to obtain fourth data, and constructing a target noise set from the fourth data" is described in detail below in conjunction with an embodiment.

[0070] This technical feature involves how, during the noise processing process, statistical analysis of the third data is performed to set an appropriate noise threshold, thereby filtering out qualified data from the second data and ultimately constructing a target noise set. The statistical results of the third data refer to the third data obtained after filtering out the second data from the first data in the previous step and further eliminating extremely noisy data. This dataset typically contains relatively stable, non-noisy service quality indicator data. By performing statistical analysis on the third data (for example, calculating statistics such as the mean, standard deviation, and variance), we can understand the data distribution and fluctuation, providing a basis for setting the noise threshold.

[0071] The noise threshold is determined based on the statistical results of the third data. The noise threshold can typically be determined in the following ways: Based on the mean and standard deviation: The noise threshold might be set as a multiple of the mean ± standard deviation of the third data. For example, if the data's fluctuation exceeds a certain multiple of the standard deviation, it can be considered noise. Alternatively, based on volatility analysis: If the data's fluctuation exceeds a certain threshold (e.g., a certain range of standard deviations), the data is considered noise.

[0072] As an implementable method, obtaining a target noise set based on a noise calculation set includes: calculating the maximum value, minimum value, mean value and noise fluctuation factor of the third data; setting a noise threshold based on the maximum value, minimum value, mean value and noise fluctuation factor; and based on the noise threshold, screening out data in the second data that meets the noise threshold as fourth data, and the fourth data constitutes the target noise set.

[0073] Specifically, first, the maximum, minimum, mean, and noise fluctuation factor of the third data are calculated. The third data is a relatively stable data set of service quality indicators obtained after preliminary screening. This data is typically preprocessed using low-pass, high-pass, or band-pass filtering, and therefore contains less extreme noise. Statistical analysis of this data reveals its maximum, minimum, and mean. Further analysis of the data's volatility yields the noise fluctuation factor. The noise fluctuation factor generally reflects the degree of data fluctuation; greater fluctuations may indicate a higher noise component. The maximum and minimum values ​​represent the largest and smallest values ​​in the data set, respectively; the mean reflects the central tendency of the data set, or the approximate average level of the data. The noise fluctuation factor can be calculated based on the data's standard deviation or variance and is generally used to measure the magnitude of data fluctuation. Larger data fluctuations indicate a higher noise fluctuation factor, while smaller fluctuations indicate a lower noise fluctuation factor.

[0074] Preferably, the noise fluctuation factor is calculated by the following formula:

[0075] (1)

[0076] Wherein, nc is the number of data of the third data, is the value of the third data, is the mean.

[0077] Next, a noise threshold is set based on the maximum, minimum, mean, and noise fluctuation factor. The noise threshold is set to filter out data that does not conform to a normal fluctuation range based on certain criteria. Typically, the noise threshold is adjusted based on the mean and noise fluctuation factor. For example, the noise threshold can be defined as the mean ± a certain number of standard deviations, or the threshold range can be dynamically set based on the data's fluctuation factor. By adjusting the threshold range, the screening criteria can be flexibly controlled to eliminate anomalous data that falls outside the reasonable fluctuation range.

[0078] Preferably, the noise threshold It is defined by the following formula:

[0079] (2)

[0080] in, is the weight of the noise fluctuation factor, is the noise fluctuation factor, is the mean value;

[0081] The weight of the noise fluctuation factor is adjusted according to the formula so that the noise threshold is greater than the minimum value and less than the maximum value. That is:

[0082] (3)

[0083] The larger the value, the weaker the noise reduction intensity, and the more data samples can be retained; conversely, the greater the noise reduction intensity, the more samples are removed. It is recommended to use noise reduction, and the noise reduction ratio is related to the data distribution.

[0084] The noise threshold is used to filter out the data that meets the conditions from the second data, which is called the fourth data. That is:

[0085] (4)

[0086] in, is the value in the second data, This is the fourth data.

[0087] The fourth data set contains data identified as abnormal after passing the noise threshold. This process further refines data processing, ensuring that only valid, noise-free service quality indicator data is used for subsequent analysis and processing. Ultimately, the resulting fourth data set constitutes the target noise set, which is used for further noise reduction or accurate calculation of service quality indicators.

[0088] The above step 106, i.e., "using the target noise set to perform noise reduction processing on the first data to obtain a mean calculation set, and calculating the mean of the service quality indicator based on the mean calculation set" is described in detail below in conjunction with an embodiment.

[0089] First, the first data is subjected to noise reduction processing using the target noise set. In the previous step, the target noise set was selected using a noise threshold. By applying the target noise set to the first data, the noise components can be removed, making the signal portion of the first data more prominent. Specifically, the data in the target noise set can be directly removed from the first data to form the mean calculation set. Alternatively, the data in the target noise set and its related data (e.g., data before and after it) can be removed from the first data to form the mean calculation set.

[0090] Next, the mean calculation set is obtained. After noise reduction, the noise-free data set becomes the "mean calculation set." This mean calculation set is the noise-reduced data set used to calculate the mean of the service quality indicator. By averaging the data in the mean calculation set, the final mean service quality indicator can be obtained.

[0091] The mean can be calculated using the following formula:

[0092] (5)

[0093] in, is the number of values ​​in the fourth data, s is the number of values ​​in the first data, is the value in the second data, is the value in the fourth data.

[0094] In order to reflect the technical effect of the method proposed in this application, the experimental verification results are shown below.

[0095] Figure 2 To estimate the statistical results of the noise set, including the mean , minimum value , maximum value and data fluctuation range This chart is used to analyze the distribution characteristics of noise and its impact on noise reduction, and to verify the effectiveness of noise identification. Figure 3 The results show that the mean of the estimated noise set is close to the minimum value of the data, indicating that the noise is concentrated in the lower range; the fluctuation value converges, indicating that the distribution of the noise data is limited. Furthermore, the noise is scattered near the upper bound of the data, supporting the theoretical analysis that the majority of the noise is located at the edge of the estimated range. Therefore, the statistical analysis of the estimated noise set supports the effectiveness of the noise reduction analysis method from both theoretical and data perspectives, providing a scientific basis for the subsequent construction of the noise calculation set.

[0096] Figure 3 The comparison results of the network downlink TCP round-trip delay data before and after noise reduction are used to evaluate the impact of the noise reduction algorithm on the delay data, especially the improvement of the degree of proximity between data fluctuation and expected value. The experiment uses the network downlink TCP round-trip delay data as a sample, and the extreme value ratio is set to 5%. Among them, when setting the noise threshold When L1 is The denoised mean data, L2 is The denoised mean data, L3 is = The denoised mean data. The value of can determine the intensity of noise reduction, and The value of is negatively correlated with the noise reduction strength. As can be seen from the figure, after noise reduction, the fluctuations in the latency data are significantly reduced, and the data's temporal trend becomes more stable. Furthermore, the noise-reduced data is numerically closer to the expected value than the unprocessed data. As the noise reduction depth increases, the data approaches the expected value further, indicating that increasing noise reduction strength helps further eliminate the effects of noise. Therefore, noise reduction significantly improves the network's downlink TCP round-trip latency data. Ultimately, the processed data is more stable and closer to actual network performance indicators, providing reliable support for network performance optimization and intelligent operations and maintenance.

[0097] Figure 4 The change of denoising depth in time series under different denoising strengths reflects the adaptability of the adaptive denoising algorithm to different data distributions. The value of L1 is set. The denoised depth data, L2 is The denoised depth data, L3 is Denoising depth data. As the denoising intensity increases, the denoising depth increases accordingly, demonstrating that the algorithm can dynamically adjust the denoising parameters to suit the data characteristics. Furthermore, the fluctuations in denoising depth at different intensities are relatively stable, demonstrating that the algorithm maintains stable performance and avoids over- or under-denoising.

[0098] Figure 5 shows the changes in the data fluctuation range before and after noise reduction, which is used to evaluate the convergence effect of the noise reduction algorithm on data fluctuations. Figure 5 (a) shows the changes in the data fluctuation range before noise reduction; Figure 5 (b) shows the changes in the data fluctuation range after noise reduction. The noise reduction index fluctuation values ​​at this time are significantly reduced. The fluctuation range of the data after noise reduction has significantly converged, indicating that noise interference has been effectively suppressed. Furthermore, the converged data is more concentrated within a reasonable range, significantly reducing the impact of abnormal fluctuations on the calculation of service quality indicators. Therefore, the noise reduction algorithm performs well in reducing the range of data fluctuations, improving the accuracy and stability of service quality indicators.

[0099] The above method provided in the embodiment of the present application can be applied to a variety of application scenarios, including but not limited to: for business fault location, through real-time data analysis and noise processing, to help quickly locate business fault points and improve fault repair efficiency; for user perception analysis, by performing noise reduction processing on user experience related indicator data, accurately calculating user perceived quality, and helping enterprises optimize service quality; for intelligent operation and maintenance, during the operation and maintenance process, automatically identifying and eliminating noise data to achieve efficient and intelligent system operation and maintenance management; for quality monitoring, in scenarios such as traffic monitoring and video monitoring, more accurate quality monitoring results are provided through noise reduction processing, helping decision makers optimize resource allocation and improve service quality.

[0100] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] According to another embodiment, a device for calculating a mean service quality indicator for adaptive noise reduction is provided. Figure 6 A schematic block diagram of a device for calculating the mean service quality indicator of the adaptive noise reduction according to an embodiment is shown, as shown in FIG. Figure 6As shown, the apparatus 600 includes:

[0102] The indicator data acquisition unit 601 is configured to acquire service quality indicator data;

[0103] A full sample set construction unit 602 is configured to construct a full sample set from the service quality indicator data, where the data in the full sample set is first data;

[0104] An estimated noise set construction unit 603 is configured to extract second data from the first data based on the extreme values ​​of the entire sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values;

[0105] The noise calculation set construction unit 604 is configured to remove ultra-high noise data from the estimated noise set according to the ultra-high noise ratio, obtain third data, and construct a noise calculation set based on the third data;

[0106] a target noise set constructing unit 605 configured to set a noise threshold according to a statistical result of the third data, filter the second data using the noise threshold to obtain fourth data, and construct a target noise set based on the fourth data;

[0107] The mean calculation unit 606 is configured to perform noise reduction processing on the first data using the target noise set to obtain a mean calculation set, and calculate the mean of the service quality indicator based on the mean calculation set.

[0108] As an implementable method, the estimated noise set construction unit 603 can be configured to perform low-pass filtering, high-pass filtering or band-pass filtering on the first data to obtain extreme value data; based on the extreme value proportion of the extreme value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed based on the second data.

[0109] As an implementable method, the estimated noise set construction unit 603 can be configured to: perform band-pass filtering on the first data to obtain extreme value data; extract second data from the first data based on the sum of the extreme value proportion of the maximum value data in the first data and the extreme value proportion of the minimum value data in the first data, and construct an estimated noise set based on the second data.

[0110] As an implementable method, the estimated noise set construction unit 603 can be configured to extract the second data from the first data according to the proportion of the extreme value data in the first data as follows: extract the second data from the first data according to the extraction proportion N; wherein the extraction proportion N is determined by the following formula: N=1.3×M, wherein M is the extreme value proportion of the extreme value data in the first data.

[0111] As an implementable manner, the noise calculation set construction unit 604 may be configured to determine the ultra-high noise ratio according to the extreme value ratio.

[0112] As an implementable method, the target noise set construction unit 605 can be configured to: calculate the maximum value, minimum value, mean value and noise fluctuation factor of the third data; set a noise threshold based on the maximum value, minimum value, mean value and noise fluctuation factor; based on the noise threshold, filter out the data in the second data that meets the noise threshold as the fourth data, and the fourth data constitutes the target noise set.

[0113] As an implementable manner, the target noise set construction unit 605 can be configured as follows when setting the noise threshold according to the maximum value, minimum value, mean value and noise fluctuation factor: the noise threshold T It is defined by the following formula: ;in, is the weight of the noise fluctuation factor, is the noise fluctuation factor, is the mean; and adjusting the weight of the noise fluctuation factor according to the formula so that the noise threshold is greater than the minimum value and less than the maximum value.

[0114] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0116] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0117] And an electronic device comprising:

[0118] one or more processors; and

[0119] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

[0120] The present application also provides a computer program product, comprising a computer program, which implements the steps of any one of the methods described in the aforementioned method embodiments when executed by a processor.

[0121] in, Figure 7 The electronic device architecture is shown as an example, and may include a processor 710, a video display adapter 711, a disk drive 712, an input / output interface 713, a network interface 714, and a memory 720. The processor 710, the video display adapter 711, the disk drive 712, the input / output interface 713, the network interface 714, and the memory 720 may be communicatively connected via a communication bus 730.

[0122] The processor 710 may be implemented as a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and may be used to execute relevant programs to implement the technical solutions provided in this application.

[0123] The memory 720 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 720 can store an operating system 721 for controlling the operation of the electronic device 700 and a basic input and output system (BIOS) 722 for controlling the low-level operations of the electronic device 700. In addition, a web browser 723, a data storage management system 724, and a mean service quality indicator calculation device for adaptive noise reduction 725, etc. can also be stored. The above-mentioned mean service quality indicator calculation device for adaptive noise reduction 725 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 720 and is called and executed by the processor 710.

[0124] The input / output interface 713 is used to connect to input / output modules to enable information input and output. The input / output modules can be configured as components within the device (not shown) or externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, and various sensors, while output devices may include a display, speaker, vibrator, indicator light, and the like.

[0125] The network interface 714 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, Wi-Fi, Bluetooth, etc.).

[0126] The bus 730 comprises a pathway for transmitting information between the various components of the device (eg, the processor 710 , the video display adapter 711 , the disk drive 712 , the input / output interface 713 , the network interface 714 , and the memory 720 ).

[0127] It should be noted that although the above device only shows the processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, memory 720, bus 730, etc., in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may also include only the components necessary to implement the solution of the present application, and does not necessarily include all the components shown in the figure.

[0128] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer program product. The computer program product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

[0129] The above is a detailed introduction to the technical solutions provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be understood as limiting this application.

Claims

1. A method for calculating a mean service quality indicator for adaptive noise reduction, characterized in that: The method comprises: Obtain service quality indicator data; Constructing a full sample set from the service quality indicator data, where the data in the full sample set is first data; Extracting second data from the first data based on the extreme values ​​of the entire sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values; Eliminating ultra-high noise data from the estimated noise set according to the ultra-high noise ratio to obtain third data, and constructing a noise calculation set based on the third data; A noise threshold is set according to a statistical result of the third data, fourth data is obtained by filtering the second data using the noise threshold, and a target noise set is constructed from the fourth data. The process of constructing the target noise set from the fourth data includes: Calculating the maximum value, minimum value, mean value, and noise fluctuation factor of the third data; Setting a noise threshold according to the maximum value, minimum value, mean value and noise fluctuation factor; According to the noise threshold, filter out data in the second data that meets the noise threshold as fourth data, and use the fourth data to form the target noise set; Performing noise reduction processing on the first data using the target noise set to obtain a mean calculation set, and calculating a mean of the service quality indicator based on the mean calculation set; Setting the noise threshold according to the maximum value, minimum value, mean value, and noise fluctuation factor includes: The noise threshold T is defined by the following formula: ; in, is the weight of the noise fluctuation factor, is the noise fluctuation factor, is the mean value; Adjusting the weight of the noise fluctuation factor according to the formula so that the noise threshold is greater than the minimum value and less than the maximum value; The noise fluctuation factor is calculated by the following formula: ; Wherein, nc is the number of data of the third data, is the value of the third data, is the mean.

2. The method according to claim 1, wherein extracting second data from the first data based on the extreme values ​​of the full sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values, comprises: performing low-pass filtering, high-pass filtering, or band-pass filtering on the first data to obtain extreme value data; According to the extreme value proportion of the extreme value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed based on the second data.

3. The method according to claim 2, wherein when performing bandpass filtering on the first data to obtain extreme value data, the method comprises: performing band-pass filtering on the first data to obtain maximum value data and minimum value data; According to the sum of the extreme value proportion of the maximum value data in the first data and the extreme value proportion of the minimum value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed according to the second data.

4. The method according to claim 2, characterized in that Extracting second data from the first data according to the proportion of the extreme value data in the first data includes: The second data is extracted from the first data according to the extraction ratio N; wherein the extraction ratio N is determined by the following formula: N=1.3×M, wherein M is the extreme value ratio of the extreme value data in the first data.

5. The method according to claim 3, characterized in that The method comprises: The ultra-high noise ratio is determined according to the extreme value ratio.

6. A mean service quality indicator calculation device for adaptive noise reduction, characterized in that: The device comprises: An indicator data acquisition unit, configured to acquire service quality indicator data; a full sample set construction unit, configured to construct a full sample set from the service quality indicator data, wherein the data in the full sample set is first data; an estimated noise set construction unit, configured to extract second data from the first data based on the extreme values ​​of the full sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values; a noise calculation set construction unit configured to remove ultra-high noise data from the estimated noise set according to the ultra-high noise ratio, obtain third data, and construct a noise calculation set based on the third data; a target noise set construction unit configured to set a noise threshold according to a statistical result of the third data, filter the second data using the noise threshold to obtain fourth data, and construct a target noise set based on the fourth data; The mean calculation unit is configured to perform noise reduction processing on the first data using the target noise set to obtain a mean calculation set, and calculate the mean of the service quality indicator based on the mean calculation set.

7. An electronic device, characterized in that: include: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images