Mean service quality index calculation method and device for adaptive noise reduction

Abstract

The embodiment of the invention discloses a method and a device for calculating a mean service quality index of adaptive noise reduction. The method comprises the following steps: acquiring service quality index data; constructing a full sample set by the service quality index data; extracting second data from the first data based on the extreme value of the full sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the ratio of the extreme value; according to the ultra-high noise ratio, eliminating ultra-high noise data from the pre-estimated noise set to obtain third data, and constructing a noise calculation set by the third data; setting a noise threshold according to a statistical result of the third data, screening the second data by using the noise threshold to obtain fourth data, and constructing a target noise set by the fourth data; and performing noise reduction processing on the first data by using the target noise set to obtain a mean value calculation set, and calculating a mean value of the service quality indexes based on the mean value calculation set. According to the method, the pre-estimation noise set, the noise calculation set and the target noise set are constructed, so that layered screening and accurate recognition of the noise in the data are realized.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and device for calculating a mean service quality indicator with adaptive noise reduction. Background Art

[0002] In the current digital and information environment, the complexity and data volume of business systems are increasing day by day. The statistical methods of traditional business quality indicators usually do not fully deal with the noise in the data, which may lead to a decrease in the accuracy of the calculation results. For example, extreme values, random fluctuations and environmental interference will have a significant impact on indicators in areas such as picture clarity indicators in video monitoring, bandwidth usage indicators in traffic monitoring, and system load analysis indicators in intelligent operation and maintenance.

[0003] In addition, improving user perception has become an important goal of business development. Timely discovering problems and optimizing user experience through quality monitoring is particularly critical for video services, network traffic services and other fields. However, existing noise reduction methods such as fixed threshold methods, simple quantile statistics and box-and-whisker plots are usually difficult to achieve satisfactory results in complex and changing data scenarios, especially in real-time computing or large-scale data processing, where the problem of high computational complexity is more prominent. Summary of the invention

[0004] The present application provides a method and device for calculating a mean service quality indicator with adaptive noise reduction, which realizes hierarchical screening and accurate identification of noise in data.

[0005] This application provides the following solutions: According to a first aspect, a method for calculating a mean service quality indicator with adaptive noise reduction is provided, the method comprising: obtaining service quality indicator data; constructing a full sample set from the service quality indicator data, the data in the full sample set being first data; extracting second data from the first data based on the extreme value of the full sample set to construct an estimated noise set, the size of the estimated noise set being determined according to the proportion of extreme values; according to the proportion of ultra-high noise, removing ultra-high noise data from the estimated noise set to obtain third data, and constructing a noise calculation set from the third data; setting a noise threshold according to the statistical result of the third data, using the noise threshold to filter out fourth data from the second data, and constructing a target noise set from the fourth data; using the target noise set to perform noise reduction processing on the first data to obtain a mean calculation set, and calculating the mean of the service quality indicator based on the mean calculation set.

[0006] According to an achievable method in an embodiment of the present application, the second data is extracted from the first data based on the extreme values ​​of the full sample set to construct an estimated noise set, and the size of the estimated noise set is determined according to the proportion of extreme values, including: low-pass filtering, high-pass filtering or band-pass filtering of the first data to obtain extreme value data; according to the proportion of extreme values ​​of the extreme value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed based on the second data.

[0007] According to an achievable method in an embodiment of the present application, when the first data is band-pass filtered to obtain extreme value data, the method includes: band-pass filtering the first data to obtain maximum value data and minimum value data; extracting second data from the first data based on the sum of the extreme value proportion of the maximum value data in the first data and the extreme value proportion of the minimum value data in the first data, and constructing an estimated noise set based on the second data.

[0008] According to an achievable method in an embodiment of the present application, extracting second data from the first data according to the proportion of the extreme value data in the first data includes: extracting second data from the first data according to an extraction proportion N; wherein the extraction proportion N is determined by the following formula: N=1.3×M, wherein M is the extreme value proportion of the extreme value data in the first data.

[0009] According to an achievable manner in an embodiment of the present application, the method includes: the ultra-high noise proportion is determined according to the extreme value proportion.

[0010] According to an implementable method in an embodiment of the present application, the noise threshold is set according to the statistical result of the third data, the fourth data is obtained by filtering out the second data using the noise threshold, and the target noise set is constructed from the fourth data, including: calculating the maximum value, minimum value, mean value and noise fluctuation factor of the third data; setting the noise threshold according to the maximum value, minimum value, mean value and noise fluctuation factor; according to the noise threshold, filtering out the data in the second data that meets the noise threshold as the fourth data, and the fourth data constitutes the target noise set.

[0011] According to an achievable method in an embodiment of the present application, setting the noise threshold according to the maximum value, minimum value, mean value and noise fluctuation factor includes: the noise threshold T It is defined by the following formula: ;in, is the weight of the noise fluctuation factor, is the noise fluctuation factor, is the mean; and the weight of the noise fluctuation factor is adjusted according to the formula so that the noise threshold is greater than the minimum value and less than the maximum value.

[0012] According to a second aspect, a mean service quality indicator calculation device with adaptive noise reduction is provided, characterized in that the device comprises: an indicator data acquisition unit, configured to acquire service quality indicator data; a full sample set construction unit, configured to construct a full sample set from the service quality indicator data, wherein the data in the full sample set is first data; an estimated noise set construction unit, configured to extract second data from the first data based on the extreme value of the full sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the extreme value proportion; a noise calculation set construction unit, configured to remove ultra-high noise data from the estimated noise set according to the ultra-high noise proportion to obtain third data, and construct a noise calculation set from the third data; a target noise set construction unit, configured to set a noise threshold according to a statistical result of the third data, use the noise threshold to filter out fourth data from the second data, and construct a target noise set from the fourth data; a mean calculation unit, configured to use the target noise set to perform noise reduction processing on the first data to obtain a mean calculation set, and calculate the mean of the service quality indicator based on the mean calculation set.

[0013] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0014] According to a fourth aspect, an electronic device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, and the program instructions, when read and executed by the one or more processors, execute the steps of any one of the methods described in the first aspect.

[0015] According to the specific embodiments provided in this application, this application discloses the following technical effects: (1) This application realizes hierarchical screening and precise identification of noise in data by constructing an estimated noise set, a noise calculation set and a target noise set. The size of the estimated noise set is dynamically determined based on the proportion of extreme values, so that the noise range initially screened out is sufficient to cover possible anomalies in the data while avoiding including too much normal data, thereby ensuring the efficiency and accuracy of subsequent processing. On this basis, a noise calculation set is constructed by eliminating ultra-high noise data, further narrowing the noise range and optimizing the statistical analysis results of noise characteristics. Finally, the target noise set is screened out from the second data using the noise threshold. The size and range of the target noise set are strictly calculated and dynamically adjusted to ensure that it accurately covers the actual noise area in the data, providing reliable support for the noise reduction processing of the first data, thereby improving the accuracy and stability of the mean calculation of the service quality indicators.

[0016] (2) This application uses low-pass filtering, high-pass filtering or band-pass filtering to effectively extract extreme value data from the entire sample set and construct an estimated noise set, thereby achieving preliminary identification and screening of potential noise. This process can dynamically adapt to different data distribution characteristics, making the constructed estimated noise set more accurate, laying the foundation for subsequent noise reduction processing.

[0017] (3) Based on bandpass filtering, the present application extracts maximum value data and minimum value data respectively, and dynamically adjusts the construction process of the estimated noise set according to the proportion of their extreme values ​​in the first data. This method can capture the distribution characteristics of noise in a more detailed manner, especially in a data environment with asymmetric extreme value distribution, further improving the accuracy of noise identification.

[0018] (4) This application dynamically calculates the extraction ratio N through the formula N=1.3×M, so that the size of the estimated noise set can be automatically adjusted as the extreme value ratio M changes. This dynamic adjustment method avoids the limitations brought by the fixed noise ratio, making the noise screening range neither too large nor too small, thereby improving the rationality and effectiveness of noise removal.

[0019] (5) This application dynamically determines the proportion of extremely high noise based on the proportion of extreme values, and can flexibly adjust the intensity of noise processing to ensure that extremely high noise data can be accurately removed under different noise distributions. This method further reduces the interference of extreme values ​​on data analysis and improves the adaptability and effectiveness of noise removal.

[0020] (6) The present application sets the noise threshold based on the statistical results (maximum value, minimum value, mean value and noise fluctuation factor) of the third data, which can more accurately define the distinction criteria between normal data and noise data in the noise identification process. By dynamically adjusting the threshold range, the fourth data that meets the conditions is screened out from the second data, and the target noise set is constructed, so that the noise removal process has higher accuracy and stability.

[0021] (7) This application sets the noise threshold through a formula and uses the noise fluctuation factor and its weight to flexibly adjust it to ensure that the threshold range is always between the maximum and minimum values. This process further optimizes the identification and processing methods of noise data, and can more accurately remove noise while retaining normal data that is important for the calculation of service quality indicators.

[0022] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A flowchart of a method for calculating a mean service quality indicator for adaptive noise reduction provided in an embodiment of the present application; Figure 2 is the statistical result of estimating the noise set; Figure 3 This is the comparison result before and after the noise reduction processing of the network downlink TCP round-trip delay data; Figure 4 The change of denoising depth in time series under different denoising strengths; Figure 5 (a) shows the change of data fluctuation range before noise reduction; Figure 5 (b) shows the change of data fluctuation range after noise reduction; Figure 6 A structural block diagram of a mean service quality indicator calculation device for adaptive noise reduction provided in an embodiment of the present application; Figure 7 A schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0026] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0027] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0028] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0029] At present, there are some existing technologies, which usually have defects such as excessive computational complexity, poor adaptability to data distribution or unsuitability for real-time calculation. For example, quantile statistics, that is, only the best quantile samples are counted, but in reality, it is difficult to determine the appropriate quantile, either the noise reduction is insufficient, or there are still a large number of data in the samples that do not participate in the statistics that belong to reasonable samples; threshold noise reduction statistics, this method is to exclude data that exceeds the threshold from the statistics, and the threshold will fail when the overall characteristics of the data population sample change; the box plot (Interquartile Range, IQR) algorithm is too simple, and the algorithm is not ideal when the data samples are asymmetric, and its fixed threshold factor has a relatively large deviation in the adaptability of the data set; the three sigma rule is not ideal when the data samples are non-Gaussian distributed; the Mahalanobis distance algorithm may exaggerate the role of variables with small changes in the calculation process, resulting in the neglect of some important information in the analysis and over-reliance on the stability of the covariance matrix; the ABOD algorithm (Angle-Based Outlier Detection) and the local outlier factor (Local The Outlier Factor (LOF) algorithm has the disadvantage that the algorithm complexity is too high and is not suitable for real-time calculation.

[0030] In view of this, the present application provides a new idea. Figure 1The flowchart of the method for calculating the mean service quality index of the adaptive noise reduction provided in the embodiment of the present application is as follows: Figure 1 As shown in , the method may include the following steps: Step 101: Obtain service quality indicator data.

[0031] Step 102: construct a full sample set from the service quality indicator data, where the data in the full sample set is the first data.

[0032] Step 103: extracting second data from the first data based on the extreme values ​​of the entire sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values.

[0033] Step 104: according to the proportion of ultra-high noise, remove ultra-high noise data from the estimated noise set to obtain third data, and construct a noise calculation set based on the third data.

[0034] Step 105: setting a noise threshold according to the statistical result of the third data, using the noise threshold to filter out fourth data from the second data, and constructing a target noise set from the fourth data.

[0035] Step 106: Use the target noise set to perform noise reduction processing on the data sample set to obtain a mean calculation set, and calculate the mean of the service quality indicator based on the mean calculation set.

[0036] It can be seen from the above process that the present application realizes the hierarchical screening and precise identification of noise in the data by constructing an estimated noise set, a noise calculation set and a target noise set. The size of the estimated noise set is dynamically determined based on the proportion of extreme values, so that the noise range initially screened out is sufficient to cover the anomalies that may exist in the data, and avoids including too much normal data, thereby ensuring the efficiency and accuracy of subsequent processing. On this basis, a noise calculation set is constructed by eliminating ultra-high noise data, further narrowing the noise range and optimizing the statistical analysis results of noise characteristics. Finally, the target noise set is screened out from the second data using the noise threshold. The size and range of the target noise set are strictly calculated and dynamically adjusted to ensure that it accurately covers the actual noise area in the data, providing reliable support for the noise reduction processing of the first data, thereby improving the accuracy and stability of the mean calculation of the service quality indicators.

[0037] First, the above step 101, namely "obtaining service quality indicator data", is described in detail in conjunction with the embodiment.

[0038] Obtain raw data reflecting service quality from various monitoring devices, systems or platforms. These data usually involve various aspects of business operations, service quality and user experience. In actual applications, the types and sources of service quality indicator data are diverse, covering different monitoring areas, such as video surveillance, network traffic monitoring, user feedback, system performance, equipment health, etc.

[0039] For example, in the field of video surveillance, service quality indicators may include video clarity, delay, frame rate, packet loss rate, etc. By deploying video surveillance equipment in different locations, real-time video quality data can be obtained and used as service quality indicators for analysis.

[0040] The above step 102, namely "constructing a full sample set from the service quality indicator data, wherein the data in the full sample set is the first data", is described in detail below in conjunction with an embodiment.

[0041] This technical feature refers to extracting all available business quality indicators from multi-dimensional and multi-source data to form a complete data set (full sample set), and treating all data in the set as preliminary raw data (first data) for subsequent processing and analysis.

[0042] The present application performs mean calculation on the business quality indicator data, and the business indicator data obtained from various monitoring devices, systems or platforms can be directly used as the first data, or some preprocessing operations can be performed on the business indicator data, such as outlier identification, missing value completion, normalization, etc., and then the preprocessed data can be used as the first data.

[0043] Taking traffic monitoring as an example, assuming that the monitoring data includes network response time, by aggregating these indicator data in chronological order and performing appropriate normalization or standardization, a set of data that fully reflects the network response time can be obtained, and these data are regarded as the first data.

[0044] The above step 103, i.e., "extracting second data from the first data based on the extreme values ​​of the entire sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values", is described in detail below in conjunction with an embodiment.

[0045] Extreme values ​​refer to extremely high or low values ​​in the data, which usually represent abnormal fluctuations or errors, and are therefore often considered to be potential sources of noise. In specific operations, it is first necessary to find the maximum and minimum values ​​in the entire sample set through statistical analysis. Then, based on these extreme values, corresponding data points are extracted from the first data to construct an estimated noise set. The data in the estimated noise set represents possible noise, but not all extreme value data are necessarily noise. In order to ensure the accuracy of the noise, the present invention determines the size of the estimated noise set by the proportion of extreme values. Usually, this ratio is a preset parameter used to limit the number or proportion of noise data to avoid too much data being misjudged as noise. For example, this ratio can be a specific value directly between 5% and 8%, preferably, 5% or 6% can be selected.

[0046] As an implementable method, the present application performs low-pass filtering, high-pass filtering or band-pass filtering on the first data to obtain extreme value data; based on the extreme value proportion of the extreme value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed based on the second data.

[0047] Specifically, low-pass filtering (LPF), high-pass filtering (HPF) and band-pass filtering (BPF) are three commonly used filtering methods, which are used to handle different types of data anomalies. The specific filtering method can be set according to the specific type of service quality indicator data. Low-pass filtering is suitable for situations where the data mainly contains low-frequency changes, while high-frequency changes are considered to be noise. For example, in video monitoring, the overall change of video quality often shows a smooth trend, while sudden high-frequency fluctuations (such as short-term screen freezes or delays) may be noise. High-pass filtering is suitable for situations where there are low-frequency trend changes in the data, while high-frequency fluctuations are considered to be noise. For example, in network traffic monitoring, some short-term network fluctuations or sudden traffic changes may be noise, while the long-term change trend of the overall network traffic is more valuable for reference. Band-pass filtering is suitable for data types that contain both low-frequency and high-frequency fluctuations, and is usually used in situations where it is necessary to balance the removal of low-frequency and high-frequency noise. For example, in traffic monitoring, the system's traffic data may contain both short-term fluctuations (high-frequency noise) and long-term trend changes (low-frequency noise).

[0048] After applying these filtering methods, the extreme data obtained represent the parts that are considered abnormal during the processing. These extreme data reflect the extreme changes in the data, which are usually related to noise or errors. Next, based on the proportion of these extreme data in the first data, it is determined which data points are noise, and the second data set, namely the estimated noise set, is extracted.

[0049] For example, suppose in a video monitoring system, the system records the quality indicators of the video stream (such as clarity, latency, frame loss rate, etc.) once a minute. By low-pass filtering and removing frequent spikes and short fluctuations, we may find some extremely high or low quality values, which may be caused by equipment failure or instantaneous network fluctuations. According to the proportion of these extreme values ​​in the total data (for example, the maximum and minimum values ​​account for 10%), these extreme values ​​can be extracted from the first data set to construct an estimated noise set. The data in these noise sets are possible abnormal data, which will be further eliminated or denoised.

[0050] When band-pass filtering is performed on the first data to obtain extreme value data, the first data can be band-pass filtered to obtain maximum value data and minimum value data; based on the sum of the extreme value proportions of the maximum value data in the first data and the extreme value proportions of the minimum value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed based on the second data.

[0051] Specifically, first, the first data is bandpass filtered to obtain two important results: maximum data and minimum data. Maximum data usually represents the peak value of the service quality index within a certain period of time. These values ​​often represent abnormal fluctuations, such as quality degradation caused by system overload or emergencies. Minimum data represents the trough value of the service quality index, which may be caused by system failure or unforeseen factors. Through bandpass filtering, these maximum and minimum data will be effectively extracted for further analysis. Next, the proportion of maximum data and minimum data in the entire first data is calculated. Based on the proportion of the above maximum data and minimum data, the second data is extracted from the first data. This second data set is the estimated noise set, which contains the extreme data part that is considered to be noise. By removing these extreme maximum and minimum data from the original data set, the estimated noise set helps us further eliminate the impact on the service quality index, making the subsequent mean calculation more accurate and stable.

[0052] The present application extracts the second data from the first data according to the proportion of the extreme value data in the first data. This can be implemented in a variety of ways, for example, M is the extreme value proportion of the extreme value data in the first data, N is the size of the estimated noise set, and as an implementable method, N=M+a; a is an adaptive factor, and a specific value can be set according to the type of service quality indicator data.

[0053] Preferably, the second data is extracted from the first data according to the extraction proportion N; wherein the extraction proportion N is determined by the following formula: N=1.3×M, wherein M is the extreme value proportion of the extreme value data in the first data. M is obtained by analyzing the proportion of these extreme values ​​in the first data set. The extraction proportion N is calculated by the formula N=1.3×M. The core of this formula is to expand the proportion of extreme value data by the constant 1.3, in order to ensure that when the second data is extracted, sufficient extreme value data is included to effectively identify noise and perform further processing. Through this calculation, the extraction proportion N will be slightly larger than the simple extreme value proportion M, thereby ensuring that potential outliers are not missed in data processing.

[0054] For example, there is a set of video monitoring data. Some maximum and minimum values ​​are detected in the data set through low-pass filtering. It is calculated that these extreme value data account for 5% of the overall data (i.e., M=0.05). The extraction ratio is calculated according to the formula: N=1.3×0.05=0.065. Therefore, the extraction ratio N is 6.5%. This means that from the first data set, we will select a subset containing 6.5% of the data as the second data, and the noise data or extreme data in it will be further identified and processed.

[0055] The above step 104, i.e., "eliminating ultra-high noise data from the estimated noise set according to the ultra-high noise ratio to obtain third data, and constructing a noise calculation set from the third data" is described in detail below in conjunction with an embodiment.

[0056] This technical feature describes how to further screen and remove noise data during data processing to ensure data quality and accuracy in order to build a more accurate noise calculation set. In this process, you first need to determine an ultra-high noise ratio. The ultra-high noise ratio refers to the proportion of noise data in the data set that deviates abnormally from the normal data range and may greatly interfere with the results. The fluctuation range of these data is very large, usually manifested as abnormal extreme values, which may be caused by hardware failure, data collection errors, or other factors.

[0057] A specific ratio value for the ultra-high noise ratio can be preset, or it can be flexibly set according to the data volume, data type, etc. of the service quality indicator data. Preferably, the ultra-high noise ratio can be dynamically determined according to the extreme value ratio. For example, the value of the extreme value ratio is directly used as the value of the ultra-high noise ratio, or the value after the extreme value ratio is expanded or reduced is used as the ultra-high noise ratio.

[0058] Assuming that the proportion of ultra-high noise is Y, based on the proportion of ultra-high noise, the ultra-high noise data is removed from the estimated noise set to obtain the third data, that is, the second data is sorted and the first Y parts of the data are removed. The remaining second data after the removal is used as the third data, and the third data constitutes the noise calculation set. After this screening, these data have removed extreme noise compared to the original estimated noise set, so they are more in line with the actual situation and can more accurately reflect the actual characteristics of the data.

[0059] The above step 105, i.e., "setting a noise threshold according to the statistical result of the third data, using the noise threshold to filter out fourth data from the second data, and constructing a target noise set from the fourth data" is described in detail below in conjunction with an embodiment.

[0060] This technical feature involves how to set appropriate noise thresholds by statistically analyzing the third data during the noise processing process, thereby filtering out qualified data from the second data and finally constructing a target noise set. The statistical result of the third data refers to the third data obtained after the second data has been filtered out from the first data in the previous step and the ultra-high noise data has been further eliminated. This data set usually contains relatively stable, non-noise business quality indicator data. By performing statistical analysis on the third data (for example, calculating statistics such as mean, standard deviation, variance, etc.), the distribution and fluctuation of the data can be understood, thereby providing a basis for setting the noise threshold.

[0061] The setting of the noise threshold is determined based on the statistical results of the third data. The noise threshold can usually be determined in the following ways: Based on the mean and standard deviation: The noise threshold may be set as a certain multiple of the mean ± standard deviation of the third data. For example, if the fluctuation of the data is greater than a certain multiple of the standard deviation, it can be determined as noise. Or based on volatility analysis: If the fluctuation of the data is greater than a certain threshold (such as a certain standard deviation range), the data is considered to be noise.

[0062] As an implementable method, obtaining a target noise set based on a noise calculation set includes: calculating the maximum value, minimum value, mean value and noise fluctuation factor of the third data; setting a noise threshold based on the maximum value, minimum value, mean value and noise fluctuation factor; based on the noise threshold, screening out data in the second data that meets the noise threshold as fourth data, and the fourth data constitutes the target noise set.

[0063] Specifically, first, the maximum value, minimum value, mean value and noise fluctuation factor of the third data are calculated. The third data is a relatively stable service quality indicator data set obtained after preliminary screening. These data are usually preprocessed by low-pass, high-pass or band-pass filtering, so they contain less extreme noise. By statistically analyzing these data, their maximum value, minimum value and mean value are obtained, and the volatility of these data is further analyzed to obtain the noise fluctuation factor. The noise fluctuation factor usually reflects the degree of fluctuation of the data, and a larger fluctuation may mean a higher noise component. The maximum and minimum values ​​represent the largest and smallest values ​​in the data set, respectively; the mean value reflects the central trend of the data set, that is, the approximate average level of the data; the noise fluctuation factor can be calculated based on the standard deviation or variance of the data, and is usually used to measure the amplitude of data fluctuation. If the data fluctuates greatly, the noise fluctuation factor is higher, otherwise it is lower.

[0064] Preferably, the noise fluctuation factor is calculated by the following formula: (1) Wherein, nc is the number of data of the third data, is the value of the third data, is the mean.

[0065] Next, a noise threshold is set according to the maximum value, minimum value, mean value and noise fluctuation factor. The noise threshold is set to filter out data that does not conform to the normal fluctuation range through certain standards. Usually, the noise threshold is adjusted based on the mean value and the noise fluctuation factor. For example, the noise threshold can be defined as the mean ± several times the standard deviation, or the threshold interval can be dynamically set according to the fluctuation factor of the data. By adjusting the threshold range, the screening criteria can be flexibly controlled to eliminate abnormal data that exceeds the reasonable fluctuation range.

[0066] Preferably, the noise threshold It is defined by the following formula: (2) in, is the weight of the noise fluctuation factor, is the noise fluctuation factor, is the mean value; The weight of the noise fluctuation factor is adjusted according to the formula so that the noise threshold is greater than the minimum value and less than the maximum value. That is: (3) The larger the value, the smaller the noise reduction intensity, and the more data samples can be retained; conversely, the greater the noise reduction intensity, the more samples are removed; noise reduction is recommended, and the noise reduction ratio is related to the data distribution.

[0067] The noise threshold is used to filter out the qualified data from the second data, which is called the fourth data. That is: (4) in, is the value in the second data, It is the fourth data.

[0068] The fourth data contains data that is identified as abnormal after passing the noise threshold. This process is actually further refining data processing to ensure that only valid and noise-free service quality indicator data can be used for subsequent analysis and processing. Finally, the fourth data set obtained after screening constitutes the target noise set. The target noise set is used for further noise reduction processing or accurate calculation of service quality indicators.

[0069] The above step 106, i.e., "using the target noise set to perform noise reduction processing on the first data to obtain a mean calculation set, and calculating the mean of the service quality indicator based on the mean calculation set" is described in detail below in conjunction with an embodiment.

[0070] First, the first data is subjected to noise reduction processing using the target noise set. In the previous step, the target noise set is screened out by the noise threshold. By applying the target noise set to the first data, the noise components therein can be removed, making the signal part in the first data more prominent. Specifically, the data in the target noise set in the first data can be directly removed to form a mean calculation set. Alternatively, the data in the target noise set in the first data and its related data (such as the data before and after it) are removed to form a mean calculation set.

[0071] Next, the mean calculation set is obtained. After the noise reduction process, the data set without the influence of noise becomes the "mean calculation set". The mean calculation set is the data set after noise reduction, which is used to calculate the mean of the service quality index. By averaging the data in the mean calculation set, the final mean of the service quality index can be obtained.

[0072] The mean can be calculated using the following formula: (5) in, is the number of values ​​in the fourth data, s is the number of values ​​in the first data, is the value in the second data, is the value in the fourth data.

[0073] In order to reflect the technical effect of the method proposed in this application, the experimental verification results are shown below.

[0074] Figure 2To estimate the statistical results of the noise set, including the mean , minimum , maximum value and data fluctuation range This chart is used to analyze the distribution characteristics of noise and its impact on noise reduction, and to verify the effectiveness of noise identification. Figure 3 From the results, we can see that the mean of the estimated noise set is close to the minimum value of the data, indicating that the noise is concentrated in a lower range; the fluctuation value converges, indicating that the distribution range of the noise data is limited. In addition, the noise is sporadically distributed near the upper limit of the data, which supports the theoretical analysis that most of the noise is located at the edge of the estimated range. Therefore, the statistical analysis of the estimated noise set supports the effectiveness of the noise reduction analysis method from the perspective of theory and data performance, and provides a scientific basis for the subsequent construction of the noise calculation set.

[0075] Figure 3 The comparison results of the network downstream TCP round-trip delay data before and after noise reduction are used to evaluate the impact of the noise reduction algorithm on the delay data, especially the improvement of the degree of proximity between data fluctuation and expected value. The experiment uses the network downstream TCP round-trip delay data as a sample, and the extreme value ratio is set to 5%. Among them, when setting the noise threshold When L1 is The denoised mean data, L2 is The denoised mean data, L3 is = The denoised mean data. The value of can determine the intensity of noise reduction, and The value of is negatively correlated with the noise reduction intensity. As can be seen from the figure, after noise reduction processing, the fluctuation of the delay data is significantly reduced, and the time series trend of the data becomes more stable. In addition, the noise-reduced data is numerically closer to the expected value than the unprocessed data. As the noise reduction depth increases, the data moves closer to the expected value, indicating that the increase in noise reduction intensity helps to further eliminate the impact of noise. Therefore, noise reduction processing has a significant improvement effect on the network downlink TCP round-trip delay data. In the end, the processed data is more stable and closer to the actual network performance indicators, providing a reliable support basis for network performance optimization and intelligent operation and maintenance.

[0076] Figure 4 The change of denoising depth in time series under different denoising strengths reflects the adaptability of the adaptive denoising algorithm to different data distributions. The value of L1 is set. The denoised depth data, L2 is Denoised depth data, L3 is Denoising depth data. As the denoising intensity increases, the denoising depth increases accordingly, showing that the algorithm can dynamically adjust the denoising parameters to adapt to the data characteristics. In addition, the fluctuation of denoising depth at different intensities is relatively stable, indicating that the algorithm can maintain stable performance and avoid over-denoising or under-denoising.

[0077] Figure 5 shows the change of data fluctuation range before and after noise reduction, which is used to evaluate the convergence effect of noise reduction algorithm on data fluctuation. Among them, Figure 5 (a) shows the change of data fluctuation range before noise reduction; Figure 5 (b) shows the change of data fluctuation range after noise reduction. Among them, Figure 5 (b) shows the change of data fluctuation range after noise reduction. The fluctuation value of the noise reduction index at that time. The fluctuation range of the data after noise reduction has obviously converged, indicating that the noise interference has been effectively suppressed. At the same time, the converged data is more concentrated in a reasonable range, which significantly reduces the impact of abnormal fluctuations on the calculation of service quality indicators. Therefore, the noise reduction algorithm performs well in reducing the data fluctuation range and improves the accuracy and stability of service quality indicators.

[0078] The above method provided in the embodiment of the present application can be applied to a variety of application scenarios, including but not limited to: for business fault location, through real-time data analysis and noise processing, to help quickly locate business fault points and improve fault repair efficiency; for user perception analysis, by performing noise reduction processing on user experience related indicator data, accurately calculating user perceived quality, and helping enterprises optimize service quality; for intelligent operation and maintenance, in the operation and maintenance process, automatically identifying and eliminating noise data to achieve efficient and intelligent system operation and maintenance management; for quality monitoring, in scenarios such as traffic monitoring and video monitoring, more accurate quality monitoring results are provided through noise reduction processing, helping decision makers optimize resource allocation and improve service quality.

[0079] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0080] According to an embodiment of another aspect, a device for calculating a mean service quality indicator for adaptive noise reduction is provided. Figure 6 A schematic block diagram of a mean service quality indicator calculation device for adaptive noise reduction according to an embodiment is shown, as shown in FIG. Figure 6 As shown, the device 600 includes: The indicator data acquisition unit 601 is configured to acquire service quality indicator data; A full sample set construction unit 602 is configured to construct a full sample set from the service quality indicator data, where the data in the full sample set is first data; An estimated noise set construction unit 603 is configured to extract second data from the first data based on the extreme values ​​of the full sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values; The noise calculation set construction unit 604 is configured to remove the ultra-high noise data from the estimated noise set according to the ultra-high noise ratio, obtain third data, and construct the noise calculation set based on the third data; A target noise set construction unit 605 is configured to set a noise threshold according to a statistical result of the third data, filter the second data using the noise threshold to obtain fourth data, and construct a target noise set using the fourth data; The mean calculation unit 606 is configured to perform noise reduction processing on the first data using the target noise set to obtain a mean calculation set, and calculate the mean of the service quality indicator based on the mean calculation set.

[0081] As an implementable method, the estimated noise set construction unit 603 can be configured to perform low-pass filtering, high-pass filtering or band-pass filtering on the first data to obtain extreme value data; based on the extreme value proportion of the extreme value data in the first data, extract second data from the first data, and construct an estimated noise set based on the second data.

[0082] As an implementable manner, when the estimated noise set construction unit 603 performs band-pass filtering on the first data to obtain extreme value data, it can be configured as follows: performing band-pass filtering on the first data to obtain maximum value data and minimum value data; extracting second data from the first data based on the sum of the extreme value proportions of the maximum value data in the first data and the extreme value proportions of the minimum value data in the first data, and constructing an estimated noise set based on the second data.

[0083] As an implementable manner, when extracting the second data from the first data according to the proportion of the extreme value data in the first data, the estimated noise set construction unit 603 can be configured as follows: extracting the second data from the first data according to the extraction proportion N; wherein the extraction proportion N is determined by the following formula: N=1.3×M, wherein M is the extreme value proportion of the extreme value data in the first data.

[0084] As an implementable manner, the noise calculation set construction unit 604 may be configured to determine the ultra-high noise proportion according to the extreme value proportion.

[0085] As an implementable manner, the target noise set construction unit 605 can be configured to: calculate the maximum value, minimum value, mean value and noise fluctuation factor of the third data; set a noise threshold according to the maximum value, minimum value, mean value and noise fluctuation factor; and according to the noise threshold, filter out the data in the second data that meets the noise threshold as the fourth data, and the fourth data constitutes the target noise set.

[0086] As an implementable manner, the target noise set construction unit 605 can be configured as follows when setting the noise threshold according to the maximum value, the minimum value, the mean value and the noise fluctuation factor: T It is defined by the following formula: ;in, is the weight of the noise fluctuation factor, is the noise fluctuation factor, is the mean; and the weight of the noise fluctuation factor is adjusted according to the formula so that the noise threshold is greater than the minimum value and less than the maximum value.

[0087] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0088] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0089] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0090] And an electronic device, comprising: one or more processors; and A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

[0091] The present application also provides a computer program product, including a computer program, which implements the steps of any one of the methods in the aforementioned method embodiments when executed by a processor.

[0092] in, Figure 7 The architecture of the electronic device is shown as an example, which may include a processor 710, a video display adapter 711, a disk drive 712, an input / output interface 713, a network interface 714, and a memory 720. The processor 710, the video display adapter 711, the disk drive 712, the input / output interface 713, the network interface 714, and the memory 720 may be communicatively connected via a communication bus 730.

[0093] The processor 710 may be implemented by a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in this application.

[0094] The memory 720 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 720 can store an operating system 721 for controlling the operation of the electronic device 700, and a basic input and output system (BIOS) 722 for controlling the low-level operation of the electronic device 700. In addition, a web browser 723, a data storage management system 724, and a mean service quality indicator calculation device 725 for adaptive noise reduction, etc. can also be stored. The above-mentioned mean service quality indicator calculation device 725 for adaptive noise reduction can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 720 and is called and executed by the processor 710.

[0095] The input / output interface 713 is used to connect the input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0096] The network interface 714 is used to connect to a communication module (not shown) to achieve communication interaction between the device and other devices. The communication module can achieve communication through a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0097] The bus 730 comprises a pathway for transmitting information between the various components of the device (eg, the processor 710 , the video display adapter 711 , the disk drive 712 , the input / output interface 713 , the network interface 714 , and the memory 720 ).

[0098] It should be noted that, although the above device only shows a processor 710, a video display adapter 711, a disk drive 712, an input / output interface 713, a network interface 714, a memory 720, a bus 730, etc., in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0099] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a computer program product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0100] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A method for calculating a mean service quality indicator for adaptive noise reduction, characterized in that: The method comprises: Obtaining service quality indicator data; Constructing a full sample set from the service quality indicator data, where the data in the full sample set is first data; Extracting second data from the first data based on the extreme values ​​of the entire sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values; According to the proportion of ultra-high noise, ultra-high noise data is removed from the estimated noise set to obtain third data, and a noise calculation set is constructed based on the third data; Setting a noise threshold according to the statistical result of the third data, using the noise threshold to filter the second data to obtain fourth data, and constructing a target noise set from the fourth data; The first data is subjected to noise reduction processing by using the target noise set to obtain a mean calculation set, and the mean of the service quality indicator is calculated based on the mean calculation set.

2. According to the method of claim 1, extracting second data from the first data based on the extreme values ​​of the full sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values, and comprises: Performing low-pass filtering, high-pass filtering or band-pass filtering on the first data to obtain extreme value data; According to the extreme value proportion of the extreme value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed according to the second data.

3. The method according to claim 2, when performing bandpass filtering on the first data to obtain extreme value data, the method comprises: Performing bandpass filtering on the first data to obtain maximum value data and minimum value data; According to the sum of the extreme value proportion of the maximum value data in the first data and the extreme value proportion of the minimum value data in the first data, second data is extracted from the first data, and an estimated noise set is constructed according to the second data.

4. The method according to claim 2, characterized in that: The extracting second data from the first data according to the proportion of the extreme value data in the first data comprises: According to the extraction ratio N, the second data is extracted from the first data; wherein the extraction ratio N is determined by the following formula: N=1.3×M, wherein M is the extreme value ratio of the extreme value data in the first data.

5. The method according to claim 3, characterized in that: The method comprises: The ultra-high noise ratio is determined according to the extreme value ratio.

6. The method according to claim 1, characterized in that The step of setting a noise threshold according to the statistical result of the third data, using the noise threshold to filter the second data to obtain fourth data, and constructing a target noise set from the fourth data comprises: Calculating the maximum value, minimum value, mean value and noise fluctuation factor of the third data; Setting a noise threshold according to the maximum value, minimum value, mean value and noise fluctuation factor; According to the noise threshold, data satisfying the noise threshold in the second data is screened out as fourth data, and the target noise set is constituted by the fourth data.

7. The method according to claim 6, characterized in that The setting of the noise threshold according to the maximum value, the minimum value, the mean value and the noise fluctuation factor comprises: The noise threshold T is defined by the following formula: ; in, is the weight of the noise fluctuation factor, is the noise fluctuation factor, is the mean value; The weight of the noise fluctuation factor is adjusted according to the formula so that the noise threshold is greater than the minimum value and less than the maximum value.

8. A mean service quality indicator calculation device for adaptive noise reduction, characterized in that: The device comprises: An indicator data acquisition unit, configured to acquire service quality indicator data; A full sample set construction unit is configured to construct a full sample set from the service quality indicator data, wherein the data in the full sample set is first data; an estimated noise set construction unit, configured to extract second data from the first data based on the extreme values ​​of the full sample set to construct an estimated noise set, wherein the size of the estimated noise set is determined according to the proportion of extreme values; A noise calculation set construction unit is configured to remove ultra-high noise data from the estimated noise set according to the ultra-high noise ratio, obtain third data, and construct a noise calculation set based on the third data; a target noise set construction unit, configured to set a noise threshold according to a statistical result of the third data, filter the second data using the noise threshold to obtain fourth data, and construct a target noise set using the fourth data; The mean calculation unit is configured to perform noise reduction processing on the first data using the target noise set to obtain a mean calculation set, and calculate the mean of the service quality indicator based on the mean calculation set.

9. An electronic device, characterized in that: include: one or more processors; And a memory associated with the one or more processors, the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the steps of the method described in any one of claims 1 to 7 are executed.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images