Index threshold value determination method and apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to an index threshold determination method and device, electronic equipment and computer readable storage medium, and relates to the technical field of network operation, which can be applied to the scene of screening target service forms according to determined service indexes. The method comprises: determining an index dynamic threshold, the index dynamic threshold being used for screening target service forms from to-be-processed service forms; the index dynamic threshold is generated based on a threshold determination model, the threshold determination model being trained based on one or more of a time dimension, a geographical dimension, a user dimension, a network dimension and a service dimension of historical service forms, and being dynamically updated based on real-time service forms. The present disclosure provides a scheme of automatically generating an index dynamic threshold, automatically determines and adjusts the index dynamic threshold of various indexes through the threshold determination model, improves the network operation capability, and can reduce manual intervention and improve work efficiency.

Description

[0001] This application claims priority to Chinese Patent Application No. 202311320233.5, filed on October 12, 2023, entitled “Method and Apparatus for Determining Index Thresholds, Electronic Device and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of network operation technology, and more specifically, to a method for determining an indicator threshold, an apparatus for determining an indicator threshold, an electronic device, and a computer-readable storage medium. Background Technology

[0003] Network failure refers to a state in which a network cannot provide normal service or has reduced service quality due to hardware problems, software vulnerabilities, virus intrusion, etc. Before root cause localization of network failures, thresholds for indicators such as network quality, network performance, and service quality are usually set based on expert experience to screen out poor-quality call detail records (CDRs). Then, root cause localization and segmentation analysis are performed using technologies such as artificial intelligence and big data analysis.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a method, device, electronic device, and computer-readable storage medium for determining indicator thresholds, thereby overcoming, to at least some extent, the problem that the configuration of service indicator thresholds that cause network failures needs to be manually set and cannot be adaptively adjusted according to real-time network changes.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0007] According to a first aspect of this disclosure, a method for determining an indicator threshold is provided, comprising: determining a dynamic threshold for an indicator, the dynamic threshold being used to filter target business forms from business forms to be processed; the dynamic threshold for an indicator being generated based on a threshold determination model, the threshold determination model being trained based on one or more dimensions of historical business forms, including time dimension, geographic dimension, user dimension, network dimension, and business dimension, and being dynamically updated based on real-time business forms.

[0008] In one exemplary embodiment of this disclosure, the method further includes: collecting network performance data, service quality data, configuration data, and service operation data from the network through a deep packet inspection probe and a production system to generate an initial historical service form; the production system includes one or more of a network management system, a customer relationship management system, a work order system, and a customer service system.

[0009] In one exemplary embodiment of this disclosure, after generating the initial historical business form, the method further includes: performing a review process on the initial historical business form to obtain the historical business form.

[0010] In one exemplary embodiment of this disclosure, the threshold determination model is generated through the following steps: using the historical business forms with classification labels, a pre-built initial model is trained to obtain the threshold determination model. The threshold determination model uses probability density analysis and the n*σ criterion to determine the dynamic threshold of each business-related indicator in the historical business forms. The dynamic threshold is used to determine the classification label of the business forms to be processed.

[0011] In one exemplary embodiment of this disclosure, the step of determining the dynamic threshold of each of the business-related indicators using probability density analysis and the n*σ criterion includes: using the n*σ criterion to determine the upper limit and lower limit of the threshold of the business-related indicators; determining the dynamic threshold of the indicators based on the upper limit and lower limit of the threshold; the upper limit and lower limit of the threshold are determined based on the probability distribution of the business-related indicators, and the probability distribution is determined based on the mean, variance, and probability density function of the business-related indicators.

[0012] In one exemplary embodiment of this disclosure, determining the dynamic threshold of the indicator based on the upper limit and the lower limit of the indicator threshold includes: determining the normal distribution of the business-related indicator, wherein the normal distribution is determined based on the upper limit of the indicator threshold, the lower limit of the indicator threshold, and the group interval of the indicator group of the business-related indicator, and the group interval of the indicator group is determined based on the upper limit of the indicator threshold, the lower limit of the indicator threshold, and the number of threshold groups; and determining the dynamic threshold of the indicator based on the inverse function of the cumulative distribution function of the normal distribution of the indicator.

[0013] In one exemplary embodiment of this disclosure, the method further includes: using an association mining algorithm to perform secondary verification and optimization processing on the dynamic threshold of the indicator to obtain the optimized dynamic threshold of the indicator.

[0014] In one exemplary embodiment of this disclosure, the step of using an association mining algorithm to perform secondary verification and optimization processing on the dynamic threshold of the indicator to obtain the optimized dynamic threshold of the indicator includes: classifying the historical business forms according to business types to obtain frequent one-itemsets; determining the minimum support threshold and minimum confidence threshold corresponding to each of the classified business indicators; and determining whether to trigger a threshold update operation on the dynamic threshold of the indicator based on the frequent one-itemsets, the minimum support threshold, and the minimum confidence threshold to obtain the optimized dynamic threshold of the indicator.

[0015] In one exemplary embodiment of this disclosure, determining whether to trigger a threshold update operation on the dynamic threshold of the indicator based on the frequent one-itemset, the minimum support threshold, and the minimum confidence threshold to obtain the optimized dynamic threshold of the indicator includes: performing a frequent itemset determination operation based on the frequent one-itemset to obtain all frequent k-itemsets in the historical business form; calculating association rules based on all the frequent k-itemsets to obtain the indicator confidence; when the indicator confidence is greater than or equal to the minimum confidence threshold, using the dynamic threshold of the indicator as the optimized dynamic threshold of the indicator; when the indicator confidence is less than the minimum confidence threshold, triggering an update operation on the dynamic threshold of the indicator to obtain the optimized dynamic threshold of the indicator.

[0016] In one exemplary embodiment of this disclosure, the step of performing a frequent itemset determination operation based on the frequent 1-itemsets to obtain all frequent k-itemsets in the historical business form includes: scanning the frequent 1-itemsets; determining frequent k-itemsets and the support of each frequent k-itemset based on the scanning results of the frequent 1-itemsets; determining a frequent k-itemset based on the minimum support among the support of multiple frequent k-itemsets; repeating the above steps until the number of determined frequent k+j itemsets is less than or equal to 1, thereby obtaining all the frequent k-itemsets.

[0017] In one exemplary embodiment of this disclosure, the method further includes: using the real-time business form to perform model update processing on the threshold determination model to obtain an updated threshold determination model, wherein the updated threshold determination model is used to adjust the dynamic threshold of the indicator, and the real-time business form is a business form obtained by periodic sampling based on the time dimension.

[0018] According to a second aspect of this disclosure, an indicator threshold determination device is provided, comprising: an indicator threshold determination module, configured to determine a dynamic threshold for an indicator, wherein the dynamic threshold is used to filter target business forms from business forms to be processed, the dynamic threshold is generated based on a threshold determination model, the threshold determination model is trained based on one or more dimensions of historical business forms, including time dimension, geographic dimension, user dimension, network dimension, and business dimension, and is dynamically updated based on real-time business forms.

[0019] In one exemplary embodiment of this disclosure, the indicator threshold determination device further includes an initial historical form collection module, used to: collect network performance data, service quality data, configuration data, and service operation data from the network through a deep packet inspection probe and a production system to generate an initial historical service form; the production system includes one or more of a network management system, a customer relationship management system, a work order system, and a customer service system.

[0020] In one exemplary embodiment of this disclosure, the indicator threshold determination device further includes a historical form generation module, used to: review the initial historical business form to obtain the historical business form.

[0021] In one exemplary embodiment of this disclosure, the indicator threshold determination device further includes a threshold model generation module, configured to: train a pre-built initial model using the historical business form with classification labels to obtain the threshold determination model; the threshold determination model uses probability density analysis and the n*σ criterion to determine the dynamic threshold of each business-related indicator in the historical business form; the dynamic threshold is used to determine the classification label of the business form to be processed.

[0022] In one exemplary embodiment of this disclosure, the threshold model generation module includes a threshold determination unit, configured to: determine the upper limit and lower limit of the threshold of the business-related indicator using the n*σ criterion; determine the dynamic threshold of the indicator based on the upper limit and lower limit of the threshold; the upper limit and lower limit of the threshold are determined based on the probability distribution of the business-related indicator, and the probability distribution is determined based on the mean, variance and probability density function of the business-related indicator.

[0023] In one exemplary embodiment of this disclosure, the threshold determination unit includes a threshold determination subunit, configured to: determine the indicator normal distribution of the business-related indicator, wherein the indicator normal distribution is determined based on the upper limit of the indicator threshold, the lower limit of the indicator threshold, and the indicator grouping interval of the business-related indicator, wherein the indicator grouping interval is determined based on the upper limit of the indicator threshold, the lower limit of the indicator threshold, and the number of threshold groups; and determine the indicator dynamic threshold based on the inverse function of the cumulative distribution function of the indicator normal distribution.

[0024] In one exemplary embodiment of this disclosure, the indicator threshold determination device further includes a threshold update module, configured to: perform secondary verification and optimization processing on the indicator dynamic threshold using an association mining algorithm to obtain the optimized indicator dynamic threshold.

[0025] In one exemplary embodiment of this disclosure, the threshold update module includes a threshold update unit, configured to: classify the historical business forms according to business types to obtain frequent one-itemsets; determine the minimum support threshold and minimum confidence threshold corresponding to each classified business indicator; and determine whether to trigger a threshold update operation on the dynamic threshold of the indicator based on the frequent one-itemsets, the minimum support threshold, and the minimum confidence threshold, so as to obtain the optimized indicator dynamic threshold.

[0026] In one exemplary embodiment of this disclosure, the threshold update unit includes a threshold update subunit, configured to: perform a frequent itemset determination operation based on the frequent 1-itemsets to obtain all frequent k-itemsets in the historical business form; calculate association rules based on all the frequent k-itemsets to obtain the indicator confidence level; when the indicator confidence level is greater than or equal to the minimum confidence level threshold, use the indicator dynamic threshold as the optimized indicator dynamic threshold; when the indicator confidence level is less than the minimum confidence level threshold, trigger the execution of the indicator dynamic threshold update operation to obtain the optimized indicator dynamic threshold.

[0027] In one exemplary embodiment of this disclosure, the threshold update subunit includes a frequent itemset determination subunit, configured to: scan the frequent 1-itemsets; determine frequent k-itemsets and the support of each frequent k-itemsets based on the scanning results of the frequent 1-itemsets; determine a frequent k-itemset based on the minimum support among the support of multiple frequent k-itemsets; repeat the above steps until the number of determined frequent k+j itemsets is less than or equal to 1, thereby obtaining all the frequent k-itemsets.

[0028] In one exemplary embodiment of this disclosure, the indicator threshold determination device further includes a model update module, configured to: use the real-time business form to perform model update processing on the threshold determination model to obtain an updated threshold determination model, wherein the updated threshold determination model is used to adjust the indicator dynamic threshold, and the real-time business form is a business form obtained by periodic sampling based on the time dimension.

[0029] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory storing computer-readable instructions that, when executed by the processor, implement the index threshold determination method according to any one of the preceding claims.

[0030] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the index threshold determination method according to any one of the preceding claims.

[0031] The technical solution provided in this disclosure may include the following beneficial effects:

[0032] The indicator threshold determination method in the exemplary embodiments of this disclosure, on the one hand, trains a threshold generation model based on historical business forms. This threshold generation model can adapt to multiple dimensions in the historical business forms to obtain dynamic thresholds for various indicators, thereby improving network operation capabilities. On the other hand, since the threshold generation model can be dynamically updated according to real-time business forms, it can automatically adjust the dynamic thresholds of indicators, reducing manual intervention, improving work efficiency, and lowering operation and maintenance costs.

[0033] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0034] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0035] Figure 1 A flowchart illustrating an index threshold determination method according to an exemplary embodiment of the present disclosure is shown schematically;

[0036] Figure 2 A system architecture diagram illustrating an implementation of an index threshold determination method according to an exemplary embodiment of the present disclosure is shown.

[0037] Figure 3The flowchart illustrates a process for determining dynamic thresholds of various business-related indicators using probability density analysis and the n*σ criterion according to an exemplary embodiment of this disclosure.

[0038] Figure 4 The flowchart illustrating the determination of a dynamic threshold of an indicator based on an upper limit and a lower limit of an indicator threshold according to an exemplary embodiment of the present disclosure is shown in the illustration.

[0039] Figure 5 A block diagram of an index threshold determination apparatus according to an exemplary embodiment of the present disclosure is shown schematically;

[0040] Figure 6 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown schematically;

[0041] Figure 7 The illustration shows a schematic diagram of a computer-readable storage medium according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0042] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0043] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details described, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known structures, methods, apparatuses, implementations, materials, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0044] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more software-hardened modules, or in different network and / or processor devices and / or microcontroller devices.

[0045] The relevant technical solutions have the following technical problems in terms of indicator threshold configuration: (1) The setting of indicator thresholds is mainly based on fixed thresholds given by experts' experience. It cannot make accurate judgments and quick adjustments in a timely manner according to changes in the network and the customer's business experience, which leads to the evaluation results not matching the actual situation, resulting in the phenomenon of missed detection of problems or excessive negative reviews. Ultimately, it leads to misjudgment or missed judgment of factors affecting customers, and there is a risk of inaccurate fault location.

[0046] (2) Experience thresholds are often considered from an overall perspective, which leads to a one-size-fits-all problem. They are generally trained and modeled using historical data from a certain time series, and lack consideration for factors such as uneven regional development, asynchronous network construction maturity, and differences in customer service usage.

[0047] (3) The configuration of experience thresholds requires manual processing and cannot be adjusted adaptively, which not only consumes manpower costs but also affects network operation efficiency.

[0048] Based on this, according to the embodiments of the present disclosure, a method for determining an index threshold, an apparatus for determining an index threshold, a computer-readable storage medium, and an electronic device are proposed.

[0049] In this paper, the variable 'x' in a specific service detail record (xDetailed Record, xDR) represents different service types, such as voice services and data services. The xDR records detailed service data, including dimensions such as time, geography, network / network element, user, service, and terminal.

[0050] In this example embodiment, a method for determining an indicator threshold is first provided. The method for determining the indicator threshold of this disclosure can be implemented using a server or using a terminal device. The terminal described in this disclosure may include mobile terminals such as mobile phones, tablets, laptops, handheld computers, and personal digital assistants (PDAs), as well as fixed terminals such as desktop computers. Figure 1 A schematic diagram illustrating a method flow for determining index thresholds according to some embodiments of the present disclosure is provided. Reference Figure 1 The method for determining the threshold of this indicator may include the following steps:

[0051] Step S110: Determine the dynamic threshold of the indicator. The dynamic threshold of the indicator is used to filter the target business forms from the business forms to be processed. The dynamic threshold of the indicator is generated based on the threshold determination model. The threshold determination model is trained based on one or more dimensions of the historical business forms, including time dimension, geographical dimension, user dimension, network dimension, and business dimension, and is dynamically updated based on the real-time business forms.

[0052] According to some exemplary embodiments of this disclosure, the dynamic threshold for an indicator can be a reference threshold corresponding to indicator values ​​of different dimensions. The dynamic threshold can be used to measure whether a certain indicator in a business form meets the requirements. The business form to be processed can be a business form to be classified using the dynamic threshold. The target business form can be a business form that does not meet specific requirements and is filtered out from the business forms to be processed through a screening process. The threshold determination model can be an intelligent algorithm model used to determine the dynamic threshold for the indicator. Historical business forms can be business forms generated within a specific past time period. Real-time business forms can be new business forms generated within the current time period.

[0053] To determine dynamic thresholds for different metrics in business forms, historical business forms can be used as training data to train a model, resulting in a threshold determination model. Historical business forms can be call forms of various business types collected from multiple systems, and the data dimensions in these forms are formed by comprehensively processing data dimensions from multiple systems. Historical business forms can include multi-dimensional data such as time, geographic, user, network, and business dimensions.

[0054] Specifically, the time dimension can represent information related to the time period in historical business forms. The geographic dimension can represent information related to the geographic region to which the historical business forms belong. The user dimension can represent the specific categories of users included in the historical business forms. The network dimension can represent network-related information included in the historical business forms; for example, the network dimension can include information related to network quality, network element information, and network performance. The business dimension can represent the specific business categories involved in the historical business forms; business categories can include various services such as video, games, and voice.

[0055] Historical business forms containing the aforementioned data dimensions are used as training data to train the model, resulting in a threshold determination model. This model can adaptively determine dynamic thresholds for various business metrics across different data dimensions. This allows for subsequent classification and processing of business forms based on these dynamic thresholds. If a business metric in a form is below its dynamic threshold, it is designated as a target form—that is, a form that does not meet specific business quality requirements is filtered out. This allows for subsequent segmentation and localization analysis of the target form to pinpoint the root causes of its poor performance.

[0056] To adapt to the constantly changing real-time network environment and determine dynamic thresholds for metrics that better reflect real-time conditions, real-time acquired business forms (i.e., real-time business forms) can be used as model training data. The thresholds are incrementally optimized and updated to determine the model. The updated thresholds are then used to automatically adjust the dynamic thresholds for each business metric, which are then used as the basis for selecting target business forms. Based on these adjusted dynamic thresholds, low-quality business forms are selected and used as input data for determining the root cause of quality issues, thereby improving the accuracy of root cause identification.

[0057] According to the indicator threshold determination method in this example embodiment, on the one hand, a threshold determination model is trained based on historical business forms. This model can adapt to multiple dimensions in the historical business forms to obtain dynamic thresholds for various indicators, thereby improving network operation capabilities. On the other hand, since the threshold generation model can be dynamically updated according to real-time business forms, it can automatically adjust the dynamic thresholds of indicators, reducing manual intervention, improving work efficiency, and lowering operation and maintenance costs.

[0058] The method for determining the index threshold in this example embodiment will be further explained below.

[0059] In one exemplary embodiment of this disclosure, a deep packet inspection probe and a production system are used to collect network performance data, service quality data, configuration data, and service operation data from the network to generate an initial historical service form; the production system includes one or more of a network management system, a customer relationship management system, a work order system, and a customer service system.

[0060] Deep Packet Inspection (DPI) probes are application-layer traffic detection and control technologies that add application protocol identification, packet content inspection, and deep decoding to traditional Internet Protocol (IP) packet inspection techniques. Production systems refer to information systems that support the daily business operations of an organization under normal circumstances. Network performance data can be metrics measuring network performance, including jitter, packet loss rate, and latency. Service quality data can be customer service quality feedback data provided by communication service providers, which may include video download speeds for video services and page loading latency for browsing services. Configuration data can be relevant system configuration data from various systems. Business operation data can be network management data, customer service data, etc. Initial historical business forms can be data collected directly from deep packet inspection probes or production systems.

[0061] refer to Figure 2 , Figure 2The diagram schematically illustrates a system architecture diagram of an implementation indicator threshold determination method according to an exemplary embodiment of this disclosure. When determining historical business forms, the data acquisition module 210 can first collect customer experience-related xDR form data from network management systems, customer relationship management systems, work order systems, customer service systems, etc. For example, the data acquisition module 210 can use DPI probes and various types of production systems to collect network performance data, service quality data, configuration data, and business operation data from the network. After collecting the above data, the data is merged into a single data table to obtain the corresponding xDR form data wide table, which serves as the initial historical business form 211.

[0062] Specifically, a Network Management System (NMS) is a system that combines software and hardware to adjust network status, responsible for the management and operation of network resources such as the core network, wireless network, and transmission network. A Customer Relationship Management (CRM) system helps enterprises collect, organize, update, and store customer information, enabling customer data sharing and collaboration, and improving the accuracy and timeliness of customer information. A work order system can be used to record, process, and track the completion status of a task. A customer service system can be a system that coordinates personnel, business processes, technology, and strategy; based on user complaint data provided by the customer service system, it can perform operations such as user complaint processing and user behavior analysis.

[0063] Through the above processing steps, an initial historical business form is obtained, encompassing multiple dimensions such as time, geography, user, network, and business. This initial historical business form is then used to generate subsequent historical business forms. For example, network performance data, service quality data, configuration data, and business operation data obtained from the network can be business data related to customer experience. After collecting this customer experience-related business data from the network, customer experience analysis can be performed based on this data.

[0064] In one exemplary embodiment of this disclosure, the initial historical business form is reviewed to obtain the historical business form.

[0065] Continue to refer to Figure 2To obtain form data that conforms to standard data specifications, after obtaining the initial historical business form 211, it is input into the data verification module 220. The data verification module 220 automatically verifies and improves the data integrity and accuracy of the initial historical business form 211, and uses the processed business form as the historical business form. During the verification process of the initial historical business form 211, the data verification module 220 can delete or add average values ​​to business forms that are missing specific values, resulting in a historical business form that meets the requirements. Through the above steps, the obtained historical business form can be used as model training data for the threshold determination model during the model training process.

[0066] In one exemplary embodiment of this disclosure, the threshold determination model is generated through the following steps: using historical business forms with classification labels, a pre-built initial model is trained to obtain the threshold determination model. The threshold determination model uses probability density analysis and the n*σ criterion to determine the dynamic threshold of each business-related indicator in the historical business forms. The dynamic threshold is used to determine the classification label of the business form to be processed.

[0067] Among them, business-related indicators can be indicators that are specifically related to the specific business of the network service. Category tags can be tags that indicate whether a certain business form meets the business requirements. If a business form does not meet the specific business requirements, its category tag is configured as a non-compliant form.

[0068] Continue to refer to Figure 2 The threshold training module 230 acquires a pre-built initial model, which can be a pre-built artificial intelligence (AI) model. The model training module 231 uses the obtained historical business forms to train the initial model, resulting in a threshold determination model. During the model training process, the pre-built initial model uses probability density analysis and the n*σ criterion to determine the dynamic thresholds of each business-related indicator in the historical business forms.

[0069] After obtaining the dynamic threshold of each business-related indicator, the historical business forms are classified using the dynamic threshold. For example, if the value of a certain business-related indicator in a historical business form is less than the dynamic threshold, the classification label of the historical business form is set to 1, indicating that it does not meet the specific business requirements; if the value of a certain business-related indicator in a historical business form is greater than or equal to the dynamic threshold, the classification label of the historical business form is set to 0, indicating that it meets the specific business requirements.

[0070] During model usage, the threshold determination model also employs probability density analysis and the n*σ criterion to determine the dynamic thresholds of each business-related indicator in the business form to be processed. These dynamic thresholds are then used as reference values ​​to determine the classification labels for each business form, thus obtaining the classification label for each form. The threshold determination model obtained through the above processing steps can adapt to multiple different data dimensions to obtain dynamic thresholds for different business indicators, which are then used in the classification process of the business forms to be processed.

[0071] In one exemplary embodiment of this disclosure, probability density analysis and the n*σ criterion are used to determine the dynamic threshold of each business-related indicator, including: using the n*σ criterion to determine the upper limit and lower limit of the threshold of the business-related indicator; determining the dynamic threshold of the indicator based on the upper limit and lower limit of the threshold; the upper limit and lower limit of the threshold are determined based on the probability distribution of the business-related indicator, and the probability distribution is determined based on the mean, variance and probability density function of the business-related indicator.

[0072] The upper limit of the indicator threshold can be the maximum value that an indicator can achieve. The lower limit of the indicator threshold can be the minimum value that an indicator can achieve. The indicator probability distribution can be the probability distribution of all values ​​that an indicator can take.

[0073] refer to Figure 3 , Figure 3 This illustration schematically depicts a flowchart illustrating the determination of dynamic thresholds for various business-related indicators using probability density analysis and the n*σ criterion according to an exemplary embodiment of this disclosure. Reviewed historical business forms are input into a pre-built initial model. The data dimensions of these historical business forms cover multiple dimensions such as time, region, user, network, and business. For example, based on the region dimension, historical business forms (total number N) from the same quarter are subjected to correlation analysis, and probability density analysis is performed on the values ​​of various customer experience-related indicators. In step S310, the n*σ criterion is used to determine the upper and lower thresholds of the business-related indicators. The n*σ criterion (where n is an integer between 1 and 6, adaptively adjusted based on indicator data) is used to determine a reasonable value range for each business-related indicator, including the upper and lower thresholds of the business-related indicators.

[0074] For each business-related metric, the mean and variance of the corresponding metric can be calculated, along with its probability density function. Based on the calculated mean, variance, and probability density function, the probability distribution of the business-related metric can be determined. After obtaining the corresponding probability distribution, the upper and lower thresholds for the business-related metric can be determined.

[0075] In step S320, the dynamic threshold of the business-related indicator is determined based on the upper and lower limits of the indicator threshold. For example, for a single business-related indicator, historical data of the business-related indicator is obtained, and the corresponding indicator mean μ and indicator variance σ are determined based on the historical data, and then the probability density function is applied. Analyze its probability distribution and use the n*σ criterion to obtain the upper limit of the index threshold T1 = μ + n*σ and the lower limit of the index threshold T0 = μ - n*σ for the reasonable value of the index.

[0076] Historical business forms can be categorized by labeling them according to dynamic thresholds. These categorized historical business forms are then used as training data to train an initial model, resulting in a threshold determination model. This threshold determination model, trained through the above steps, can adaptively determine the dynamic thresholds for different business-related indicators to classify and process the business forms in question.

[0077] In one exemplary embodiment of this disclosure, determining the dynamic threshold of an indicator based on the upper limit and lower limit of the indicator threshold includes: determining the normal distribution of business-related indicators, wherein the normal distribution of indicators is determined based on the upper limit and lower limit of the indicator threshold and the group interval of the indicator groups of business-related indicators, and the group interval of the indicator groups is determined based on the upper limit and lower limit of the indicator threshold and the number of threshold groups; and determining the dynamic threshold of the indicator based on the inverse function of the cumulative distribution function of the normal distribution of the indicator.

[0078] Among them, the group distance of the indicator group is the distance between the highest and lowest values ​​of each group after the indicator values ​​are grouped. The number of threshold groups can be the number obtained by grouping the upper and lower limits of the indicator threshold values.

[0079] refer to Figure 4 , Figure 4 This illustration schematically shows a flowchart of determining a dynamic threshold for an indicator based on an upper and lower threshold according to an exemplary embodiment of the present disclosure. In step S410, the normal distribution of the business-related indicator is determined. For a single business-related indicator, after calculating the upper and lower thresholds of the indicator, the grouping interval of the indicator can be further determined. Specifically, based on the upper and lower thresholds, the interval between the upper and lower thresholds of the business-related indicator is divided into x groups, i.e., the number of threshold groups is determined. Then, the corresponding grouping interval is determined based on the number of threshold groups, such as the grouping interval d = (T1 - T0) / (x - 1); where T1 can represent the upper threshold, T0 can represent the lower threshold, and x can represent the number of threshold groups. After calculating the grouping interval, the normal distribution of the indicator is obtained.

[0080] In step S420, the dynamic threshold of the indicator is determined based on the inverse function of the cumulative distribution function of the normal distribution of the indicator. This is based on the normal cumulative distribution function of the business-related indicators. The inverse function is used to calculate the dynamic threshold T of the relevant business indicator. Business forms with indicator values ​​below the dynamic threshold T are labeled as "poor quality," resulting in corresponding category labels. Similarly, all relevant business indicators are labeled using the same method to obtain tagged historical business forms. Through these processing steps, business forms can be tagged based on their dynamic thresholds, completing the classification process for business forms.

[0081] In one exemplary embodiment of this disclosure, the method further includes: using an association mining algorithm to perform secondary verification and optimization processing on the dynamic threshold of the indicator to obtain an optimized dynamic threshold of the indicator.

[0082] The secondary verification and optimization process can be a process of adjusting and optimizing the dynamic threshold of the indicator.

[0083] Continue to refer to Figure 2 For historical business forms with category tags, further improved association rule algorithms (such as the Apriori algorithm) are used to mine association rules for indicators at dimensions such as cell, user, and slice, verifying the rationality of the dynamic thresholds for the indicators. For the determined dynamic thresholds for the indicators, the threshold adjustment module 232 can use an association mining algorithm to perform secondary verification and optimization processing on the dynamic thresholds for the indicators to obtain optimized dynamic thresholds for the indicators. Through the above processing steps, indicator thresholds that conform to the association rules can be obtained.

[0084] In one exemplary embodiment of this disclosure, an association mining algorithm is used to perform secondary verification and optimization processing on the dynamic threshold of the indicator to obtain an optimized dynamic threshold. This includes: classifying historical business forms according to business types to obtain frequent one-itemsets; determining the minimum support threshold and minimum confidence threshold corresponding to each category of business indicator; and determining whether to trigger a threshold update operation on the dynamic threshold of the indicator based on the frequent one-itemsets, the minimum support threshold, and the minimum confidence threshold to obtain the optimized dynamic threshold.

[0085] The business category can be any category of all specific businesses involved in the historical business forms. The minimum support threshold can be the minimum support value corresponding to a frequent itemset. The minimum confidence threshold can be the minimum confidence value of the association rules contained in the frequent itemset. The threshold update operation can be an automated operation that dynamically updates the dynamic thresholds of the optimization metrics.

[0086] For a historical business form dataset with a total of N business forms, the historical business forms can be classified according to business type to obtain frequent one-itemsets. For example, indicators can be divided into categories A, B, C, D, ... according to business type. The indicator features that each business type focuses on are represented by "business category + number". For example, the indicators for category A are represented by Ai (i = 1, 2, 3, ...). Scan the dataset to obtain all the indicators that have appeared, classify the indicators according to business, and select the business category containing multiple business indicators as candidate frequent one-itemsets. For example, the frequent one-itemset for category A is {A} = {A1, A2, A3, ...}.

[0087] Association rules are implications of the form A => B, where A and B are subsets of all itemsets and are not empty, and A intersects B to form an empty set. The support of an association rule is defined as: support(A => B) = P(A ∪ B). The confidence of an association rule is defined as: For each category of business indicators obtained from the division, the minimum support threshold and the minimum confidence threshold corresponding to each category of business indicators can be determined. The minimum support threshold can be labeled as Ts, and the minimum confidence threshold can be labeled as Tc.

[0088] Furthermore, based on the identified frequent one-itemsets and their corresponding minimum support and minimum confidence thresholds, it is determined whether to trigger a threshold update operation on the dynamic threshold of the indicator to obtain the optimized dynamic threshold. The improved Apriori algorithm mainly reflects its comprehensive consideration of frequent itemsets from a business perspective, eliminating the need to consider individual indicators. This significantly reduces the overhead caused by permutations and combinations and scanning the dataset, improving computational efficiency while ensuring accurate location of frequent itemsets.

[0089] In one exemplary embodiment of this disclosure, determining whether to trigger a threshold update operation on the dynamic threshold of an indicator to obtain an optimized dynamic threshold is based on frequent one-itemsets, a minimum support threshold, and a minimum confidence threshold. This includes: performing a frequent itemset determination operation based on frequent one-itemsets to obtain all frequent k-itemsets in historical business forms; calculating association rules based on all frequent k-itemsets to obtain indicator confidence; when the indicator confidence is greater than or equal to the minimum confidence threshold, using the dynamic threshold of the indicator as the optimized dynamic threshold; and when the indicator confidence is less than the minimum confidence threshold, triggering an update operation on the dynamic threshold of the indicator to obtain the optimized dynamic threshold.

[0090] Here, association rules can be specific rules that establish an association relationship with frequent itemsets. Indicator confidence can be the confidence level of association rules between different indicators.

[0091] The frequent itemset determination operation is performed based on frequent 1-itemsets to obtain all frequent k-itemsets in the historical business forms, where k can take the value 2, 3, 4, ..., N. All association rules are calculated from the frequent k-itemsets, and the indicator confidence score is calculated. If the indicator confidence score is greater than or equal to the minimum confidence threshold Tc, the indicator threshold of the items contained in the frequent itemset is considered reasonable, and the dynamic indicator threshold can be used as the optimized dynamic indicator threshold. If the indicator confidence score is less than the minimum confidence threshold Tc, the currently calculated dynamic indicator threshold is considered unreasonable, triggering an update operation until the updated dynamic indicator threshold meets the conditions, resulting in the optimized dynamic indicator threshold. During the optimization of the dynamic indicator threshold, the association rules between multiple business-related indicators are considered, making the calculated optimized dynamic indicator threshold more reasonable.

[0092] In one exemplary embodiment of this disclosure, a frequent itemset determination operation is performed based on frequent 1-itemsets to obtain all frequent k-itemsets in historical business forms, including: scanning frequent 1-itemsets; determining frequent k-itemsets and the support of each candidate frequent k-itemset based on the scanning results of frequent 1-itemsets; determining a frequent k-itemset based on the minimum support among multiple frequent k-itemsets; repeating the above steps until the number of determined frequent k+j itemsets is less than or equal to 1, thereby obtaining all frequent k-itemsets.

[0093] Suppose that the set itemset = {item_1, item_2, ..., item_m} is the set of all items, where item_k (k = 1, 2, ..., m) is called an item, the set of items is called an itemset, and an itemset containing k items is called a k-itemset. Frequent itemsets can be any itemset that satisfies the minimum support. The properties of frequent itemsets include: (1) all non-empty subsets of a frequent itemset are also frequent itemsets; (2) if itemset A is not a frequent itemset, then other itemsets or the union of itemset A with other itemsets or transactions are also not frequent itemsets. Frequent k-itemsets can be any k-itemsets that satisfy the minimum support. After scanning the entire business form dataset after classifying it according to business type, all the form data that have appeared and contain a single business type are taken as candidate frequent 1-itemsets, and then frequent k-itemsets are mined based on the candidate frequent 1-itemsets.

[0094] The specific process for mining frequent k-itemsets based on frequent 1-itemsets is as follows: Scan the data and calculate the support of candidate frequent k-itemsets. Consider that all subsets of frequent k-itemsets are also frequent itemsets, apply pruning rules to obtain all candidate frequent k-itemsets. Scan the dataset for candidate frequent k-itemsets to obtain the support of all candidate frequent k-itemsets, and calculate the minimum support threshold Ts based on the support of all candidate frequent k-itemsets. Based on the minimum support threshold Ts, obtain the frequent k+1 itemsets. Specifically, delete candidate frequent k-itemsets with support less than the minimum support threshold Ts to obtain the frequent k+1 itemsets.

[0095] Scan the dataset to obtain each item, generating a set C1 of candidate 1-itemsets. Then, count each item and remove items from C1 that do not meet the minimum support requirement, thus obtaining a frequent 1-itemset L1. Perform a pruning strategy on the set generated by self-joins of L1 to produce a set C2 of candidate 2-itemsets. Then, scan all transactions and count each item in set C2. Similarly, remove items from C2 that do not meet the minimum support requirement, thus obtaining a frequent 2-itemset L2. Continue in this manner to obtain frequent k-itemsets.

[0096] Repeat the above steps until the number of frequent itemsets generated at the (k+j)th layer is 0 or 1, thus obtaining all frequent k-itemsets. The value of k can be 2, 3, 4, ..., N; and the value of j can be 0, 1, 2, 3, 4, ..., N. Through these processing steps, all frequent k-itemsets contained in historical business forms can be obtained, making the determination process of dynamic thresholds for indicators more reasonable as it considers the association rules between multiple indicators.

[0097] In one exemplary embodiment of this disclosure, a real-time business form is used to update the threshold determination model, resulting in an updated threshold determination model. The updated threshold determination model is used to adjust the dynamic threshold of the indicator. The real-time business form is a business form obtained by periodically sampling based on the time dimension.

[0098] The updated threshold determination model can be a model obtained by optimizing and updating the threshold determination model using real-time business forms as model training data.

[0099] Continue to refer to Figure 2During network operation, new business forms will be continuously generated. The data acquisition module 210 can perform periodic data sampling operations based on the time dimension, such as sampling data according to a pre-configured time period to obtain real-time business forms. The time period can be configured to be one week, one month, or three months. After obtaining the real-time business forms, the real-time business forms are used as model training data for the threshold determination model. The threshold adjustment module 231 can further update the threshold determination model based on the real-time business forms to obtain an updated threshold determination model.

[0100] After obtaining the updated threshold determination model, the dynamic thresholds of business-related indicators are adjusted using this model. The form classification module 240 can then classify the business forms to be processed based on the adjusted dynamic thresholds, determining the corresponding classification labels for each form. Updating the threshold determination model using real-time business forms and then using the updated model for indicator threshold determination ensures that the determined dynamic thresholds are more accurate and reasonable.

[0101] In summary, the indicator threshold determination method disclosed herein determines dynamic thresholds for indicators, which are used to filter target business forms from pending business forms. These dynamic thresholds are generated based on a threshold determination model, which is trained on one or more dimensions of historical business forms, including time, geography, user, network, and business dimensions, and is dynamically updated based on real-time business forms. On one hand, the threshold determination model, trained on historical business forms, can adapt to multiple dimensions in historical business forms, obtaining dynamic thresholds for various indicators, thus improving network operation capabilities. On the other hand, since the threshold generation model can be dynamically updated based on real-time business forms, it can automatically adjust the dynamic thresholds, reducing manual intervention, improving work efficiency, and lowering operation and maintenance costs. Furthermore, the threshold determination model uses the n*σ criterion to flexibly address the differences in threshold values ​​for different indicators, making the generated thresholds more reasonable and effective. Finally, when performing secondary optimization on the dynamic thresholds, frequent itemsets are considered holistically according to the business dimension, eliminating the need to consider individual indicators, significantly reducing the overhead caused by permutations and combinations and scanning datasets, thus improving computational efficiency while ensuring accurate location of frequent itemsets.

[0102] It should be noted that although the steps of the method in this invention are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0103] Furthermore, in this example embodiment, an indicator threshold determination device is also provided. (Reference) Figure 5 The indicator threshold determination device 500 may include an indicator threshold determination module 510.

[0104] Specifically, the indicator threshold determination module 510 is used to determine the dynamic threshold of indicators. The dynamic threshold of indicators is used to filter target business forms from the business forms to be processed. The dynamic threshold of indicators is generated based on the threshold determination model. The threshold determination model is trained based on one or more dimensions of historical business forms, including time dimension, geographical dimension, user dimension, network dimension, and business dimension, and is dynamically updated based on real-time business forms.

[0105] In one exemplary embodiment of this disclosure, the indicator threshold determination device 500 further includes an initial historical form acquisition module, used to: collect network performance data, service quality data, configuration data and service operation data from the network through a deep packet inspection probe and a production system to generate an initial historical service form; the production system includes one or more of a network management system, a customer relationship management system, a work order system, and a customer service system.

[0106] In one exemplary embodiment of this disclosure, the indicator threshold determination device 500 further includes a historical form generation module, used to: review the initial historical business form to obtain a historical business form.

[0107] In one exemplary embodiment of this disclosure, the indicator threshold determination device 500 further includes a threshold model generation module, used to: train a pre-built initial model using historical business forms with classification labels to obtain a threshold determination model, wherein the threshold determination model uses probability density analysis and the n*σ criterion to determine the dynamic threshold of each business-related indicator in the historical business forms, and the dynamic threshold is used to determine the classification label of the business form to be processed.

[0108] In one exemplary embodiment of this disclosure, the threshold model generation module includes a threshold determination unit, used to: determine the upper limit and lower limit of the threshold of the business-related indicators using the n*σ criterion; determine the dynamic threshold of the indicators based on the upper limit and lower limit of the threshold; the upper limit and lower limit of the threshold are determined based on the probability distribution of the business-related indicators, and the probability distribution is determined based on the mean, variance and probability density function of the business-related indicators.

[0109] In one exemplary embodiment of this disclosure, the threshold determination unit includes a threshold determination subunit, used to: determine the indicator normal distribution of business-related indicators, wherein the indicator normal distribution is determined based on the upper limit of the indicator threshold, the lower limit of the indicator threshold and the group interval of the indicator group of business-related indicators, wherein the group interval of the indicator group is determined based on the upper limit of the indicator threshold, the lower limit of the indicator threshold and the number of threshold groups; and determine the indicator dynamic threshold according to the inverse function of the cumulative distribution function of the indicator normal distribution.

[0110] In one exemplary embodiment of this disclosure, the indicator threshold determination device 500 further includes a threshold update module, used to: perform secondary verification and optimization processing on the indicator dynamic threshold using an association mining algorithm to obtain an optimized indicator dynamic threshold.

[0111] In one exemplary embodiment of this disclosure, the threshold update module includes a threshold update unit, configured to: classify historical business forms according to business type to obtain frequent one-itemsets; determine the minimum support threshold and minimum confidence threshold corresponding to each category of business indicator; and determine whether to trigger a threshold update operation on the dynamic threshold of the indicator based on the frequent one-itemsets, the minimum support threshold, and the minimum confidence threshold, so as to obtain an optimized dynamic threshold for the indicator.

[0112] In one exemplary embodiment of this disclosure, the threshold update unit includes a threshold update subunit, configured to: perform a frequent itemset determination operation based on frequent 1-itemsets to obtain all frequent k-itemsets in the historical business form; calculate association rules based on all frequent k-itemsets to obtain the indicator confidence level; when the indicator confidence level is greater than or equal to the minimum confidence level threshold, use the indicator dynamic threshold as the optimized indicator dynamic threshold; when the indicator confidence level is less than the minimum confidence level threshold, trigger the execution of the indicator dynamic threshold update operation to obtain the optimized indicator dynamic threshold.

[0113] In one exemplary embodiment of this disclosure, the threshold update subunit includes a frequent itemset determination subunit, configured to: scan frequent 1-itemsets; determine frequent k-itemsets and the support of each frequent k-itemset based on the scanning results of the frequent 1-itemsets; determine a frequent k-itemset based on the minimum support among the support of multiple frequent k-itemsets; repeat the above steps until the number of determined frequent k+j itemsets is less than or equal to 1, thereby obtaining all frequent k-itemsets.

[0114] In one exemplary embodiment of this disclosure, the indicator threshold determination device 500 further includes a model update module, which is used to: use a real-time business form to perform model update processing on the threshold determination model to obtain an updated threshold determination model. The updated threshold determination model is used to adjust the dynamic threshold of the indicator. The real-time business form is a business form obtained by periodically sampling based on the time dimension.

[0115] The specific details of the virtual modules of the above-mentioned indicator threshold determination devices have been described in detail in the corresponding indicator threshold determination methods, so they will not be repeated here.

[0116] It should be noted that although several modules or units of the indicator threshold determination device have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0117] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0118] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented as entirely hardware embodiments, entirely software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, collectively referred to herein as “circuit,” “module,” or “system.”

[0119] The following is for reference. Figure 6 To describe an electronic device 600 according to such an embodiment of the present disclosure. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0120] like Figure 6 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including storage unit 620 and processing unit 610), and a display unit 640.

[0121] The storage unit stores program code that can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of this disclosure.

[0122] Storage unit 620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 621 and / or cache memory 622, and may further include read-only memory (ROM) 623.

[0123] Storage unit 620 may include a program / utility 624 having a set (at least one) program module 625, such program module 625 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0124] Bus 630 can represent one or more of several bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0125] Electronic device 600 can also communicate with one or more external devices 670 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. As shown, network adapter 660 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0126] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0127] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section above.

[0128] refer to Figure 7 As shown, a program product 700 for implementing the above-described method according to an embodiment of the present invention is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0129] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0130] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0131] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0132] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0133] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0134] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0135] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for determining an indicator threshold, characterized in that, include: Determine the dynamic threshold of the indicator, which is used to filter target business forms from the business forms to be processed; The dynamic threshold of the indicator is generated based on a threshold determination model and dynamically updated based on real-time business forms. The threshold determination model is trained based on one or more dimensions of historical business forms, including time dimension, geographic dimension, user dimension, network dimension, and business dimension, and is dynamically updated based on the real-time business forms. The dynamic threshold of the indicator is determined based on the following steps: using n The σ criterion determines the upper and lower limits of the threshold values ​​for each business-related indicator. The dynamic threshold of the indicator is determined based on the upper limit and the lower limit of the indicator threshold. The step of determining the dynamic threshold of the indicator based on the upper limit and the lower limit of the indicator threshold includes: Determine the normal distribution of the business-related indicators. The normal distribution is determined based on the upper limit of the indicator threshold, the lower limit of the indicator threshold, and the group interval of the indicator group of the business-related indicators. The group interval is determined based on the upper limit of the indicator threshold, the lower limit of the indicator threshold, and the number of threshold groups. The dynamic threshold of the index is determined by the inverse function of the cumulative distribution function of the normal distribution of the index.

2. The method according to claim 1, characterized in that, The method further includes: By using deep packet inspection probes and the production system, network performance data, service quality data, configuration data, and business operation data are collected from the network to generate initial historical business forms; the production system includes one or more of the following: network management system, customer relationship management system, work order system, and customer service system.

3. The method according to claim 2, characterized in that, After generating the initial historical business form, the method further includes: The initial historical business form is reviewed to obtain the historical business form.

4. The method according to claim 1, characterized in that, The threshold determination model is generated through the following steps: Using the historical business forms with classification labels, a pre-built initial model is trained to obtain the threshold determination model. The threshold determination model employs probability density analysis and n... The σ criterion is used to determine the dynamic threshold of each business-related indicator in the historical business form, and the dynamic threshold is used to determine the classification label of the business form to be processed.

5. The method according to claim 4, characterized in that, The upper and lower limits of the indicator thresholds are determined based on the probability distribution of the business-related indicators, which is determined based on the indicator mean, indicator variance, and probability density function of the business-related indicators.

6. The method according to claim 1, characterized in that, The method further includes: The dynamic threshold of the indicator is subjected to secondary verification and optimization using an association mining algorithm to obtain an optimized dynamic threshold.

7. The method according to claim 6, characterized in that, The step of using an association mining algorithm to perform secondary verification and optimization on the dynamic threshold of the indicator to obtain an optimized dynamic threshold includes: The historical business forms are categorized according to business type to obtain frequent item sets; Determine the minimum support threshold and minimum confidence threshold for each category of business indicators; Based on the frequent 1-itemset, the minimum support threshold, and the minimum confidence threshold, determine whether to trigger a threshold update operation for the dynamic threshold of the indicator to obtain the optimized dynamic threshold of the indicator.

8. The method according to claim 7, characterized in that, The step of determining whether to trigger a threshold update operation on the dynamic threshold of the indicator based on the frequent 1-itemset, the minimum support threshold, and the minimum confidence threshold, in order to obtain the optimized dynamic threshold of the indicator, includes: Based on the frequent 1-itemset, perform a frequent itemset determination operation to obtain all frequent k-itemsets in the historical business form; The association rules are calculated based on all the frequent k-itemsets, and the confidence level of the index is obtained. When the confidence level of the indicator is greater than or equal to the minimum confidence threshold, the dynamic threshold of the indicator is used as the dynamic threshold of the optimized indicator. When the confidence level of the indicator is less than the minimum confidence threshold, the dynamic threshold update operation of the indicator is triggered to obtain the optimized dynamic threshold of the indicator.

9. The method according to claim 8, characterized in that, The frequent itemset determination operation based on the frequent 1-itemsets yields all frequent k-itemsets in the historical business forms, including: Scan the frequent 1-itemsets, and determine the frequent k-itemsets and the support of each frequent k-itemsets based on the scan results; The frequent k-itemset is determined based on the minimum support among the support of multiple frequent k-itemsets; Repeat the above steps until the number of frequent k+j itemsets is less than or equal to 1, thus obtaining all the frequent k-itemsets.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: The real-time business form is used to update the threshold determination model to obtain an updated threshold determination model. The updated threshold determination model is used to adjust the dynamic threshold of the indicator. The real-time business form is a business form obtained by periodically sampling based on the time dimension.

11. A device for determining an index threshold, characterized in that, include: The indicator threshold determination module is used to determine the dynamic threshold of the indicator. The dynamic threshold of the indicator is used to filter the target business form from the business forms to be processed. The dynamic threshold of the indicator is generated based on the threshold determination model and dynamically updated based on the real-time business forms. The threshold determination model is trained based on one or more dimensions of the historical business forms, namely time dimension, geographical dimension, user dimension, network dimension, and business dimension, and is dynamically updated based on the real-time business forms. The indicator threshold determination module is also used to use n The σ criterion determines the upper and lower limits of the threshold values ​​for each business-related indicator. The dynamic threshold of the indicator is determined based on the upper limit and the lower limit of the indicator threshold. The step of determining the dynamic threshold of the indicator based on the upper limit and the lower limit of the indicator threshold includes: Determine the normal distribution of the business-related indicators. The normal distribution is determined based on the upper limit of the indicator threshold, the lower limit of the indicator threshold, and the group interval of the indicator group of the business-related indicators. The group interval is determined based on the upper limit of the indicator threshold, the lower limit of the indicator threshold, and the number of threshold groups. The dynamic threshold of the index is determined by the inverse function of the cumulative distribution function of the normal distribution of the index.

12. An electronic device, characterized in that, include: processor; as well as A memory storing computer-readable instructions that, when executed by the processor, implement the index threshold determination method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the index threshold determination method according to any one of claims 1 to 10.

Images