Network service quality evaluation method and device, equipment and readable storage medium

Abstract

The embodiment of the invention provides a network service quality evaluation method and device, equipment and a readable storage medium. The method comprises the following steps: acquiring abnormal traffic data and target light splitting data; performing calculation processing according to the target light splitting data and the abnormal flow data, and determining a first network service operator set and first evaluation data of each network service operator in the set; determining a second network service operator set and second evaluation data of each network service operator in the second network service operator set according to abnormal information in the abnormal information set; and determining target evaluation data of each network service operator in the first network service operator set and the second network service operator set according to the evaluation weight data, and determining the network service operator with the maximum target evaluation data as an abnormal network service operator. According to the embodiment of the invention, the abnormal network service operator can be quickly and accurately determined in combination with multiple methods, and the efficiency of network service quality evaluation can be improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a network service quality evaluation method, a network service quality evaluation device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Internet Data Center (IDC) refers to a platform that can provide customers with Internet basic platform services (such as server hosting, email caching, etc.) and various value-added services (such as domain name system services, load balancing systems, etc.). IDC usually uses the Border Gateway Protocol (BGP) to interconnect with network service operators. Network service operators can be, for example, Internet Service Providers (ISPs). ISPs can provide customers with Internet access, information services, and value-added services.

[0003] A network service system usually includes an IDC and multiple ISPs. When an ISP in the network service system is abnormal, the on-duty personnel need to check the ISPs one by one to determine the abnormal ISP. This method relies heavily on the professional skills and experience of the on-duty personnel, and the processing efficiency is very low. Therefore, how to accurately and efficiently determine the abnormal network service operator is a problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a network service quality assessment method, apparatus, device, and readable storage medium, which can combine multiple methods to use assessment data to quickly determine abnormal network service operators, thereby accurately determining abnormal network service operators, which is conducive to improving the efficiency of network service quality assessment.

[0005] On the one hand, an embodiment of the present application provides a method for evaluating network service quality, the method comprising:

[0006] Acquire abnormal traffic data between the data center and the service user and target optical splitting data corresponding to the abnormal traffic data, wherein the target optical splitting data includes network service operators on the transmission path of the abnormal traffic data;

[0007] Performing calculations and processing according to the target optical splitting data and the abnormal traffic data to determine a first set of network service operators and first evaluation data of each network service operator in the first set of network service operators;

[0008] Determine a second network service operator set and second evaluation data of each network service operator in the second network service operator set according to abnormal information in the abnormal information set, wherein the abnormal information is one or more of network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, and direct network service operator quality abnormal information of the service user;

[0009] The first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators are integrated and calculated according to the evaluation weight data to obtain the target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators, and the network service operator with the largest target evaluation data is determined as an abnormal network service operator.

[0010] On the one hand, an embodiment of the present application provides a network service quality evaluation device, the device comprising:

[0011] An acquisition unit, used to acquire abnormal traffic data between a data center and a service user and target optical splitting data corresponding to the abnormal traffic data, wherein the target optical splitting data includes network service operators on a transmission path of the abnormal traffic data;

[0012] a processing unit, configured to perform calculations based on the target optical splitting data and the abnormal traffic data to determine a first set of network service operators and first evaluation data of each network service operator in the first set of network service operators;

[0013] The processing unit is further used to determine a second network service operator set and second evaluation data of each network service operator in the second network service operator set according to abnormal information in the abnormal information set, wherein the abnormal information is one or more of network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, and direct network service operator quality abnormal information of the service user;

[0014] a calculation unit, configured to integrate and calculate the first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators according to the evaluation weight data, to obtain the target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators, and to determine the network service operator with the largest target evaluation data as an abnormal network service operator.

[0015] On the one hand, an embodiment of the present application provides a computer device, comprising: a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores an executable program code, and the processor is used to call the executable program code to implement the network service quality evaluation method provided in the embodiment of the present application.

[0016] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is executed on a computer, the computer implements the network service quality evaluation method provided in the embodiment of the present application.

[0017] Accordingly, an embodiment of the present application further provides a computer program product, the computer program product comprising a computer program or a computer instruction, the computer program or the computer instruction being stored in a computer-readable storage medium. A processor of a computer device reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the computer device implements the network service quality evaluation method provided in the embodiment of the present application.

[0018] In the present application, abnormal traffic data and target spectroscopic data are obtained; calculation and processing are performed based on the target spectroscopic data and the abnormal traffic data to determine a first set of network service operators and first evaluation data of each network service operator in the set; a second set of network service operators and second evaluation data of each network service operator in the set are determined based on abnormal information in the abnormal information set; target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators are determined based on evaluation weight data, and the network service operator with the largest target evaluation data is determined as an abnormal network service operator. Through the network service quality evaluation method provided by the embodiment of the present application, multiple evaluation data of network service operators can be determined by using multiple different methods, and abnormal network service operators can be accurately determined by combining multiple evaluation data; the first network service operator set and the first evaluation data of each network service operator in the set can be determined by using abnormal traffic data and spectral data, and the abnormal traffic data and spectral data are fully utilized to determine the network service operators associated with the abnormal traffic data, effectively narrowing the screening range of screening operators; the second network service operator set and the second evaluation data of each network service operator in the set can be determined by using multiple abnormal information in the abnormal information set, and the abnormal information of multiple dimensions is comprehensively considered, which is conducive to more accurate determination of abnormal network service operators; the determined network service operators can be integrated and calculated according to the evaluation weight data to determine the target evaluation data of each network service operator, so that the abnormal network service operators can be intuitively and accurately determined, effectively improving the efficiency of network service quality evaluation; compared with the method of manually determining abnormal network service operators, the method provided by the present application realizes automatic judgment of abnormal network service operators, effectively reducing the resources consumed by network service quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 It is a system architecture diagram of a network service quality evaluation system provided by an embodiment of the present application;

[0021] Figure 2 It is a flowchart of a network service quality evaluation method provided by an embodiment of the present application;

[0022] Figure 3A schematic diagram showing abnormal network connection information provided by an embodiment of the present application is shown;

[0023] Figure 4 This is a schematic diagram of data flow alarm information provided by an embodiment of the present application;

[0024] Figure 5 This is a schematic diagram of quality abnormality information of a designated network service operator provided in an embodiment of the present application;

[0025] Figure 6 This is a schematic diagram of abnormal quality information of a direct-connected network service operator provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of target evaluation data provided by an embodiment of the present application is shown;

[0027] Figure 8 It is a flowchart of another network service quality evaluation method provided by an embodiment of the present application;

[0028] Fig. 9 A method for determining an abnormal network service operator provided by an embodiment of the present application is shown;

[0029] Fig.10 The method for determining the first network service operator set provided by an embodiment of the present application is shown;

[0030] Fig.11 A method for determining target evaluation data provided by an embodiment of the present application is shown;

[0031] Fig.12 It is a schematic diagram of a network service operator scheduling method provided by the implementation of this application;

[0032] Fig.13 The embodiment of the present application provides a schematic diagram of another network service operator scheduling method;

[0033] Fig.14 A network service scheduling method provided by an embodiment of the present application is shown;

[0034] Fig.15 It is a structural block diagram of a network service quality evaluation device provided in an embodiment of the present application;

[0035] Fig.16 It is a structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] It should be noted that the descriptions of "first", "second", etc. involved in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0038] In a network service system composed of an Internet Data Center (IDC) and network service operators (such as Internet Service Providers (ISPs)), when a network service operator is abnormal, in order to ensure the normal operation of the network service system, the solution is generally: the on-duty personnel check all network service operators in the network service system one by one, find the abnormal network service operator, and deactivate the abnormal network service operator. However, this method is heavily dependent on the professional skills of the on-duty personnel, is time-consuming, and has low efficiency, which easily causes the network service system to be in an abnormal state for a long time.

[0039] Based on this, the embodiment of the present application provides a network service quality evaluation method, which can obtain abnormal traffic data between a data center and a business user and target spectral data corresponding to the abnormal traffic data; calculate and process the target spectral data and the abnormal traffic data to determine the first network service operator set and the first evaluation data of each network service operator in the first network service operator set; determine the second network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the abnormal information in the abnormal information set; integrate and calculate the first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the evaluation weight data, obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set, and determine the network service operator with the largest target evaluation data as the abnormal network service operator. Through the method provided by the embodiment of the present application, the evaluation data of each network service operator can be determined in combination with multiple methods, so that the abnormal network service operator can be determined intuitively and accurately, effectively improving the efficiency of network service quality evaluation; compared with the method of manually determining abnormal network service operators, the method provided by the present application realizes automatic judgment of abnormal network service operators, effectively reducing the resources consumed by network service quality evaluation.

[0040] The network service quality evaluation method provided in the embodiment of the present application can be applied to the field of cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices. In the embodiment of the present application, the data center is an application scenario of cloud computing technology. The data center can form a network service system with multiple network service operators to provide Internet access services, information services, value-added services, etc. to a large number of users. When an abnormal network service operator appears in the network service system, the data center can use the network service quality evaluation method provided in the embodiment of the present application to accurately and efficiently determine the abnormal network service operator, effectively save resources, and ensure the user experience.

[0041] The following will introduce the architecture of the network service quality evaluation system provided by the embodiment of the present application in conjunction with the accompanying drawings.

[0042] See also Figure 1, which is a schematic diagram of the system architecture of a network service quality evaluation system provided in an embodiment of the present application, wherein the network service quality evaluation system includes a data center 101, a plurality of network service operators 102 ( Figure 1 The system shown includes network service operators 1021, network service operators 1022, network service operators 1023, network service operators 1024, network service operators 1025) and service use objects 103 ( Figure 1 The system shown includes a service usage object 1031, a service usage object 1032, and a service usage object 1033), and the network service operator 102 can perform data interaction with the data center 101 and the service usage object 103. Among them:

[0043] Data center 101 can be an application service platform that can provide Internet basic platform services (for example: server hosting, virtual hosting, mail cache, virtual mail, etc.) and various value-added services (for example: site rental services, domain name system services, load balancing systems, database systems, data backup services, etc.). The equipment corresponding to data center 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0044] The network service operator 102 may be a telecommunications operator that provides Internet access services, information services, and value-added services to service users. Splitters may be deployed between any two network service operators, between any network service operator and a data center, and between any network service operator and a service user. The splitters may be used to obtain split data containing traffic data transmission paths.

[0045] The business user 103 is an object that uses network business services, and may be a large cross-regional enterprise organization, an application software service provider, a system integration provider, a multimedia service provider, a website design and hosting provider, etc.

[0046] The following will explain in detail Figure 1 The working principle of the network service quality evaluation system shown:

[0047] When an abnormality occurs in the network service quality assessment system, the data center 101 can make an abnormality judgment: obtain the abnormal traffic data between the data center and each service user and the target spectroscopic data corresponding to the abnormal traffic data, and the target spectroscopic data includes the network service operators on the transmission path of the abnormal traffic data; the data center 101 can perform calculations and processing based on the target spectroscopic data and the abnormal traffic data to determine the first network service operator set and the first assessment data of each network service operator in the first network service operator set. After an abnormality occurs, the data center can also obtain a variety of abnormal information, and determine the second network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the abnormal information in the abnormal information set. The abnormal information can be one or more of network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, and service user's direct network service operator quality abnormal information; the data center 101 can integrate and calculate the first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the evaluation weight data, obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set, and determine the network service operator with the largest target evaluation data as the abnormal network service operator. After the data center 101 determines the abnormal network service operator, it can perform adaptive self-healing scheduling according to the abnormal severity of the abnormal network service operator, so that the entire network service quality evaluation system can operate normally. Through the network service quality evaluation method provided in the embodiment of the present application, the evaluation data of the network service operator can be determined from multiple dimensions, and the abnormal network service operator can be comprehensively and accurately determined based on the evaluation data of multiple dimensions, which can effectively improve the efficiency of anomaly detection and reduce the cost required for anomaly repair.

[0048] It is understandable that the architecture diagram of the network service quality evaluation system described in the embodiment of the present application is to more clearly illustrate the network service quality evaluation method of the embodiment of the present application, and does not constitute a limitation on the network service quality evaluation method provided in the embodiment of the present application. For example, the network service quality evaluation method provided in the embodiment of the present application can be executed by the data center 101, and can also be executed by other devices that are different from the data center 101 and can obtain abnormal traffic data, target spectral data, abnormal information, etc. A person of ordinary skill in the art will know that Figure 1The number of data centers 101, network service operators 102, and service users 103 is only for illustration. Any number of devices and nodes may be configured according to the needs of service implementation. Moreover, with the evolution of system architecture and the emergence of new service scenarios, the network service quality evaluation method provided in the embodiment of the present application is also applicable to similar technical problems.

[0049] See also Figure 2 , Figure 2 A flowchart of a network service quality evaluation method provided in an embodiment of the present application. The network service quality evaluation method can be implemented by the above-mentioned data center 101, or by other devices. The process of the network service quality evaluation method provided in the embodiment of the present application includes but is not limited to:

[0050] S201. Acquire abnormal traffic data between a data center and a service user and target optical splitting data corresponding to the abnormal traffic data, wherein the target optical splitting data includes network service operators on a transmission path of the abnormal traffic data.

[0051] In the embodiment of the present application, a data center, multiple network service operators and multiple business users together constitute a network service quality evaluation system. The data center can provide Internet basic platform services (for example: server hosting, virtual hosting, mail cache, virtual mail, etc.) and various value-added services for business users. The network service operator is usually located between the data center and the business user, and is used to forward data between the data center and the business user. The network service operator can be an Internet service provider (ISP). A network service quality evaluation system usually includes a large number of network service operators. When a network service operator is abnormal, the data center can obtain abnormal traffic data between the data center and each business user and target split data corresponding to the abnormal traffic data. The abnormal traffic data can be traffic data in which abnormal situations such as packet loss occur in the network service quality evaluation system. The target split data of the abnormal traffic data includes the network service operators on the transmission path of the abnormal traffic data. For example: for an abnormal traffic data transmitted from a business user to a data center, its corresponding target split data may include the network service operator M1 and the network service operator M2 on the transmission path of the abnormal traffic data. The method provided in the embodiment of the present application can accurately obtain the spectral data associated with the abnormal traffic data, which is convenient for subsequent abnormal analysis and processing using the spectral data to obtain accurate results.

[0052] S202: Perform calculations based on the target optical splitting data and the abnormal traffic data to determine a first network service operator set and first evaluation data of each network service operator in the first network service operator set.

[0053] In the embodiment of the present application, the target optical splitting data of each abnormal traffic data and the abnormal traffic data can be calculated and processed to determine the first network service operator set, which may include multiple network service operators, and these network service operators are all on the transmission path of one or some abnormal traffic data, and the first evaluation data of each network service operator in the set can be determined, and the first evaluation data can indicate the probability of the network service operator being an abnormal network service operator to a certain extent. Through the method provided in the embodiment of the present application, the abnormal traffic data and the optical splitting data can be effectively used to determine the first evaluation data of the network service operator on the transmission path of the abnormal traffic data, which is conducive to the subsequent accurate determination of the abnormal network service operator based on the first evaluation data.

[0054] S203. Determine a second network service operator set and second evaluation data of each network service operator in the second network service operator set according to the abnormal information in the abnormal information set, wherein the abnormal information is one or more of network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, and direct network service operator quality abnormal information of the service user.

[0055] In the embodiment of the present application, when an abnormality occurs in the network service quality evaluation system, the data center will receive abnormal information of different dimensions, and can determine the network service operator that may have the abnormality based on the abnormal information. The abnormal information set of the data center may include one or more abnormal information, and these abnormal information may be network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, or service user's direct network service operator quality abnormal information. The network connection abnormal information may be network connection abnormal alarm information in the network service quality evaluation system, and the type of network connection abnormality may be disconnection, "BGP down", etc. Please refer to Figure 3 , which shows a schematic diagram of network connection abnormality information provided in an embodiment of the present application. Figure 3In the network connection abnormality information, the "Alarm Number" is "602187421864" and the "Details" are "Network device (Operator A in Region A and Region B) had a GBP down alarm at 12:00 on January 2, 2023." Through the network connection abnormality information, the abnormal network service operator "Operator A in Region A and Region B" can be added to the second network service operator set, and the second evaluation data of the network service operator can be determined. When the change rate of the transmission traffic value of a network service operator is too large within a period of time, the system will automatically generate data traffic alarm information. The data traffic alarm information includes the network service operator with abnormal transmission traffic value changes, and the second evaluation data of the network service operator can be determined based on the data traffic alarm information. Please refer to Figure 4 , which is a schematic diagram of data flow alarm information provided in an embodiment of the present application. Figure 4 In the data flow alarm information, the network service operator with abnormal traffic is "Operator B in Region A, Region B", and the outbound alarm is "The value of the traffic transmitted in 3 minutes has dropped by 3327M, and the amount of change is about 1.0 times the value of the traffic at the most recent normal time". The outbound direction generally refers to the direction from the data center to the service user. The outbound evaluation data of this network service operator is 100; the inbound direction generally refers to the method of transmission from the service user to the data center. The inbound alarm of this network service operator is "The value of the traffic transmitted in 3 minutes has dropped by 3041M, and the amount of change is about 1.0 times the value of the traffic at the most recent normal time". The inbound evaluation data is 100. According to Figure 4 The data traffic alarm information shown in can add the network service operator "Operator B in Region A and Region B" to the second network service operator set, and the second evaluation data of the network service operator "Operator B in Region A and Region B" can be determined based on the data traffic alarm information. In some cases, through manual settings, the traffic data can be transmitted through a few fixed network service operators, so that the service user can obtain network services. Therefore, the quality abnormality information of the specified network service operator can be obtained, which can indicate whether the specified network service operator has an abnormality. For example: by setting it so that the service user object User1 and the data center exchange data through network service operator Y1 and network service operator Y2, then when the service user object User1 cannot obtain network services normally, it means that the abnormal network service operator is more likely to be one or two of network service operator Y1 and network service operator Y2. Please refer to Figure 5 , which is a schematic diagram of quality abnormality information of a designated network service operator provided in an embodiment of the present application.

[0056] Figure 5In the “Abnormal Diagnosis Result”, the abnormal analysis result is determined based on the abnormal quality information of the designated network service operator, which is “Operator C in Region A and Region B is likely to be an abnormal network service operator”. The “Abnormal Diagnosis Details” is used to indicate the detailed information determined based on the abnormal quality information of the designated network service operator. When the service user cannot be used normally, an alarm message will be sent to the data center. The data center will analyze the received alarm message and obtain the “Abnormal Diagnosis Details”, that is, “A total of 26 alarm messages of service users were analyzed this time, of which about 25 alarm messages for Operator C in Region A and Region B exceeded the established attribute value and were determined to be abnormal.” Figure 5 In the data, the quality analysis data of the designated network service operator can also reflect the abnormal quality of operator C in region A and region B: "Location" refers to the location of the network service operator, which is "region A and region B", "Destination" refers to the location of the service user corresponding to the network service operator, which is "region A and region C", "Number of service user objects" indicates the object of the service user corresponding to the network service operator, which is 15751, "Type" is "invalid", "Transit1" corresponds to an abnormal evaluation value of "48.85", and "Transit2" corresponds to an abnormal evaluation value of "37.84". The values ​​corresponding to each item of the network service operator are relatively large, so it can be roughly determined that the network service operator is an abnormal network service operator. Therefore, the network service operator "Operator C in region A and region B" can be added to the second network service operator set, and the second evaluation data of the network service operator "Operator C in region A and region B" can be determined according to the data traffic alarm information. The network service operator with which the service user object directly interacts with data is called the direct-connected network service operator of the service user object. Assume that a network service operator is directly connected to 100 service users. If all 100 service users are unable to obtain network services, the quality of the direct-connected network service operator can be roughly determined. Figure 6 , which is a schematic diagram of abnormal quality information of a direct network service operator provided in an embodiment of the present application. Figure 6 The displayed result is the result of the quality anomaly analysis of the direct network service operator, which is "obtained based on the alarm information analysis of the service user". The operator name is "Operator D in Region A and Region B", the "Network Module" indicates the module to which the operator belongs, which is "Region A and Region B Network Module", the "Region" is "Region A", the "Region B" is "Region B", and the "Quality Value" is "100". Figure 6The results of the abnormal quality analysis of the directly connected network service operator shown can add the network service operator "Operator D in Region A, Region B" to the second set of network service operators, and can determine the second evaluation data of the network service operator "Operator D in Region A, Region B" based on the data traffic alarm information. Through the method provided in the embodiment of the present application, the second set of network service operators including multiple network service operators can be determined using abnormal information of multiple dimensions, and the second evaluation data of each network service operator can be determined based on the abnormal information, which is conducive to the subsequent accurate quality judgment of the network service operator based on the second evaluation data.

[0057] S204. Integrate and calculate the first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators according to the evaluation weight data to obtain target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators, and determine the network service operator with the largest target evaluation data as an abnormal network service operator.

[0058] In the embodiment of the present application, the first network service operator set may include multiple network service operators, the second network service operator set may include multiple network service operators, and there may be overlapping network service operators in the first network service operator set and the second network service operator set. The target evaluation data of each network service operator in the first network service operator set and the second network service operator set can be obtained by integrating and calculating the evaluation weight data. Since the evaluation data can reflect the probability that the network service operator is an abnormal network service operator to a certain extent, the network service operator with the largest target evaluation data can be determined as an abnormal network service operator. Please refer to Figure 7 , which shows a schematic diagram of target evaluation data provided in an embodiment of the present application. Figure 7 In the "Diagnosis Result", the target evaluation data of each network service operator is displayed, where the target evaluation data corresponding to the "Name" of "Network Service Operator M3" is "40"; the target evaluation data corresponding to the "Name" of "Network Service Operator M4" is "16.6"; and the target evaluation data corresponding to the "Name" of "Network Service Operator M5" is "8.4". The method provided in the embodiment of the present application effectively integrates the evaluation data of different dimensions, converts the abnormal information into the evaluation data of the network service operator, so that the abnormal network service operator can be accurately determined according to the evaluation data.

[0059] Based on the above embodiments, the beneficial effects of the present application are as follows: the network service quality evaluation method provided by the embodiments of the present application can perform a comprehensive analysis based on the abnormal traffic data in the entire system to determine the first set of network service operators and the evaluation data of each network service operator in the set; it can obtain abnormal information of multiple dimensions, analyze and process the abnormal information, and convert a large amount of abnormal information into second evaluation data of network service operators, which is convenient for subsequent integrated calculation, and this process does not require human assistance, so the analysis efficiency can be effectively improved; the target evaluation data of each network service operator can be determined by combining the evaluation data of multiple dimensions, so that the target evaluation data can accurately reflect the abnormal probability of the network service operator; the method provided by the embodiments of the present application can accurately and efficiently determine abnormal network service operators, and the entire determination process does not require human participation, so it effectively saves resources and reduces the resources consumed by network service quality evaluation.

[0060] See also Figure 8 , Figure 8 A flowchart of another network service quality evaluation method provided in an embodiment of the present application. The network service quality evaluation method can be implemented by the above-mentioned data center 101, or by other devices. The process of the network service quality evaluation method provided in the embodiment of the present application includes but is not limited to:

[0061] S801. Acquire abnormal traffic data including a plurality of fault traffic data between a data center and a business user object, and determine data identification information of any fault traffic data among the plurality of fault traffic data.

[0062] In the embodiment of the present application, a data center, a plurality of network service operators and a plurality of business users jointly constitute a network service quality evaluation system. The data center can provide Internet basic platform services and various value-added services for business users. The network service operator is usually located between the data center and the business user, and is used to forward data between the data center and the business user. The network service operator can be an Internet service provider (ISP). A network service quality evaluation system usually includes a large number of network service operators. When a network service operator is abnormal, the data center can obtain abnormal traffic data between the data center and each business user and target spectrometry data corresponding to the abnormal traffic data. The abnormal traffic data can be traffic data in which abnormal situations such as packet loss occur in the network service quality evaluation system, and the abnormal traffic data can include multiple fault traffic data. The data identification information of the fault traffic data can be determined according to any one of the multiple fault traffic data. Through the method provided in the embodiment of the present application, the data identification information of the fault traffic data can be determined, which is convenient for accurately determining the abnormal network service operator according to the data identification information in the subsequent process.

[0063] S802. Determine the first type of split data and the second type of split data corresponding to any fault traffic data according to the data identification information, and use the first type of split data and the second type of split data corresponding to the multiple fault traffic data as the target split data corresponding to the abnormal traffic data, wherein the target split data includes the network service operator on the transmission path of the abnormal traffic data.

[0064] In the embodiment of the present application, the first type of split data and the second type of split data corresponding to the fault flow data can be determined according to the data identification information, and the acquisition time of the first type of split data is earlier than the acquisition time of the fault flow data, for example: the acquisition time of the fault flow data is 12:00 on January 3, 2023, then the acquisition time of the first type of split data can be 9:00 on January 3, 2023. The acquisition time of the second type of split data is later than the acquisition time of the fault flow data, for example: the acquisition time of the fault flow data is 12:00 on January 3, 2023, then the acquisition time of the second type of split data can be 12:05 on January 3, 2023. The first type of split data and the second type of split data corresponding to each fault flow data can be used as the target split data corresponding to the abnormal flow data. The target split data can include the network service operator on the transmission path of the abnormal flow data, for example: for an abnormal flow data transmitted by a service user to a data center, its corresponding target split data can include the network service operator M6 and the network service operator M7 on the transmission path of the abnormal flow data. Through the method provided in the embodiment of the present application, spectral data in two time dimensions can be obtained, which facilitates subsequent accurate and comprehensive abnormality analysis based on these spectral data.

[0065] In one embodiment, the implementation method of determining the first type of split data and the second type of split data corresponding to any fault flow data according to the data identification information can be: determining a split data table according to the data identification information, wherein the split data table contains a plurality of candidate split data associated with any fault flow data collected before the acquisition time of any fault flow data; randomly extracting the candidate split data in the split data table to obtain the first type of split data corresponding to any fault flow data; sending a detection data packet to a network service operator on the transmission path of any fault flow data, and receiving the second type of split data corresponding to any fault flow data returned by the network service operator on the transmission path of any fault flow data according to the detection data packet. The data center can obtain the split data in each splitter in the system every minute, and store the obtained split data in the split data table. The split data table corresponding to the fault flow data can be determined according to the data identification information, wherein the split data table contains a plurality of candidate split data associated with the fault flow data collected before the acquisition time of the fault flow data. Randomly extracting the candidate split data in the split data table to obtain the first type of split data corresponding to the fault flow data. Since in the network service quality evaluation system, the transmission paths of the flow data with the same data identification information are most likely the same, the corresponding spectroscopic data table is determined according to the data identification information of the fault flow data, and the spectroscopic data in the spectroscopic data table can be considered to be the spectroscopic data corresponding to the fault flow data to a certain extent. This method of determining the spectroscopic data of the fault flow data is applicable to abnormalities such as network disconnection and communication congestion in the system: Assuming that a communication congestion abnormality occurs in the system, after the data center receives the fault flow data, it cannot quickly obtain the spectroscopic data from the spectrometer corresponding to the fault flow data due to the system abnormality, but can quickly obtain the spectroscopic data table corresponding to the fault flow data from the database of the data center, thereby determining the first type of spectroscopic data of the fault flow data. Please refer to Table 1, which shows the performance data of two methods for obtaining the spectroscopic data of the fault flow data.

[0066] Table 1:

[0067]

[0068] As shown in Table 1 above, when the network service quality evaluation system has an abnormality of communication congestion, the "historical optical splitting data" method (i.e., the method of determining the first type of optical splitting data of the fault flow data according to the optical splitting data table) has an "acquisition speed" of "historical optical splitting data can be obtained before the abnormality occurs and can be used directly when the abnormality occurs", and the "data validity" of this method is "high data validity"; the "real-time optical splitting data" method (i.e., the method of determining the second type of optical splitting data of the fault flow data according to the data returned by the detection data packet based on the network service operator on the transmission path of the fault flow data) has an "acquisition speed" of "needs to be obtained during the abnormal period, affecting the overall diagnosis speed", and the "data validity" of this method is "abnormal network service operator may cause a large amount of path information not to be echoed, and the data is difficult to verify". Therefore, when an abnormality such as communication congestion occurs, the method of determining the first type of optical splitting data of the fault flow data according to the optical splitting data table can be used to accurately and effectively determine the optical splitting data of the fault flow data. The data center can also send a detection data packet to the network service operator on the transmission path of the fault flow data, and receive the second type of optical splitting data corresponding to the fault flow data returned by the network service operator on the transmission path of the fault flow data according to the detection data packet. This method of obtaining spectral data is applicable to abnormalities such as traffic concentration. For example, if a traffic concentration abnormality occurs in the system, most of the traffic flows to a certain network service operator, rather than being evenly distributed among various network service operators. At this time, the data center obtains fault traffic data. If the spectral data determined by the spectral data table method cannot accurately reflect the transmission path of the fault traffic data, the data center can send a detection data packet to the network service operator on the transmission path of the fault traffic data, and receive the second type of spectral data corresponding to the fault traffic data returned by the network service operator on the transmission path of the fault traffic data according to the detection data packet. Through the method provided in the embodiment of the present application, spectral data of multiple dimensions can be obtained, so that no matter what type of abnormality occurs, the corresponding network service operator can be determined based on the obtained multi-dimensional spectral data, so that the method provided by the present application has good universality and can cope with various abnormal situations.

[0069] S803. Determine multiple first candidate network service operators based on the first type of split data of each of the fault traffic data, and perform proportional calculation based on the total number of the abnormal traffic data and the number of abnormal traffic data corresponding to each of the first candidate network service operators to obtain evaluation data of each of the first candidate network service operators.

[0070] In an embodiment of the present application, multiple first candidate network service operators can be determined based on the first type of spectral data of each fault traffic data. For example, the abnormal traffic data includes three fault traffic data. According to the first type of spectral data of the first fault traffic data, network service operator S1 and network service operator S2 can be determined; according to the first type of spectral data of the second fault traffic data, network service operator S1 and network service operator S3 can be determined; according to the first type of spectral data of the third fault traffic data, network service operator S1 and network service operator S4 can be determined; then network service operator S1, network service operator S2, network service operator S3, and network service operator S4 are all first candidate network service operators. In the system, the common network service operator of multiple fault traffic data is more likely to be an abnormal network service operator. Therefore, the evaluation data of each first candidate network service operator can be obtained by performing a proportional calculation based on the total number of abnormal traffic data and the number of abnormal traffic data corresponding to each first candidate network service operator. Please refer to Fig. 9 , which shows a method for determining an abnormal network service operator provided by an embodiment of the present application. Fig. 9 Includes data center 901, network service operator 902, network service operator 903, network service operator 904, network service operator 905, network service operator 906, network service operator 907 and multiple business users. Assume that the abnormal traffic data includes four fault traffic data, among which the first type of splitting data of the first fault traffic data indicates that the transmission path of the fault traffic is from the data center to the network service operator 903, then to the network service operator 905, and finally to the business user; the first type of splitting data of the second fault traffic data indicates that the transmission path of the fault traffic is from the data center to the network service operator 903, then to the network service operator 906, and finally to the business user; the first type of splitting data of the third fault traffic data indicates that the transmission path of the fault traffic is from the data center to the network service operator 903, then to the network service operator 907, and finally to the business user; the first type of splitting data of the fourth fault traffic data indicates that the transmission path of the fault traffic is from the data center to the network service operator 904, then to the network service operator 906, and finally to the business user. By Fig. 9It can be seen that the total number of abnormal traffic data is 4, and the number of abnormal traffic data corresponding to network service operator 902 is 0. After proportional calculation, it can be obtained that the evaluation data of network service operator 903 is 0 (the specific calculation formula is 100*0 / 4); the number of abnormal traffic data corresponding to network service operator 903 is 3, and after proportional calculation, it can be obtained that the evaluation data of network service operator 903 is 75 (the specific calculation formula is 100*3 / 4); the number of abnormal traffic data corresponding to network service operator 904 is 1, and after proportional calculation, it can be obtained that the evaluation data of network service operator 904 is 25 (the specific calculation formula is 1 00*1 / 4); the number of abnormal traffic data corresponding to network service operator 905 is 1, then after proportional calculation, it can be obtained that the evaluation data of network service operator 905 is 25 (the specific calculation formula is 100*1 / 4); the number of abnormal traffic data corresponding to network service operator 906 is 2, then after proportional calculation, it can be obtained that the evaluation data of network service operator 906 is 50 (the specific calculation formula is 100*2 / 4); the number of abnormal traffic data corresponding to network service operator 907 is 2, then after proportional calculation, it can be obtained that the evaluation data of network service operator 907 is 25 (the specific calculation formula is 100*1 / 4). Through the method provided in the embodiment of the present application, the evaluation data of each candidate network service operator can be accurately determined by the proportional calculation method, so that accurate abnormal judgment can be made based on the determined evaluation data in the future, effectively improving the efficiency of quality evaluation.

[0071] In one embodiment, the implementation method of obtaining the evaluation data of each first candidate network service operator by performing a proportional calculation based on the total number of abnormal traffic data and the number of abnormal traffic data corresponding to each first candidate network service operator in the data outbound direction can be: performing a proportional calculation based on the total number of abnormal traffic data in the data outbound direction and the number of abnormal traffic data corresponding to each first candidate network service operator in the data outbound direction to obtain the outbound evaluation data of each first candidate network service operator, where the data outbound direction is the direction from the network service operator to the service user; performing a proportional calculation based on the total number of abnormal traffic data in the data inbound direction and the number of abnormal traffic data corresponding to each first candidate network service operator in the data inbound direction to obtain the inbound evaluation data of each first candidate service object, where the data inbound direction is the direction from the service user to the network service operator; and determining the evaluation data of the corresponding first candidate network service operator based on the outbound evaluation data and the inbound evaluation data of each first candidate network service operator. The data transmission direction of the network service operator is generally divided into data outbound and data inbound, where data outbound generally refers to the data transmission direction from the data center to the service user, and data inbound generally refers to the data transmission direction from the service user to the data center. The same network service operator may have different numbers of abnormal traffic data transmitted in different data transmission directions, so it is necessary to calculate separately in order to roughly determine which data transmission direction of the network service operator has an abnormality. The total number of abnormal traffic data in the data outbound direction and the number of abnormal traffic data corresponding to each first candidate network service operator in the data outbound direction can be proportionally calculated to obtain the outbound evaluation data of each first candidate network service operator; the total number of abnormal traffic data in the data inbound direction and the number of abnormal traffic data corresponding to each first candidate network service operator in the data inbound direction can be proportionally calculated to obtain the inbound evaluation data of each first candidate service object; the evaluation data of the corresponding first candidate network service operator can be determined based on the largest evaluation data in the outbound evaluation data and the inbound evaluation data of each first candidate network service operator. The method provided in the embodiment of the present application refines the abnormal judgment to the outbound or inbound direction of the network service operator, and determines the evaluation data of the candidate network service operator based on the evaluation data in different data transmission directions, which can make the final abnormal judgment result more refined and more accurate.

[0072] S804. Determine multiple second candidate network service operators based on the second type of split data of each of the fault traffic data, and perform proportional calculation based on the total number of the abnormal traffic data and the number of abnormal traffic data corresponding to each of the second candidate network service operators to obtain evaluation data of each of the second candidate network service operators.

[0073] In the embodiment of the present application, multiple second candidate network service operators can be determined according to the second type of split data of each fault traffic data, and the evaluation data of each second candidate network service operator can be obtained by performing a proportional calculation according to the total amount of abnormal traffic data and the amount of abnormal traffic data corresponding to each second candidate network service operator. The implementation method in the embodiment of the present application is the same as the implementation method for determining the evaluation data of the first candidate network service operator described above.

[0074] S805. Compare the evaluation data of each of the first candidate network service operators and the evaluation data of each of the second candidate network service operators to obtain a comparison result, and use the comparison result to determine a first network service operator set and first evaluation data of each network service operator in the first network service operator set according to the multiple first candidate network service operators or the multiple second candidate network service operators.

[0075] In an embodiment of the present application, the evaluation data of each first candidate network service operator and the evaluation data of each second candidate network service operator are compared and processed to obtain a comparison result; the comparison result can be used to determine the first network service operator set and the first evaluation data of each network service operator in the first network service operator set based on multiple first candidate network service operators or multiple second candidate network service operators. For example: three first candidate network service operators and their evaluation data (79, 18 and 3 respectively), two second candidate network service operators and their evaluation data (52 and 48 respectively) can be determined by the method described in the above embodiment, which means that the first candidate network service operator transmits abnormal traffic data more concentratedly. Therefore, if the comparison result is the first candidate network service operator, the first network service operator set and the first evaluation data of each network service operator in the set (79, 18 and 3 respectively) can be determined based on the three first candidate network service operators. Please refer to Fig.10 , which shows a method for determining a first network service operator set provided in an embodiment of the present application. Fig.10 In FIG. 1 , a plurality of first candidate network service operators and their evaluation data, a plurality of second candidate network service operators and their evaluation data determined according to the method provided in the above embodiment are displayed. Fig.10Among them, the "outbound count" (i.e. the number of abnormal traffic data transmitted by the network service operator in the data outbound direction) of the first candidate network service operator named "Operator E in Region A and Region B" is "5616", the "outbound evaluation data" is "42.5", the "inbound count" (i.e. the number of abnormal traffic data transmitted by the network service operator in the data inbound direction) is "5776", and the "inbound evaluation data" is "42.2"; the "outbound count" of the first candidate network service operator "Operator F in Region A and Region B" is "5561", the "outbound evaluation data" is "41.9", the "inbound count" is "5535", and the "inbound evaluation data" is "40.4". The second candidate network service operator "Carrier G in Region A and Region B" has an "outbound count" of "4836", an "outbound assessment data" of "39.1", an "inbound count" of "771", and an "inbound assessment data" of "42.1"; the second candidate network service operator "Carrier H in Region A and Region B" has an "outbound count" of "4542", an "outbound assessment data" of "36.7", an "inbound count" of "686", and an "inbound assessment data" of "37.5". Then the Fig.10 It can be seen that the two first candidate network service operators are network service operators in the first network service operator set, and the first evaluation data of the two network service operators are 42.5 and 41.9 respectively. Through the method provided in the embodiment of the present application, the first network service operator set and the first evaluation data of each network service operator in the set can be determined more reliably, which is conducive to the subsequent determination of abnormal network service operators based on these data.

[0076] S806. Determine a second network service operator set and second evaluation data of each network service operator in the second network service operator set based on the abnormal information in the abnormal information set, wherein the abnormal information is one or more of network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, and direct network service operator quality abnormal information of the service user.

[0077] In the embodiment of the present application, when an abnormality occurs in the network service quality evaluation system, the data center will receive abnormal information of different dimensions, and the network service operator that may have an abnormality can be determined based on the abnormal information. The abnormal information set of the data center may include one or more abnormal information, and these abnormal information may be network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, or service user object direct network service operator quality abnormal information. The network connection abnormal information may be network connection abnormal alarm information in the network service quality evaluation system, and the type of network connection abnormality may be disconnection, "BGP down", etc. When the rate of change of the transmission flow value of the network service operator is too large within a period of time, the system will automatically generate data flow alarm information, and the data flow alarm information includes the network service operator with abnormal transmission flow value change, and the second evaluation data of the network service operator can be determined based on the data flow alarm information. In some cases, through artificial setting, the flow data can be transmitted through a fixed number of network service operators, so that the service user object obtains network services. Therefore, the quality abnormal information of the designated network service operator can be obtained, and the information can indicate whether the designated network service operator has an abnormality. For example: by setting up the data exchange between the service user object User1 and the data center through the network service operator Y1 and the network service operator Y2, when the service user object User1 cannot obtain the network service normally, it means that the abnormal network service operator is more likely to be one or two of the network service operator Y1 and the network service operator Y2. The network service operator with which the service user object directly exchanges data is called the directly connected network service operator of the service user object. Assuming that a certain network service operator is directly connected to 100 service users, if these 100 service users are unable to obtain network services, the quality of the directly connected network service operator can be roughly determined. Through the method provided in the embodiment of the present application, the second network service operator set including multiple network service operators can be determined using abnormal information of multiple dimensions, and the second evaluation data of each network service operator can be determined based on the abnormal information, which is conducive to the subsequent accurate quality judgment of the network service operator based on the second evaluation data.

[0078] In one embodiment, the abnormal information is data traffic alarm information, and the implementation method of determining the second evaluation data of the second network service operator set and each network service operator in the second network service operator set according to the abnormal information in the abnormal information set may be: analyzing and processing the data traffic alarm information to obtain abnormal change information of outbound traffic and abnormal change information of inbound traffic of the second network service operator set and each network service operator in the second network service operator set, the abnormal change information of outbound traffic is used to indicate the change amount of data traffic in the data outbound direction of the network service operator during the statistical period, and the abnormal change information of inbound traffic is used to indicate the change amount of data traffic in the data inbound direction of the network service operator during the statistical period; determining the outbound traffic change evaluation data of each network service operator in the second network service operator set according to the abnormal change information of outbound traffic, and determining the inbound traffic change evaluation data of each network service operator in the second network service operator set according to the abnormal change information of inbound traffic; and taking the largest evaluation data of the outbound traffic change evaluation data and the inbound traffic change evaluation data of each network service operator in the second network service operator set as the second evaluation data of the corresponding network service operator in the second network service operator set. When the abnormal information is data flow alarm information, the data flow alarm information can be analyzed and processed to obtain the abnormal change information of the outbound flow and the abnormal change information of the inbound flow of the second network service operator set and each network service operator in the second network service operator set. The abnormal change information of the outbound flow is used to indicate the change amount of the data flow of the network service operator in the data outbound direction during the statistical period, and the abnormal change information of the inbound flow is used to indicate the change amount of the data flow of the network service operator in the data inbound direction during the statistical period. As mentioned above Figure 4As shown, the second network service set is determined according to the data traffic alarm information, and the set includes the network service operator "Operator B in Region A and Region B", and the abnormal change information of the outbound traffic of the network service operator is "3327M", and the abnormal change information of the inbound traffic is "3041M". The outbound traffic change assessment data of the network service operator can be determined according to the abnormal change information of the outbound traffic, for example: the abnormal change information of the outbound traffic is "3327M", and it is proportionally calculated with the normal traffic data to obtain the outbound traffic change assessment data of the network service operator as 100 (the specific calculation formula can be 100*normal traffic data / abnormal change information of outbound traffic). The inbound traffic change assessment data of each network service operator in the second network service operator set can also be determined according to the abnormal change information of the inbound traffic. The largest assessment data of the outbound traffic change assessment data and the inbound traffic change assessment data of the network service operator is used as the second assessment data of the network service operator in the second network service operator set. For example, if the outbound traffic change assessment data of a network service operator is 100 and the inbound traffic change assessment data is 80, the outbound traffic change assessment data of the network service operator can be determined as the second assessment data of the network service operator. Through the method provided in the embodiment of the present application, the second assessment data of the network service operator can be effectively determined based on the abnormal information, so that the abnormal information in the system can be effectively utilized, and the efficiency of service quality assessment can be improved.

[0079] It should be noted that when the abnormal information is network connection abnormal information, designated network service operator quality abnormal information or direct network service operator quality abnormal information of the service user object, the method for determining the second network service operator set and the second evaluation data of each network service operator in the second network service operator set based on the abnormal information is similar to the method described in the above embodiment.

[0080] S807. Integrate and calculate the first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators according to the evaluation weight data to obtain target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators, and determine the network service operator with the largest target evaluation data as an abnormal network service operator.

[0081] In the embodiment of the present application, the first network service operator set may include multiple network service operators, the second network service operator set may include multiple network service operators, and there may be overlapping network service operators in the first network service operator set and the second network service operator set, for example: network service operator 1 is included in the first network service operator set and in the second network service operator set. Each network service operator in the first network service operator set and each network service operator in the second network service operator set can be integrated and calculated according to the evaluation weight data to obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set. If a network service operator is included in the first network service operator set and not included in the second network service operator set, the second evaluation data of the network service operator is 0; if a network service operator is included in the second network service operator set and not included in the first network service operator set, the first evaluation data of the network service operator is 0. Since the evaluation data can reflect the probability that the network service operator is an abnormal network service operator to a certain extent, the network service operator with the largest target evaluation data can be determined as an abnormal network service operator. The method provided in the embodiment of the present application effectively integrates evaluation data of different dimensions and converts abnormal information into evaluation data of network service operators, so that abnormal network service operators can be accurately determined based on the evaluation data.

[0082] In one embodiment, the abnormal information in the abnormal information set is network connection abnormal information, and the evaluation weight data includes a first evaluation weight and a second evaluation weight; then the first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set are integrated and calculated according to the evaluation weight data to obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set, which can be implemented as follows: the network connection abnormal information is analyzed and processed to obtain the network abnormality level; the first evaluation weight and the second evaluation weight are determined according to the network abnormality level; the first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set are integrated and calculated according to the first evaluation weight and the second evaluation weight to obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set. The network connection abnormal information can be analyzed and processed to obtain the network abnormality level, which can reflect the severity of the current abnormality to a certain extent, for example: the network abnormality level can be divided into level 1, level 2 and level 3. The first evaluation weight and the second evaluation weight can be determined according to the network abnormality level, that is, the value of the evaluation weight data can be appropriately adjusted according to the abnormal information. For example, if the network anomaly level is level 3, it means that a relatively serious network connection anomaly has occurred in the system. Then, the second network service operator set determined based on the network connection anomaly information has a high probability of including abnormal network service operators. Therefore, the value of the second evaluation weight can be appropriately increased, while the value of the first evaluation weight can be appropriately reduced. The first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set can be integrated and calculated based on the first evaluation weight and the second evaluation weight to obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set.When the abnormal information set includes two abnormal information, and the two abnormal information are network connection abnormal information and data flow alarm information respectively, network service operator X1, network service operator X2 and their second evaluation data in the second network service operator set are determined according to the network connection abnormal information, and network service operator X3 and network service operator X4 in the second network service operator set are determined according to the data flow alarm information; the first evaluation weight, the second evaluation weight and the third evaluation weight can be determined according to the network connection abnormal information and the data flow alarm information, and the first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set are integrated and calculated according to the first evaluation weight, the second evaluation weight and the third evaluation weight to obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set.

[0083] In one embodiment, the first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators are integrated and calculated according to the first evaluation weight and the second evaluation weight to obtain the target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators. The implementation method can be as follows: for each network service operator in the first set of network service operators and the second set of network service operators, the first evaluation data and the second evaluation data of each network service operator are obtained; the third evaluation data of each network service operator is obtained by multiplying the first evaluation weight and the first evaluation data of each network service operator; the fourth evaluation data of each network service operator is obtained by multiplying the second evaluation weight and the second evaluation data of each network service operator; and the third evaluation data of each network service operator and the fourth evaluation data are summed to obtain the target evaluation data of each network service operator. After determining the first evaluation weight and the second evaluation weight, for each network service operator in the first set of network service operators and the second set of network service operators, obtain the first evaluation data and the second evaluation data of each network service operator (if the network service operator is only included in the first set of network service operators or only included in the second set of network service operators, the second evaluation data or the first evaluation data of the network service operator is zero); perform product calculation based on the first evaluation weight and the first evaluation data of the network service operator to obtain the third evaluation data of the network service operator; perform product calculation based on the second evaluation weight and the second evaluation data of the network service operator to obtain the fourth evaluation data of the network service operator; sum the third evaluation data and the fourth evaluation data of the network service operator to obtain the target evaluation data of the network service operator. When there are multiple abnormal information, the corresponding evaluation weights can be determined based on the multiple abnormal information, and weighted processing is performed based on the evaluation weights and the second evaluation data of the network service operator determined based on the abnormal information to obtain the weighted evaluation data. Please refer to Fig.11 , which shows a method for determining target evaluation data provided in an embodiment of the present application. Fig.11In the “Diagnosis Result Summary”, it is shown that the evaluation weight data includes the first evaluation weight and the second evaluation weight, the first evaluation weight (including the split data weight (outbound) and the split data weight (inbound)) is 0.2, and the second evaluation weight is 0.4; as can be seen from the figure, the first network service operator set includes three network service operators, namely “network service operator N1”, “network service operator N2” and “network service operator N3”, and their outbound evaluation data are 42.5, 41.9 and 15.6 respectively (that is, for the split data weight (outbound), the first score is 42.5 corresponding to “network service operator N1” , the second score is 41.9 corresponding to "network service operator N2", and the third score is 15.6 corresponding to "network service operator N3"), the inbound evaluation data of "network service operator N3" is "42.2", the inbound evaluation data of "network service operator N1" is "40.4", and the inbound evaluation data of "network service operator N2" is "17.4" (that is, for the split data weight (inbound), the first score is 42.2 corresponding to "network service operator N3", the second score is 40.4 corresponding to "network service operator N1", and the third score is 17.4 corresponding to "network service operator N2"). The second network service operator set includes a network service operator "network service operator N4", and its second evaluation data is "100". The target evaluation data of the above four network service operators can be determined by the method provided in the embodiment of the present application. Fig.11 As shown in the operator total score statistics table, the target evaluation data for the operator named "Network Service Operator N4" is "40"; the target evaluation data for the operator named "Network Service Operator N1" is "16.58"; the target evaluation data for the operator named "Network Service Operator N2" is "11.86"; the target evaluation data for the operator named "Network Service Operator N3" is "11.56". Therefore, it can be determined that the network service operator named "Network Service Operator N4" is an abnormal network service operator. Through the method provided in the embodiment of the present application, abnormal network service operators can be accurately and efficiently determined, which can effectively save the resources consumed by abnormal diagnosis and reduce the losses caused by abnormalities in the system.

[0084] In one embodiment, after determining the abnormal network service operator, it is also possible to: obtain the operator category of the abnormal network service operator; determine the abnormal level of the abnormal network service operator according to the abnormal information related to the abnormal network service operator in the abnormal information set; if the abnormal level reaches the preset level, the abnormal network service operator is switched according to the operator category; if the abnormal level does not reach the preset level, the network service operator connected to the target service user corresponding to the abnormal network service operator is switched according to the operator category. After determining the abnormal network service operator, the method provided in the embodiment of the present application can enable the data center to determine the scheduling scheme according to the operator type of the abnormal network service operator, thereby realizing system self-healing; after an abnormality occurs, the normal operation of the system can be guaranteed through self-healing scheduling. The operator category of the abnormal network service operator can be obtained, and different operators have different characteristics and need to adopt different scheduling methods. The abnormal level of the abnormal network service operator is determined according to the abnormal information related to the abnormal network service operator in the abnormal information set. Since the network service operator will cause losses to a certain extent, in order to minimize the scheduling loss, it is necessary to determine the abnormal level of the abnormal network service operator. If the abnormality level reaches the preset level, the network service operator switching process is performed on the abnormal network service operator according to the operator category, that is, if a serious fault occurs, the abnormal network service operator is switched and disabled; if the abnormality level does not reach the preset level, the network service operator connected to the target service user corresponding to the abnormal network service operator can be switched according to the operator category, that is, if a less serious fault occurs, only the network service operator connected to some service users affected by the abnormal network service operator is switched, so that these partial service users can obtain network services normally. Through the method provided in the embodiment of the present application, system self-healing scheduling can be carried out in a targeted manner, which not only ensures the normal operation of the system, but also reduces the operation and maintenance cost of the entire system.

[0085] In one embodiment, the abnormal level of the abnormal network service operator can be determined according to preset rules. An autonomous system (AS) refers to a group of networks that use a unified internal routing protocol. If the network router of a member unit is ready to use the BGP (Border Gateway Protocol) protocol, it can apply for an AS number, which is an autonomous system number. It is used to identify an independent autonomous system. Within the same autonomous system, the same internal routing protocol is used, and external routing protocols (usually BGP protocols) are used between autonomous systems. Therefore, the AS number can represent a business usage object to a certain extent. For example: if the operator type of the abnormal network service operator is "transit" type (i.e., general network type), the preset rules may be: in the user object alarms received by the data center, the business user objects (i.e., AS) affected by the fault cover more than five regions, and at least two business user objects in each region are affected by the fault; the data center receives abnormal traffic alarm information, the traffic fluctuation range in the alarm information exceeds 20%, and the network service operator exit in the alarm information is consistent with the abnormal network service operator exit; the number of business user objects affected by the fault reaches one-sixth of the total number of business user objects in the system; it is determined based on the spectroscopic data that the evaluation data of a certain network service operator is greater than 90 and the network service operator is an abnormal network service operator; the system generates a level 1 work order based on the fault situation; there is no recovery within 5 minutes after the alarm occurs. Since the "transit" type of network service operators are very important and cannot be switched at will, only when all the above preset rules are met can it be determined that the abnormal level meets the preset level, and the "transit" type of network service operators can be switched and scheduled. After the switching and scheduling, the abnormal network service operator will stop being used, and the data originally transmitted by the abnormal network service operator will be transmitted by other "transit" type network service operators. Since the number of "transit" type network service operators is small and very important, there are other restrictions when scheduling abnormal network service operators of the "transit" type: in the same area, only one abnormal network service operator of the "transit" type can be switched at a time; only a level 1 work order can switch abnormal network service operators of the "transit" type; after switching abnormal network service operators of the "transit" type, the area is locked, and no automatic scheduling and switching will be performed until the abnormal network service operator failure is repaired and put back into use.For another example: the operator type of the abnormal network service operator is the "PNI" type (i.e., a dedicated network type), then the preset rules may be: the data center receives abnormal traffic alarm information, the traffic fluctuation range in the alarm information exceeds 20%, and the network service operator exit in the alarm information is consistent with the abnormal network service operator exit; the abnormal network service operator is included in the multiple network service operators indicated by the object alarm received by the data center; the type of the abnormal network service operator is the "PNI" type. The "PNI" type of network service operator has a smaller impact and a relatively lower switching cost, so there are fewer preset conditions. When the above preset conditions are met, it can be determined that the abnormal level meets the preset level, and the abnormal network service operator of the "PNI" type can be switched. For another example: the operator type of the abnormal network service operator is the "IXP" type (i.e., the exchange network type), then the preset rule may be: the data center receives traffic abnormality alarm information, the traffic fluctuation range in the alarm information exceeds 20%, and the network service operator exit in the alarm information is consistent with the abnormal network service operator exit; the network service operator with the largest evaluation data determined based on the spectroscopic data of the abnormal traffic data is the abnormal network service operator; the type of the abnormal network service operator is the "IXP" type. The "IXP" type of network service operator has a smaller impact and a relatively lower switching cost, so there are fewer preset conditions. When the above preset conditions are met, it can be determined that the abnormal level meets the preset level, and the abnormal network service operator of the "IXP" type can be switched.

[0086] See also Fig.12 , which is a schematic diagram of a network service operator scheduling method provided by the implementation of this application. Fig.12 In the embodiment, the data center can perform system scheduling according to the method described in the above embodiment; after the scheduling is completed, the monitoring system is called to confirm the recovery status and determine whether the network condition has improved. Regardless of whether the network condition has improved, the scheduling information is recorded; after determining the recovery status, automatic fault reporting is performed, that is, the relevant technical personnel of the abnormal network service operator are contacted to notify them to repair the abnormal network service operator. After the abnormality is recovered, the monitoring system is called to determine the recovery status. After confirming the recovery, the scheduling operation is switched back and the order is closed, so that the system returns to normal. The method provided in the embodiment of the present application can effectively complete the self-healing scheduling of abnormal network service operators, effectively improve the system's abnormality handling efficiency, and reduce losses.

[0087] In one embodiment, when the abnormality level does not meet the preset level, in order to ensure the normal operation of the system, refined scheduling can be performed, that is, the network service operator connected to the target service user corresponding to the abnormal network service operator is switched according to the operator category. For example: the rules that need to be followed for refined scheduling are: alarm work orders of delay type are not automatically scheduled; alarm information in the same area within 4 hours, if diagnosed as the same abnormal network service operator, will no longer be repeatedly scheduled. The method provided in the embodiment of the present application can determine the abnormal data transmission direction of the abnormal network service operator, and targeted scheduling can be performed according to the operator category and the operator's abnormal data transmission direction (data outbound or data inbound). For example: for abnormal network service operators of the "transit" type in the data inbound direction, the abnormal network service operator is blacklisted and banned, and the dispatching effect is detected by the detector; all ASs in the area corresponding to the abnormal network service operator are detected in real time within 5 minutes after the dispatching, and the packet loss ratio of these ASs within 5 minutes after the dispatching is calculated to determine the number of ASs that have improved (i.e., the packet loss ratio has decreased after the dispatching), the number of ASs that have remained the same (i.e., the packet loss ratio has remained unchanged after the dispatching), and the number of ASs that have deteriorated (i.e., the packet loss ratio has increased after the dispatching). If the number of ASs that have improved is greater than the number of ASs that have deteriorated 5 minutes after the dispatching is completed, the dispatching is considered valid and the work order is confirmed to be valid; if the number of ASs that have improved is less than the number of ASs that have deteriorated 5 minutes after the dispatching is completed, the dispatching is considered invalid, the aforementioned dispatching operation is returned, and outbound dispatching is performed (i.e., if the abnormal situation has not improved after the inbound dispatching, outbound dispatching is attempted). Specifically, the relevant rules for blacklist ban processing are as follows: determine the area with the most packet loss according to the spectral number of abnormal traffic data (the number of packet losses in this area must be greater than 20 pairs), and select this area as the detection area for blacklist scheduling; for the selected area (the area contains two monitoring machines, monitoring machine A and monitoring machine B), perform blacklist network segment calculation, if the number of packet losses of monitoring machine A and monitoring machine B in the area is greater than 20% of the number of packet losses in the area, then ban the Internet Protocol addresses (Internet Protocol addresses) corresponding to monitoring machine A and monitoring machine B at the same time. ProtocolAddress, IP address), if the number of packet losses of monitoring machine A and monitoring machine B is less than 20% of the number of packet losses in the area, the IP addresses corresponding to monitoring machine A and monitoring machine B will not be banned; determine whether monitoring machine A and monitoring machine B are in the same / 24 network segment, if so, select any IP address of monitoring machine A and monitoring machine B to ban, so as to save space, otherwise both IP addresses will be banned; if the parent network segment mask of monitoring machine A and monitoring machine B is less than or equal to / 20, the blacklist of the area will not be performed, but a fixed IP blacklist will be used to directly ban the relevant IP addresses one by one.If the inbound scheduling is invalid, then outbound scheduling can be attempted. The outbound scheduling rules for abnormal network service operators of the "transit" type may include: the data center obtains all affected ASs from the service usage object alarm and compares them with the AS whitelist address library (the address library is extracted and established, and the ASs in the address library are important service usage objects, and it is necessary to give priority to ensuring that they can obtain network services normally); select all ASs included in the AS whitelist address library from the affected ASs, and if the number of selected ASs is greater than or equal to 5, then 5 are randomly selected from the selected multiple ASs; if the number of selected ASs is less than 5, then ASs are randomly selected from the affected ASs so that the number of ASs finally selected is 5; after selecting 5 ASs, static outbound scheduling is performed on the ASs so that the next hop of the AS points to a normal "transit" type network service operator; schedule and select ASs one by one, with an interval of 2 minutes between two adjacent AS schedulings; perform quality detection on the ASs after scheduling, if the status of the AS is not optimized 10 minutes after scheduling, it means that this scheduling is invalid, and the operation is returned without any scheduling. If it is an abnormal network service operator of the "PNI" type or "IXP" type for data inbound, its refined scheduling process is roughly similar to the scheduling process of the abnormal network service operator of the "transit" type for data inbound, and the inbound scheduling is performed first, and the outbound scheduling is performed after the inbound scheduling is invalid. If it is an abnormal network service operator of the "transit" type for data outbound, its refined scheduling process is to perform outbound scheduling first (the scheduling method is the same as the outbound scheduling method of the abnormal network service operator of the "transit" type for data inbound), and if the outbound scheduling is invalid, the inbound scheduling is performed (the scheduling method is the same as the inbound scheduling method of the abnormal network service operator of the "transit" type for data inbound). If it is an abnormal network service operator of the "PNI" type or "IXP" type for data outbound, its refined scheduling process is roughly similar to the scheduling process of the abnormal network service operator of the "transit" type for data outbound, and the outbound scheduling is performed first, and the inbound scheduling is performed after the outbound scheduling is invalid. Through the method provided in the embodiment of the present application, refined scheduling can be achieved, thereby effectively ensuring that important business users can obtain network services normally. At the same time, the method provided in the embodiment of the present application has good universality and can better handle abnormal situations of different types and different data transmission directions, effectively improving the efficiency of exception handling.

[0088] See also Fig.13 , which is a schematic diagram of another network service operator scheduling method provided in an embodiment of the present application. Fig.13In the process, the data center performs refined scheduling according to the method described in the above embodiment: the data center calls the AS whitelist and determines the execution direction according to the positioning result. If it is outbound, outbound scheduling is performed; if it is inbound, inbound scheduling is performed; after the scheduling is completed, wait for a period of time, call the monitoring system to confirm the recovery situation, and determine whether the network situation has improved. Regardless of whether it has improved, the abnormal data transmission direction is recorded. After confirming the network recovery situation, automatic fault reporting is performed; after the abnormal recovery, the monitoring system is called to determine the network recovery situation. If the recovery is confirmed, the operation is switched back and the order is closed to restore the system to normal. The method provided in the embodiment of the present application can perform refined scheduling for less serious anomalies, effectively reducing the cost of system scheduling, while ensuring the overall normal operation of the system.

[0089] In one embodiment, the overall application process of the method provided by the present application can be as follows: Fig.14 shown. Fig.14 A network service scheduling method provided by an embodiment of the present application is shown. In the network service scheduling method, the technical developer first determines the scope of automation, plans the automation process and builds the automation process, so that the entire network service quality evaluation system can automatically execute the abnormal diagnosis and scheduling method provided by the present application. During use, the business monitoring personnel do not need to intervene in the processing process too much. They only need to receive the alarm information and view the details of the alarm message. The data center will automatically locate, make decisions, and schedule according to the alarm information and observe the effect. After the scheduling is completed, the business monitoring personnel can automatically report the fault to the operator-related personnel so that they can repair the abnormal network service operator; after the system determines that the network quality has returned to normal, it will be cut back to make the system return to normal. The method provided by the implementation of this application can realize the automatic abnormal location, automatic scheduling, and automatic observation of the scheduling results of the system, effectively improving the efficiency of network service quality evaluation and system scheduling, and reducing the required manpower costs.

[0090] Based on the above embodiments, the beneficial effects of the present application are as follows: the network service quality evaluation method provided by the embodiment of the present application can comprehensively determine the evaluation data of the network service operator based on the spectral data of multiple dimensions, so that the evaluation data can more comprehensively and accurately reflect the quality of the network service operator, and also make the method provided by the present application have good universality and can be used to determine a variety of abnormal situations; the evaluation data of the network service operator can be determined based on the abnormal data of multiple dimensions, so that the final abnormal network service operator is more accurate and reliable; the weight data can be adaptively adjusted according to the abnormal information, so that the determined abnormal network service operator is more reasonable; different scheduling strategies can be executed according to the determined abnormal network service operator and the abnormal level, while ensuring the overall normal operation of the system, reducing the scheduling cost, saving human resources, and effectively improving the efficiency and accuracy of network service quality evaluation and abnormal scheduling.

[0091] See also Fig.15 , Fig.15 This is a structural block diagram of a network service quality evaluation device provided in an embodiment of the present application. The device includes:

[0092] An acquisition unit 1501 is used to acquire abnormal traffic data between a data center and a service user and target optical splitting data corresponding to the abnormal traffic data, wherein the target optical splitting data includes network service operators on a transmission path of the abnormal traffic data;

[0093] A processing unit 1502 is configured to perform calculation processing according to the target optical splitting data and the abnormal traffic data to determine a first network service operator set and first evaluation data of each network service operator in the first network service operator set;

[0094] The acquisition unit 1501 is further configured to determine a second network service operator set and second evaluation data of each network service operator in the second network service operator set according to the abnormal information in the abnormal information set, wherein the abnormal information is one or more of network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, and direct network service operator quality abnormal information of the service user;

[0095] The calculation unit 1503 is used to integrate and calculate the first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators according to the evaluation weight data, to obtain the target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators, and determine the network service operator with the largest target evaluation data as an abnormal network service operator.

[0096] In one embodiment, when the acquisition unit 1501 acquires abnormal traffic data between a data center and a business usage object and target splitting data corresponding to the abnormal traffic data, it is specifically used to: acquire abnormal traffic data between a data center and a business usage object, wherein the abnormal traffic data includes multiple fault traffic data; for any fault traffic data among the multiple fault traffic data, determine data identification information of the any fault traffic data; determine first type splitting data and second type splitting data corresponding to the any fault traffic data according to the data identification information, wherein an acquisition time of the first type splitting data is earlier than an acquisition time of the any fault traffic data, and an acquisition time of the second type splitting data is later than an acquisition time of the any fault traffic data; and use the first type splitting data and the second type splitting data corresponding to the multiple fault traffic data as target splitting data corresponding to the abnormal traffic data.

[0097] In one embodiment, when the acquisition unit 1501 determines the first type of spectral data and the second type of spectral data corresponding to any fault flow data according to the data identification information, it is specifically used to: determine a spectral data table according to the data identification information, wherein the spectral data table contains multiple candidate spectral data associated with any fault flow data collected before the acquisition time of any fault flow data; randomly extract the candidate spectral data in the spectral data table to obtain the first type of spectral data corresponding to any fault flow data; send a detection data packet to a network service operator on the transmission path of any fault flow data, and receive the second type of spectral data corresponding to any fault flow data returned by the network service operator on the transmission path of any fault flow data according to the detection data packet.

[0098] In one embodiment, when the acquisition unit 1501 performs calculation processing according to the target optical splitting data and the abnormal traffic data to determine the first network service operator set and the first evaluation data of each network service operator in the first network service operator set, it is specifically used to: determine a plurality of first candidate network service operators according to the first type optical splitting data of each fault traffic data; perform proportional calculation according to the total number of the abnormal traffic data and the number of abnormal traffic data corresponding to each first candidate network service operator to obtain the evaluation data of each first candidate network service operator; determine a plurality of second candidate network service operators according to the second type optical splitting data of each fault traffic data; perform proportional calculation according to the total number of the abnormal traffic data and the number of abnormal traffic data corresponding to each second candidate network service operator to obtain the evaluation data of each second candidate network service operator; compare the evaluation data of each first candidate network service operator and the evaluation data of each second candidate network service operator to obtain a comparison result, and determine the first evaluation data of each network service operator in the first network service operator set and the first network service operator set according to the plurality of first candidate network service operators or the plurality of second candidate network service operators using the comparison result.

[0099] In one embodiment, when the acquisition unit 1501 performs a proportional calculation based on the total number of the abnormal traffic data and the number of abnormal traffic data corresponding to each of the first candidate network service operators to obtain the evaluation data of each of the first candidate network service operators, it is specifically used to: perform a proportional calculation based on the total number of abnormal traffic data in the data outbound direction and the number of abnormal traffic data corresponding to each of the first candidate network service operators in the data outbound direction to obtain the outbound evaluation data of each of the first candidate network service operators, where the data outbound direction is the direction from the network service operator to the service user; perform a proportional calculation based on the total number of abnormal traffic data in the data inbound direction and the number of abnormal traffic data corresponding to each of the first candidate network service operators in the data inbound direction to obtain the inbound evaluation data of each of the first candidate service objects, where the data inbound direction is the direction from the service user to the network service operator; and determine the evaluation data of the corresponding first candidate network service operator based on the outbound evaluation data and the inbound evaluation data of each of the first candidate network service operators.

[0100] In one embodiment, the abnormal information is the data traffic alarm information; when the processing unit 1502 determines the second network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the abnormal information in the abnormal information set, it is specifically used to: analyze and process the data traffic alarm information to obtain the second network service operator set and the outbound traffic abnormal change information and the inbound traffic abnormal change information of each network service operator in the second network service operator set, the outbound traffic abnormal change information is used to indicate the change amount of the data traffic of the network service operator in the data outbound direction within the statistical period, and the in ... The change information is used to indicate the change in the data flow in the data inbound direction of the network service operator during the statistical period; the outbound flow change assessment data of each network service operator in the second network service operator set is determined according to the abnormal change information of the outbound flow, and the inbound flow change assessment data of each network service operator in the second network service operator set is determined according to the abnormal change information of the inbound flow; the largest assessment data of the outbound flow change assessment data of each network service operator in the second network service operator set and the inbound flow change assessment data is used as the second assessment data of the corresponding network service operator in the second network service operator set.

[0101] In one embodiment, the abnormal information in the abnormal information set is the network connection abnormal information, and the evaluation weight data includes a first evaluation weight and a second evaluation weight; when the processing unit 1502 integrates and calculates the first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the evaluation weight data to obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set, it is specifically used to: analyze and process the network connection abnormal information to obtain a network abnormality level; determine the first evaluation weight and the second evaluation weight according to the network abnormality level; integrate and calculate the first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the first evaluation weight and the second evaluation weight to obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set.

[0102] In one embodiment, when the calculation unit 1503 integrates and calculates the first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators according to the first evaluation weight and the second evaluation weight to obtain the target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators, it is specifically used to: for each network service operator in the first set of network service operators and the second set of network service operators, obtain the first evaluation data and the second evaluation data of each network service operator; perform product calculation based on the first evaluation weight and the first evaluation data of each network service operator to obtain the third evaluation data of each network service operator; perform product calculation based on the second evaluation weight and the second evaluation data of each network service operator to obtain the fourth evaluation data of each network service operator; and sum the third evaluation data of each network service operator with the fourth evaluation data to obtain the target evaluation data of each network service operator.

[0103] In one embodiment, the calculation unit 1503 is further used to: obtain the operator category of the abnormal network service operator; determine the abnormal level of the abnormal network service operator according to the abnormal information related to the abnormal network service operator in the abnormal information set; if the abnormal level reaches a preset level, perform network service operator switching processing on the abnormal network service operator according to the operator category; if the abnormal level does not reach the preset level, switch the network service operator connected to the target service user corresponding to the abnormal network service operator according to the operator category.

[0104] It can be understood that the functions of each functional unit of the network service quality evaluation device in the embodiment of the present application can be specifically implemented according to the network service quality evaluation method in the above method embodiment, and its specific implementation process can refer to the relevant description in the above network service quality evaluation method embodiment, which will not be repeated here.

[0105] Based on the above embodiments, the beneficial effects of the present application are as follows: the network service quality evaluation method provided by the embodiment of the present application can comprehensively determine the evaluation data of the network service operator based on the spectral data of multiple dimensions, so that the evaluation data can more comprehensively and accurately reflect the quality of the network service operator, and also make the method provided by the present application have good universality and can be used to determine a variety of abnormal situations; the evaluation data of the network service operator can be determined based on the abnormal data of multiple dimensions, so that the final abnormal network service operator is more accurate and reliable; the weight data can be adaptively adjusted according to the abnormal information, so that the determined abnormal network service operator is more reasonable; different scheduling strategies can be executed according to the determined abnormal network service operator and the abnormal level, while ensuring the overall normal operation of the system, reducing the scheduling cost, saving human resources, and effectively improving the efficiency and accuracy of network service quality evaluation and abnormal scheduling.

[0106] See also Fig.16 , Fig.16 A block diagram of a computer device provided in an embodiment of the present application. The computer device described in the embodiment of the present application includes: a processor 1601, a communication interface 1602, and a memory 1603. The processor 1601, the communication interface 1602, and the memory 1603 may be connected via a bus or other means, and the embodiment of the present application takes the connection via a bus as an example.

[0107] Among them, the processor 1601 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which can parse various instructions in the computer device and process various data of the computer device. For example, the CPU can be used to parse the power on and off instructions sent by the user to the computer device, and control the computer device to perform power on and off operations; for another example, the CPU can transmit various interactive data between the internal structures of the computer device, and so on. The communication interface 1602 can optionally include a standard wired interface, a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), which is controlled by the processor 1601 to send and receive data. The memory 1603 (Memory) is a memory device in the computer device for storing programs and data. It can be understood that the memory 1603 here can include both the built-in memory of the computer device and the extended memory supported by the computer device. The memory 1603 provides a storage space, which stores the operating system of the computer device, which may include but is not limited to: Android system, iOS system, Windows Phone system, etc., and this application does not limit this.

[0108] In the embodiment of the present application, the processor 1601 performs the following operations by running the executable program code in the memory 1603:

[0109] Acquire abnormal traffic data between the data center and the service user and target optical splitting data corresponding to the abnormal traffic data, wherein the target optical splitting data includes network service operators on the transmission path of the abnormal traffic data;

[0110] Performing calculations and processing according to the target optical splitting data and the abnormal traffic data to determine a first set of network service operators and first evaluation data of each network service operator in the first set of network service operators;

[0111] Determine a second network service operator set and second evaluation data of each network service operator in the second network service operator set according to abnormal information in the abnormal information set, wherein the abnormal information is one or more of network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, and direct network service operator quality abnormal information of the service user;

[0112] The first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators are integrated and calculated according to the evaluation weight data to obtain the target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators, and the network service operator with the largest target evaluation data is determined as an abnormal network service operator.

[0113] In one embodiment, when the processor 1601 obtains abnormal traffic data between a data center and a business usage object and target splitting data corresponding to the abnormal traffic data, it is specifically used to: obtain abnormal traffic data between the data center and the business usage object, the abnormal traffic data including multiple fault traffic data; for any fault traffic data among the multiple fault traffic data, determine the data identification information of any fault traffic data; determine the first type of splitting data and the second type of splitting data corresponding to any fault traffic data according to the data identification information, the acquisition time of the first type of splitting data is earlier than the acquisition time of any fault traffic data, and the acquisition time of the second type of splitting data is later than the acquisition time of any fault traffic data; use the first type of splitting data and the second type of splitting data corresponding to the multiple fault traffic data as the target splitting data corresponding to the abnormal traffic data.

[0114] In one embodiment, when the processor 1601 determines the first type of spectral data and the second type of spectral data corresponding to any fault flow data according to the data identification information, it is specifically used to: determine a spectral data table according to the data identification information, wherein the spectral data table contains multiple candidate spectral data associated with any fault flow data collected before the acquisition time of any fault flow data; randomly extract the candidate spectral data in the spectral data table to obtain the first type of spectral data corresponding to any fault flow data; send a detection data packet to a network service operator on the transmission path of any fault flow data, and receive the second type of spectral data corresponding to any fault flow data returned by the network service operator on the transmission path of any fault flow data according to the detection data packet.

[0115] In one embodiment, when the processor 1601 performs calculation processing according to the target optical splitting data and the abnormal traffic data to determine the first network service operator set and the first evaluation data of each network service operator in the first network service operator set, it is specifically used to: determine multiple first candidate network service operators according to the first type optical splitting data of each fault traffic data; perform proportional calculation according to the total number of the abnormal traffic data and the number of abnormal traffic data corresponding to each first candidate network service operator to obtain the evaluation data of each first candidate network service operator; determine multiple second candidate network service operators according to the second type optical splitting data of each fault traffic data; perform proportional calculation according to the total number of the abnormal traffic data and the number of abnormal traffic data corresponding to each second candidate network service operator to obtain the evaluation data of each second candidate network service operator; compare the evaluation data of each first candidate network service operator and the evaluation data of each second candidate network service operator to obtain a comparison result, and use the comparison result to determine the first evaluation data of each network service operator in the first network service operator set and the first network service operator set according to the multiple first candidate network service operators or the multiple second candidate network service operators.

[0116] In one embodiment, when the processor 1601 performs a proportional calculation based on the total number of the abnormal traffic data and the number of abnormal traffic data corresponding to each of the first candidate network service operators to obtain the evaluation data of each of the first candidate network service operators, it is specifically used to: perform a proportional calculation between the total number of abnormal traffic data in the data outbound direction and the number of abnormal traffic data corresponding to each of the first candidate network service operators in the data outbound direction to obtain the outbound evaluation data of each of the first candidate network service operators, where the data outbound direction is the direction from the network service operator to the service user; perform a proportional calculation between the total number of abnormal traffic data in the data inbound direction and the number of abnormal traffic data corresponding to each of the first candidate network service operators in the data inbound direction to obtain the inbound evaluation data of each of the first candidate service objects, where the data inbound direction is the direction from the service user to the network service operator; and determine the evaluation data of the corresponding first candidate network service operator based on the outbound evaluation data and the inbound evaluation data of each of the first candidate network service operators.

[0117] In one embodiment, the abnormal information is the data traffic alarm information; when the processor 1601 determines the second network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the abnormal information in the abnormal information set, it is specifically used to: analyze and process the data traffic alarm information to obtain the second network service operator set and the outbound traffic abnormal change information and the inbound traffic abnormal change information of each network service operator in the second network service operator set, the outbound traffic abnormal change information is used to indicate the change amount of the data traffic of the network service operator in the data outbound direction during the statistical period, and the in ... The change information is used to indicate the change in the data flow in the data inbound direction of the network service operator during the statistical period; the outbound flow change assessment data of each network service operator in the second network service operator set is determined according to the abnormal change information of the outbound flow, and the inbound flow change assessment data of each network service operator in the second network service operator set is determined according to the abnormal change information of the inbound flow; the largest assessment data of the outbound flow change assessment data of each network service operator in the second network service operator set and the inbound flow change assessment data is used as the second assessment data of the corresponding network service operator in the second network service operator set.

[0118] In one embodiment, the abnormal information in the abnormal information set is the network connection abnormal information, and the evaluation weight data includes a first evaluation weight and a second evaluation weight; when the processor 1601 integrates and calculates the first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the evaluation weight data to obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set, it is specifically used to: analyze and process the network connection abnormal information to obtain a network abnormality level; determine the first evaluation weight and the second evaluation weight according to the network abnormality level; integrate and calculate the first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the first evaluation weight and the second evaluation weight to obtain the target evaluation data of each network service operator in the first network service operator set and the second network service operator set.

[0119] In one embodiment, when the processor 1601 integrates and calculates the first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators according to the first evaluation weight and the second evaluation weight to obtain the target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators, it is specifically used to: for each network service operator in the first set of network service operators and the second set of network service operators, obtain the first evaluation data and the second evaluation data of each network service operator; perform product calculation based on the first evaluation weight and the first evaluation data of each network service operator to obtain the third evaluation data of each network service operator; perform product calculation based on the second evaluation weight and the second evaluation data of each network service operator to obtain the fourth evaluation data of each network service operator; and sum the third evaluation data of each network service operator with the fourth evaluation data to obtain the target evaluation data of each network service operator.

[0120] In one embodiment, the processor 1601 is further used to: obtain the operator category of the abnormal network service operator; determine the abnormal level of the abnormal network service operator according to the abnormal information related to the abnormal network service operator in the abnormal information set; if the abnormal level reaches a preset level, perform network service operator switching processing on the abnormal network service operator according to the operator category; if the abnormal level does not reach the preset level, switch the network service operator connected to the target service user corresponding to the abnormal network service operator according to the operator category.

[0121] In a specific implementation, the processor 1601, communication interface 1602 and memory 1603 described in the embodiment of the present application can execute the implementation method of the data center described in a network service quality evaluation method provided in an embodiment of the present application, and can also execute the implementation method described in a network service quality evaluation device provided in an embodiment of the present application, which will not be repeated here.

[0122] Based on the above embodiments, the beneficial effects of the present application are as follows: the network service quality evaluation method provided by the embodiment of the present application can comprehensively determine the evaluation data of the network service operator based on the spectral data of multiple dimensions, so that the evaluation data can more comprehensively and accurately reflect the quality of the network service operator, and also make the method provided by the present application have good universality and can be used to determine a variety of abnormal situations; the evaluation data of the network service operator can be determined based on the abnormal data of multiple dimensions, so that the final abnormal network service operator is more accurate and reliable; the weight data can be adaptively adjusted according to the abnormal information, so that the determined abnormal network service operator is more reasonable; different scheduling strategies can be executed according to the determined abnormal network service operator and the abnormal level, while ensuring the overall normal operation of the system, reducing the scheduling cost, saving human resources, and effectively improving the efficiency and accuracy of network service quality evaluation and abnormal scheduling.

[0123] The embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is run on a computer, the computer executes the network service quality evaluation method as described in the embodiment of the present application. The specific implementation method can be referred to the above description, and will not be repeated here.

[0124] The embodiment of the present application also provides a computer program product, which includes a computer program or a computer instruction, and the computer program or the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the computer device executes the network service quality evaluation method as described in the embodiment of the present application. The specific implementation method can be referred to the above description, which will not be repeated here.

[0125] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0126] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0127] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A network service quality evaluation method, characterized in that: The method comprises: Acquire abnormal traffic data between the data center and the service user and target optical splitting data corresponding to the abnormal traffic data, wherein the target optical splitting data includes network service operators on the transmission path of the abnormal traffic data; Performing calculations and processing according to the target optical splitting data and the abnormal traffic data to determine a first set of network service operators and first evaluation data of each network service operator in the first set of network service operators; Determine a second network service operator set and second evaluation data of each network service operator in the second network service operator set according to abnormal information in the abnormal information set, wherein the abnormal information is one or more of network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, and direct network service operator quality abnormal information of the service user; The first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators are integrated and calculated according to the evaluation weight data to obtain the target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators, and the network service operator with the largest target evaluation data is determined as an abnormal network service operator.

2. The method according to claim 1, characterized in that The obtaining of abnormal traffic data between the data center and the service user and target optical splitting data corresponding to the abnormal traffic data includes: Acquire abnormal traffic data between the data center and the service user object, wherein the abnormal traffic data includes a plurality of fault traffic data; For any fault flow data among the plurality of fault flow data, determine data identification information of the any fault flow data; Determine, according to the data identification information, the first type of spectral data and the second type of spectral data corresponding to any fault flow data, wherein the acquisition time of the first type of spectral data is earlier than the acquisition time of any fault flow data, and the acquisition time of the second type of spectral data is later than the acquisition time of any fault flow data; The first type of spectroscopic data and the second type of spectroscopic data corresponding to the plurality of fault flow data are used as target spectroscopic data corresponding to the abnormal flow data.

3. The method according to claim 2, characterized in that The step of determining the first type of split data and the second type of split data corresponding to any fault flow data according to the data identification information includes: Determine a spectroscopic data table according to the data identification information, wherein the spectroscopic data table contains a plurality of candidate spectroscopic data associated with any fault flow data and collected before the acquisition time of any fault flow data; Randomly extracting candidate spectral data in the spectral data table to obtain first type spectral data corresponding to any fault flow data; A detection data packet is sent to a network service operator on the transmission path of any fault traffic data, and second type split data corresponding to any fault traffic data returned by the network service operator on the transmission path of any fault traffic data according to the detection data packet is received.

4. The method according to claim 2, characterized in that: The performing calculation and processing according to the target optical splitting data and the abnormal traffic data to determine a first set of network service operators and first evaluation data of each network service operator in the first set of network service operators includes: Determine a plurality of first candidate network service operators according to the first type of split data of each of the fault traffic data; Performing proportional calculation based on the total amount of the abnormal traffic data and the amount of abnormal traffic data corresponding to each of the first candidate network service operators, to obtain evaluation data of each of the first candidate network service operators; Determine a plurality of second to-be-selected network service operators according to the second type of split data of each of the fault traffic data; Performing proportional calculation based on the total amount of the abnormal traffic data and the amount of abnormal traffic data corresponding to each of the second candidate network service operators, to obtain evaluation data of each of the second candidate network service operators; The evaluation data of each of the first candidate network service operators and the evaluation data of each of the second candidate network service operators are compared and processed to obtain a comparison result, and the comparison result is used to determine a first network service operator set and the first evaluation data of each network service operator in the first network service operator set according to the multiple first candidate network service operators or the multiple second candidate network service operators.

5. The method according to claim 4, characterized in that The obtaining evaluation data of each of the first candidate network service operators by performing proportional calculation based on the total amount of the abnormal traffic data and the amount of abnormal traffic data corresponding to each of the first candidate network service operators comprises: Calculate the ratio of the total amount of abnormal traffic data in the data outbound direction to the amount of abnormal traffic data corresponding to each of the first candidate network service operators in the data outbound direction, and obtain the outbound evaluation data of each of the first candidate network service operators, where the data outbound direction is the direction from the network service operator to the service user; Calculate the ratio of the total amount of abnormal traffic data in the data inbound direction to the amount of abnormal traffic data corresponding to each of the first candidate network service operators in the data inbound direction, and obtain the inbound evaluation data of each of the first candidate service objects, where the data inbound direction is the direction from the service user to the network service operator; The evaluation data of the corresponding first candidate network service operator is determined according to the outbound evaluation data and the inbound evaluation data of each first candidate network service operator.

6. The method according to claim 1, characterized in that The abnormal information is the data traffic alarm information; and determining the second network service operator set and the second evaluation data of each network service operator in the second network service operator set according to the abnormal information in the abnormal information set includes: Analyze and process the data traffic alarm information to obtain abnormal change information of outbound traffic and abnormal change information of inbound traffic of the second network service operator set and each network service operator in the second network service operator set, wherein the abnormal change information of outbound traffic is used to indicate the change amount of data traffic in the outbound direction of the network service operator during the statistical period, and the abnormal change information of inbound traffic is used to indicate the change amount of data traffic in the inbound direction of the network service operator during the statistical period; Determine the outbound traffic change assessment data of each network service operator in the second network service operator set according to the outbound traffic abnormal change information, and determine the inbound traffic change assessment data of each network service operator in the second network service operator set according to the inbound traffic abnormal change information; The largest evaluation data among the outbound traffic change evaluation data of each network service operator in the second network service operator set and the inbound traffic change evaluation data is used as the second evaluation data of the corresponding network service operator in the second network service operator set.

7. The method according to claim 1, characterized in that The abnormal information in the abnormal information set is the network connection abnormal information, and the evaluation weight data includes a first evaluation weight and a second evaluation weight; the first evaluation data of each network service operator in the first network service operator set and the second evaluation data of each network service operator in the second network service operator set are integrated and calculated according to the evaluation weight data to obtain target evaluation data of each network service operator in the first network service operator set and the second network service operator set, including: Analyzing and processing the network connection abnormality information to obtain a network abnormality level; Determining the first evaluation weight and the second evaluation weight according to the network anomaly level; The first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators are integrated and calculated according to the first evaluation weight and the second evaluation weight to obtain target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators.

8. The method according to claim 7, characterized in that The step of integrating and calculating the first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators according to the first evaluation weight and the second evaluation weight to obtain target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators includes: For each network service operator in the first set of network service operators and the second set of network service operators, obtaining first evaluation data and second evaluation data of each network service operator; performing a product calculation based on the first evaluation weight and the first evaluation data of each network service operator to obtain third evaluation data of each network service operator; performing a product calculation based on the second evaluation weight and the second evaluation data of each network service operator to obtain fourth evaluation data of each network service operator; The third evaluation data of each network service operator and the fourth evaluation data are summed to obtain target evaluation data of each network service operator.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: Obtaining the operator category of the abnormal network service operator; Determining an abnormal level of the abnormal network service operator according to the abnormal information related to the abnormal network service operator in the abnormal information set; If the abnormal level reaches a preset level, switching the network service operator of the abnormal network service operator according to the operator category; If the abnormal level does not reach the preset level, the network service operator to which the target service user corresponding to the abnormal network service operator is connected is switched according to the operator category.

10. A network service quality evaluation device, characterized in that: The device comprises: An acquisition unit, used to acquire abnormal traffic data between a data center and a service user and target optical splitting data corresponding to the abnormal traffic data, wherein the target optical splitting data includes network service operators on a transmission path of the abnormal traffic data; a processing unit, configured to perform calculations based on the target optical splitting data and the abnormal traffic data to determine a first set of network service operators and first evaluation data of each network service operator in the first set of network service operators; The processing unit is further used to determine a second network service operator set and second evaluation data of each network service operator in the second network service operator set according to abnormal information in the abnormal information set, wherein the abnormal information is one or more of network connection abnormal information, data flow alarm information, designated network service operator quality abnormal information, and direct network service operator quality abnormal information of the service user; a calculation unit, configured to integrate and calculate the first evaluation data of each network service operator in the first set of network service operators and the second evaluation data of each network service operator in the second set of network service operators according to the evaluation weight data, to obtain the target evaluation data of each network service operator in the first set of network service operators and the second set of network service operators, and to determine the network service operator with the largest target evaluation data as an abnormal network service operator.

11. A computer device, characterized in that: include: A processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores an executable program code, and the processor is used to call the executable program code to implement the network service quality assessment method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when executed on a computer, enable the computer to implement the network service quality evaluation method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product includes a computer program or a computer instruction, and when the computer program or the computer instruction is executed by a processor, the network service quality evaluation method according to any one of claims 1 to 9 is implemented.

Images