Performance evaluation method and device of virtual network node, and electronic equipment

Abstract

The invention discloses a virtual network node performance evaluation method and device and electronic equipment, and relates to the technical field of cloud computing or other related fields, and the method comprises the steps: obtaining the performance monitoring data of a target network node in a first target time period, and the first target time period is a historical time period; a performance threshold value corresponding to a second target time period is calculated based on the performance monitoring data, and the second target time period is any time period in the future; obtaining target performance data of the target network node according to a preset frequency in a second target time period, and comparing the target performance data with the performance threshold to obtain a comparison result; and generating a performance evaluation report of the target network node in the second target time period based on the comparison result. According to the method and the device, the technical problem that the monitoring false alarm rate is high due to the fact that a static threshold configuration method cannot accurately adapt to changes of financial cloud platform network monitoring services in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing or other related fields. Specifically, it relates to a method and apparatus for evaluating the performance of virtual network nodes, and an electronic device. Background Art

[0002] With the continuous development of cloud computing technology, the financial cloud platform, as the basis for the digital transformation of financial institutions, is being given a new mission. The traditional centralized IT architecture is gradually transforming into a distributed cloud computing architecture. The network basic service resources change from "hard" to "soft", which puts forward higher requirements for the reliability of network basic services and the effectiveness of monitoring.

[0003] In the related art, at present, the performance capacity monitoring of cloud platform network elements mainly adopts the method of manually setting static thresholds. For example, for performances such as the number of connections, packet loss, and bandwidth, the monitoring is mainly carried out by manually setting constant thresholds. When the real-time performance data exceeds the set threshold, network alarms will be sent through the cloud platform.

[0004] The above technologies have the following disadvantages: 1) The configuration of static thresholds needs to be combined with the characteristics of financial cloud services and can only be completed based on the accumulation of long-term historical experience. It relies more on expert experience and has a relatively high learning cost; 2) With the continuous change of services, the pre-set thresholds may not accurately evaluate the changes in services, and it is necessary to re-evaluate the rationality of the thresholds regularly, resulting in relatively high maintenance costs and unable to guarantee the accuracy of monitoring data.

[0005] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0006] The main purpose of this application is to provide a method and apparatus for evaluating the performance of virtual network nodes, and an electronic device, so as to at least solve the technical problem in the related art that due to the static threshold configuration method not being able to accurately adapt to the changes in the network monitoring services of the financial cloud platform, the monitoring false alarm rate is high.

[0007] To achieve the above object, according to one aspect of the present application, a method for evaluating the performance of a virtual network node is provided. The method includes: obtaining performance monitoring data of a target network node within a first target time period, where the first target time period is a historical time period specified by the performance evaluation requirement; calculating a performance threshold corresponding to a second target time period based on the performance monitoring data, where the second target time period is any future time period specified by the performance evaluation requirement, and there is an association relationship between the second target time period and the first target time period; obtaining target performance data of the target network node at a preset frequency within the second target time period, and comparing the target performance data with the performance threshold to obtain a comparison result; generating a performance evaluation report of the target network node within the second target time period based on the comparison result.

[0008] Further, before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data, it further includes: calculating an abnormal data threshold based on the performance monitoring data; determining abnormal data values in the performance monitoring data based on the abnormal data threshold, where the types of the abnormal data values include: sudden increase abnormal values and sudden decrease abnormal values; deleting all the abnormal data values in the performance monitoring data to obtain the preprocessed performance monitoring data.

[0009] Further, the step of calculating the abnormal data threshold based on the performance monitoring data includes: calculating the average value and the dispersion of all the performance monitoring data; calculating the upper limit value of the abnormal data in the abnormal data threshold based on the average value and the dispersion; calculating the lower limit value of the abnormal data in the abnormal data threshold based on the average value and the dispersion, where the lower limit value of the abnormal data is less than the upper limit value of the abnormal data.

[0010] Further, the step of determining the abnormal data values in the performance monitoring data based on the abnormal data threshold includes: comparing the lower limit value of the abnormal data, the upper limit value of the abnormal data with all the abnormal data values to obtain a comparison result; determining all the abnormal data values greater than the upper limit value of the abnormal data indicated by the comparison result as the sudden increase abnormal values; determining all the abnormal data values less than the lower limit value of the abnormal data indicated by the comparison result as the sudden decrease abnormal values.

[0011] Further, before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data, the method further includes: determining a median based on all the performance monitoring data; classifying all the performance monitoring data based on the median to obtain first-class data and second-class data, where the first-class data is greater than or equal to the median, and the second-class data is less than the median; calculating a dispersion degree based on the median and the first-class data to obtain an upward dispersion degree of the performance monitoring data; calculating a dispersion degree based on the median and the second-class data to obtain a downward dispersion degree of the performance monitoring data.

[0012] Further, before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data, the method further includes: calculating a downward coefficient of variation based on the downward dispersion degree and the median, and calculating an upward coefficient of variation based on the upward dispersion degree and the median; calculating a downward abnormality degree based on all the sudden drop outliers, and calculating an upward abnormality degree based on all the sudden increase outliers; determining a downward fluctuation coefficient based on the downward coefficient of variation and the downward abnormality degree, and determining an upward fluctuation coefficient based on the upward coefficient of variation and the upward abnormality degree.

[0013] Further, the step of calculating the performance threshold corresponding to the second target time period based on the performance monitoring data includes: obtaining a preset sensitivity adjustment coefficient and a preset dispersion degree mean value; calculating an upper limit value of the performance threshold based on the median, the upward fluctuation coefficient, the upward dispersion degree, the preset sensitivity adjustment coefficient, and the preset dispersion degree mean value; calculating a lower limit value of the performance threshold based on the median, the downward fluctuation coefficient, the downward dispersion degree, the preset sensitivity adjustment coefficient, and the preset dispersion degree mean value.

[0014] To achieve the above object, according to another aspect of the present application, there is also provided a performance evaluation device for a virtual network node, the device including: an acquisition unit, configured to acquire performance monitoring data of a target network node within a first target time period, where the first target time period is a historical time period specified by a performance evaluation requirement; a calculation unit, configured to calculate a performance threshold corresponding to a second target time period based on the performance monitoring data, where the second target time period is any future time period specified by the performance evaluation requirement, and there is an association relationship between the second target time period and the first target time period; a comparison unit, configured to acquire target performance data of the target network node at a preset frequency within the second target time period, and compare the target performance data with the performance threshold to obtain a comparison result; a generation unit, configured to generate a performance evaluation report of the target network node within the second target time period based on the comparison result.

[0015] Furthermore, the performance evaluation device of the virtual network node further includes: a first calculation module, configured to calculate an abnormal data threshold based on the performance monitoring data before calculating a performance threshold corresponding to a second target time period based on the performance monitoring data; a first determination module, configured to determine abnormal data values in the performance monitoring data based on the abnormal data threshold, where the types of the abnormal data values include: sudden increase abnormal values and sudden decrease abnormal values; a deletion module, configured to delete all the abnormal data values in the performance monitoring data to obtain the preprocessed performance monitoring data.

[0016] Furthermore, the first calculation module includes: a first calculation sub-module, configured to calculate the average value and the dispersion of all the performance monitoring data; a second calculation sub-module, configured to calculate an upper limit value of abnormal data in the abnormal data threshold based on the average value and the dispersion; a third calculation sub-module, configured to calculate a lower limit value of abnormal data in the abnormal data threshold based on the average value and the dispersion, where the lower limit value of abnormal data is less than the upper limit value of abnormal data.

[0017] Furthermore, the first determination module includes: a comparison sub-module, configured to compare the lower limit value of abnormal data, the upper limit value of abnormal data with all the abnormal data values to obtain a comparison result; a first determination sub-module, configured to determine all the abnormal data values greater than the upper limit value of abnormal data indicated by the comparison result as the sudden increase abnormal values; a second determination sub-module, configured to determine all the abnormal data values less than the lower limit value of abnormal data indicated by the comparison result as the sudden decrease abnormal values.

[0018] Furthermore, the performance evaluation device of the virtual network node further includes: a selection module, configured to determine a median based on all the performance monitoring data before calculating a performance threshold corresponding to a second target time period based on the performance monitoring data; a classification module, configured to classify all the performance monitoring data based on the median to obtain a first type of data and a second type of data, where the first type of data is greater than or equal to the median, and the second type of data is less than the median; a second calculation module, configured to calculate the dispersion based on the median and the first type of data to obtain an upward dispersion of the performance monitoring data; a third calculation module, configured to calculate the dispersion based on the median and the second type of data to obtain a downward dispersion of the performance monitoring data.

[0019] Further, the performance evaluation device of the virtual network node further includes: a fourth calculation module, configured to calculate a downward coefficient of variation based on the downward dispersion and the median, and calculate an upward coefficient of variation based on the upward dispersion and the median, before calculating a performance threshold corresponding to a second target time period based on the performance monitoring data; a fifth calculation module, configured to calculate a downward abnormality degree based on all the sudden decrease outliers, and calculate an upward abnormality degree based on all the sudden increase outliers; a second determination module, configured to determine a downward fluctuation coefficient based on the downward coefficient of variation and the downward abnormality degree, and determine an upward fluctuation coefficient based on the upward coefficient of variation and the upward abnormality degree.

[0020] Further, the calculation unit includes: an acquisition module, configured to acquire a preset sensitivity adjustment coefficient and a preset dispersion mean value; a sixth calculation module, configured to calculate an upper limit value of the performance threshold based on the median, the upward fluctuation coefficient, the upward dispersion, the preset sensitivity adjustment coefficient, and the preset dispersion mean value; a seventh calculation module, configured to calculate a lower limit value of the performance threshold based on the median, the downward fluctuation coefficient, the downward dispersion, the preset sensitivity adjustment coefficient, and the preset dispersion mean value.

[0021] To achieve the above object, according to another aspect of the present application, there is also provided a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program, and when the computer program runs, it controls a device where the computer-readable storage medium is located to execute the performance evaluation method of the virtual network node described in any one of the above.

[0022] To achieve the above object, according to another aspect of the present application, there is also provided an electronic device, including one or more processors and a memory, where the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the performance evaluation method of the virtual network node described in any one of the above.

[0023] In the present invention, a performance evaluation method for a virtual network node is proposed. First, performance monitoring data of a target network node in a first target time period is acquired, where the first target time period is a historical time period specified by performance evaluation requirements. Then, a performance threshold corresponding to a second target time period is calculated based on the performance monitoring data, where the second target time period is any future time period specified by performance evaluation requirements, and there is an association relationship between the second target time period and the first target time period. Then, target performance data of the target network node is acquired at a preset frequency in the second target time period, and the target performance data is compared with the performance threshold to obtain a comparison result. Finally, a performance evaluation report of the target network node in the second target time period is generated based on the comparison result.

[0024] In the present invention, by adopting the method of dynamic threshold calculation and analyzing the historical performance monitoring data, the purpose of automatically adapting to business fluctuations and changes is achieved, thereby realizing the technical effects of improving the accuracy of network monitoring and reducing false alarms. Specifically, by analyzing the performance monitoring data of the target network node within the first target time period, the performance threshold of the second target time period (future time period) is automatically calculated. The traditional static threshold method often leads to false alarms due to improper setting (such as issuing alarms under normal performance conditions or not issuing alarms in a timely manner when performance is abnormal). The dynamic threshold setting strategy of the present invention can better adapt to the dynamic changes of the business in the financial cloud platform, more accurately judge whether the performance of the target network node exceeds the normal range, thereby effectively reducing the false alarm rate, improving the accuracy and timeliness of alarms, and further solving the technical problem in the related art that due to the static threshold configuration method not being able to accurately adapt to the changes of the network monitoring business in the financial cloud platform, the monitoring false alarm rate is high.

[0025] In addition, the dynamic threshold calculation in the present invention is automatically completed based on historical data, without the need for manual regular evaluation and adjustment, greatly reducing the workload of operation and maintenance personnel, reducing the operation and maintenance cost, and improving the efficiency of network monitoring. Since the dynamic threshold can be adjusted according to the real-time changes of the business, the monitoring system can respond more quickly to the fluctuations of network performance, discover potential problems in a timely manner, avoid missing problems or over-alarming caused by unreasonable threshold settings, and enhance the flexibility of the system and the ability to quickly respond to business changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings forming a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0027] Figure 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for evaluating the performance of a virtual network node;

[0028] Figure 2 is a flowchart of an optional method for evaluating the performance of a virtual network node according to an embodiment of the present invention;

[0029] Figure 3 is a schematic diagram of an optional device for evaluating the performance of a virtual network node according to an embodiment of the present invention;

[0030] Figure 4 is a structure block diagram of an electronic device for executing a method for evaluating the performance of a virtual network node according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] To enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0033] To facilitate the understanding of the present invention by those skilled in the art, the following explanations are made for some terms or nouns involved in the embodiments of the present invention:

[0034] VNN, Virtual Network Node, is a virtualized network function node in a virtualization and cloud environment, such as a virtual router, virtual switch, virtual firewall, etc. These virtual nodes can be flexibly deployed and configured on a cloud platform to provide the required network services.

[0035] VNE-PC, Virtual Network Elements in Private Cloud, is a virtualized network service node that realizes the functions of traditional network devices under the software-defined network architecture in the private cloud environment of financial institutions. It can run on the virtualized resources of the cloud platform in the form of software, providing network services including but not limited to virtual routers, virtual switches, virtual firewalls, and virtual load balancers.

[0036] It should be noted that the method and device for evaluating the performance of virtual network nodes in this application can be used in the field of cloud computing technology when dynamically monitoring the performance of virtual network elements in private clouds, and can also be used in any field other than the field of cloud computing technology when dynamically monitoring the performance of virtual network elements in private clouds. The application field of the method and device for evaluating the performance of virtual network nodes in this application is not limited.

[0037] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, storage, processing, transmission, provision, disclosure, use, and handling of the relevant data all comply with the laws, regulations, and standards of the relevant regions, adopt necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, there is an interface between this system and relevant users or institutions. Before obtaining relevant information, a request for acquisition needs to be sent to the aforementioned users or institutions through the interface, and after receiving the consent information feedback from the aforementioned users or institutions, the relevant information is obtained.

[0038] For the information collection (such as user voice, video, text collection) and analysis operations involved in this application, corresponding operation entrances have been provided for users to choose to agree or refuse the automated decision-making results when they are executed; if the user chooses to refuse, the expert decision-making process will be entered.

[0039] The following embodiments of the present invention can be applied to various systems / applications / devices that require network performance monitoring and dynamic threshold setting, and can achieve accurate evaluation and efficient monitoring of the performance of virtual network nodes. The present invention uses historical performance data for dynamic learning and prediction, and then, by comparing the real-time data with the predicted dynamic threshold, can better adapt to the fluctuations of business traffic, reduce monitoring false alarms and missed reports, and ensure the stability and efficiency of network services.

[0040] Specifically, the core of the present invention lies in collecting and analyzing the historical performance data of virtual network nodes (such as private cloud software network elements), and using dynamic learning algorithms to calculate the future dynamically changing performance thresholds. In this way, the combination of real-time monitoring and intelligent early warning can be realized, effectively identifying and coping with abnormal changes in network performance, thereby ensuring the continuity of financial services and user satisfaction. Through automated dynamic threshold calculation, the present invention not only reduces the workload of operation and maintenance personnel, improves the intelligence level of the monitoring system, but also can adjust the monitoring strategy in a timely manner according to changes in business requirements, avoiding the limitations and inaccuracies brought by static threshold setting.

[0041] The present invention will be described in detail below in combination with each embodiment.

[0042] Embodiment 1

[0043] According to an embodiment of the present invention, an embodiment of a method for evaluating the performance of a virtual network node is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0044] The embodiment of the method for evaluating the performance of a virtual network node provided by the first embodiment of the present invention can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing the method for evaluating the performance of a virtual network node is shown. As Figure 1 shown, the computer terminal 10 (or mobile device) may include one or more (shown as 102a, 102b,..., 102n in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0045] It should be noted that the above one or more processors 102 and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit is a processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0046] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage devices corresponding to the performance evaluation method of the virtual network node in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned performance evaluation method of the virtual network node. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.

[0047] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include the wireless network provided by the communication provider of the computer terminal 10. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0048] The display can be, for example, a touch-screen liquid crystal display (LCD), which enables the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0049] Under the above operating environment, the present invention provides a performance evaluation method of a virtual network node as shown in Figure 2 The implementation subject of this method is a financial private cloud monitoring system. Combining big data analysis and machine learning technologies, it is used for network performance monitoring scenarios in a cloud network environment, especially for the dynamic performance evaluation problem of private cloud soft network elements. Through dynamic threshold prediction means, specifically by collecting historical performance data, preprocessing outliers, training and generating a dynamic threshold model, and comparing the current performance metrics with the dynamic threshold in real time, it aims to accurately and timely identify network performance anomalies, optimize network monitoring strategies, reduce operation and maintenance costs, and improve the stability of financial services.

[0050] The embodiments of the present invention will be described in detail below in combination with each specific step.

[0051] Figure 2 is a flowchart of an optional performance evaluation method of a virtual network node according to an embodiment of the present invention, as shown in Figure 2As shown in the figure, the method includes the following steps:

[0052] Step S201: Obtain the performance monitoring data of the target network node within the first target time period, where the first target time period is the historical time period specified by the performance evaluation requirements.

[0053] It should be noted that the target network node refers to a virtualized node that provides network services for target users in a cloud network environment, including but not limited to virtual routers, virtual switches, virtual firewalls, virtual load balancing devices, etc. The target network node is implemented in a software-defined form and supports dynamic adjustment and deployment configuration according to user needs.

[0054] The performance evaluation requirements refer to the specific requirements of the target user for the performance monitoring of the target network node, including but not limited to: monitoring frequency requirements, performance index requirements (such as requirements for CPU usage rate, memory occupancy, network bandwidth, connection number, etc.), performance data storage, and data analysis and processing requirements.

[0055] The first target time period refers to the time period used to collect historical performance data, which is a historical cycle matching the future time period to be predicted. For example, in order to predict the performance threshold tomorrow, historical data within the same time cycle in the past week or month can be selected as the analysis benchmark. This historical time period is used to provide background data and patterns of performance changes and is crucial for predicting future performance trends.

[0056] The performance monitoring data refers to the data records regarding the performance status generated by the target network node within the first target time period, including but not limited to key indicators such as CPU utilization rate, memory usage, network traffic, packet loss rate, and connection number. By real-time monitoring and periodic collection of performance monitoring data, it is used to analyze the normal operating range and performance fluctuation of the network node during the calculation of the dynamic threshold.

[0057] Step S202: Calculate the performance threshold corresponding to the second target time period based on the performance monitoring data, where the second target time period is any future time period specified by the performance evaluation requirements, and there is an association relationship between the second target time period and the first target time period.

[0058] It should be noted that the second target time period is a future time period set according to the performance evaluation requirements and is used to predict the performance status of the target network node. This time period can be one day, one week, or any other specific future time period, but it needs to have a time association with the first target time period. The calculation of the performance threshold for evaluating the node health status within the second target time period is based on the historical data of the first target time period.

[0059] The performance threshold refers to the upper and lower limits of the normal range of the performance metrics of the target network node within a certain future time period (i.e., the second target time period) predicted based on historical performance monitoring data. The dynamic performance threshold can more accurately reflect the dynamics of business changes and network conditions, avoiding the problem of too high or too low false alarm rates caused by static thresholds.

[0060] In an alternative embodiment, in the private cloud environment of a certain financial institution, the target network node is a virtual load balancer device. The performance evaluation requirements include a performance data collection frequency of once per minute, and the performance metrics of concern include the number of connections and the packet loss rate. In this case, the first target time period is set to the same time period in the past 30 days (e.g., from 08:00 to 09:00), and the performance monitoring data (i.e., the number of connections and the packet loss rate) of the device during this time period is collected for analysis and learning.

[0061] After collecting the performance monitoring data, step S202 is entered to calculate the performance threshold for the second target time period (e.g., from 08:00 to 09:00 tomorrow) to predict the dynamic thresholds of the number of connections and the packet loss rate of the virtual load balancer device within the next day.

[0062] Through the above steps, the embodiments of the present invention can automatically collect the historical performance data of the target network node, automatically adjust the monitoring strategy to predict the performance threshold within the future time period, thereby realizing dynamic and intelligent network performance monitoring, adapting to the changes in business development, reducing false alarms and missed alarms, and ensuring the service quality and stability in the cloud network environment.

[0063] Optionally, before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data, it further includes: calculating the abnormal data threshold based on the performance monitoring data; determining the abnormal data values in the performance monitoring data based on the abnormal data threshold, where the types of abnormal data values include: sudden increase abnormal values and sudden decrease abnormal values; deleting all abnormal data values in the performance monitoring data to obtain the preprocessed performance monitoring data.

[0064] It should be noted that the abnormal data threshold is the upper and lower limit standard used to identify and screen out abnormal data values during the process of analyzing the performance monitoring data. It is formulated based on the statistical characteristics of historical performance data and is used to distinguish data within the normal fluctuation range from mutations. It is the key to ensuring the accuracy and effectiveness of the performance evaluation results.

[0065] An abnormal data value refers to a data point in the performance monitoring data that significantly deviates from the normal range due to network fluctuations, device failures, or other abnormal events. Abnormal data values are the root causes of false alarms or missed alarms. Accurately identifying and processing abnormal data values is crucial for node performance analysis and affects the correct assessment of the network status.

[0066] Furthermore, a sudden increase outlier refers to a data point in the abnormal data values that is higher than the upper limit of the abnormal data, indicating that the target network node has processed a large amount of additional traffic or requests in a short period of time, resulting in a sudden increase in performance metrics; conversely, a sudden decrease outlier is a data point that is lower than the lower limit of the abnormal data, indicating a decline in the performance of the target network node or a situation where the device is not fully utilized.

[0067] Before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data, it is first necessary to calculate the abnormal data threshold based on the performance monitoring data, including calculating the average value and dispersion of all performance monitoring data, determining the upper and lower threshold values of the abnormal data, comparing all performance monitoring data with the abnormal data threshold, identifying all sudden increase outliers higher than the upper threshold value and sudden decrease outliers lower than the lower threshold value, deleting all abnormal data values from the original data set, and obtaining the preprocessed performance monitoring data for subsequent performance threshold calculation and analysis.

[0068] Optionally, the steps of calculating the abnormal data threshold based on the performance monitoring data include: calculating the average value and dispersion of all performance monitoring data; calculating the upper limit value of the abnormal data in the abnormal data threshold based on the average value and dispersion; calculating the lower limit value of the abnormal data in the abnormal data threshold based on the average value and dispersion, where the lower limit value of the abnormal data is less than the upper limit value of the abnormal data.

[0069] It should be noted that calculating the abnormal data threshold includes calculating the average value and dispersion of all performance monitoring data. The average value is used to reflect the central tendency of the data set, while the dispersion is used to measure the degree of dispersion of the data points relative to the average value. By combining the average value and dispersion, a reasonable range of abnormal data thresholds can be calculated. Among them, the upper limit value of the abnormal data is set as the average value plus a certain multiple of the dispersion, and the lower limit value of the abnormal data is set as the average value minus the corresponding multiple of the dispersion. This multiple (usually called the sensitivity factor) can be adjusted according to the business scenario and monitoring requirements to adapt to the data distribution characteristics in different environments.

[0070] An optional embodiment can be based on the average value calculation formula for average value calculation, where N is the number of acquisitions of the performance monitoring data, x i refers to the i-th performance monitoring data, and u 1 refers to the calculated average value.

[0071] Furthermore, the dispersion can be calculated based on the dispersion calculation formula where σ 1 refers to the calculated dispersion.

[0072] Still further, the first abnormal calculation formula a 1 = u 1+mσ 1 Perform the calculation of the upper limit of abnormal data, based on the second abnormal calculation formula a 2 = u 1 -mσ 1 Perform the calculation of the lower limit of abnormal data, where a 1 is the calculated upper limit value of abnormal data, a 2 is the calculated lower limit value of abnormal data, m is the sensitivity factor set in the embodiment of the present invention, which is a custom variable, and the target user can adjust the sensitivity of the action of deleting abnormal values through this custom variable.

[0073] Optionally, the step of determining the abnormal data value in the performance monitoring data based on the abnormal data threshold includes: comparing the lower limit value of abnormal data, the upper limit value of abnormal data with all abnormal data values to obtain a comparison result; determining all abnormal data values greater than the upper limit value of abnormal data indicated by the comparison result as sudden increase abnormal values; determining all abnormal data values less than the lower limit value of abnormal data indicated by the comparison result as sudden decrease abnormal values.

[0074] Continuing the above embodiment, if x i > a 1 , x i can be determined as a sudden increase abnormal value. Correspondingly, if x i < a 2 , then x i can be determined as a sudden decrease abnormal value.

[0075] Another optional embodiment, assuming that in the cloud network environment of a financial institution, the target network node is a virtual switch, and the performance monitoring data includes indicators such as CPU utilization and network traffic. The implementation system can first collect the performance monitoring data of the virtual switch in the past 30 days, then calculate the average value and the dispersion, and determine the upper and lower thresholds of abnormal data based on the statistic. For example, if the average CPU utilization is 30%, the CPU utilization dispersion is 5%, the upper limit value of abnormal data can be set to 40%, and the lower limit value of abnormal data can be set to 20%, depending on the specific business requirements and sensitivity configuration.

[0076] Subsequently, the implementation system can scan all the performance monitoring data, identify all data points with CPU utilization higher than 40% or lower than 20%, mark them as sudden increase abnormal values and sudden decrease abnormal values, and delete all abnormal values in the performance monitoring data, only retaining the data within the normal range to calculate the future dynamic performance threshold.

[0077] Through the above steps, the impact of abnormal events on performance evaluation can be effectively reduced, the accuracy and reliability of the prediction model can be improved, and higher-quality decision support can be provided for the network monitoring of financial institutions.

[0078] Optionally, before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data, it further includes: determining the median based on all the performance monitoring data; classifying all the performance monitoring data based on the median to obtain the first type of data and the second type of data, where the first type of data is greater than or equal to the median, and the second type of data is less than the median; calculating the dispersion based on the median and the first type of data to obtain the upward dispersion of the performance monitoring data; calculating the dispersion based on the median and the second type of data to obtain the downward dispersion of the performance monitoring data.

[0079] In the embodiment of the present invention, the median refers to the average value of all the preprocessed performance monitoring data; the dispersion is used to describe the degree of dispersion of data points relative to a certain central tendency (such as the average value or the median, here it is the median).

[0080] In the embodiment of the present invention, the upward dispersion refers to the dispersion calculated based on the median and all the first type of data that is greater than or equal to the median, and is used to reflect the fluctuation of the performance index at a higher level, while the downward dispersion is the dispersion calculated based on the median and all the second type of data that is less than the median, and is used to reflect the fluctuation of the performance index at a lower level.

[0081] An optional embodiment can calculate the median through the median calculation formula where M is the number of performance monitoring data after deleting all abnormal data, and u 2 refers to the calculated median.

[0082] Furthermore, the upward dispersion can be calculated based on the upward dispersion calculation formula to calculate the upward dispersion σ g , where N g is the number of performance monitoring data that is greater than or equal to the median u 2 , and x ig is the i-th performance monitoring data that is greater than or equal to the median u 2 .

[0083] The downward dispersion can also be calculated based on the downward dispersion calculation formula to calculate the downward dispersion σ l , where N l is the number of performance monitoring data that is less than the median u 2 , and x il is the i-th performance monitoring data that is less than the median u 2 .

[0084] In another alternative embodiment, in the cloud network environment of a financial institution, the target network node is a virtual firewall, and the performance monitoring data includes a key metric of the number of connections. The implementation system can first collect the connection number data of the virtual firewall within a specific time window (e.g., from 10:00 to 11:00 every day) in the past 30 days and calculate the median of this data set (assuming the median is 1000 connections).

[0085] Next, compare all the connection number data with the median. Classify the data points with a connection number greater than or equal to 1000 as the first type of data, and classify the data points with a connection number less than 1000 as the second type of data. In the embodiment of the present invention, the upward dispersion can be calculated based on the median and the first type of data to analyze the fluctuation of the connection number during high-traffic periods and understand the carrying capacity and stability of the network during peak periods. Similarly, the downward dispersion can also be calculated based on the median and the second type of data to analyze the fluctuation of the connection number during low-traffic periods and understand the utilization efficiency and stability of the network during valley periods.

[0086] By calculating the upward dispersion and the downward dispersion, the performance stability of the virtual firewall can be comprehensively evaluated, including the operating conditions during peak and valley periods, providing more detailed and comprehensive analysis data for calculating the dynamic performance threshold in the second target time period (such as from 10:00 to 11:00 the next day), thereby achieving more accurate performance monitoring and early warning.

[0087] Optionally, before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data, it further includes: calculating the downward coefficient of variation based on the downward dispersion and the median, and calculating the upward coefficient of variation based on the upward dispersion and the median; calculating the downward abnormality based on all the sudden decrease outliers, and calculating the upward abnormality based on all the sudden increase outliers; determining the downward fluctuation coefficient based on the downward coefficient of variation and the downward abnormality, and determining the upward fluctuation coefficient based on the upward coefficient of variation and the upward abnormality.

[0088] It should be noted that the downward coefficient of variation and the upward coefficient of variation are statistics calculated based on the median and the downward / upward dispersion of the performance monitoring data, respectively used to measure the degree of variation of the performance metrics of the network node when below and above the median, reflecting the dispersion of the performance monitoring data below or above the mean value, and are the basis for evaluating performance stability and providing key information for dynamic threshold prediction.

[0089] The calculation steps of the downward coefficient of variation mainly focus on the fluctuation of the performance metric during the valley period or below the average level, that is, the degree of variation of the performance of the network node that may decrease, and can be calculated by the third abnormality calculation formula Calculate the downward coefficient of variation p l .

[0090] The calculation steps of the upward coefficient of variation mainly focus on the fluctuations of performance indicators during peak periods or above the average level, that is, the degree of variation in which the performance of network nodes may increase, which can be calculated by the fourth anomaly calculation formula Calculate the upward coefficient of variation p g .

[0091] Another thing to note is that the downward anomaly degree and the upward anomaly degree are statistics calculated based on all sudden drop anomaly values and sudden increase anomaly values, respectively reflecting the anomaly degrees when the performance indicators are significantly lower and higher than the normal range. Among them, the downward anomaly degree is obtained by analyzing all data points below the normal range, and is used to identify and quantify the abnormal situations where the performance of network nodes may be lower than expected; the upward anomaly degree is obtained by analyzing all data points above the normal range, and is used to identify and quantify the abnormal situations where the performance of network nodes may be higher than expected.

[0092] An optional method is to calculate the downward anomaly degree q through the fifth anomaly calculation formula Calculate the downward anomaly degree q l , where N t refers to the number of sudden drop anomaly values, and x t refers to the t-th sudden drop anomaly value.

[0093] It is also possible to calculate the upward anomaly degree q through the sixth anomaly calculation formula Calculate the upward anomaly degree q g , where N s refers to the number of sudden increase anomaly values, and x s refers to the s-th sudden increase anomaly value.

[0094] Furthermore, in order to reduce the calculation difficulty, the coefficient of variation and the anomaly degree can also be normalized, and the normalization formula is provided exemplarily where y n is the coefficient of variation or anomaly degree after normalization, y is the coefficient of variation or anomaly degree of any collection point within the target time period, max(y) is the maximum value of the coefficient of variation or anomaly degree of any collection point within the target time period, and min(y) is the minimum value of the coefficient of variation or anomaly degree of any collection point within the target time period.

[0095] Furthermore, the downward fluctuation coefficient and the upward fluctuation coefficient are comprehensive indicators obtained by further weighted calculation after determining the downward coefficient of variation, the downward anomaly degree, the upward coefficient of variation, and the upward anomaly degree, and are used to more comprehensively evaluate the fluctuation characteristics of network node performance. The downward fluctuation coefficient is used to reflect the fluctuation characteristics of network nodes when their performance is below the normal level, while the upward fluctuation coefficient is used to reflect the fluctuation characteristics of network nodes when their performance is above the normal level.

[0096] An optional method is to calculate α through the seventh anomaly calculation formula l= k 1 p ln + k 2 q ln Calculate the downward fluctuation coefficient α l , where p ln and q ln are the downward variation coefficient and downward abnormality after normalization respectively, and k 1 and k 2 are the preset first group of weighting coefficients and can take integer values.

[0097] Furthermore, the upward fluctuation coefficient α can also be calculated through the eighth abnormality calculation formula g = k 3 p gn + k 4 q gn Calculate the upward fluctuation coefficient α g , where p gn and q gn are the upward variation coefficient and upward abnormality after normalization respectively, and k 3 and k 4 are the preset second group of weighting coefficients and can also take integer values.

[0098] In another alternative embodiment, in the cloud network environment of a financial institution, the target network node is a certain virtual router, and the performance monitoring data includes the number of connections collected per minute. The real-time system can calculate the median of the number of connections within the same time window (such as from 15:00 to 16:00 every day) in the past 30 days, and calculate the downward dispersion and upward dispersion based on the median, which are respectively used to determine the variation degree of the data points below and above the median.

[0099] Calculate the downward variation coefficient based on the downward dispersion and the median, which can reflect the variation degree when the number of connections is below the median; calculate the upward variation coefficient based on the upward dispersion and the median, which can reflect the variation degree when the number of connections is above or equal to the median.

[0100] Calculate the downward abnormality based on all the sudden drop outliers (i.e., the number of connections significantly lower than the normal range), which can quantify the severity of the network performance below the expectation; calculate the upward abnormality based on all the sudden increase outliers (i.e., the number of connections significantly higher than the normal range), which can quantify the severity of the network performance above the expectation.

[0101] Finally, determine the downward fluctuation coefficient based on the downward variation coefficient, downward abnormality and a specific weighting factor, which is used to comprehensively reflect the fluctuation characteristics of the target network node at a low performance level; determine the upward fluctuation coefficient based on the upward variation coefficient, upward abnormality and the weighting factor, which is used to comprehensively reflect the fluctuation characteristics of the target network node at a high performance level.

[0102] Calculating the downward fluctuation coefficient and the upward fluctuation coefficient can more accurately evaluate the performance stability of the target network node (virtual router). Especially in the case of large fluctuations in service traffic, it is crucial for predicting future dynamic performance thresholds and optimizing network monitoring strategies.

[0103] Optionally, the step of calculating the performance threshold corresponding to the second target time period based on the performance monitoring data includes: obtaining a preset sensitivity adjustment coefficient and a preset mean dispersion; calculating the upper limit value of the performance threshold based on the median, the upward fluctuation coefficient, the upward dispersion, the preset sensitivity adjustment coefficient, and the preset mean dispersion; calculating the lower limit value of the performance threshold based on the median, the downward fluctuation coefficient, the downward dispersion, the preset sensitivity adjustment coefficient, and the preset mean dispersion.

[0104] It should be noted that the preset sensitivity adjustment coefficient is a user-defined parameter used to adjust the sensitivity when identifying abnormal performance data. It can be set according to the specific requirements of the business scenario, the stability of the network environment, and the tolerance for false alarms and missed alarms. A higher sensitivity adjustment coefficient means that the system is more sensitive to abnormal changes in performance indicators and is more likely to trigger an alarm or take action; while a lower sensitivity adjustment coefficient means that the system has a higher tolerance for performance changes and is less likely to identify fluctuations within the normal range as abnormal.

[0105] The preset mean dispersion is the average of the dispersion data at all acquisition time points calculated based on historical performance monitoring data, which is used to reflect the general fluctuation level of the performance change of the target network node. The preset mean dispersion is used as a reference baseline when calculating the dynamic threshold to evaluate the deviation of the current upward dispersion and downward dispersion from the historical average level.

[0106] When calculating the performance threshold for the future second target time period, the preset sensitivity adjustment coefficient and the mean dispersion can be obtained first, and combined with the median, the upward fluctuation coefficient, the upward dispersion of the current data set, as well as the preset sensitivity adjustment coefficient and the mean dispersion, the upper limit value of the performance threshold is calculated. The above calculation process takes into account the expected change range of the target network node performance during peak hours and the sensitivity of the business to this change. Similarly, the lower limit value of the performance threshold can be calculated based on the median, the downward fluctuation coefficient, the downward dispersion, the preset sensitivity adjustment coefficient, and the mean dispersion. This calculation process focuses on the change of network performance during valley hours and the sensitivity to low performance levels.

[0107] An optional way is to calculate the upper limit value b of the performance threshold through the upper limit calculation formula Calculate the upper limit value b of the performance threshold 1 , and the lower limit value b of the performance threshold can also be calculated through the lower limit calculation formula Calculate the lower limit value b of the performance threshold 2 , where β g and β lis a pre-customized preset sensitivity adjustment coefficient, which can take corresponding values of 1, 2, and 3 according to the distinction of low sensitivity, medium sensitivity, and high sensitivity, and is the pre-calculated preset mean of dispersion, is the upward dispersion σ g of all acquisition time points, is the downward dispersion σ l of all acquisition time points.

[0108] By integrating the preset sensitivity adjustment and the calculation of the mean of dispersion, the above steps can dynamically generate performance thresholds, accurately capture the fluctuation characteristics of network nodes, adapt to the real-time requirements of services, avoid excessive or insufficient alarms, improve the monitoring efficiency, and enhance the objectivity and reliability of threshold setting by using historical data. Specifically, calculating the upward and downward dispersions to comprehensively evaluate the change range can ensure that the upper and lower limits of the threshold reflect the possible peaks and troughs, improve the ability to predict the trend of network performance, reduce the workload of operation and maintenance, and enhance the stability of financial services and the user experience at the same time.

[0109] Step S203: Obtain the target performance data of the target network node at a preset frequency within the second target time period, and compare the target performance data with the performance threshold to obtain a comparison result.

[0110] Automatically collect the performance data of the target network node at a preset frequency (for example, once a minute) within the second target time period. The preset frequency is designed to capture the immediate changes in network performance and ensure the real-time and accuracy of monitoring; compare the real-time obtained target performance data with the previously calculated performance threshold to generate a comparison result.

[0111] The above steps can immediately identify whether the performance of the network node exceeds the expected upper limit or is lower than the expected lower limit through real-time monitoring, so as to judge whether the operating state of the network node is normal and whether further analysis or actions are needed.

[0112] Step S204: Generate a performance evaluation report of the target network node within the second target time period based on the comparison result.

[0113] Generate a performance evaluation report of the target network node within the second target time period based on these comparison results, which is used to record in detail the comparison situation between each performance data and the threshold, including but not limited to: the number of times the data exceeds the threshold, the amplitude of exceeding or being lower than the threshold, and the specific time points when the above abnormal situations occur.

[0114] Through this performance evaluation report, operation and maintenance personnel can comprehensively understand the operating conditions of network nodes within a specified time period, quickly locate problems, evaluate the health status and performance of the network, and thus make corresponding optimization decisions, such as adjusting resource allocation, optimizing network configuration, or giving early warnings of possible faults.

[0115] Through the above steps S201 to S204, the performance monitoring data of the target network node within the first target time period can be obtained first. The first target time period is a historical time period specified by the performance evaluation requirements. Then, based on the performance monitoring data, the performance threshold corresponding to the second target time period is calculated. The second target time period is any future time period specified by the performance evaluation requirements, and there is an association relationship between the second target time period and the first target time period. Then, the target performance data of the target network node is obtained at a preset frequency within the second target time period, and the target performance data is compared with the performance threshold to obtain a comparison result. Finally, a performance evaluation report of the target network node within the second target time period is generated based on the comparison result.

[0116] In the embodiment of the present invention, by adopting the method of dynamic threshold calculation and analyzing the historical performance monitoring data, the purpose of automatically adapting to business fluctuations and changes is achieved, thereby realizing the technical effects of improving the accuracy of network monitoring and reducing false alarms. Specifically, by analyzing the performance monitoring data of the target network node within the first target time period, the performance threshold of the second target time period (future time period) is automatically calculated. In the traditional static threshold method, false alarms often occur due to improper settings (such as giving an alarm when the performance is normal, or not giving an alarm in time when the performance is abnormal). The dynamic threshold setting strategy of the present invention can better adapt to the dynamic changes of the business in the financial cloud platform, more accurately judge whether the performance of the target network node exceeds the normal range, thereby effectively reducing the false alarm rate, improving the accuracy and timeliness of the alarm, and further solving the technical problem in the related art that due to the static threshold configuration method not being able to accurately adapt to the changes in the network monitoring business of the financial cloud platform, the monitoring false alarm rate is high.

[0117] In addition, the dynamic threshold calculation in the present invention is automatically completed based on historical data, without manual regular evaluation and adjustment, greatly reducing the workload of operation and maintenance personnel, reducing the operation and maintenance cost, and improving the efficiency of network monitoring. Since the dynamic threshold can be adjusted according to the real-time changes of the business, the monitoring system can respond more quickly to the fluctuations of network performance, discover potential problems in time, avoid missing problems or over-alarming due to unreasonable threshold settings, and enhance the flexibility of the system and the ability to quickly respond to business changes.

[0118] The following describes the present invention in conjunction with another optional embodiment.

[0119] Embodiment 2

[0120] An embodiment of the present invention further provides a performance evaluation device for a virtual network node. It should be noted that the performance evaluation device for the virtual network node in the embodiment of the present invention includes multiple implementation units, which can be used to execute the performance evaluation method for the virtual network node provided in the first embodiment above. Each implementation unit corresponds to each implementation step in the first embodiment above.

[0121] Figure 3 is a schematic diagram of an optional performance evaluation device for a virtual network node according to an embodiment of the present invention, as Figure 3 shown, the device may include: an acquisition unit 31, a calculation unit 32, a comparison unit 33, and a generation unit 34.

[0122] Among them, the acquisition unit 31 is used to acquire the performance monitoring data of the target network node within the first target time period, where the first target time period is a historical time period specified by the performance evaluation requirement.

[0123] The calculation unit 32 is used to calculate the performance threshold corresponding to the second target time period based on the performance monitoring data, where the second target time period is any future time period specified by the performance evaluation requirement, and there is an association relationship between the second target time period and the first target time period.

[0124] The comparison unit 33 is used to acquire the target performance data of the target network node at a preset frequency within the second target time period, and compare the target performance data with the performance threshold to obtain a comparison result.

[0125] The generation unit 34 is used to generate a performance evaluation report of the target network node within the second target time period based on the comparison result.

[0126] The above-mentioned performance evaluation device for the virtual network node can first acquire the performance monitoring data of the target network node within the first target time period through the acquisition unit 31, where the first target time period is a historical time period specified by the performance evaluation requirement, and then calculate the performance threshold corresponding to the second target time period based on the performance monitoring data through the calculation unit 32, where the second target time period is any future time period specified by the performance evaluation requirement, and there is an association relationship between the second target time period and the first target time period. Then, the comparison unit 33 acquires the target performance data of the target network node at a preset frequency within the second target time period, and compares the target performance data with the performance threshold to obtain a comparison result. Finally, the generation unit 34 generates a performance evaluation report of the target network node within the second target time period based on the comparison result.

[0127] In an embodiment of the present invention, a dynamic threshold calculation method is adopted. By analyzing historical performance monitoring data, the purpose of automatically adapting to business fluctuations and changes is achieved, thereby realizing the technical effects of improving network monitoring accuracy and reducing false alarms. Specifically, by analyzing the performance monitoring data of a target network node within a first target time period, the performance threshold for a second target time period (future time period) is automatically calculated. In the traditional static threshold method, false alarms often occur due to improper settings (such as issuing alarms under normal performance conditions or not issuing alarms in a timely manner when performance is abnormal). The dynamic threshold setting strategy of the present invention can better adapt to the dynamic changes of the business in the financial cloud platform, more accurately determine whether the performance of the target network node exceeds the normal range, thereby effectively reducing the false alarm rate, improving the accuracy and timeliness of alarms, and thus solving the technical problem in the related art that due to the inability of the static threshold configuration method to accurately adapt to the changes in the network monitoring business of the financial cloud platform, the monitoring false alarm rate is high.

[0128] In addition, the dynamic threshold calculation in the present invention is automatically completed based on historical data, without the need for manual regular evaluation and adjustment, greatly reducing the workload of operation and maintenance personnel, reducing the operation and maintenance cost, and improving the efficiency of network monitoring. Since the dynamic threshold can be adjusted according to the real-time changes of the business, the monitoring system can respond more quickly to the fluctuations of network performance, discover potential problems in a timely manner, avoid problems being overlooked or over-alarmed due to unreasonable threshold settings, and enhance the flexibility of the system and the ability to quickly respond to business changes.

[0129] Optionally, the performance evaluation device for the virtual network node further includes: a first calculation module, configured to calculate an abnormal data threshold based on the performance monitoring data before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data; a first determination module, configured to determine the abnormal data values in the performance monitoring data based on the abnormal data threshold, where the types of the abnormal data values include: sudden increase abnormal values and sudden decrease abnormal values; and a deletion module, configured to delete all the abnormal data values in the performance monitoring data to obtain the preprocessed performance monitoring data.

[0130] Optionally, the first calculation module includes: a first calculation sub-module, configured to calculate the average value and the dispersion of all the performance monitoring data; a second calculation sub-module, configured to calculate the upper limit value of the abnormal data in the abnormal data threshold based on the average value and the dispersion; and a third calculation sub-module, configured to calculate the lower limit value of the abnormal data in the abnormal data threshold based on the average value and the dispersion, where the lower limit value of the abnormal data is less than the upper limit value of the abnormal data.

[0131] Optionally, the first determination module includes: a comparison sub-module, configured to compare the lower limit value of abnormal data, the upper limit value of abnormal data with all abnormal data values to obtain a comparison result; a first determination sub-module, configured to determine all abnormal data values greater than the upper limit value of abnormal data indicated by the comparison result as sudden increase abnormal values; a second determination sub-module, configured to determine all abnormal data values less than the lower limit value of abnormal data indicated by the comparison result as sudden decrease abnormal values.

[0132] Optionally, the performance evaluation device of the virtual network node further includes: a selection module, configured to determine a median based on all performance monitoring data before calculating a performance threshold corresponding to a second target time period; a classification module, configured to classify all performance monitoring data based on the median to obtain first-class data and second-class data, where the first-class data is greater than or equal to the median, and the second-class data is less than the median; a second calculation module, configured to calculate a dispersion degree based on the median and the first-class data to obtain an upward dispersion degree of the performance monitoring data; a third calculation module, configured to calculate a dispersion degree based on the median and the second-class data to obtain a downward dispersion degree of the performance monitoring data.

[0133] Optionally, the performance evaluation device of the virtual network node further includes: a fourth calculation module, configured to calculate a downward coefficient of variation based on the downward dispersion degree and the median, and calculate an upward coefficient of variation based on the upward dispersion degree and the median before calculating a performance threshold corresponding to a second target time period; a fifth calculation module, configured to calculate a downward abnormality degree based on all sudden decrease abnormal values, and calculate an upward abnormality degree based on all sudden increase abnormal values; a second determination module, configured to determine a downward fluctuation coefficient based on the downward coefficient of variation and the downward abnormality degree, and determine an upward fluctuation coefficient based on the upward coefficient of variation and the upward abnormality degree.

[0134] Optionally, the calculation unit includes: an acquisition module, configured to acquire a preset sensitivity adjustment coefficient and a preset dispersion degree mean value; a sixth calculation module, configured to calculate an upper limit value of the performance threshold based on the median, the upward fluctuation coefficient, the upward dispersion degree, the preset sensitivity adjustment coefficient, and the preset dispersion degree mean value; a seventh calculation module, configured to calculate a lower limit value of the performance threshold based on the median, the downward fluctuation coefficient, the downward dispersion degree, the preset sensitivity adjustment coefficient, and the preset dispersion degree mean value.

[0135] It should be noted here that the above-mentioned acquisition unit 31, calculation unit 32, comparison unit 33, and generation unit 34 correspond to steps S201 to S204 in the first embodiment. The examples and application scenarios implemented by the above units and the corresponding steps are the same, but are not limited to the content disclosed in the first embodiment. It should be noted that the above modules or units may be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b,..., 102n). The above modules or units may also be part of a device and may run in the computer terminal 10 provided in the first embodiment.

[0136] The present invention will be described below in conjunction with another optional embodiment.

[0137] Embodiment 3

[0138] The embodiment of the present invention may further provide an electronic device. Figure 4 It is a structural block diagram of an electronic device for performing a performance evaluation method of a virtual network node according to an embodiment of the present invention. As Figure 4 shown, the electronic device may include: one or more ( Figure 4 only one is shown in the figure) processors 402, a memory 404, a storage controller, and a peripheral interface. Among them, the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0139] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the performance evaluation method and device of the virtual network node in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned performance evaluation method of the virtual network node. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely provided with respect to the processor, and these remote memories may be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, an enterprise internal network, a local area network, a mobile communication network, and combinations thereof.

[0140] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: Obtain the performance monitoring data of the target network node within the first target time period, where the first target time period is a historical time period specified by the performance evaluation requirements; Calculate the performance threshold corresponding to the second target time period based on the performance monitoring data, where the second target time period is any future time period specified by the performance evaluation requirements, and there is an association between the second target time period and the first target time period; Obtain the target performance data of the target network node at a preset frequency within the second target time period, and compare the target performance data with the performance threshold to obtain a comparison result; Generate a performance evaluation report of the target network node within the second target time period based on the comparison result.

[0141] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: Calculate the abnormal data threshold based on the performance monitoring data; Determine the abnormal data values in the performance monitoring data based on the abnormal data threshold, where the types of abnormal data values include: sudden increase abnormal values and sudden decrease abnormal values; Delete all abnormal data values in the performance monitoring data to obtain the preprocessed performance monitoring data.

[0142] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: Calculate the average value and dispersion of all performance monitoring data; Calculate the upper limit value of abnormal data in the abnormal data threshold based on the average value and dispersion; Calculate the lower limit value of abnormal data in the abnormal data threshold based on the average value and dispersion, where the lower limit value of abnormal data is less than the upper limit value of abnormal data.

[0143] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: Compare the lower limit value of abnormal data, the upper limit value of abnormal data with all abnormal data values to obtain a comparison result; Determine all abnormal data values greater than the upper limit value of abnormal data indicated by the comparison result as sudden increase abnormal values; Determine all abnormal data values less than the lower limit value of abnormal data indicated by the comparison result as sudden decrease abnormal values.

[0144] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: Determine the median based on all performance monitoring data; Classify all performance monitoring data based on the median to obtain the first type of data and the second type of data, where the first type of data is greater than or equal to the median, and the second type of data is less than the median; Calculate the dispersion based on the median and the first type of data to obtain the upward dispersion of the performance monitoring data; Calculate the dispersion based on the median and the second type of data to obtain the downward dispersion of the performance monitoring data.

[0145] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: calculate the downward coefficient of variation based on the downward dispersion and the median, and calculate the upward coefficient of variation based on the upward dispersion and the median; calculate the downward abnormality degree based on all the sudden drop outliers, and calculate the upward abnormality degree based on all the sudden increase outliers; determine the downward fluctuation coefficient based on the downward coefficient of variation and the downward abnormality degree, and determine the upward fluctuation coefficient based on the upward coefficient of variation and the upward abnormality degree.

[0146] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: obtain a preset sensitivity adjustment coefficient and a preset dispersion mean value; calculate the upper limit value of the performance threshold based on the median, the upward fluctuation coefficient, the upward dispersion, the preset sensitivity adjustment coefficient, and the preset dispersion mean value; calculate the lower limit value of the performance threshold based on the median, the downward fluctuation coefficient, the downward dispersion, the preset sensitivity adjustment coefficient, and the preset dispersion mean value.

[0147] By adopting the embodiment of the present invention, a performance evaluation scheme for virtual network nodes is provided. Through the method of dynamic threshold calculation and by analyzing historical performance monitoring data, the purpose of automatically adapting to business fluctuations and changes is achieved, thereby realizing the technical effects of improving the accuracy of network monitoring and reducing false alarms. Specifically, by analyzing the performance monitoring data of the target network node within the first target time period, the performance threshold for the second target time period (future time period) is automatically calculated. In the traditional static threshold method, false alarms often occur due to improper settings (such as issuing alarms under normal performance conditions or not issuing alarms in a timely manner when performance is abnormal). The dynamic threshold setting strategy of the present invention can better adapt to the dynamic changes of the business in the financial cloud platform, more accurately determine whether the performance of the target network node exceeds the normal range, thereby effectively reducing the false alarm rate, improving the accuracy and timeliness of alarms, and further solving the technical problem in the related art that the high false alarm rate of monitoring is caused by the inability of the static threshold configuration method to accurately adapt to the changes in the network monitoring business of the financial cloud platform.

[0148] Those of ordinary skill in the art can understand that Figure 4 The structure shown is only schematic, and the electronic device can also be a terminal device such as a smart phone, a tablet computer, a handheld computer, and a Mobile Internet Device (MID), a PAD, etc. Figure 4 It does not limit the structure of the above-mentioned electronic device. For example, the electronic device may further include more or fewer components (such as a network interface, a display device, etc.) than those shown Figure 4 in the figure, or have a different configuration from that shown Figure 4 in the figure.

[0149] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware of the terminal device through a program, and this program can be stored in a computer-readable storage medium. The storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disc, etc.

[0150] The present invention will be described below in conjunction with another alternative embodiment.

[0151] Embodiment 4

[0152] The embodiment of the present invention also provides a computer-readable storage medium. Optionally, in the embodiment of the present invention, the above computer-readable storage medium can be used to store the program code executed by the performance evaluation method of the virtual network node provided in the first embodiment above.

[0153] Optionally, in the embodiment of the present invention, the above storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0154] The embodiment of the present invention also provides a computer program product. When executed on a data processing device, it is adapted to execute a program for the steps of the performance evaluation method of the virtual network node: obtaining performance monitoring data of a target network node within a first target time period, where the first target time period is a historical time period specified by the performance evaluation requirement; calculating a performance threshold corresponding to a second target time period based on the performance monitoring data, where the second target time period is any future time period specified by the performance evaluation requirement, and there is an association relationship between the second target time period and the first target time period; obtaining target performance data of the target network node at a preset frequency within the second target time period, and comparing the target performance data with the performance threshold to obtain a comparison result; generating a performance evaluation report of the target network node within the second target time period based on the comparison result.

[0155] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0156] In the above embodiments of the present application, the descriptions of each embodiment have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0157] In several embodiments provided by this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0158] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional unit in various embodiments of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0160] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0161] The above is only the preferred embodiment of this application. It should be noted that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. A performance evaluation method for a virtual network node, characterized in that: include: Acquire performance monitoring data of a target network node within a first target time period, wherein the first target time period is a historical time period specified by a performance evaluation requirement; Calculating a performance threshold corresponding to a second target time period based on the performance monitoring data, wherein the second target time period is any future time period specified by the performance evaluation requirement, and the second target time period is associated with the first target time period; acquiring target performance data of the target network node at a preset frequency within the second target time period, and comparing the target performance data with the performance threshold to obtain a comparison result; A performance evaluation report of the target network node within the second target time period is generated based on the comparison result.

2. The performance evaluation method according to claim 1, characterized in that: Before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data, the method further includes: Calculating an abnormal data threshold based on the performance monitoring data; Determining an abnormal data value in the performance monitoring data based on the abnormal data threshold, wherein the types of the abnormal data value include: a sudden increase abnormal value and a sudden drop abnormal value; All the abnormal data values ​​in the performance monitoring data are deleted to obtain the performance monitoring data after preprocessing.

3. The performance evaluation method according to claim 2, characterized in that: The step of calculating an abnormal data threshold based on the performance monitoring data comprises: Calculate the average and dispersion of all the performance monitoring data; Calculate the abnormal data upper limit value in the abnormal data threshold value based on the average value and the dispersion; An abnormal data lower limit value in the abnormal data threshold is calculated based on the average value and the dispersion, wherein the abnormal data lower limit value is smaller than the abnormal data upper limit value.

4. The performance evaluation method according to claim 3, characterized in that: The step of determining the abnormal data value in the performance monitoring data based on the abnormal data threshold comprises: Compare the abnormal data lower limit value, the abnormal data upper limit value and all the abnormal data values ​​to obtain a comparison result; Determine all the abnormal data values ​​greater than the abnormal data upper limit value indicated by the comparison result as the sudden increase abnormal values; All the abnormal data values ​​that are smaller than the abnormal data lower limit value indicated by the comparison result are determined as the sudden drop abnormal values.

5. The performance evaluation method according to claim 2, characterized in that: Before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data, the method further includes: determining a median value based on all of said performance monitoring data; Classifying all the performance monitoring data based on the median to obtain first-category data and second-category data, wherein the first-category data is greater than or equal to the median, and the second-category data is less than the median; Calculate the dispersion based on the median and the first type of data to obtain the upward dispersion of the performance monitoring data; The dispersion is calculated based on the median and the second type of data to obtain the downward dispersion of the performance monitoring data.

6. The performance evaluation method according to claim 5, characterized in that: Before calculating the performance threshold corresponding to the second target time period based on the performance monitoring data, the method further includes: calculating a downward coefficient of variation based on the downward dispersion and the median, and calculating an upward coefficient of variation based on the upward dispersion and the median; Calculating downward abnormality based on all the sudden drop abnormal values, and calculating upward abnormality based on all the sudden increase abnormal values; A downward fluctuation coefficient is determined based on the downward coefficient of variation and the downward abnormality, and an upward fluctuation coefficient is determined based on the upward coefficient of variation and the upward abnormality.

7. The performance evaluation method according to claim 6, characterized in that: The step of calculating the performance threshold corresponding to the second target time period based on the performance monitoring data includes: Obtaining a preset sensitivity adjustment coefficient and a preset dispersion mean; Calculating an upper limit value of the performance threshold based on the median, the upward fluctuation coefficient, the upward dispersion, the preset sensitivity adjustment coefficient and the preset dispersion mean; The lower limit value of the performance threshold is calculated based on the median, the downward fluctuation coefficient, the downward dispersion, the preset sensitivity adjustment coefficient and the preset dispersion mean.

8. A performance evaluation device for a virtual network node, characterized in that: include: An acquisition unit, configured to acquire performance monitoring data of a target network node within a first target time period, wherein the first target time period is a historical time period specified by a performance evaluation requirement; a calculation unit, configured to calculate a performance threshold corresponding to a second target time period based on the performance monitoring data, wherein the second target time period is any future time period specified by the performance evaluation requirement, and the second target time period is associated with the first target time period; a comparing unit, configured to obtain target performance data of the target network node at a preset frequency within the second target time period, and compare the target performance data with the performance threshold to obtain a comparison result; A generating unit is used to generate a performance evaluation report of the target network node within the second target time period based on the comparison result.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the performance evaluation method for a virtual network node according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the performance evaluation method of the virtual network node described in any one of claims 1 to 7.

Images