Method and system for monitoring and analyzing data transmission performance of computer network

Abstract

The invention relates to the technical field of big data analysis and processing, in particular to a computer network data transmission performance monitoring analysis method and system, which comprises the following steps: through intelligent layout and self-adaptive configuration adjustment of monitoring points, improvement of monitoring accuracy and optimization of resource configuration are facilitated, and performance problems possibly occurring in a double-11 period are prevented; all performance indexes of the e-commerce website are monitored in real time, and performance anomaly detection is carried out, so that any problem possibly influencing website performance can be found and responded in time subsequently; by analyzing and judging whether the performance state of the website is in the normal range or is regarded as abnormal, measures are taken for intervention in time, the situation of false report or missing report is avoided, the working efficiency of an operation and maintenance team is improved, and the influence of faults on the website is reduced. The method and the device are used for solving the technical problem that the performance monitoring and optimization of the e-commerce website in the existing scheme are insufficient.

Description

Technical Field

[0001] The present invention relates to the technical field of big data analysis and processing, and in particular to a computer network data transmission performance monitoring and analysis method and system. Background Art

[0002] Big data analysis and processing technology refers to the technology of collecting, storing, processing and analyzing large data sets by using various tools, algorithms and platforms; these technologies are designed to extract valuable information and knowledge from massive and complex data to provide support for decision-making.

[0003] At present, in the specific field of e-commerce industry, there are still the following problems in performance monitoring and optimization, which need to be solved by big data analysis and processing technology: insufficient real-time and accuracy of data processing. The amount of data generated by e-commerce websites during the Double Eleven period is huge, including user access records, transaction data, log information, etc. Traditional monitoring methods may be difficult to process such large-scale data in real time, resulting in lag and decreased accuracy of monitoring results, and the data processing capacity may not be sufficient to meet this demand, affecting the monitoring effect; the intelligent level of monitoring point layout and configuration is not high. Due to the complexity of the website and the uncertainty of traffic patterns, the traditional monitoring point layout is unreasonable and difficult to respond to network changes in time; performance anomaly detection is not timely and effective. Traditional anomaly detection algorithms may not be able to accurately identify performance anomalies of e-commerce websites in complex network environments. For example, in extreme cases such as high concurrency and large traffic, the system may have false positive or false negative misjudgments. To this end, the present invention proposes a computer network data transmission performance monitoring and analysis method and system. Summary of the invention

[0004] The purpose of the present invention is to solve the problems in the background technology and to propose a computer network data transmission performance monitoring and analysis method and system.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A computer network data transmission performance monitoring and analysis method, comprising:

[0007] Step 1: Extract the data records of the e-commerce website during the Double 11 period from the database, and perform intelligent layout and adaptive configuration adjustment of monitoring points;

[0008] The steps of intelligent layout and adaptive configuration adjustment of monitoring points include:

[0009] B1. Detailed analysis of the network topology of the e-commerce website, including the connection relationship between key components such as data centers, CDN nodes, server clusters, and user access concentration areas, and generate a set of candidate monitoring points with the help of graph theory methods to effectively monitor network performance;

[0010] B2. Analyze the traffic data of e-commerce websites during the historical Double 11 shopping festivals to identify traffic patterns and trends, and use time series analysis to predict the peak traffic period during the Double 11 shopping festival.

[0011] B3. Analyze user access patterns by calculating resource access frequency and access duration, and thus select user access hotspots;

[0012] B4. Combine the network topology, predicted traffic peak time and user access hotspots, and use a multi-objective optimization algorithm to automatically recommend the optimal monitoring point location;

[0013] B5. Dynamically adjust the monitoring point configuration results or status according to the real-time network status and preset adjustment rules;

[0014] Step 2: Monitor all performance indicators of the e-commerce website in real time and perform performance anomaly detection;

[0015] Step 3: Further analyze and determine whether the website performance status is within the normal range or is considered abnormal, so as to take timely intervention measures.

[0016] It should be noted that the application objects of a computer network data transmission performance monitoring and analysis method and system in the embodiments of the present invention can be widely applicable to various computer network environments, including but not limited to enterprise intranets, data centers, cloud computing platforms, Internet of Things networks, etc. Specifically, it can be used to monitor the performance of the Double Eleven network of e-commerce websites through big data analysis and machine learning technology, which is conducive to timely discovering and solving potential performance bottlenecks, ensuring the high availability and user experience of e-commerce websites during the Double Eleven period, and optimizing resource allocation. Reduce business losses caused by network problems.

[0017] Furthermore, the network topology of the e-commerce website is analyzed in detail, and the process of generating a set of candidate monitoring points with the help of graph theory methods includes:

[0018] The network topology is an undirected graph labeled G = (V, E); where V is the set of all nodes in the network, representing data centers, CDN nodes, key server clusters, and user access concentration areas, and E is the edge set, representing the connection between nodes;

[0019] For each node v∈V, calculate the degree centrality of v In the formula, d(v) represents the degree of node v; at the same time, the betweenness centrality of v is calculated In the formula, u represents any starting node in the network except v, r represents any target node in the network except v and u, σ ur represents the total number of shortest paths between nodes u and r, σ ur (v) represents the number of paths passing through node v among all shortest paths;

[0020] It can be understood that the number of paths passing through v in all such shortest paths is compared with the total number of all shortest paths between u and r to obtain the betweenness centrality value of v, which is used to measure the frequency of node v appearing in all shortest paths; through node importance evaluation (such as degree centrality, betweenness centrality), we can better understand the network structure, optimize network performance, prevent risk propagation, improve information dissemination efficiency, and enhance network security;

[0021] Set the degree centrality threshold θ DC and the betweenness centrality threshold θ BC , select DC(v)>θ DC And BC(v)>θ BC The nodes are taken as the candidate monitoring point set X.

[0022] Furthermore, the process of analyzing the traffic data of the e-commerce website during the historical Double Eleven period to identify traffic patterns and trends and using time series analysis to predict the peak traffic period during the Double Eleven period includes:

[0023] Collect the time series traffic data during the historical Double Eleven period and mark it as {T t}; where t represents the time unit;

[0024] It is understandable that these flow data exist in the form of time series, and by choosing hours as the time unit, it is possible to observe the flow changes in more detail;

[0025] Use the time series analysis method to fit the time series traffic data during the historical Double Eleven period and make a forecast: Assume that there is a time series Y t Indicates the daily traffic before and after Double 11 (it can be the number of days from the start of Double 11, negative numbers indicate before Double 11, positive numbers indicate after Double 11) using the formula Y t =β0+β1t+S t +ε t Fitting and prediction; where β0 is the intercept term, indicating the baseline level of the time series, β1 is the slope term, indicating the time trend, S t is a seasonal adjustment term, which adjusts the forecast value according to the value of t to reflect the special traffic pattern during the Double Eleven period, ε t is the error term, indicating possible random fluctuations;

[0026] Based on historical data, the traffic value at each time point during the Double Eleven period is predicted, that is, the predicted traffic value Where n represents the number of future time units predicted;

[0027] It is understandable that if we want to predict the traffic on Double Eleven, n is the number of hours from the beginning to the end of Double Eleven;

[0028] The preset flow peak threshold θ P ;

[0029] It is understandable that the preset traffic peak threshold is used to identify which time periods have potential peak traffic;

[0030] The predicted flow value and the preset flow peak threshold θ P Compare and traverse the predicted traffic values Identify predicted flow values period, and take this period as the potential peak period P.

[0031] Furthermore, by calculating the resource access frequency and access duration, analyzing the user access pattern, and selecting the user access hotspot, the process includes:

[0032] Get the user access record set A = {(a, b, c)}; where a represents the user ID, b represents the resource ID, and c represents the access time;

[0033] For each resource b, calculate the access frequency and average access time; where the access frequency is f(b) = ∑ (a,b,c)∈A 1. The average visit duration is g(a,b,c) represents the duration that user a accesses resource b at time c;

[0034] Set the access hotspot threshold θ f and Filter high-frequency and long-duration resources, that is, select f(b)>θ f and The resources are taken as the access hotspot H; wherein the user access hotspot represents the user's high-frequency access area, including popular product pages, shopping cart pages, etc.

[0035] Furthermore, the process of automatically recommending the optimal monitoring point location using a multi-objective optimization algorithm based on the network topology, predicted traffic peak time period, and user access hotspots includes:

[0036] Define the optimization objective function F(x); where x represents the monitoring point configuration scheme, including the selection, layout, number, type and possible other configuration parameters of the monitoring points, which is used to guide the subsequent monitoring point configuration work;

[0037] The optimization objectives include maximizing the monitoring coverage Cov(x,H), i.e., the range of hotspot areas monitored, minimizing the monitoring cost Cost(x), and ensuring a high sampling frequency during the potential peak period P;

[0038] The multi-objective optimization algorithm minF(x)=(-Cov(H,x),Cost(x)) is used to perform optimization operations, thereby obtaining the optimal monitoring point position; where minF(x) represents the minimization result of the optimization objective function F(x);

[0039] It is understandable that in order to unify the optimization direction, it is transformed into a minimization problem and the monitoring coverage is negative; the specific steps include: generating a set of initial monitoring point configuration schemes x1, x2, …, x m , as the starting point of the algorithm, where m represents the total number of monitoring point configuration schemes; for each configuration scheme x k , calculate its optimization objective function value F(x k ), where k represents the index of the monitoring point configuration scheme; the schemes are non-dominated sorted according to the optimization objective function value, and the crowding distance of each scheme is calculated (to maintain the diversity of solutions); according to the non-dominated sorting and crowding distance, a part of excellent schemes are selected as parents to generate the next generation of offspring; new configuration schemes are generated through operations such as crossover and mutation; iterative iterations are repeated until the termination condition is met; and finally a set of optimal solution sets are obtained, that is, a series of monitoring point configuration schemes that achieve different balances between monitoring coverage and monitoring cost.

[0040] Furthermore, the process of dynamically adjusting the monitoring point configuration result or state according to the real-time network status and the preset adjustment rules includes:

[0041] The adjustment is directly triggered by the real-time monitored network status W, where the network status W is a multi-dimensional vector, including indicators such as traffic changes and device load;

[0042] According to the preset adjustment rule R and the monitoring point configuration scheme x, the monitoring point configuration result or state h is updated, that is, h'=R(x(h),W); wherein the adjustment rule R is used to determine whether adaptive adjustment needs to be triggered, and h' represents the updated monitoring point configuration result or state, reflecting the actual status of the monitoring network at a certain moment, including increasing the sampling frequency of hotspot areas, reducing the sampling frequency of non-hotspot areas, or replacing the monitoring indicator set.

[0043] Furthermore, the process of real-time monitoring of all performance indicators of e-commerce websites and detecting performance anomalies includes:

[0044] The performance indicator data of page loading time Pl, server response time Sr, network delay Nl and concurrent user number Cu are collected in real time, and each performance data is stored in the performance data set D, that is, Each data point corresponds to a timestamp and a corresponding performance indicator value; i represents the index and N represents the total number of performance indicator data. It can be understood that page loading time refers to the time from when the user initiates a page request to when the page is fully loaded and displayed on the e-commerce webpage; server response time refers to the time from when the server receives the request to when it starts to send a response to the client; network latency refers to the time required for a data packet to be transmitted in the network; the number of concurrent users refers to the number of users accessing the e-commerce website at the same time, which is an indicator that directly reflects the load capacity of the e-commerce website;

[0045] Statistical analysis is performed for each performance indicator, where the statistical analysis formula is:

[0046]

[0047] In the formula, μD represents the average value of the performance index, and τD represents the standard deviation of the performance index;

[0048] Based on the mean and standard deviation of the performance indicators, the standard score method is used to quantify the abnormality of the performance indicators, and the concurrent user number threshold comparison method is adopted:

[0049]

[0050] In the formula, Z Pl , Z Sr , Z Nl Respectively represent the standard score value of each performance index;

[0051] The maximum absolute value of the standard score value of the performance indicator and the comparison result of the number of concurrent users Cu and the concurrent user threshold BY are extracted, and they are combined into a composite anomaly score:

[0052]

[0053] Where AS represents the composite anomaly score. The higher the score, the higher the degree of anomaly. max(|Z Pl |,|Z Sr |,|Z Nl |) represents the maximum absolute value of the standard score value of the performance indicator; Indicates the comparison result between the number of concurrent users and the concurrent user threshold. If the number of concurrent users is greater than the concurrent user threshold, then the calculation Proportional difference, if the number of concurrent users is not greater than the concurrent user threshold, the indicator function is 0; I(Cu>BY) is the indicator function, when the number of concurrent users is greater than the concurrent user threshold, the indicator function value is 1, when the number of concurrent users is not greater than the concurrent user threshold, the indicator function value is 0.

[0054] Furthermore, the process of further analyzing and determining whether the website performance status is within the normal range or is considered abnormal, so as to take timely intervention measures includes:

[0055] Set the abnormal detection value (ADmin, ADmax), and ADmin<ADmax;

[0056] The performance status of the e-commerce website is evaluated according to the composite anomaly score value to obtain the performance evaluation result to deal with possible performance bottlenecks: the composite anomaly score value AS is compared with the anomaly detection value (ADmin, ADmax) to determine whether the overall performance status of the e-commerce website is within the normal range or is considered abnormal; if the composite anomaly score value AS is less than or equal to the anomaly detection value ADmin, it indicates that the overall performance status of the current e-commerce website is within the normal range and is marked as highly normal; if the composite anomaly score value AS is greater than or equal to the anomaly detection value ADmax, it indicates that the overall performance status of the current e-commerce website is in a highly abnormal state and an abnormal signal is generated; if the composite anomaly score value AS is not less than the anomaly detection value ADmin and not greater than the anomaly detection value ADmax, it indicates that the overall performance status of the current e-commerce website is suspected to be abnormal, but has not yet reached the level of high abnormality, and an inquiry signal is generated;

[0057] For the abnormal signals generated, an early warning is immediately issued to the operation and maintenance personnel to quickly locate the root cause of the problem and take maintenance measures in advance to avoid service interruption; the early warning includes the time when the abnormality occurred, the affected system components, the possible scope of impact, etc.

[0058] For the generated abnormal signal, the query signal is associated with the composite abnormal score value to determine the performance indicator that may have abnormalities. If the indicator shows an upward trend, it is necessary to pay attention to whether there is a risk of resource exhaustion. At the same time, the current query signal is compared with the corresponding historical data of the performance indicator to check whether there are abnormal fluctuations or trend changes for predictive maintenance.

[0059] A computer network data transmission performance monitoring and analysis system, comprising:

[0060] The data extraction and processing analysis module is used to extract the historical data of the data center from the database, and classify and analyze it, so as to obtain the normal data group and the fault data group; trace the normal data group and the fault data group to obtain the first data group and the second data group;

[0061] The data extraction and processing module is used to extract the data records of the e-commerce website during the Double 11 period from the database, and to perform intelligent layout and adaptive configuration adjustment of monitoring points;

[0062] The performance monitoring and anomaly detection module is used to monitor all performance indicators of the e-commerce website in real time and perform performance anomaly detection;

[0063] The performance status judgment and analysis module is used to further analyze and determine whether the website performance status is within the normal range or is considered abnormal, so as to take timely intervention measures.

[0064] Compared with the existing technology, the advantages of the computer network data transmission performance monitoring and analysis method and system provided by the present invention are:

[0065] 1. The present invention extracts data records of the e-commerce website during the Double 11 period from the database, performs intelligent layout and adaptive configuration adjustment of monitoring points, ensures that the monitoring points cover the most critical network areas, and generates a set of candidate monitoring points to help make the best choice under limited resources; by using a multi-objective optimization algorithm to automatically recommend the optimal monitoring point location, it can achieve the best monitoring effect under limited resource conditions and optimize resource allocation;

[0066] 2. The present invention can quickly discover performance anomalies by monitoring all performance indicators of e-commerce websites in real time and performing performance anomaly detection, and can predictably identify potential performance bottlenecks or failures, providing the possibility for rapid response and problem solving;

[0067] 3. The present invention determines whether the website performance status is within the normal range or is considered abnormal by analyzing, and can accurately determine the website performance status, so as to take timely intervention measures to prevent the problem from further deteriorating.

[0068] To sum up, the various steps of the present invention together constitute a complete process for performance monitoring and optimization of e-commerce websites during the Double Eleven period. Its beneficial effects are reflected in improving monitoring efficiency, optimizing resource allocation, enhancing user experience, enhancing system adaptability and reducing failure frequency, thereby ensuring the normal operation of a subsequent computer network data transmission performance monitoring and analysis method and system. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 The present invention provides a flow chart of a computer network data transmission performance monitoring and analysis method.

[0070] Figure 2 This is a module diagram of a computer network data transmission performance monitoring and analysis system proposed by the present invention.

[0071] Figure 3 This is a flow chart of the intelligent layout and adaptive configuration adjustment of monitoring points proposed by the present invention. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the implementation regulations described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] See also Figure 1 The present invention provides a computer network data transmission performance monitoring and analysis method, comprising:

[0074] Step 1: Extract the data records of the e-commerce website during the Double 11 period from the database, and perform intelligent layout and adaptive configuration adjustment of monitoring points;

[0075] Step 2: Monitor all performance indicators of the e-commerce website in real time and perform performance anomaly detection;

[0076] Step 3: Further analyze and determine whether the website performance status is within the normal range or is considered abnormal, so as to take timely intervention measures.

[0077] It should be noted that the application objects of a computer network data transmission performance monitoring and analysis method and system in the embodiments of the present invention can be widely applicable to various computer network environments, including but not limited to enterprise intranets, data centers, cloud computing platforms, Internet of Things networks, etc. Specifically, it can be used to monitor the performance of the Double Eleven network of e-commerce websites through big data analysis and machine learning technology, which is conducive to timely discovering and solving potential performance bottlenecks, ensuring the high availability and user experience of e-commerce websites during the Double Eleven period, and optimizing resource allocation. Reduce business losses caused by network problems.

[0078] See also Figure 1 and Figure 3 The present invention provides a computer network data transmission performance monitoring and analysis method. In the step 1, the data records of the e-commerce website during the historical Double Eleven period are extracted from the database, and the steps of performing intelligent layout and adaptive configuration adjustment of monitoring points include:

[0079] Step 101: Analyze the network topology of the e-commerce website in detail, including the connection relationship between key components such as data centers, CDN nodes, server clusters, and user access concentration areas, and generate a set of candidate monitoring points with the help of graph theory methods to effectively monitor network performance. The specific steps are as follows:

[0080] The network topology is an undirected graph labeled G = (V, E); where V is the set of all nodes in the network, representing data centers, CDN nodes, key server clusters, and user access concentration areas, and E is the edge set, representing the connection between nodes;

[0081] For each node v∈V, calculate the degree centrality of v In the formula, d(v) represents the degree of node v; at the same time, the betweenness centrality of v is calculated In the formula, u represents any starting node in the network except v, r represents any target node in the network except v and u, σ ur represents the total number of shortest paths between nodes u and r, σ ur (v) represents the number of paths passing through node v among all shortest paths;

[0082] It can be understood that the number of paths passing through v in all such shortest paths is compared with the total number of all shortest paths between u and r to obtain the betweenness centrality value of v, which is used to measure the frequency of node v appearing in all shortest paths; through node importance evaluation (such as degree centrality, betweenness centrality), we can better understand the network structure, optimize network performance, prevent risk propagation, improve information dissemination efficiency, and enhance network security;

[0083] Set the degree centrality threshold θ DC and the betweenness centrality threshold θ BC , select DC(v)>θ DC And BC(v)>θ BC The nodes are taken as the candidate monitoring point set X;

[0084] Step 102: Analyze the traffic data of the e-commerce website during the historical Double 11 shopping festival to identify traffic patterns and trends, and use time series analysis to predict the traffic peak period during the Double 11 shopping festival. The specific steps are as follows:

[0085] Collect the time series traffic data during the historical Double Eleven period and mark it as {T t}; where t represents the time unit;

[0086] It is understandable that these flow data exist in the form of time series, and by choosing hours as the time unit, it is possible to observe the flow changes in more detail;

[0087] Use the time series analysis method to fit the time series traffic data during the historical Double Eleven period and make a forecast: Assume that there is a time series Y t Indicates the daily traffic before and after Double 11 (it can be the number of days from the start of Double 11, negative numbers indicate before Double 11, positive numbers indicate after Double 11) using the formula Y t=β0+β1t+S t +ε t Fitting and prediction; where β0 is the intercept term, indicating the baseline level of the time series, β1 is the slope term, indicating the time trend, S t is a seasonal adjustment term, which adjusts the forecast value according to the value of t to reflect the special traffic pattern during the Double Eleven period, ε t is the error term, indicating possible random fluctuations;

[0088] Based on historical data, the traffic value at each time point during the Double Eleven period is predicted, that is, the predicted traffic value Where n represents the number of future time units predicted;

[0089] It is understandable that if we want to predict the traffic on Double Eleven, n is the number of hours from the beginning to the end of Double Eleven;

[0090] The preset flow peak threshold θ P ;It is understandable that the preset traffic peak threshold is used to identify which time periods are potential peak traffic;

[0091] The predicted flow value and the preset flow peak threshold θ P Compare and traverse the predicted traffic values Identify predicted flow values The period is taken as the potential peak period P;

[0092] Step 103: Analyze the user access pattern by calculating the resource access frequency and access duration, and thus select the user access hotspot; wherein the specific steps are as follows:

[0093] Get the user access record set A = {(a, b, c)}; where a represents the user ID, b represents the resource ID, and c represents the access time;

[0094] For each resource b, calculate the access frequency and average access time; where the access frequency is f(b) = ∑ (a,b,c)∈A 1. The average visit duration is g(a,b,c) represents the duration that user a accesses resource b at time c;

[0095] Set the access hotspot threshold θ f and Filter high-frequency and long-duration resources, that is, select f(b)>θ f and The resources are taken as the access hotspot H; wherein the user access hotspot represents the user's high-frequency access area, including popular product pages, shopping cart pages, etc.;

[0096] Step 104: Combine the network topology, predicted traffic peak time period and user access hotspots, and use a multi-objective optimization algorithm to automatically recommend the optimal monitoring point location; wherein the specific steps are as follows:

[0097] Define the optimization objective function F(x); where x represents the monitoring point configuration scheme, including the selection, layout, number, type and possible other configuration parameters of the monitoring points, which is used to guide the subsequent monitoring point configuration work;

[0098] The optimization objectives include maximizing the monitoring coverage Cov(x,H), i.e., the range of hotspot areas monitored, minimizing the monitoring cost Cost(x), and ensuring a high sampling frequency during the potential peak period P;

[0099] The multi-objective optimization algorithm minF(x)=(-Cov(H,x),Cost(x)) is used to perform optimization operations, thereby obtaining the optimal monitoring point position; where minF(x) represents the minimization result of the optimization objective function F(x);

[0100] In step 104, in order to unify the optimization direction, it is converted into a minimization problem, and the monitoring coverage is negative. The specific steps include: generating a set of initial monitoring point configuration schemes x1, x2, ..., x m , as the starting point of the algorithm, where m represents the total number of monitoring point configuration schemes; for each configuration scheme x k , calculate its optimization objective function value F(x k ), where k represents the index of the monitoring point configuration scheme; the schemes are non-dominated sorted according to the optimization objective function value, and the crowding distance of each scheme is calculated (to maintain the diversity of solutions); according to the non-dominated sorting and crowding distance, a part of excellent schemes are selected as parents to generate the next generation of offspring; new configuration schemes are generated through operations such as crossover and mutation; iterative iterations are repeated until the termination condition is met; and finally a set of optimal solution sets are obtained, that is, a series of monitoring point configuration schemes that achieve different balances between monitoring coverage and monitoring cost;

[0101] Step 105: dynamically adjust the monitoring point configuration result or state according to the real-time network status and preset adjustment rules; wherein the specific steps are as follows:

[0102] The adjustment is directly triggered by the real-time monitored network status W, where the network status W is a multi-dimensional vector, including indicators such as traffic changes and device load;

[0103] According to the preset adjustment rule R and the monitoring point configuration scheme x, the monitoring point configuration result or state h is updated, that is, h'=R(x(h),W); wherein the adjustment rule R is used to determine whether adaptive adjustment needs to be triggered, and h' represents the updated monitoring point configuration result or state, reflecting the actual status of the monitoring network at a certain moment, including increasing the sampling frequency of hotspot areas, reducing the sampling frequency of non-hotspot areas, or replacing the monitoring indicator set.

[0104] See also Figure 1 The present invention provides a computer network data transmission performance monitoring and analysis method. In step 2, the step of real-time monitoring of all performance indicators of the e-commerce website and performing performance anomaly detection includes:

[0105] Step 201: collect performance indicator data of page loading time Pl, server response time Sr, network delay Nl and concurrent user number Cu in real time, and store each performance data in performance data set D, that is, Each data point corresponds to a timestamp and a corresponding performance indicator value, where i represents the index and N represents the total number of performance indicator data.

[0106] In step 201, page loading time refers to the time from when a user initiates a page request to when the page is fully loaded and displayed on the e-commerce webpage; server response time refers to the time from when the server receives a request to when it starts to send a response to the client; network latency refers to the time required for a data packet to be transmitted in the network; the number of concurrent users refers to the number of users accessing the e-commerce website at the same time, which is an indicator that directly reflects the load capacity of the e-commerce website;

[0107] Step 202: Perform statistical analysis on each performance indicator, wherein the statistical analysis formula is:

[0108]

[0109] In the formula, μD represents the average value of the performance index, and τD represents the standard deviation of the performance index;

[0110] Step 203: Based on the average value and standard deviation of the performance indicator, the standard score method is used to quantify the abnormality of the performance indicator, and the concurrent user number threshold comparison method is adopted:

[0111]

[0112] In the formula, Z Pl , Z Sr , Z Nl Respectively represent the standard score value of each performance index;

[0113] Step 204: extract the maximum value of the absolute value of the standard score value of the performance indicator and the comparison result of the number of concurrent users Cu and the concurrent user threshold BY, and combine them into a composite anomaly score:

[0114]

[0115] Where AS represents the composite anomaly score. The higher the score, the higher the degree of anomaly. max(|Z Pl |,|Z Sr |,|Z Nl |) represents the maximum absolute value of the standard score value of the performance indicator; Indicates the comparison result between the number of concurrent users and the concurrent user threshold. If the number of concurrent users is greater than the concurrent user threshold, then the calculation Proportional difference, if the number of concurrent users is not greater than the concurrent user threshold, the indicator function is 0; I(Cu>BY) is the indicator function, when the number of concurrent users is greater than the concurrent user threshold, the indicator function value is 1, when the number of concurrent users is not greater than the concurrent user threshold, the indicator function value is 0.

[0116] See also Figure 1 The present invention provides a computer network data transmission performance monitoring and analysis method. In step three, the step of further analyzing and determining whether the website performance status is within a normal range or is considered abnormal, so as to take timely intervention measures includes:

[0117] Step 301, setting an abnormal detection value (ADmin, ADmax), and ADmin<ADmax;

[0118] Step 302: Evaluate the performance status of the e-commerce website according to the composite anomaly score value to obtain a performance evaluation result to deal with possible performance bottlenecks: compare the composite anomaly score value AS with the anomaly detection value (ADmin, ADmax) to determine whether the overall performance status of the e-commerce website is within the normal range or is considered abnormal; if the composite anomaly score value AS is less than or equal to the anomaly detection value ADmin, it indicates that the overall performance status of the current e-commerce website is within the normal range and is marked as highly normal; if the composite anomaly score value AS is greater than or equal to the anomaly detection value ADmax, it indicates that the overall performance status of the current e-commerce website is in a highly abnormal state, and an abnormal signal is generated; if the composite anomaly score value AS is not less than the anomaly detection value ADmin and not greater than the anomaly detection value ADmax, it indicates that the overall performance status of the current e-commerce website is suspected to be abnormal, but has not yet reached the level of high abnormality, and an inquiry signal is generated;

[0119] Step 303: For the generated abnormal signal, an early warning is immediately issued to the operation and maintenance personnel to quickly locate the root cause of the problem and take maintenance measures in advance to avoid service interruption; wherein the early warning includes the time when the abnormality occurs, the affected system components, the possible impact range, etc.;

[0120] Step 304: For the generated abnormal signal, the query signal is associated with the composite abnormality score value to determine the performance indicator that may have abnormalities. If the indicator shows an upward trend, it is necessary to pay attention to whether there is a risk of resource exhaustion; at the same time, the current query signal is compared with the corresponding performance indicator historical data to check whether there are abnormal fluctuations or trend changes for predictive maintenance.

[0121] In the embodiment of the present invention, by deeply analyzing the data of the historical Double Eleven period of the e-commerce website, using intelligent layout and adaptive configuration adjustment technology, a solid foundation is laid for subsequent monitoring and performance optimization. By analyzing the historical traffic data and time series analysis, the peak traffic period during the Double Eleven period can be predicted more accurately, which is helpful for early resource deployment and load balancing optimization. By identifying user access hotspots, it is helpful to add monitoring points in hotspot areas, monitor and optimize user experience in real time, and reduce delays or interruptions caused by large access volume. By using a multi-objective optimization algorithm to automatically recommend the optimal monitoring point location, reduce manual intervention, and improve configuration efficiency and accuracy, by dynamically adjusting the monitoring point configuration according to real-time network conditions and preset rules, the flexibility and adaptability of the system are enhanced, ensuring that a stable monitoring effect can be maintained during peak traffic periods. By real-time monitoring of all performance indicators of the e-commerce website, it is ensured that any problems that may affect the performance of the website can be discovered in time during the Double Eleven period. By conducting in-depth analysis of the monitored performance data, it is determined whether the website performance status is within the normal range, and once an abnormality is found, timely measures are taken to intervene. In summary, the present invention solves the technical problems that the e-commerce website in the existing scheme has deficiencies in performance monitoring and optimization. In actual situations, more data and contextual information may be needed to make specific decisions and optimize solutions.

[0122] See also Figure 2 The present invention provides a computer network data transmission performance monitoring and analysis system, comprising:

[0123] The data extraction and processing module is used to extract the data records of the e-commerce website during the Double 11 period from the database, and to perform intelligent layout and adaptive configuration adjustment of monitoring points;

[0124] The performance monitoring and anomaly detection module is used to monitor all performance indicators of the e-commerce website in real time and perform performance anomaly detection;

[0125] The performance status judgment and analysis module is used to further analyze and determine whether the website performance status is within the normal range or is considered abnormal, so as to take timely intervention measures.

[0126] In addition, the formulas involved in the above are all calculated by removing dimensions and taking their numerical values. They are a formula that is closest to the actual situation obtained by collecting a large amount of data and performing software simulation. The proportional coefficient in the formula and the various preset thresholds in the analysis process are set by technical personnel in this field according to actual conditions or obtained by simulating a large amount of data; the size of the proportional coefficient is to quantify each parameter to obtain a specific value for subsequent comparison. The size of the proportional coefficient depends on the amount of sample data and the preliminary setting of the corresponding processing coefficient for each group of sample data by technical personnel in this field; as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0127] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically based on the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0128] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0129] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0133] Secondly: In the drawings of the embodiments disclosed in the present invention, only the structures related to the embodiments disclosed in the present invention are involved, and other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of the present invention can be combined with each other;

[0134] Finally: The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A computer network data transmission performance monitoring and analysis method, characterized in that: Step 1: Extract the data records of the e-commerce website during the Double 11 period from the database, and perform intelligent layout and adaptive configuration adjustment of monitoring points; The steps of intelligent layout and adaptive configuration adjustment of monitoring points include: B1. Analyze the network topology of the e-commerce website in detail and generate a set of candidate monitoring points with the help of graph theory methods; B2. Analyze the traffic data of e-commerce websites during the historical Double 11 shopping festivals to identify traffic patterns and trends, and use time series analysis to predict the peak traffic period during the Double 11 shopping festival. B3. Analyze user access patterns by calculating resource access frequency and access duration, and thus select user access hotspots; B4. Combine the network topology, predicted traffic peak time and user access hotspots, and use a multi-objective optimization algorithm to automatically recommend the optimal monitoring point location; B5. Dynamically adjust the monitoring point configuration results or status according to the real-time network status and preset adjustment rules; Step 2: Monitor all performance indicators of the e-commerce website in real time and perform performance anomaly detection; Step 3: Further analyze and determine whether the website performance status is within the normal range or is considered abnormal, so as to take timely intervention measures.

2. A computer network data transmission performance monitoring and analysis method according to claim 1, characterized in that: The process of analyzing the network topology of the e-commerce website in detail and generating a set of candidate monitoring points with the help of graph theory methods includes: The network topology is an undirected graph labeled G = (V, E); where V is the set of all nodes in the network, and E is the edge set, representing the connection between nodes; For each node v∈V, calculate the degree centrality of v In the formula, d(v) represents the degree of node v; at the same time, the betweenness centrality of v is calculated In the formula, u represents any starting node in the network except v, and r represents any target node in the network except v and u. σur represents the total number of shortest paths between nodes u and r, σur (v) represents the number of paths passing through node v among all shortest paths; Set the degree centrality threshold θ DC and the betweenness centrality threshold θ BC , select DC(v)>θ DC And BC(v)>θ BC The nodes are taken as the candidate monitoring point set X.

3. A computer network data transmission performance monitoring and analysis method according to claim 1, characterized in that: The process of analyzing the traffic data of the e-commerce website during the Double 11 shopping festival to identify traffic patterns and trends and using time series analysis to predict the peak traffic period during the Double 11 shopping festival includes: Collect the time series traffic data during the historical Double Eleven period and mark it as {T t }; where t represents the time unit; Use the time series analysis method to fit the time series traffic data during the historical Double Eleven period and make a forecast: Assume that there is a time series Y t Indicates the daily traffic before and after Double Eleven, using the formula Y t =β0+β1t+S t +ε t Fitting and prediction; where β0 is the intercept term, indicating the baseline level of the time series, β1 is the slope term, indicating the time trend, S t is a seasonal adjustment term, which adjusts the forecast value according to the value of t to reflect the special traffic pattern during the Double Eleven period, ε t is the error term; Based on historical data, the traffic value at each time point during the Double Eleven period is predicted, that is, the predicted traffic value Where n represents the number of future time units predicted; The preset flow peak threshold θ P ; The predicted flow value and the preset flow peak threshold θ P Compare and traverse the predicted traffic values Identify predicted flow values period, and take this period as the potential peak period P.

4. A computer network data transmission performance monitoring and analysis method according to claim 1, characterized in that: The process of calculating resource access frequency and access duration, analyzing user access patterns, and selecting user access hotspots includes: Get the user access record set A = {(a, b, c)}; where a represents the user ID, b represents the resource ID, and c represents the access time; For each resource b, calculate the access frequency and average access time; where the access frequency is f(b) = ∑ (a,b,c)∈A 1. The average visit duration is g(a,b,c) represents the duration that user a accesses resource b at time c; Set the access hotspot threshold θ f and Filter high-frequency and long-duration resources, that is, select f(b)>θ f and The resource is used as the access hotspot H.

5. A computer network data transmission performance monitoring and analysis method according to claim 1, characterized in that: The process of automatically recommending the optimal monitoring point location using a multi-objective optimization algorithm based on the network topology, predicted traffic peak time period, and user access hotspots includes: Define the optimization objective function F(x); where x represents the monitoring point configuration scheme; The optimization objectives include maximizing the monitoring coverage Cov(x,H), i.e., the range of hotspot areas monitored, minimizing the monitoring cost Cost(x), and ensuring a high sampling frequency during the potential peak period P; The multi-objective optimization algorithm minF(x)=(-Cov(H,x),Cost(x)) is used to perform optimization operations to obtain the optimal monitoring point position; where minF(x) represents the minimization result of the optimization objective function F(x).

6. A computer network data transmission performance monitoring and analysis method according to claim 1, characterized in that: The process of dynamically adjusting the monitoring point configuration result or status according to the real-time network status and preset adjustment rules includes: The adjustment is directly triggered by the network status W monitored in real time; wherein the network status W is a multi-dimensional vector; According to the preset adjustment rule R and the monitoring point configuration scheme x, the monitoring point configuration result or state h is updated, that is, h'=R(x(h),W); wherein the adjustment rule R is used to determine whether adaptive adjustment needs to be triggered, and h' represents the updated monitoring point configuration result or state, reflecting the actual status of the monitoring network at a certain moment.

7. A computer network data transmission performance monitoring and analysis method according to claim 1, characterized in that: The process of real-time monitoring of all performance indicators of e-commerce websites and detecting performance anomalies includes: The performance indicator data of page loading time Pl, server response time Sr, network delay Nl and concurrent user number Cu are collected in real time, and each performance data is stored in the performance data set D, that is, Each data point corresponds to a timestamp and a corresponding performance indicator value; i represents the index and N represents the total number of performance indicator data. It can be understood that page loading time refers to the time from when the user initiates a page request to when the page is fully loaded and displayed on the e-commerce webpage; server response time refers to the time from when the server receives the request to when it starts to send a response to the client; network latency refers to the time required for a data packet to be transmitted in the network; the number of concurrent users refers to the number of users accessing the e-commerce website at the same time, which is an indicator that directly reflects the load capacity of the e-commerce website; Statistical analysis is performed for each performance indicator, where the statistical analysis formula is: In the formula, μD represents the average value of the performance index, and τD represents the standard deviation of the performance index; Based on the mean and standard deviation of the performance indicators, the standard score method is used to quantify the abnormality of the performance indicators, and the concurrent user number threshold comparison method is adopted: In the formula, Z Pl , Z Sr , Z Nl Respectively represent the standard score value of each performance index; The maximum absolute value of the standard score value of the performance indicator and the comparison result of the number of concurrent users Cu and the concurrent user threshold BY are extracted, and they are combined into a composite anomaly score: In the formula, AS represents the composite anomaly score. The higher the score, the higher the degree of anomaly. max(Z Pl ,|Z Sr ,|Z Nl ) represents the maximum value among the absolute values ​​of the standard scores of the performance indicators; Indicates the comparison result between the number of concurrent users and the concurrent user threshold. If the number of concurrent users is greater than the concurrent user threshold, then the calculation Proportional difference, if the number of concurrent users is not greater than the concurrent user threshold, the indicator function is 0; I(Cu>BY) is the indicator function, when the number of concurrent users is greater than the concurrent user threshold, the indicator function value is 1, when the number of concurrent users is not greater than the concurrent user threshold, the indicator function value is 0.

8. A computer network data transmission performance monitoring and analysis method according to claim 1, characterized in that: The process of further analyzing and determining whether the website performance status is within the normal range or is considered abnormal, so as to take timely intervention measures, includes: Set the abnormal detection value (ADmin, ADmax), and ADmin<ADmax; The performance status of the e-commerce website is evaluated according to the composite anomaly score value to obtain the performance evaluation result: the composite anomaly score value AS is compared with the anomaly detection value (ADmin, ADmax) to determine whether the overall performance status of the e-commerce website is within the normal range or is considered abnormal; if the composite anomaly score value AS is less than or equal to the anomaly detection value ADmin, it indicates that the overall performance status of the current e-commerce website is within the normal range and is marked as highly normal; if the composite anomaly score value AS is greater than or equal to the anomaly detection value ADmax, it indicates that the overall performance status of the current e-commerce website is in a highly abnormal state and an abnormal signal is generated; if the composite anomaly score value AS is not less than the anomaly detection value ADmin and not greater than the anomaly detection value ADmax, it indicates that the overall performance status of the current e-commerce website is suspected to be abnormal, but has not yet reached the level of high abnormality, and an inquiry signal is generated; For abnormal signals generated, an early warning is immediately issued to the operation and maintenance personnel to quickly locate the root cause of the problem and take maintenance measures in advance; For the generated abnormal signal, the query signal is associated with the composite abnormal score value to determine the performance indicator that may have abnormalities. If the indicator shows an upward trend, it is necessary to pay attention to whether there is a risk of resource exhaustion. At the same time, the current query signal is compared with the corresponding historical data of the performance indicator to check whether there are abnormal fluctuations or trend changes for predictive maintenance.

9. A computer network data transmission performance monitoring and analysis system, characterized in that: A computer network data transmission performance monitoring and analysis method as described in any one of claims 1 to 8, the system comprising: The data extraction and processing module is used to extract the data records of the e-commerce website during the Double 11 period from the database, and to perform intelligent layout and adaptive configuration adjustment of monitoring points; The performance monitoring and anomaly detection module is used to monitor all performance indicators of the e-commerce website in real time and perform performance anomaly detection; The performance status judgment and analysis module is used to further analyze and determine whether the website performance status is within the normal range or is considered abnormal, so as to take timely intervention measures.

Images