Performance analysis method based on call scene large model and log collection

Abstract

The invention discloses a performance analysis method based on a call scene large model and log collection, and particularly relates to the field of computer system operation analysis, comprising log collection, log processing and storage, log model training, data injection, log reasoning and training, performance billboard and interactive question and answer, and performance billboard construction by using Grafana. According to the step of billboard design, a new Dashboard is created in Grafana, the layout of a billboard is designed, the layout comprises the arrangement, size and color of a panel, a corresponding Panel is created for each performance index, and a data source, a query statement and a chart type are configured. The method has the advantages of high automation degree, comprehensive and accurate analysis and high expandability.

Description

Technical Field

[0001] The present invention relates to the field of computer system operation analysis, and in particular to a performance analysis method based on a large call scenario model and log collection. Background Art

[0002] With the rapid development of computer technology, the overall operation of the system has become an important research direction. Traditional performance analysis methods often rely on manual experience and simple statistical methods, which cannot fully and accurately reflect the performance of the system. In recent years, with the continuous development of big data and artificial intelligence technology, the application of big models in the field of performance analysis has gradually attracted attention.

[0003] However, how to effectively collect and utilize log data to support performance analysis of large models remains an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a performance analysis method based on a large call scenario model and log collection, to solve the efficiency problems of existing operation and maintenance log analysis methods when processing large-scale data, as well as the deficiencies in scenarios such as predictability, performance indicator statistics, operation indicator statistics and background auditing.

[0005] To achieve the above object, the present invention provides the following technical solutions: A performance analysis method based on a large call scenario model and log collection includes the following steps: 1) Log collection: collects log data from various call scenarios, including system logs, application logs, and security logs; 2) Log processing and storage: The collected log data is cleaned, filtered and converted using Pandas or DataCleaner to eliminate redundant information, standardize the data format, and extract valuable information. The processed data is stored in a database or distributed storage system. 3) Log model training, use K-means clustering or DBSCAN algorithm to train the processed log data: DBSCAN algorithm steps: Initialization: Assume that there is a data set containing multiple data points. Set two key parameters: eps and MinPts; Core point identification: traverse each point in the data set and calculate the number of points in its eps neighborhood; if the number of points in the eps neighborhood of a point is greater than or equal to MinPts, the point is marked as a core point.

[0006] Clustering start: randomly select a core point as the starting point to start a new clustering process; add all points in the eps neighborhood of the core point to the current cluster; density reachability and density connection: starting from the points that have been added to the cluster, find all density reachable points; if there is a core point sequence between two points, so that they can be density reachable through the sequence, then the two points are called density connected, and all density connected points are added to the current cluster; Continue clustering: Repeat the above process until all core points have been visited and the clusters they belong to have been determined, and the remaining points are marked as noise points; Results: Finally, the data set is divided into multiple clusters and a set of noise points. Clusters are composed of points connected by density; 4) Data injection: data injection is used to supplement missing data or simulate log data in certain specific scenarios; 5) Log reasoning and training: Through log reasoning, the system performance can be deeply analyzed. The performance analysis indicators include response time, throughput, CPU utilization, and memory utilization. 6) Performance dashboard and interactive Q&A. Use Grafana to build a performance dashboard. The steps to build a performance dashboard include demand analysis, data source preparation, dashboard design, performance monitoring, user interaction, optimization and iteration. The steps of dashboard design are to create a new Dashboard in Grafana, design the layout of the dashboard, including the arrangement, size and color of the panels, create a corresponding panel for each performance indicator, and configure the data source, query statement and chart type.

[0007] Preferably, the useful information is extracted in 2) by parsing data using regular expressions, log analysis tools or custom scripts, and the log analysis tool adopts ELK or Splunk.

[0008] Preferably, the 3) model needs to be evaluated and hyperparameters tuned. During the training process, the model needs to be evaluated using a validation set to ensure the generalization ability of the model; the performance of the model is optimized by adjusting the hyperparameters of the model, which include learning rate, batch size, and number of iterations. Model selection is based on the specific needs of the problem and the characteristics of the data.

[0009] Preferably, the performance dashboard in 6) is: The performance indicators of demand analysis include response time, throughput, CPU usage, and memory usage; Data source preparation is to organize the results of log analysis into time series or JSON data formats supported by Grafana; data source configuration is to connect Grafana to the database or system that stores log analysis results, including Elasticsearch, Prometheus, and MySQL; Performance monitoring: Update the data in the Panel in real time to display the real-time operating status of the system; set alarms to notify relevant personnel in a timely manner when performance indicators exceed preset thresholds; User interaction: Use Grafana's variables and link functions to achieve cross-Panel or cross-Dashboard interaction.

[0010] Compared with the prior art, the present invention has the following beneficial effects: High degree of automation: Through the automated processing of log collection tools and large models, the automated collection and analysis of system log data is realized, which improves the efficiency and accuracy of performance analysis; Comprehensive and accurate analysis: The large model has deep learning and natural language processing capabilities, which can automatically identify and analyze key information in log data, providing comprehensive and accurate performance analysis basis for operation and maintenance personnel; Strong scalability: The method of the present invention can be applied to different types of systems and scenarios, and only the configuration of the log collection tool and the large model needs to be adjusted according to actual needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 The figure shows a flow chart of call scenario log collection performance of the present invention; Figure 2 Shown is a screenshot of log collection of the present invention; Figure 3 Shown is a screenshot of log processing of the present invention; Figure 4 Shown are screenshots of log reasoning and training of the present invention; Figure 5 Shown is a screenshot of the performance dashboard of the present invention. DETAILED DESCRIPTION

[0012] Example: Figure 1-5 As shown, a performance analysis method based on a large call scenario model and log collection includes the following steps: 1) Log Collection Log collection is the first step of log analysis and the basis of the entire process. At this stage, we need to collect log data from various call scenarios, including system logs, application logs, security logs, etc. These data will serve as the raw material for subsequent analysis and are crucial to ensuring the accuracy and effectiveness of the analysis. Figure 2 shown.

[0013] 2) Log processing and storage The collected log data needs to go through processing steps such as cleaning, filtering, and conversion to eliminate redundant information, standardize the data format, and extract valuable information. Redundant information can be eliminated by using data cleaning tools such as Pandas (Python library) and DataCleaner, writing scripts or programs, using database deduplication, indexing and view functions, and data standardization. To extract information, you can parse the data by using regular expressions, log analysis tools (such as ELK, Splunk, etc.) or custom scripts. The processed log data will be stored in a database or distributed storage system for subsequent quick query and analysis.

[0014] The special definition of call scenario log is as follows: Dial request log; Status push log; Call record push log; Recording download log.

[0015] The special definitions of call scenario business indicator logs are as follows: The total number of call records, success rate, and error rate; The total amount of recordings, success rate, and error rate; The total number of requests, success rate, and error rate.

[0016] The special definitions of call scenario business segmentation scenarios include the following: Call log indicators, failure error type, network, non-push, service abnormality status, other abnormalities, etc. Recording indicators, failed error types, network, file not generated, file type error, other exceptions, etc.

[0017] Log processing screenshots are as follows Figure 3 shown.

[0018] 3) Log model training In the log model training phase, the processed log data is trained using machine learning algorithms to build a model that can identify system anomalies and predict system performance trends. Through continuous iteration and optimization, the accuracy and generalization ability of the model can be improved, making it better able to adapt to various complex call scenarios. Among them, the machine learning algorithm uses K-means clustering and DBSCAN.

[0019] Prepare and select the algorithm based on the following aspects: Data preprocessing: Before training the model, log data needs to be cleaned, formatted, and feature extracted; Model evaluation: During the training process, the model needs to be evaluated using a validation set to ensure the generalization ability of the model; Hyperparameter tuning: Optimize the performance of the model by adjusting the model's hyperparameters (such as learning rate, batch size, number of iterations, etc.).

[0020] Model selection: Choose an appropriate machine learning algorithm based on the specific requirements of the problem and the characteristics of the data.

[0021] DBSCAN algorithm steps: 1. Initialization: Assume that there is a data set containing multiple data points. Set two key parameters: eps (neighborhood radius) and MinPts (minimum number of points in the neighborhood).

[0022] 2. Core point identification: traverse each point in the data set and calculate the number of points in its eps neighborhood. If the number of points in the eps neighborhood of a point is greater than or equal to MinPts, then mark the point as a core point.

[0023] 3. Clustering start: Randomly select a core point as the starting point and start a new clustering process. Add all points in the eps neighborhood of the core point (including other core points and non-core points) to the current cluster. Density reachable and density connected: Starting from the points that have been added to the cluster, find all density reachable points (that is, points connected by a series of core points). If there is a core point sequence between two points so that they can both be density-reachable through the sequence, then the two points are said to be density-connected. Add all density-connected points to the current cluster.

[0024] 4. Continue clustering: Repeat the above process until all core points have been visited and the clusters they belong to have been determined. The remaining points (i.e., points that are neither core points nor in the eps neighborhood of any core point) are marked as noise points.

[0025] 5. Results: Finally, the data set is divided into multiple clusters and a set of noise points. Clusters are composed of points connected by density and usually have arbitrary shapes (such as circles, ellipses, irregular shapes, etc.).

[0026] 4) Data injection In some cases, we may need to use data injection to supplement missing data or simulate log data in certain scenarios. This helps us understand the operation status of the system more comprehensively and discover potential performance issues.

[0027] 5) Log reasoning and training Through log reasoning, we can deeply analyze the system performance and find potential performance bottlenecks and failure points. Specific analysis methods include log collection and preprocessing, including cleaning, formatting, deduplication and other operations; log parsing, parsing log information into structured data, extracting key fields such as timestamp, operation type, error code, etc.; performance indicator analysis, such as response time, throughput, CPU utilization, memory utilization, etc., analyze the changing trends of these indicators, identify abnormal fluctuations and potential performance bottlenecks. At the same time, log training can also be used to further optimize the model to improve its prediction accuracy and response speed; analysis screenshots are shown below. Figure 4 shown.

[0028] 6) Build performance dashboards and Q&A Based on the results of log analysis, an intuitive performance dashboard can be built to display the real-time operating status and performance indicators of the system. In addition, through interactive Q&A, convenient performance query and problem-solving services can be provided to users to help them better understand the system performance status and optimization suggestions.

[0029] Using Grafana to build a performance dashboard to display the real-time operating status and performance indicators of the system is an efficient and commonly used method.

[0030] Here are the specific steps to build a performance dashboard based on the analysis results: Demand Analysis: Specify the performance indicators that need to be displayed, such as response time, throughput, CPU usage, memory usage, etc. Data source preparation: Organize the results of log analysis into data formats supported by Grafana, such as time series data, JSON, etc. Configure the data source to connect Grafana to the database or system that stores log analysis results, such as Elasticsearch, Prometheus, MySQL, etc.

[0031] Board design: Create a new Dashboard in Grafana.

[0032] Design the layout of the dashboard, including the arrangement, size, color, etc. of the panels.

[0033] Create a corresponding Panel for each performance indicator and configure the data source, query statement, chart type, etc.

[0034] Performance Monitoring: Update the data in the Panel in real time to display the real-time operating status of the system.

[0035] Set up alerts to notify relevant personnel in a timely manner when performance indicators exceed preset thresholds.

[0036] User Interaction: Through Grafana's variables and links, cross-panel or cross-dashboard interactions are achieved. Users are allowed to query performance data through interactive question-and-answer methods, which usually requires the combination of Grafana's Explore function or external systems.

[0037] Optimization and iteration: Continuously optimize the content and layout of the dashboard based on user feedback and actual needs.

[0038] Update data sources and query statements regularly to ensure that the data displayed on the dashboard is up to date.

[0039] Through the above steps, you can build an intuitive performance dashboard based on the results of log analysis. The performance dashboard screenshot is as follows: Figure 5 As shown in the figure, Grafana is used to display the real-time operation status and performance indicators of the system. At the same time, through interactive Q&A and performance monitoring functions, convenient performance query and problem solving services are provided to users.

Claims

1. A performance analysis method based on a large call scenario model and log collection, characterized in that: The steps include: 1) Log collection: collects log data from various call scenarios, including system logs, application logs, and security logs; 2) Log processing and storage: The collected log data is cleaned, filtered and converted using Pandas or DataCleaner to eliminate redundant information, standardize the data format, and extract valuable information. The processed data is stored in a database or distributed storage system. 3) Log model training, use K-means clustering or DBSCAN algorithm to train the processed log data: DBSCAN algorithm steps: Initialization: Assume that there is a data set containing multiple data points. Set two key parameters: eps and MinPts; Core point identification: traverse each point in the data set and calculate the number of points in its EPS neighborhood; If the number of points in the eps neighborhood of a point is greater than or equal to MinPts, the point is marked as a core point; Clustering start: randomly select a core point as the starting point to start a new clustering process; Add all points in the eps neighborhood of the core point to the current cluster; density reachability and density connection: starting from the points that have been added to the cluster, find all density reachable points; if there is a core point sequence between two points, so that they can be density-reachable through the sequence, then the two points are said to be density-connected, and all density-connected points are added to the current cluster; Continue clustering: Repeat the above process until all core points have been visited and the clusters they belong to have been determined, and the remaining points are marked as noise points; Results: Finally, the data set is divided into multiple clusters and a set of noise points. Clusters are composed of points connected by density; 4) Data injection: data injection is used to supplement missing data or simulate log data in certain specific scenarios; 5) Log reasoning and training: Through log reasoning, the system performance can be deeply analyzed. The performance analysis indicators include response time, throughput, CPU utilization, and memory utilization. 6) Performance dashboard and interactive Q&A. Use Grafana to build a performance dashboard. The steps to build a performance dashboard include demand analysis, data source preparation, dashboard design, performance monitoring, user interaction, optimization and iteration. The steps of dashboard design are to create a new Dashboard in Grafana, design the layout of the dashboard, including the arrangement, size and color of the panels, create a corresponding panel for each performance indicator, and configure the data source, query statement and chart type.

2. A performance analysis method based on a large call scenario model and log collection according to claim 1, characterized in that: Useful information is extracted from the data in 2) by using regular expressions, log analysis tools, or custom scripts to parse the data. The log analysis tool uses ELK or Splunk.

3. A performance analysis method based on a call scenario large model and log collection according to claim 1, characterized in that: The model mentioned in 3) needs to be evaluated and hyperparameters tuned. During the training process, the model needs to be evaluated using a validation set to ensure the generalization ability of the model. The performance of the model can be optimized by adjusting the hyperparameters of the model. The hyperparameters include learning rate, batch size, and number of iterations. Model selection is based on the specific needs of the problem and the characteristics of the data.

4. A performance analysis method based on a call scenario large model and log collection according to claim 1, characterized in that: The performance dashboard in 6) is as follows: The performance indicators of demand analysis include response time, throughput, CPU usage, and memory usage; Data source preparation is to organize the results of log analysis into time series or JSON data formats supported by Grafana; Configuring the data source is to connect Grafana to the database or system that stores log analysis results, including Elasticsearch, Prometheus, and MySQL; Performance monitoring: Update the data in the Panel in real time to display the real-time operating status of the system; Set up alerts to notify relevant personnel when performance indicators exceed preset thresholds; User interaction: Use Grafana's variables and link functions to achieve cross-Panel or cross-Dashboard interaction.

Images