Full-link front-end performance monitoring method based on multi-scene intelligent analysis

Abstract

The invention provides a full-link front-end performance monitoring method based on multi-scene intelligent analysis, relates to the field of front-end performance monitoring, and aims to improve real-time monitoring, data analysis depth and user experience through multi-scene intelligent analysis. The Performance API is used for collecting key indexes such as user behaviors and response time, and the Apache Kafka is used for realizing millisecond-level data transmission; the TensorFlow.js is used for front-end data cleaning and preprocessing work, the processed data is sent to a rear end, TensorFlow or PyTorch is used for deep learning model training, and a front-end performance data mode is learned; cNN feature extraction and LSTM sequence data analysis are combined, so that the prediction accuracy of the potential performance problem is improved. Comprehensive monitoring from a client side to a server side is achieved, and CI / CD processes, automatic testing and rapid iteration updating in an agile development environment are supported; an AI algorithm is introduced to analyze a performance test result, a report is automatically generated, optimization suggestions are provided, user feedback is integrated, an NLP technology is used to identify pain points, a targeted optimization task is triggered, and user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of front-end performance monitoring, real-time data stream processing, and full-link monitoring technology, and particularly to a full-link front-end performance monitoring method based on multi-scenario intelligent analysis. Background Art

[0002] With the rapid development of Internet technology, front-end applications are becoming more and more complex, and the requirements for user experience are becoming higher and higher. As one of the key technologies to ensure the smooth operation of front-end applications, front-end performance monitoring has evolved from simple page loading time monitoring to all-round monitoring including user experience, which helps improve user experience and helps developers quickly locate and solve problems.

[0003] In this process, some key technical terms frequently appear, and their definitions and applications are essential to understanding the technology in this field: 1. Convolutional Neural Network (CNN): A deep learning model for feature extraction, but in this paper, we will use CNN to extract feature patterns in front-end performance monitoring to better understand and predict performance bottlenecks; 2. Long Short-Term Memory (LSTM): A recurrent neural network used to process sequence data. 3. Apache Kafka - message queue system for real-time data stream processing; Apache Flink - a framework for low-latency streaming data processing. 5. Google Lighthouse: An open source automated tool for improving the quality of web pages. 6. WebPageTest: used to test the performance and loading speed of the website; 7. New Relic - Comprehensive APM (application performance management) solution. 8. Dynatrace: provides front-end performance monitoring capabilities; 9. Pingdom: A website monitoring tool that monitors whether the website is online and the loading speed; 10. Performance API: provides a rich interface to obtain performance indicators such as front-end page loading time, resource loading size, rendering time, etc.

[0004] The current status of front-end performance monitoring of intelligent analysis presents the characteristics of automation, intelligence, real-time monitoring and multi-source data fusion, but it still has the following shortcomings: 1. Lack of real-time performance: Google Lighthouse provides performance scores and optimization suggestions, but it is mainly aimed at static page analysis and lacks support for real-time performance monitoring. WebPageTest provides detailed performance reports, but it is also not suitable for real-time monitoring because it requires manual triggering of tests; 2. Limited depth of data analysis: New Relic can monitor both front-end and back-end performance, but it mainly relies on basic indicators such as page loading time and error rate for front-end performance monitoring, and has limited in-depth analysis capabilities for user experience. Dynatrace mainly focuses on basic performance indicators, and does not analyze fine-grained data such as user interaction and DOM rendering in depth; 3. Insufficient consideration of user experience: Pingdom monitors whether the website is online and the loading speed, but it mainly focuses on page loading time and lacks sufficient attention to other aspects of user experience, such as interactive response time and visual integrity.

[0005] Therefore, a more comprehensive, real-time and in-depth full-link front-end performance monitoring method with multi-scenario intelligent analysis is needed. This new technology should be able to improve the shortcomings of existing technologies and significantly improve the performance and user experience of front-end applications. By introducing advanced data processing technology, deep learning algorithms and artificial intelligence technology, more accurate anomaly detection, deeper performance analysis and more comprehensive user experience optimization can be achieved, thereby helping developers quickly locate and solve problems and improve user satisfaction. Summary of the invention

[0006] In view of the shortcomings of the existing technology, the present invention introduces advanced artificial intelligence technology to provide an innovative full-link front-end performance monitoring method based on multi-scenario intelligent analysis, aiming to overcome the limitations of the existing technology and achieve more comprehensive, real-time and accurate performance monitoring. This method not only improves the intelligence level and accuracy of monitoring, but also realizes in-depth analysis and anomaly detection of front-end performance through the application of deep learning and machine learning technology, thereby providing developers with more effective optimization suggestions and significantly improving user experience and application performance.

[0007] Specifically, the present invention aims to use advanced data processing technology to quickly collect and analyze performance data, achieve instant performance insights, and use deep learning algorithms to conduct in-depth analysis from user interface to server response to accurately locate performance bottlenecks; by integrating machine learning and deep learning technologies, it can automatically adjust monitoring strategies and optimization suggestions according to different usage scenarios and user behavior patterns; by integrating front-end and back-end monitoring data, end-to-end performance monitoring; and finally, combining strategies such as rapid trial and error and adjustment, continuous integration and deployment, iteration planning and priority setting to achieve agile iteration to improve development efficiency and application performance.

[0008] To achieve the above objectives, a full-link front-end performance monitoring method based on multi-scenario intelligent analysis is implemented through the following technical solution, including the following steps: S1: Real-time performance monitoring, through efficient data collection and processing technology, to achieve instant performance insights. First, integrate efficient real-time monitoring technology to instantly capture key performance indicators such as user behavior and application response time to ensure the real-time and accuracy of data. Combined with deep analysis capabilities, it mainly observes traditional performance indicators such as page loading time and resource loading time, and deeply analyzes more granular data such as user interaction, DOM rendering, resource download, etc., to provide a more comprehensive perspective for performance analysis. Quickly identify and resolve performance bottlenecks, effectively avoiding misjudgments caused by data delays and shallow analysis; S2: Multi-scenario intelligent analysis, using machine learning and deep learning technology to automatically adjust monitoring strategies according to different application scenarios. Through intelligent algorithms, it deeply analyzes performance data and can autonomously identify performance bottlenecks and potential problems. For different types of application scenarios, such as mobile terminals, web applications, and applets, it provides personalized monitoring solutions and optimization guidance to ensure the accuracy and pertinence of monitoring work; S3: End-to-end performance monitoring, full-link performance monitoring from front-end to back-end, to ensure consistent user experience. By extending from monitoring front-end performance to back-end services and network layers, full-link performance monitoring from user end to server end is achieved to ensure that the application can provide a consistent user experience in all links. Based on the processing of front-end and back-end monitoring data, a complete performance monitoring system is formed to provide more comprehensive performance analysis and optimization suggestions; S4: Agile iteration and rapid feedback, combined with rapid trial and error and adjustment, continuous integration and deployment, iteration planning and priority setting, support the development team to efficiently solve performance problems and accelerate product iteration. Through cross-departmental collaboration and user feedback collection and analysis, real-time acquisition of user needs and market changes, timely adjustment of optimization strategies, and further improvement of the overall performance of the product and user satisfaction.

[0009] Preferably, in said S1, it also includes: Set different collection frequencies to ensure that performance fluctuations can be captured in a timely manner without causing excessive impact on front-end performance. Use big data processing technology and algorithms to quickly analyze the collected real-time data and extract key performance indicators (KPIs) and abnormal data. Use correlation analysis technology to associate data from different sources and dimensions to reveal their inherent connections and mutual influences, helping to deeply understand the root causes of performance problems. Store the collected performance data in the database for historical data backtracking and trend analysis. By analyzing historical data, predict the changing trend of front-end performance in the future, and provide a basis for future performance optimization and capacity planning.

[0010] Preferably, in said S2, further comprising: Use convolutional neural networks (CNN), long short-term memory networks (LSTM), etc. to achieve more accurate anomaly detection and optimization suggestions. By building a CNN model, accepting data of the shape of (number of samples, time steps, number of features), using convolution kernels of different sizes to extract features, and using the ReLU activation function; using maximum pooling to reduce feature dimensions; flattening the output of the convolution layer, and using a fully connected layer for classification or regression analysis; using sigmoid or softmax functions for anomaly detection classification. Select a suitable loss function (cross entropy loss) and optimizer (Adam), train CNN with training set data, use the validation set for model selection and hyperparameter adjustment, and use early stopping to prevent overfitting. Evaluate the performance of the model through the test set, use indicators such as accuracy, recall, and F1 score, deploy the trained model to the production environment, monitor front-end performance data in real time, and automatically detect anomalies. These models can extract features from massive data and identify potential performance problem patterns; Identify different usage scenarios (mobile, web applications, mini programs) through machine learning models. For mobile, pay special attention to network latency and power consumption, and collect performance data by integrating mobile device-specific monitoring tools: Xcode Instruments and Android Profiler. Use machine learning models to analyze user behavior data, identify performance bottlenecks under different network conditions, reduce application response time, improve battery efficiency, and thus improve user satisfaction. For the WEB side, consider factors such as browser compatibility and CDN acceleration, and collect performance data by integrating web performance monitoring tools: Google PageSpeed ​​Insights and Web Page Test. Use deep learning models to analyze resource loading time during page loading, optimize resource loading strategies, improve page loading speed, reduce server response time, and enhance user experience. In view of the characteristics of mini programs, integrate mini program performance monitoring APIs, collect page loading time and user interaction data, apply machine learning algorithms to analyze user behavior patterns, and automatically adjust mini program performance optimization strategies. Apply machine learning algorithms to analyze user behavior patterns and automatically adjust mini program performance optimization strategies.

[0011] Preferably, in said S3, further comprising: In the end-to-end, network performance evaluation tools are integrated to monitor network latency, bandwidth usage, packet loss and other network-level performance indicators, identify the impact of network problems on front-end performance, analyze the resource dependencies of front-end applications, such as CDN resource loading, third-party service calls, etc., evaluate the impact of these dependencies on front-end performance, and provide optimization suggestions. For applications using containerization and microservice architecture, monitor the performance status of each microservice and the resource usage of the container to optimize service performance. Establish a performance impact evaluation model to quantify the impact of front-end changes on back-end services, provide a basis for front-end and back-end collaborative optimization, improve the efficiency of problem solving, and reduce the time to locate and solve problems.

[0012] Preferably, in said S4, further comprising: In the agile iteration and rapid feedback loop, through the rapid iteration development process, new versions are quickly deployed and feedback is collected immediately. If performance problems are found, the code can be quickly rolled back or adjusted to shorten the time to solve the problem. Through the continuous integration and continuous deployment process, ensure that each change can be automatically tested; then deployed to the production environment. Use AI algorithms to analyze performance test results, automatically generate test reports, and provide data-driven performance optimization suggestions. Perform performance regression analysis after each iteration to ensure that new code changes do not introduce performance problems and maintain the stability of application performance. Integrate user feedback into the performance monitoring system, use natural language processing technology to analyze user comments, etc., to identify performance problems and user pain points. Improve the response speed to performance issues and enhance the flexibility and adaptability of the entire development process.

[0013] Compared with the prior art, the present invention discloses a full-link front-end performance monitoring method based on multi-scenario intelligent analysis. The present invention has the following beneficial effects: 1. Through instant insight and deep analysis, it solves the problems of insufficient real-time performance, limited data analysis depth, unfriendly user interface, lack of prediction ability and insufficient user behavior analysis in existing front-end performance monitoring methods. By integrating Performance API and real-time data stream processing tools (Apache Kafka), the present invention can achieve millisecond-level monitoring of front-end performance, capture performance fluctuations in real time, and monitor page loading time 50% faster than traditional tools; 2. AI-driven multi-scenario precise monitoring solves the problems of traditional front-end performance monitoring methods being unable to adapt to changing usage scenarios, fixed performance thresholds, lack of flexibility in resource management, reliance on manual operations for anomaly detection, and lack of personalization in user experience optimization. By using deep learning algorithms, especially CNN and LSTM, the present invention can extract deep features from massive performance data, identify potential performance bottlenecks and abnormal patterns, and improve the accuracy of identifying performance bottlenecks by 30% compared with traditional methods. 3. End-to-end monitoring solves the problems existing in existing front-end performance monitoring methods, such as limited monitoring scope, separation of front-end and back-end performance data, insufficient identification of network and resource dependency issues, disconnection between security and performance monitoring, and difficulty in locating the root cause of problems, thus expanding the comprehensive monitoring scope of the entire system performance; 4. Through agile iteration and rapid feedback loop, we solved the problems of performance monitoring being out of touch with the development process, slow response to problems, lack of data support for optimization work, low efficiency of team collaboration, and difficulty in inheriting experience. We integrated performance monitoring tightly into the agile development process and used AI technology to improve the automation level of analysis and optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments; Figure 1 It is a flow chart of the instant insight and in-depth analysis of data of the present invention; Figure 2 It is a flow chart of AI-driven multi-scenario precise monitoring of the present invention; Figure 3 It is a flowchart of the specific implementation of the end-to-end monitoring of agile iteration and rapid feedback loop of the present invention. DETAILED DESCRIPTION

[0015] Step 1: Instant Insight and In-depth Analysis S11 integrates efficient real-time monitoring technology, which can instantly capture key performance indicators such as user behavior and application response time to ensure the real-time and accuracy of data. Use the Performance API provided by the browser to collect data. When the front-end page is loaded, introduce the performance object through JavaScript code to access the performance indicators provided by the browser; create a new PerformanceObserver instance, listen to specific types of performance entries (navigation, resource, etc.), and define the callback function to be executed when new performance entries are available. Use performance.getEntries() or performance. getEntriesByType() to obtain all recorded performance entries or entries of a specific type; S12 focuses on traditional performance indicators such as page loading time and resource loading time, and deeply analyzes fine-grained data such as user interaction, DOM rendering, and resource downloading, providing a more comprehensive perspective for performance analysis, quickly identifying and solving performance bottlenecks, and effectively avoiding misjudgments caused by data delays and shallow analysis; S13 uses correlation analysis technology to correlate data from different sources and dimensions, revealing the inherent connections and mutual influences between them, and helping to deeply understand the root causes of performance problems. Configure a Kafka Producer and connect it to the front-end page to read data from the Performance API and send it to the Kafka cluster. Configure a Kafka Consumer to listen to messages in the Kafka cluster in the back-end server and save them to the database or for further processing; S14 stores the collected performance data in the database for historical data backtracking and trend analysis. By analyzing historical data, the changing trend of front-end performance in the future can be predicted, providing a basis for future performance optimization and capacity planning.

[0016] Step 2: AI-driven multi-scenario precision monitoring S21 uses advanced machine learning and deep learning technologies to accurately monitor and detect anomalies in front-end performance. By using the TensorFlow.js framework, you can write custom data cleaning and preprocessing logic to remove outliers, fill in missing values, and standardize data. The cleaned data is transferred to the back-end server and model training is performed using deep learning frameworks such as TensorFlow or PyTorch. Learn the rules and patterns in the front-end performance data and provide customized monitoring strategies and optimization suggestions; S22 uses the advantages of the combination of convolutional neural networks (CNN) and long short-term memory networks (LSTM) to improve the accuracy of time series data prediction. Select the appropriate number of CNN layers according to data complexity and computing resources, set the size, number and step size of the convolution kernel to extract local features of time series data, and select activation functions such as ReLU and Sigmoid to increase nonlinearity. Select the number of LSTM layers according to the dependency length and computing resources of the time series, set the number of LSTM units, which affects the memory capacity and training speed of the model, and adjust the parameters of the forget gate to control the degree of retention of historical information. Use the output of CNN as the input of LSTM, or transition through a fully connected layer (Dense), ensure that the input data meets the input requirements of CNN and LSTM, select the appropriate optimizer (Adam), loss function (MSE) and number of training rounds, and train the model; S23 identifies abnormal behaviors by training models, automatically adjusts the thresholds and rules for anomaly detection according to different scenarios and contexts, and analyzes user behavior data to understand users' usage habits and preferences in different scenarios.

[0017] Step 3: End-to-end monitoring S31 monitors the front-end performance and extends it to the back-end service and network layers to achieve full-link performance monitoring from the user end to the server end, ensuring that the application can provide a consistent user experience in all links; S32 uses the network performance evaluation tool Wireshark to analyze network-level performance indicators such as network latency, bandwidth usage, and packet loss, and uses MTR and traceroute tools to analyze packet transmission paths and identify problematic nodes in the network. Use tools such as Wireshark and Tcpdump to capture and analyze network traffic, understand network usage and traffic patterns, and use professional monitoring tools such as PRTG Network Monitor and SolarWinds to monitor network traffic in real time and detect abnormal traffic or problems in a timely manner; S33 analyzes the resource dependencies of front-end applications, CDN resource loading, third-party service calls, etc., evaluates the impact of these dependencies on front-end performance, and provides optimization suggestions; S34 uses containerized and microservice architecture applications to monitor the performance status of each microservice and the resource usage of the container to help developers optimize service performance.

[0018] Step 4: Agile Iteration and Rapid Feedback Loop S41 uses a fast iterative development process to quickly deploy new versions and collect feedback immediately. When performance issues are discovered, the code can be quickly rolled back or adjusted to shorten the time to solve the problem. S42 ensures that every change can be automatically tested and deployed to the production environment through continuous integration and continuous deployment processes. Using distributed version control systems such as Git to manage code bases, all developers work on the same main branch and create feature branches to develop new features or fix problems. When the code is submitted, the automated build process is triggered, which includes compiling the code, running static code analysis tools, and building front-end resources. Unit testing, integration testing, and end-to-end testing are used to ensure that new code changes do not break existing functions. Test results are fed back to developers in real time, and any failed tests will block subsequent deployment processes, forcing developers to fix problems. Once the code passes all automated tests and quality checks, it will be automatically deployed to the pre-production or production environment. Use deployment tools (Jenkins, GitLab CI / CD) to automate the deployment process. Perform performance testing in the pre-production environment to ensure that the newly deployed code does not degrade application performance. Use Blue-Green Deployment or Rolling Update to minimize downtime and ensure that users can always access the latest stable version until the new version is confirmed to be correct before switching completely; S43 uses AI algorithms to analyze performance test results, automatically generates test reports, and provides data-driven performance optimization suggestions. It performs performance regression analysis after each iteration to ensure that new code changes do not introduce performance issues and maintain the stability of application performance. S44 integrates user feedback into the performance monitoring system and uses natural language processing technology to analyze user comments and support tickets to identify performance issues and user pain points. Remove useless information such as HTML tags, special characters, and stop words, and restore vocabulary to basic form for easy processing. Use sentiment analysis algorithms to identify emotional tendencies in user feedback and distinguish between positive, negative, or neutral feedback. Use algorithms such as LDA to identify common topics and concerns in user feedback, and classify large amounts of feedback into different topics to identify hot issues discussed by users. Use NER technology to identify and extract specific entities in text, and automatically classify user feedback into predefined categories. Analyze the time series data of user feedback and generate a summary of user feedback, highlighting the most important issues and areas of greatest concern to users. Based on the results of the NLP analysis, automatically generate action suggestions or trigger performance optimization tasks.

Claims

1. A full-link front-end performance monitoring method based on multi-scenario intelligent analysis, characterized in that: The method comprises the following steps: S1: Integrate efficient real-time monitoring technology to instantly capture key performance indicators such as user behavior and application response time through tools such as Performance API; S2: Use machine learning and deep learning technologies to automatically adjust monitoring strategies according to different application scenarios; S3: Full-link performance monitoring from frontend to backend to ensure consistent user experience; S4: Combine strategies such as rapid trial and error, adjustment, and continuous integration and deployment to support development teams in efficiently solving performance problems.

2. The method according to claim 1, characterized in that The S1 further comprises: Set different acquisition frequencies to ensure that performance fluctuations can be captured in a timely manner without causing excessive impact on front-end performance; Use the Performance API for data collection, including importing the performance object, creating a PerformanceObserver instance, listening for specific types of performance items (navigation, resource, etc.), and defining callback functions to be executed when new performance items are available; Use big data processing technology and correlation analysis technology to quickly analyze real-time data and extract key performance indicators (KPIs) and abnormal data; Correlate data from different sources and dimensions to reveal their inherent connections and mutual influences; The collected performance data is stored in the database to facilitate historical data backtracking and trend analysis, and to predict the changing trend of front-end performance in the future.

3. The method according to claim 1, characterized in that The S2 further includes: Use convolutional neural networks (CNN), a deep learning model for feature extraction; and long short-term memory networks (LSTM), a recurrent neural network that excels at processing sequence data; By building a CNN model, accepting data of shape (number of samples, time steps, number of features), using convolution kernels of different sizes to extract features, and using the ReLU activation function; using max pooling to reduce feature dimensions; flattening the output of the convolution layer and using a fully connected layer for classification or regression analysis; using sigmoid or softmax functions for classification of anomaly detection; Select the appropriate loss function (cross entropy loss) and optimizer (Adam), train CNN with the training set data, select the model and adjust the hyperparameters with the validation set, and use the early stopping method to prevent overfitting; For mobile devices, we pay special attention to network latency and power consumption, and collect performance data by integrating mobile device-specific monitoring tools such as Xcode Instruments and Android Profiler; For web applications, consider factors such as browser compatibility and CDN acceleration, and collect performance data by integrating tools such as Google PageSpeedInsights and WebPageTest; In view of the characteristics of mini programs, the mini program performance monitoring API is integrated to collect page loading time and user interaction data, and machine learning algorithms are applied to analyze user behavior patterns to automatically adjust the mini program's performance optimization strategy.

4. The method according to claim 1, characterized in that: The S3 further includes: Integrate network performance evaluation tools such as Wireshark, MTR, and traceroute to analyze network-level performance indicators and identify problematic nodes in the network; Analyze resource dependencies of front-end applications, such as CDN resource loading, third-party service calls, etc., evaluate the impact of these dependencies on front-end performance, and provide optimization suggestions; For applications using containerization and microservice architecture, monitor the performance status of each microservice and its resource usage to help developers optimize service performance; Establish a performance impact assessment model to quantify the impact of front-end changes on back-end services, and provide a basis for front-end and back-end collaborative optimization.

5. The method according to claim 1, characterized in that The S4 further comprises: Through the rapid iterative development process, new versions are quickly deployed and feedback is collected immediately. If performance problems are found, the code is quickly rolled back or adjusted to shorten the time to solve the problem. Ensure that every change can be automatically tested and deployed to the production environment through continuous integration and continuous deployment (CI / CD) processes; Leverage AI algorithms - specifically, use the TensorFlow.js framework to write custom data cleaning and preprocessing logic, combine deep learning models to perform data analysis, generate test reports, and provide data-driven performance optimization recommendations; Perform performance regression analysis after each iteration to ensure that new code changes do not introduce performance issues and maintain the stability of application performance; Use natural language processing (NLP) technology - involving text preprocessing (removing HTML tags, special characters and stop words, vocabulary restoration), sentiment analysis (identifying emotional tendencies), topic modeling (such as LDA algorithm to identify hot issues) and entity recognition (NER technology to identify specific entities) to analyze user comments and support tickets, and automatically generate action suggestions or trigger performance optimization tasks based on the analysis results.

6. A computer system for executing the method according to any one of claims 1 to 5, characterized in that include; Data collection module, used to obtain front-end performance data in real time, including Performance API and KafkaProducer components; Data processing module, used to quickly analyze and process the collected data, including big data processing technology and correlation analysis technology; Intelligent analysis module, including machine learning and deep learning algorithms, including CNN model building submodule, LSTM model building submodule, anomaly detection submodule and personalized optimization submodule; Network performance evaluation module, which is used to monitor network-level performance indicators, including tools such as Wireshark, MTR, and traceroute; CI / CD module, which is used for automated testing and deployment, including tools such as Jenkins and GitLab CI / CD; User feedback analysis module, used to analyze user feedback and generate performance optimization suggestions, specifically including text preprocessing submodule, sentiment analysis submodule, topic identification submodule and entity recognition submodule; The performance prediction module is used to predict the changing trend of front-end performance in the future, including historical data analysis and time series prediction models.

Images