Webpage performance monitoring method and system based on Java language and cdp4j open source component

Abstract

The invention relates to the technical field of cloud monitoring, in particular to a webpage performance monitoring method and system based on a Java language and a cdp4j open source component, and the method comprises the following steps: building an infrastructure; the monitoring function is enhanced; a collection end is configured in detail; the functions of the server are perfected; technical details are realized; the webpage performance monitoring method and system based on the Java language and the cdp4j open source component have the beneficial effects that key indexes such as webpage loading duration and element file size can be accurately obtained based on the Java language and the cdp4j open source component, and through multi-micro-service design, the webpage performance monitoring method and system based on the Java language and the cdp4j open source component can be used for monitoring the webpage performance. Webpage request response index acquisition and data analysis and storage are divided into two independent micro-service processing, so that the business coupling degree is effectively reduced, and the maintainability and expandability of the system are improved. And meanwhile, the real-time performance and the accuracy of data are ensured by synchronously deploying information through a heartbeat mechanism.

Description

Technical Field

[0001] The present invention relates to the field of cloud monitoring technology, and in particular to a web page performance monitoring method and system based on Java language and cdp4j open source components. Background Art

[0002] With the rapid development of Internet technology, the importance of web page performance to user experience has become increasingly prominent. Performance indicators such as web page loading speed and response time directly affect user satisfaction and loyalty. Therefore, developers need to continuously monitor web page performance in order to discover and resolve potential problems.

[0003] However, existing web page monitoring tools have certain limitations. For example, although tools such as Google PageSpeedInsights and WebPageTest can provide web page performance evaluation, most tools can only provide regular monitoring reports and cannot monitor the loading process and performance changes of web pages in real time. Moreover, these tools usually only provide limited performance indicators and cannot fully reflect the performance status of web pages.

[0004] In addition, most existing web monitoring tools rely on third-party services, which may have data security and privacy protection issues. Moreover, the monitoring results of these tools may be affected by factors such as network environment and geographical location, resulting in inaccurate and unreliable monitoring results. Summary of the invention

[0005] The purpose of the present invention is to provide a web page performance monitoring method and system based on Java language and cdp4j open source components to solve the problems raised in the above-mentioned background technology. By monitoring performance indicators, such as page rendering time, server response time, etc., bottlenecks can be identified and optimized, thereby improving user satisfaction; when the web page loading time is abnormal or the performance is degraded, the system can issue an alarm in time to notify the operation and maintenance personnel to troubleshoot the problem and prevent users from encountering access barriers. In addition, the purpose of the present invention also includes providing a web page monitoring system that is efficient, compatible and scalable. Monitoring indicators can be expanded according to needs, such as whether to disable the cache, whether to ignore certificate errors and other indicators, and page loading performance tests can also be performed across different geographical locations and network conditions to ensure that our web pages provide a consistent experience for different users.

[0006] To achieve the above object, the present invention provides the following technical solution: a web page performance monitoring method based on Java language and cdp4j open source component, the method comprising the following steps:

[0007] Building infrastructure;

[0008] Enhanced monitoring capabilities;

[0009] Configure the acquisition end in detail;

[0010] Improve server functions;

[0011] Implement technical details, use Java language to implement all logic on the server and collection side, and use cdp4j open source components to interact with Google Chrome to obtain web page performance monitoring data.

[0012] Preferably, the specific operations of building the infrastructure include:

[0013] Set up a collection server, install Google Chrome and run the cdp4j component;

[0014] Deploy multiple collection terminals to perform monitoring tasks and collect monitoring indicator data;

[0015] Set up a server to manage collection nodes, issue monitoring tasks, collect and process data, and display it to customers in a visual form;

[0016] Implement the heartbeat mechanism to synchronously collect node deployment information and coordinate task delivery with the server.

[0017] Preferably, the specific operations of enhancing the monitoring function include:

[0018] Introduce monitoring strategies and performance evaluation methods to comprehensively monitor web page loading events and obtain loading indicators for each element;

[0019] Establish an alarm mechanism to issue an alarm and notify operation and maintenance personnel when the web page loading time is abnormal or the performance is degraded.

[0020] Preferably, the specific operations of configuring the acquisition end in detail include:

[0021] Use the curl function to check the response status code of the webpage address and decide whether to continue the subsequent steps based on the status code;

[0022] Launch Google Chrome in headless mode;

[0023] Depending on the server configuration, set whether to disable caching and ignore certificate errors;

[0024] Monitor web page loading events and record key loading indicators;

[0025] Automatically take a screenshot of the web page when the page load event times out.

[0026] Preferably, the specific operations for improving the server functions include:

[0027] Obtain the information of the collection node and issue the dial test collection task as needed;

[0028] Collect data from various acquisition terminals, summarize, process and analyze them;

[0029] Automatically generate and send alarm information according to the alarm configuration.

[0030] A web page performance monitoring system based on Java language and cdp4j open source components, the system includes the following modules:

[0031] Collection server, used to install Google Chrome and run cdp4j components;

[0032] Multiple collection terminals are used to regularly obtain monitoring tasks from the server and obtain monitoring indicator data based on the tasks;

[0033] The server is used to manage the collection nodes, issue the dial test collection tasks, collect data, summarize, process and analyze the data, and display it to customers in a visual form;

[0034] The heartbeat mechanism is used to synchronize the deployment information of the collection nodes and issue collection tasks to the server.

[0035] Preferably, it also includes: a monitoring strategy and a performance evaluation method for comprehensively monitoring web page loading events and obtaining loading indicators for each element.

[0036] Preferably, the collection end also includes: using the curl function to check the response status code of the web page address; deciding whether to continue to execute subsequent steps based on the response status code; starting Google Chrome in headless mode; setting whether to disable caching and ignore certificate errors based on server configuration settings; monitoring web page loading events and recording key loading indicators; and taking screenshots when the page loading event times out.

[0037] Preferably, the server also includes: obtaining collection node information and issuing dial test collection tasks; collecting data from various collection terminals, summarizing, processing and analyzing the data; and generating alarm information according to the alarm configuration.

[0038] Preferably, it also includes: an alarm mechanism for issuing an alarm when the web page loading time is abnormal or the performance is degraded, and notifying the operation and maintenance personnel to troubleshoot the problem.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The web page performance monitoring method and system based on Java language and cdp4j open source components proposed in the present invention can accurately obtain key indicators such as web page loading time and element file size based on Java language and cdp4j open source components. Through multi-microservice design, web page request response indicator collection and data analysis and storage are divided into two separate microservices for processing, which effectively reduces the business coupling degree and improves the maintainability and scalability of the system. At the same time, the deployment information is synchronized through the heartbeat mechanism to ensure the real-time and accuracy of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of the method of the present invention;

[0042] Figure 2 It is a business flow chart of the system of the present invention;

[0043] Figure 3 This is a flow chart of the collection end of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose and technical solution of the present invention clearly and completely described, and the advantages more clearly understood, the embodiments of the present invention are further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, rather than all of the embodiments, and are only used to explain the embodiments of the present invention, and are not used to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] For example, see Figures 1 to 3 The present invention provides a technical solution: a web page performance monitoring method based on Java language and cdp4j open source components, the method comprising the following steps:

[0046] Build the infrastructure; specific operations include:

[0047] Set up a collection server, install Google Chrome and run the cdp4j component;

[0048] Deploy multiple collection terminals to perform monitoring tasks and collect monitoring indicator data;

[0049] Set up a server to manage collection nodes, issue monitoring tasks, collect and process data, and display it to customers in a visual form;

[0050] Implement the heartbeat mechanism to synchronously collect node deployment information and coordinate task delivery with the server.

[0051] Enhanced monitoring capabilities; specific operations include:

[0052] Introduce monitoring strategies and performance evaluation methods to comprehensively monitor web page loading events and obtain loading indicators for each element;

[0053] Establish an alarm mechanism to issue an alarm and notify operation and maintenance personnel when the web page loading time is abnormal or the performance is degraded.

[0054] Configure the acquisition end in detail; specific operations include:

[0055] Use the curl function to check the response status code of the webpage address and decide whether to continue the subsequent steps based on the status code;

[0056] Launch Google Chrome in headless mode;

[0057] Depending on the server configuration, set whether to disable caching and ignore certificate errors;

[0058] Monitor web page loading events and record key loading indicators;

[0059] Automatically take a screenshot of the web page when the page load event times out.

[0060] Improve the server functions; specific operations include:

[0061] Obtain the information of the collection node and issue the dial test collection task as needed;

[0062] Collect data from various acquisition terminals, summarize, process and analyze them;

[0063] Automatically generate and send alarm information according to the alarm configuration.

[0064] Implement technical details, use Java language to implement all logic on the server and collection side, and use cdp4j open source components to interact with Google Chrome to obtain web page performance monitoring data.

[0065] Embodiment 2: Based on Embodiment 1, in order to improve the accuracy and efficiency of the web page performance monitoring system, we adopt the following technical solutions:

[0066] Java language and cdp4j component: The core of the system is developed in Java because Java is cross-platform, stable and mature. cdp4j is a lightweight Java client that communicates with the Chrome DevTools Protocol (CDP), allowing us to deeply obtain browser performance data and network request information.

[0067] Multi-microservice design: The system architecture adopts microservice design, which divides web page request response indicator collection and data analysis and storage into two independent services. The advantage of this is that it reduces business coupling, improves the maintainability and scalability of the system, and also allows each service to focus on its own functions and improve efficiency.

[0068] Heartbeat mechanism: Through the heartbeat mechanism, the collection node regularly sends signals to the server and reports its deployment information to the server. The server manages and maintains the collection nodes based on this information and issues dial-up collection tasks to them. This mechanism ensures the real-time and accuracy of the data.

[0069] Monitoring strategies and performance evaluation methods: On the acquisition side, we implemented a series of monitoring strategies and performance evaluation methods to comprehensively monitor web page loading events and obtain loading indicators for each element. This includes tracking and recording key loading indicators, such as the first screen time, DOMContentLoaded event time, and the time point when the entire page is loaded.

[0070] Data processing and analysis: On the service side, we are responsible for collecting data from various acquisition terminals and aggregating, processing and analyzing the reported data. This includes using a variety of techniques such as statistical analysis, data mining, and machine learning algorithms to discover patterns, trends and associations in the data.

[0071] Visualization: The system displays the processed and analyzed data to customers in a visual form. This is usually achieved through data visualization tools, such as charts, dashboards, and interactive maps. Through these visualization tools, customers can quickly grasp the key information of the data.

[0072] Alarm mechanism: The system can generate alarm information according to the alarm configuration. When the web page loading time is abnormal or the performance is degraded, such as the response code is not 200, the time taken to complete the page loading exceeds the set threshold, etc., the system can issue an alarm in time to notify the operation and maintenance personnel to troubleshoot the problem and prevent users from encountering access barriers.

[0073] 1. Preparation

[0074] The system of the present invention requires the installation of Google Chrome on the collection server.

[0075] 2. Collection-side business implementation steps

[0076] Step 1: Obtain the monitoring tasks of the collection end from the server end at regular intervals, and obtain the monitoring indicator data from the monitoring web page at regular intervals according to the tasks.

[0077] Step 2: Before obtaining the monitoring indicator data, use the curl function to check the response status code of the web address. The default HTTP status code is 200, which means that the address can be requested successfully. It also supports setting multiple response status codes as request success status codes on the server. If the response is successful, continue to the next step, otherwise the monitoring result is returned as failed.

[0078] Step 3. Start Google Chrome in headless mode, controlled by command line parameters like --disable-gpu and --headless.

[0079] Step 4: Configure the request add-on based on whether to disable cache and ignore certificate errors according to the server configuration. Do not use cache session.getCommand().getNetwork().setCacheDisabled(true); ignore certificate errors session.getCommand().getSecurity().setIgnoreCertificateErrors(true);

[0080] Step 5: Monitor web page loading events and record key loading indicators, such as the firstPaint time, DOMContentLoaded time, and InteractiveTime of the entire web page, as well as the network request size, file size, queue joining time, request start time, request end time, etc. of each element.

[0081] Step 6: Get a screenshot when the page load event times out. In order to understand the effect of the page displayed to the user when the page load time is long, we can consider getting a screenshot of the current page when the page load event times out to understand the user experience and help developers troubleshoot problems. It should be noted that when implementing this function, we must ensure that the generation of the screenshot will not affect the normal loading and performance of the page to avoid secondary delays.

[0082] 3. Server-side business implementation steps

[0083] Step 1: Get the collection node and issue the dial test collection task. In order to improve the efficiency of information collection, the heartbeat mechanism can be used to synchronize deployment information. The collection node sends a heartbeat signal to the server regularly and reports its own deployment information to the server. The server is responsible for managing and maintaining this information and issuing the dial test collection task to the collection end.

[0084] Step 2: Data collection, processing and display. The server is responsible for collecting data from various acquisition terminals, and then summarizing, processing and analyzing the reported data, and finally presenting it to customers in a visual form.

[0085] Step 3: Generate an alarm based on the alarm configuration. When the web page loading time is abnormal or the performance is degraded, such as when the response code is not 200 or the time taken to load the page exceeds the set threshold, the system can issue an alarm in time to notify the operation and maintenance personnel to troubleshoot the problem and prevent users from encountering access barriers.

[0086] 4 System advantages:

[0087] Based on Java language and cdp4j open source components, it can accurately obtain key indicators such as web page loading time and element file size.

[0088] Multi-microservice design reduces business coupling and improves system maintainability and scalability.

[0089] The heartbeat mechanism ensures the real-time and accuracy of data.

[0090] Comprehensive monitoring strategy and performance evaluation method, comprehensively monitor web page loading events and obtain loading indicators for each element.

[0091] Visual display shows data in an intuitive way, helping customers quickly grasp key information.

[0092] The timely alarm mechanism helps operation and maintenance personnel to troubleshoot problems in a timely manner and prevent users from encountering access barriers.

[0093] Embodiment 3, based on embodiment 2, proposes a web page performance monitoring system based on Java language and cdp4j open source components, the system includes the following modules:

[0094] The collection server is used to install Google Chrome and run cdp4j components; obtain collection node information and issue dial-up collection tasks; collect data from various collection terminals, summarize, process and analyze the data; and generate alarm information based on the alarm configuration.

[0095] Multiple acquisition terminals are used to periodically obtain monitoring tasks from the server and obtain monitoring indicator data based on the tasks; use the curl function to check the response status code of the web page address; decide whether to continue with the subsequent steps based on the response status code; start Google Chrome in headless mode; set whether to disable caching and ignore certificate errors based on the server configuration; listen to web page loading events and record key loading indicators; and take screenshots when the page loading event times out.

[0096] The server is used to manage the collection nodes, issue the dial test collection tasks, collect data, summarize, process and analyze the data, and display it to customers in a visual form;

[0097] Heartbeat mechanism, used to synchronize the deployment information of the collection nodes and issue collection tasks to the server;

[0098] Also included: monitoring strategies and performance evaluation methods for comprehensively listening to web page loading events and obtaining loading indicators for each element;

[0099] It also includes: an alarm mechanism that issues an alarm when the web page loading time is abnormal or the performance is degraded, notifying the operation and maintenance personnel to troubleshoot the problem.

[0100] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A web page performance monitoring method based on Java language and cdp4j open source component, characterized in that: The method comprises the following steps: Building infrastructure; Enhanced monitoring capabilities; Configure the acquisition end in detail; Improve server functions; Implement technical details, use Java language to implement all logic on the server and collection side, and use cdp4j open source components to interact with Google Chrome to obtain web page performance monitoring data.

2. A web page performance monitoring method based on Java language and cdp4j open source component according to claim 1, characterized in that: The specific operations of building the infrastructure include: Set up a collection server, install Google Chrome and run the cdp4j component; Deploy multiple collection terminals to perform monitoring tasks and collect monitoring indicator data; Set up a server to manage collection nodes, issue monitoring tasks, collect and process data, and display it to customers in a visual form; Implement the heartbeat mechanism to synchronously collect node deployment information and coordinate task delivery with the server.

3. A web page performance monitoring method based on Java language and cdp4j open source component according to claim 1, characterized in that: The specific operations of the enhanced monitoring function include: Introduce monitoring strategies and performance evaluation methods to comprehensively monitor web page loading events and obtain loading indicators for each element; Establish an alarm mechanism to issue an alarm and notify operation and maintenance personnel when the web page loading time is abnormal or the performance is degraded.

4. A web page performance monitoring method based on Java language and cdp4j open source component according to claim 1, characterized in that: The specific operations of detailed configuration of the acquisition end include: Use the curl function to check the response status code of the webpage address and decide whether to continue the subsequent steps based on the status code; Launch Google Chrome in headless mode; Depending on the server configuration, set whether to disable caching and ignore certificate errors; Monitor web page loading events and record key loading indicators; Automatically take a screenshot of the web page when the page load event times out.

5. The web page performance monitoring method based on Java language and cdp4j open source component according to claim 1, characterized in that: The specific operations to improve the server functions include: Obtain the information of the collection node and issue the dial test collection task as needed; Collect data from various acquisition terminals, summarize, process and analyze them; Automatically generate and send alarm information according to the alarm configuration.

6. A web page performance monitoring system based on Java language and cdp4j open source component according to the web page performance monitoring method based on Java language and cdp4j open source component according to any one of claims 1 to 5, characterized in that: The system includes the following modules: Collection server, used to install Google Chrome and run cdp4j components; Multiple collection terminals are used to regularly obtain monitoring tasks from the server and obtain monitoring indicator data based on the tasks; The server is used to manage the collection nodes, issue the dial test collection tasks, collect data, summarize, process and analyze the data, and display it to customers in a visual form; The heartbeat mechanism is used to synchronize the deployment information of the collection nodes and issue collection tasks to the server.

7. A web page performance monitoring system based on Java language and cdp4j open source components according to claim 6, characterized in that: Also includes: Monitoring strategies and performance evaluation methods are used to comprehensively listen to web page loading events and obtain loading indicators for each element.

8. A web page performance monitoring system based on Java language and cdp4j open source components according to claim 6, characterized in that: The collection end also includes: using the curl function to check the response status code of the web page address; deciding whether to continue with the subsequent steps based on the response status code; starting Google Chrome in headless mode; setting whether to disable caching and ignore certificate errors based on the server configuration; monitoring web page loading events and recording key loading indicators; and taking screenshots when the page loading event times out.

9. A web page performance monitoring system based on Java language and cdp4j open source components according to claim 6, characterized in that: The server also includes: obtaining collection node information and issuing dial-up collection tasks; collecting data from various collection terminals, summarizing, processing and analyzing the data; and generating alarm information according to the alarm configuration.

10. The web page performance monitoring system based on Java language and cdp4j open source component according to claim 6, characterized in that: Also includes: The alarm mechanism is used to issue an alarm when the web page loading time is abnormal or the performance is degraded, notifying the operation and maintenance personnel to troubleshoot the problem.

Images