Method and system for monitoring H5 page in real time based on headless browser

Abstract

The invention belongs to the technical field of computer technology, and discloses an H5 page real-time monitoring method based on a headless browser, which comprises the following specific steps: step 1, processing flow: the system processing flow is divided into a user configuration flow and a simulated user behavior processing flow; in the user configuration process, a user configures a page for monitoring configuration, the user fills in the page needing to be monitored in real time, and the behavior of the user is simulated after the page is loaded; and step 2, page monitoring. According to the headless browser-based H5 page real-time monitoring method and system provided by the invention, a user can clearly operate in a short time through simple configuration; according to the invention, the system engineering is extensible, each component is in a pluggable design, and a data source, an alarm mode and a display mode can be replaced according to needs; according to different scenes, abstract configuration, rapid trial and error and self-inspection discovery can be realized as much as possible; and a solution scheme for monitoring the H5 page in real time is realized.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and specifically relates to a method and system for real-time monitoring of H5 pages based on a headless browser. Background Art

[0002] The advent of the Internet age has brought us into a brand new information age. The network connects people from all over the world, making the flow and exchange of information more convenient and faster. With the help of the Internet, we can get news, learn knowledge, shop, entertain, etc. anytime and anywhere. At the same time, the Internet also provides us with more opportunities, allowing us to start a business, work, make friends, etc. on the Internet. It can be said that the advent of the Internet has changed our lifestyle and the face of the world, and has brought us into a more open, free and diversified era.

[0003] While bringing convenience, it also brings challenges. The products and services of the IT industry have penetrated into all aspects of our lives and have a significant impact on our lives and work. Daily monitoring includes: Logging, Metrics. With the rise of microservice architecture, Tracing has become an indispensable form of data. It records a certain RPC call information, and it can also be understood as a special access log. It is more inclined to be classified as a type of Logging. Traditional Web Internet services can be divided into five layers (hardware, middleware, back-end programs, business processes, UI). Monitoring should cover each layer, and problems such as imperfect functions, many bugs, and easy errors seriously affect normal use and user experience, causing the product to lose reputation and reduce trust. Therefore, it is necessary to complete testing and verification of functions, performance, stability, security, etc. to ensure product quality and reliability. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for real-time monitoring of H5 pages based on a headless browser to solve the problems raised in the above background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solution: a method for real-time monitoring of H5 pages based on a headless browser, the specific steps are as follows:

[0006] Step 1: Processing flow

[0007] The system processing flow is divided into two parts: user configuration flow and simulated user behavior processing flow;

[0008] User configuration process: The user configures the monitoring configuration on the user configuration page. The user fills in the page that needs to be monitored in real time. After the page is loaded, the user's behavior is simulated and the user's customized user behavior is configured. The system processes this data, saves it to the database, and feeds back the saved status to the user.

[0009] Simulate user behavior processing flow: The system reads user configuration regularly, calls puppeteer based on the configuration information, runs the loading page in the headless browser, checks the elements on the page after a specified time, and compares them with the preset expected values. If the result does not meet the expectation, an alarm will be called. Then, according to the configuration, simulate user behavior, and an alarm will be triggered if the expected value is not met.

[0010] Step 2: Page monitoring

[0011] Define the loaded page and the simulated behavior;

[0012] According to the configuration information, puppeteer is called to load the page, simulate user behavior, and compare the expected value with the feedback result. If it is not satisfied, an alarm will be issued. This process continues to achieve real-time monitoring.

[0013] As a preferred technical solution of the present invention, the user behavior described in step one refers to various operations performed by the user on the page, including clicking, filling in a form, mouse sliding, etc.

[0014] As a preferred technical solution of the present invention, the headless browser described in step one refers to a browser that can run in a graphical interface. It accesses the web page by rendering the page in memory and then sending the result back to the user or program that requested it, without displaying the web page on the screen.

[0015] As a preferred technical solution of the present invention, the page described in step 1 and step 2 is an H5 page. The H5 page refers to a web page developed based on HTML5 technology. HTML5 is a standardized markup language that provides new functions and semantic elements and can be used to create richer and more dynamic Web content and applications. H5 pages generally have better accessibility, interactivity, responsiveness and mobile support.

[0016] A system for real-time monitoring of H5 pages based on a headless browser, comprising:

[0017] Configuration page module: Vue-element-admin is used as the configuration page framework. As a fast-developed front-end framework, it can quickly build a set of pages for backend management.

[0018] Information saving and reading module: Sequelize is used as the ORM tool of the programming environment to save page configuration information to the database and read database data;

[0019] Database module: Use Sqlite3 database as the database for storing page configuration and alarm information;

[0020] Basic framework module: Puppeteer framework is used as the basic framework for calling headless browsers, which can be used for automated form submission, UI testing, keyboard input, etc.

[0021] User notification module: used to send notifications to users.

[0022] As a preferred technical solution of the present invention, the Vue-element-admin is a production-ready front-end solution for the management interface, which is based on vue and uses the UI toolkit element-ui, based on the latest development stack of vue.

[0023] As a preferred technical solution of the present invention, Sequelize is an easy-to-use and commitment-based Node.js ORM tool suitable for Postgres, MySQL, MariaDB, SQLite, DB2, Microsoft SQL Server, Snowflake, Oracle DB and Db2 for IBM i.

[0024] As a preferred technical solution of the present invention, the Sqlite3 is a lightweight database and a relational database management system that complies with ACID. It is contained in a relatively small C library.

[0025] As a preferred technical solution of the present invention, Puppeteer is a Node.js library that provides a high-level API to control Chrome / Chromium through the DevTools protocol.

[0026] As a preferred technical solution of the present invention, the programming environment adopts Node.js, which is a JavaScript runtime environment based on the Chrome V8 engine.

[0027] The beneficial effects of the present invention are as follows:

[0028] The method and system for real-time monitoring of H5 pages based on a headless browser proposed in the present invention can be simply configured so that users can understand the operation in a short time; the system engineering of the present invention is expandable, and each component is pluggable, and the data source, alarm mode, and display mode can be replaced as needed; it can also realize abstract configuration according to different scenarios, quickly trial and error, and self-check and discovery as much as possible; a real-time monitoring H5 page solution is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a system block diagram of the present invention;

[0030] Figure 2 This is a schematic diagram of the present invention;

[0031] Figure 3 Configure a flow chart for the user of the present invention;

[0032] Figure 4 This is a process flow chart of the system of the present invention;

[0033] Figure 5 This is a flow chart of the monitoring process of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] like Figures 1 to 5 As shown, the embodiment of the present invention provides a method for real-time monitoring of H5 pages based on a headless browser, and the specific steps are as follows:

[0036] Step 1: Processing flow

[0037] The system processing flow is divided into two parts: user configuration flow and simulated user behavior processing flow;

[0038] User configuration process: The user configures the monitoring configuration on the user configuration page. The user fills in the page that needs to be monitored in real time. After the page is loaded, the user's behavior is simulated and the user's customized user behavior is configured. The system processes this data, saves it to the database, and feeds back the saved status to the user.

[0039] Simulate user behavior processing flow: The system reads user configuration regularly, calls puppeteer based on the configuration information, runs the loading page in the headless browser, checks the elements on the page after a specified time, and compares them with the preset expected values. If the result does not meet the expectation, an alarm will be called. Then, according to the configuration, simulate user behavior, and an alarm will be triggered if the expected value is not met.

[0040] Step 2: Page monitoring

[0041] Define the loaded page and the simulated behavior;

[0042] According to the configuration information, puppeteer is called to load the page, simulate user behavior, and compare the expected value with the feedback result. If it is not satisfied, an alarm will be issued. This process continues to achieve real-time monitoring.

[0043] The present invention provides a configuration monitoring page to configure the H5 page that needs to be monitored and the user's behavior after opening the page. According to the configured information, the user's behavior is simulated in a headless browser, and the configured expected value is compared with the feedback result. For the behavior that does not meet the expected value, a monitoring alarm notification is implemented to achieve real-time monitoring. The present invention can realize the concrete configuration of the abstract user behavior of the H5 page; call the headless browser to simulate the user behavior according to the custom configuration to achieve real-time monitoring; and realize notification and alarm according to the comparison of page feedback and expected value.

[0044] Among them, the user behavior in step one refers to various operations performed by users on the page, including clicks, form filling, mouse sliding, etc.

[0045] The refined division of user behavior facilitates the identification and subsequent simulation of user behavior, providing basic data support.

[0046] Among them, the headless browser in step 1 refers to a browser that can run in a graphical interface. It accesses the web page by rendering the page in memory and then sending the result back to the user or program that requested it, without displaying the web page on the screen.

[0047] This makes headless browsers not only suitable for automated tasks such as web scraping and testing, but also safer for web browsing because it does not display your activities on the screen. In addition, headless browsers can also be used for tasks such as network data collection, website monitoring, and automated form filling.

[0048] Among them, the pages in step 1 and step 2 are H5 pages. H5 pages refer to web pages developed based on HTML5 technology. HTML5 is a standardized markup language that provides new functions and semantic elements and can be used to create richer and more dynamic web content and applications. H5 pages usually have better accessibility, interactivity, responsiveness and mobile support.

[0049] Compared with traditional PC web pages, H5 pages are lighter, more beautiful, more interactive, and more suitable for browsing on mobile devices, providing a better user experience. H5 pages can contain multiple elements such as text, pictures, audio, video, and animation, and can also interact with users through forms, voting, and sharing.

[0050] A system for real-time monitoring of H5 pages based on a headless browser, comprising:

[0051] Configuration page module: Vue-element-admin is used as the configuration page framework. As a fast-developed front-end framework, it can quickly build a set of pages for backend management.

[0052] Information saving and reading module: Sequelize is used as the ORM tool of the programming environment to save page configuration information to the database and read database data;

[0053] Database module: Use Sqlite3 database as the database for storing page configuration and alarm information;

[0054] Basic framework module: Puppeteer framework is used as the basic framework for calling headless browsers, which can be used for automated form submission, UI testing, keyboard input, etc.

[0055] User notification module: used to send notifications to users.

[0056] The present invention proposes a system for real-time monitoring of H5 pages based on a headless browser, which mainly monitors the UI layer. By monitoring the page display content and the interaction process, the normal operation of the page is ensured. The UI is closest to the user and can most directly affect the user experience. At the same time, due to the complex network environment and differences in device versions, UI problems occur frequently. The system proposed in the present invention includes the internal processing flow of the system, the overall process of calling the headless browser and alarming according to the feedback results, and abstracting the specific user behavior verifier. Although the system uses multiple open source components, it combines the monitoring configuration process and the simulation of user behavior and the alarm according to the results.

[0057] Among them, Vue-element-admin is a production-ready front-end solution for the management interface, which is based on vue and uses the UI toolkit element-ui, based on the latest development stack of vue.

[0058] Vue-element-admin has a built-in i18n solution, typical templates for enterprise applications, and many great features to help you build large and complex single-page applications.

[0059] Among them, Sequelize is an easy-to-use and promise-based Node.js ORM tool for Postgres, MySQL, MariaDB, SQLite, DB2, Microsoft SQL Server, Snowflake, Oracle DB, and Db2 for IBM i.

[0060] Sequelize features solid transaction support, relationships, eager and lazy loading, read replication, and more.

[0061] Among them, Sqlite3 is a lightweight database and a relational database management system that complies with ACID. It is contained in a relatively small C library.

[0062] Sqlite3 is a public domain project established by D. Richard Hipp. It is designed for embedded systems and has been used in many embedded products. It takes up very little resources. In embedded devices, only a few hundred KB of memory is needed.

[0063] Among them, Puppeteer is a Node.js library that provides a high-level API to control Chrome / Chromium through the DevTools Protocol.

[0064] Puppeteer runs in headless mode by default, i.e. a headless browser, but can be configured to run in full (headless) Chrome / Chromium.

[0065] The programming environment uses Node.js, which is a JavaScript runtime environment based on the ChromeV8 engine.

[0066] Node.js enables JavaScript to run on the server side and is used to develop high-performance, scalable network applications, such as web applications, back-end APIs, real-time communication applications, microservices, command-line tools, etc. Compared with traditional web servers, Node.js has features such as non-blocking I / O and event-driven, and can handle a large number of concurrent connections in a very efficient way.

[0067] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0068] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for real-time monitoring of H5 pages based on a headless browser, characterized in that: The specific steps are as follows: Step 1: Processing flow The system processing flow is divided into two parts: user configuration flow and simulated user behavior processing flow; User configuration process: The user configures the monitoring configuration on the user configuration page. The user fills in the page that needs to be monitored in real time. After the page is loaded, the user's behavior is simulated and the user's customized user behavior is configured. The system processes this data, saves it to the database, and feeds back the saved status to the user. Simulate user behavior processing flow: The system reads user configuration regularly, calls puppeteer based on the configuration information, runs the loading page in the headless browser, checks the elements on the page after a specified time, and compares them with the preset expected values. If the result does not meet the expectation, an alarm will be called. Then, according to the configuration, simulate user behavior, and an alarm will be triggered if the expected value is not met. Step 2: Page monitoring Define the loaded page and the simulated behavior; According to the configuration information, puppeteer is called to load the page, simulate user behavior, and compare the expected value with the feedback result. If it is not satisfied, an alarm will be issued. This process continues to achieve real-time monitoring.

2. According to claim 1, a method for real-time monitoring of H5 pages based on a headless browser is characterized in that: The user behavior described in step 1 refers to the various operations performed by users on the page, including clicks, form filling, mouse sliding, etc.

3. According to claim 1, a method for real-time monitoring of H5 pages based on a headless browser is characterized in that: The headless browser described in step 1 refers to a browser that can run in a graphical interface. It accesses the web page by rendering the page in memory and then sending the result back to the user or program that requested it, without displaying the web page on the screen.

4. According to claim 1, a method for real-time monitoring of H5 pages based on a headless browser is characterized in that: The pages described in step 1 and step 2 are H5 pages. H5 pages refer to web pages developed based on HTML5 technology. HTML5 is a standardized markup language that provides new functions and semantic elements and can be used to create richer and more dynamic web content and applications. H5 pages usually have better accessibility, interactivity, responsiveness and mobile support.

5. A system for real-time monitoring of H5 pages based on a headless browser, characterized in that: include: Configuration page module: Vue-element-admin is used as the configuration page framework. As a fast-developed front-end framework, it can quickly build a set of pages for backend management. Information saving and reading module: Sequelize is used as the ORM tool of the programming environment to save page configuration information to the database and read database data; Database module: Use Sqlite3 database as the database for storing page configuration and alarm information; Basic framework module: Puppeteer framework is used as the basic framework for calling headless browsers, which can be used for automated form submission, UI testing, keyboard input, etc. User notification module: used to send notifications to users.

6. The system for real-time monitoring of H5 pages based on a headless browser according to claim 5 is characterized in that: The Vue-element-admin is a production-ready front-end solution for the management interface, which is based on vue and uses the UI toolkit element-ui, based on the latest development stack of vue.

7. The system for real-time monitoring of H5 pages based on a headless browser according to claim 5, characterized in that: Sequelize is an easy-to-use, promise-based Node.js ORM tool for Postgres, MySQL, MariaDB, SQLite, DB2, Microsoft SQL Server, Snowflake, Oracle DB, and Db2 for IBM i.

8. The system for real-time monitoring of H5 pages based on a headless browser according to claim 5, characterized in that: The Sqlite3 is a lightweight database and a relational database management system that complies with ACID. It is contained in a relatively small C library.

9. The system for real-time monitoring of H5 pages based on a headless browser according to claim 5, characterized in that: The Puppeteer is a Node.js library that provides a high-level API to control Chrome / Chromium via the DevTools Protocol.

10. The system for real-time monitoring of H5 pages based on a headless browser according to claim 5, characterized in that: The programming environment uses Node.js, which is a JavaScript runtime environment based on the Chrome V8 engine.

Images