A resource loading monitoring method, device, equipment and storage medium

Abstract

This application discloses a resource loading monitoring method, apparatus, device, and storage medium. The technical solution provided in this application involves subscribing to web resource loading events under a specified monitoring domain name through an extension plugin installed on a browser. The extension plugin listens for these events, generating resource loading information upon detecting them. This information is then sent to a server for storage. Staff can analyze and process the saved resource loading information to optimize web resource loading performance, improving overall performance. This approach eliminates the need for excessive source code intrusion into the content security audit system, reducing maintenance workload and improving the quality of web resource loading monitoring.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a resource loading monitoring method, apparatus, device, and storage medium. Background Technology

[0002] With the continuous development of web page and browser technologies, more and more applications are choosing web pages to implement product functions. For example, in content security auditing systems, staff need to review a large number of online resources such as images and videos. Currently, web resources are distributed through CDNs (Content Delivery Networks). However, since staff working on content security auditing systems are located in different locations and network conditions vary, it is necessary to monitor the loading performance of web resources for targeted optimization.

[0003] Currently, monitoring the performance of network resource loading typically involves inserting listener functions for loading completion / failure events into the front-end code of the content security audit system. However, this approach is highly intrusive to the source code of the content security audit system, increasing the difficulty of maintaining the system. Summary of the Invention

[0004] This application provides a resource loading monitoring method, apparatus, device, and storage medium to solve the technical problem in the prior art where intrusive modifications to the source code of a content security review system increase the maintenance difficulty of the content security review system. It reduces intrusion into the source code of the content security review system, reduces the maintenance workload of the content security review system, and improves the quality of web page resource loading monitoring.

[0005] In a first aspect, embodiments of this application provide a resource loading monitoring method, including:

[0006] Subscribe to webpage resource loading events for a specified monitoring domain in your browser's extension;

[0007] The extension plugin listens for web page resource loading events in the browser, and generates resource loading information when it detects a web page resource loading event for the specified monitored domain.

[0008] The resource loading information generated by the extension plugin is obtained and sent to the server, whereby the server saves the resource loading information.

[0009] In a second aspect, embodiments of this application provide a resource loading monitoring device, including a domain name subscription module, a loading monitoring module, and an information storage module, wherein:

[0010] The domain subscription module is used to subscribe to webpage resource loading events under a specified monitoring domain in the browser's extension plugin;

[0011] The loading monitoring module is used to monitor the browser's web page resource loading events through the extension plugin. When the extension plugin detects the browser's web page resource loading event for the specified monitoring domain, it generates resource loading information.

[0012] The information storage module is used to obtain the resource loading information generated by the extension plugin, and send the resource loading information to the server, whereby the server stores the resource loading information.

[0013] In a third aspect, embodiments of this application provide a resource loading monitoring device, including: a memory and one or more processors;

[0014] The memory is used to store one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the resource loading monitoring method as described in the first aspect.

[0016] In a fourth aspect, embodiments of this application provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the resource loading monitoring method as described in the first aspect.

[0017] This application embodiment subscribes to web resource loading events under a specified monitoring domain through an extension plugin installed on the browser. The extension plugin listens for these events and generates resource loading information upon detecting them. This information is then sent to a server for storage. Staff can analyze the loading performance of web resources based on this information to optimize them and improve loading efficiency. Monitoring web resource loading events under a specified monitoring domain through the extension plugin reduces the need for invasive modifications to the content security audit system's source code, thus minimizing maintenance complexity and improving the quality of web resource loading monitoring. Attached Figure Description

[0018] Figure 1 This is a flowchart of a resource loading monitoring method provided in an embodiment of this application;

[0019] Figure 2 This is a flowchart of another resource loading monitoring method provided in the embodiments of this application;

[0020] Figure 3 This is a flowchart illustrating a web resource provider provided in an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the communication timing between a browser and an extension plugin provided in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of a resource loading monitoring device provided in an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the structure of a resource loading monitoring device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but additional steps not included in the drawings may also be present. The above processes can correspond to methods, functions, procedures, subroutines, subroutines, etc.

[0025] Figure 1 A flowchart of a resource loading monitoring method provided in an embodiment of this application is given. The resource loading monitoring method provided in this embodiment of the application can be executed by a resource loading monitoring device, which can be implemented by hardware and / or software and integrated into a resource loading monitoring equipment.

[0026] The following description uses a resource loading monitoring device to perform a resource loading monitoring method as an example. (Reference) Figure 1 The resource loading monitoring method includes:

[0027] S101: Subscribe to web resource loading events for a specified monitoring domain in your browser extension.

[0028] In this embodiment, the browser is installed in the resource loading monitoring device and can be used to request web resources from a web resource provider based on a configured domain name. The web resource provider can be a Content Delivery Network (CDN). After receiving the web resource request from the browser, the CDN allocates a corresponding resource server based on the domain name of the request. The browser accesses the allocated resource server, which then provides the web resource to the browser (different resource servers correspond to different IP addresses). The web resource is an HTTP resource loaded through a webpage, such as video, images, audio, and JavaScript.

[0029] The browser provided in this embodiment can be Chrome, Firefox, Microsoft Edge, etc., and the specific browser type can be selected according to the actual situation. The browser is configured with a browser extension API, which can be used to write extension programs to extend the browser's functionality. Hooks in the browser extension API are used to achieve non-intrusive monitoring of the browser's web resource requests. Only a small amount of SDK code needs to be embedded in the browser to activate the browser's resource monitoring function. Specifically, this solution utilizes the browser's extension API to install and configure an extension plugin in the browser and connect to the browser's extension API. This extension plugin has the function of monitoring the browser's web resource loading status and communicates with the resource loading monitoring device based on a set network protocol. The extension plugin can be permanently installed in the browser, or it can be activated and connected to the browser's extension API only when resource loading monitoring is needed. The extension plugin can subscribe to web resource loading events under multiple different specified monitoring domains, that is, simultaneously monitor the resource loading performance of multiple specified monitoring domains.

[0030] For example, based on the specified monitoring domain name corresponding to the resource loading to be monitored, the extension plugin installed on the browser subscribes to the web resource loading events under the specified monitoring domain name. When the extension plugin detects the web resource loading event of the specified monitoring domain name in the browser, it will generate resource loading information.

[0031] S102: The extension plugin listens for web page resource loading events in the browser. When the extension plugin listens for web page resource loading events of the specified monitored domain, it generates resource loading information.

[0032] For example, after subscribing to web resource loading events under a specified monitoring domain in a browser extension, the extension listens for these events. The extension indirectly acquires all resource loading requests initiated by the browser. When a browser request to load web resources corresponding to the specified domain triggers a web resource loading event, the extension fetches the domain corresponding to that request. If the domain is not the specified monitoring domain, the web resource loading event is ignored. However, if the fetched domain is the specified monitoring domain, the extension confirms that it has detected the browser's web resource loading event for that domain and monitors it. The browser then sends the web resource loading status to the extension, which generates resource loading information based on this information.

[0033] In one embodiment, the resource loading information includes a combination of one or more of the following: the response status code (HTTP status code) corresponding to the browser loading the web page resource of the specified monitored domain name; the response status header (e.g., HTTP Header request / response header information sent and received by the web page resource); the web page address (IP address) responding to the web page resource loading event; the size of the web page resource; the resource loading time (e.g., the time interval between the start and completion of loading the web page resource); the loading result (e.g., loading successful or loading failed); and the loading failure reason. The loading failure reason includes one or more of the following: network timeout, status code error, inability to establish a connection, and domain name lookup failure. The specific loading failure reason can be obtained through the browser's extension interface.

[0034] S103: Obtain resource loading information generated by the extension plugin and send the resource loading information to the server, which then saves the resource loading information.

[0035] For example, the resource loading information generated by the extension plugin is obtained, and the resource loading information is sent to the server through a set communication protocol. The server then persistently saves the received resource loading information so that staff can analyze and process the loading performance of web page resources based on the resource loading information saved on the server.

[0036] Taking the content moderation device used by content moderators as an example of the resource loading monitoring device provided in this embodiment, since users of the content moderation backend are located all over the world, and the network conditions of different content moderation backends vary, the content delivery network provider needs to optimize the network lines based on the webpage resource loading status (resource loading information). Content moderators need to run the content moderation backend on the monitoring device. The content moderation backend uses a browser to request webpage resources corresponding to the specified domain name from the content delivery network, and downloads the webpage resources according to the webpage address specified by the content delivery network. Content moderators then perform content moderation based on the downloaded webpage resources. At the same time, an extension plugin is installed on the browser of the monitoring device, and the extension plugin subscribes to webpage resource loading events under the specified monitoring domain name. When the extension plugin listens for the browser's webpage resource loading event for the specified monitoring domain name, it generates resource loading information and sends the resource loading information to the server, where the server saves the resource loading information.

[0037] The above describes a method that involves subscribing to web resource loading events under a specified monitoring domain through a browser extension. The extension listens for these events and generates resource loading information upon detecting them. This information is then sent to a server for storage. Staff can analyze the loading performance of web resources based on this information to optimize performance and improve loading efficiency. Monitoring web resource loading events under a specified domain via an extension reduces the need for invasive modifications to the content security audit system's source code, thus minimizing maintenance complexity and improving the quality of web resource loading monitoring.

[0038] Based on the above embodiments, Figure 2 A flowchart of another resource loading monitoring method provided in an embodiment of this application is given, which is a concretization of the above-described resource loading monitoring method. (Reference) Figure 2 The resource loading monitoring method includes:

[0039] S201: In response to the opening of the monitored webpage, register the monitored webpage with the extension plugin.

[0040] S202: Subscribe to web resource loading events for a specified monitoring domain in your browser extension.

[0041] Specifically, the content moderation backend of the resource loading monitoring device uses a pre-defined webpage (e.g., a JavaScript- or CSS-based webpage) to load webpage resources corresponding to a specified domain for content moderation. This pre-defined webpage is the monitored webpage for which resource loading performance monitoring is required in this solution. After installing the extension plugin in the browser, the monitored webpage is specified in the browser. The browser listens for the opening status of the monitored webpage. Once the monitored webpage is opened, the browser initiates a registration event with the extension plugin to register the monitored webpage in the extension plugin and subscribes to the webpage resource loading events (webpage resource loading event stream) under the specified monitoring domain in the browser's extension plugin.

[0042] S203: Listen to the browser's web page resource loading events through an extension plugin. When the extension plugin detects a web page resource loading event for a specified monitored domain, it generates resource loading information.

[0043] Specifically, after registering the monitored webpages and subscribing to webpage resource loading events under the specified monitoring domain in the extension plugin, the extension plugin will listen for webpage resource loading events on the registered monitored webpages. When it detects a webpage resource loading event on the specified monitoring domain, it will generate resource loading information based on the webpage resource loading status provided by the monitored webpage. This resource loading information can be generated for the corresponding webpage resource segments. For example, for a video sent in segments, multiple resource loading information segments can be generated for each video segment.

[0044] In one embodiment, when the extension plugin detects a webpage resource loading event on a specified monitored domain, it generates resource loading information based on the resource loading time corresponding to the webpage resource loading event. Furthermore, the extension plugin can determine whether to directly upload the resource loading information or partially upload the resource loading information based on the webpage resource loading time. Therefore, when generating resource loading information based on the resource loading time corresponding to the webpage resource loading event, the extension plugin provided in this solution includes at least one of the following:

[0045] S2031: When the resource loading time corresponding to the webpage resource loading event reaches the set time threshold, report the resource loading information.

[0046] S2032: When the resource loading time corresponding to the webpage resource loading event has not reached the set time threshold, report the resource loading information according to the set ratio.

[0047] Specifically, when the extension plugin receives the resource loading status from the browser's (monitored webpage) extension interface, it determines the resource loading time corresponding to the webpage resource loading event based on the resource loading status and compares the resource loading time with the set time threshold.

[0048] Furthermore, when the resource loading time reaches a set time threshold, it is considered that the resource loading time of the current webpage is too long, which may indicate an abnormality in the corresponding network line. In this case, resource loading information is generated directly based on the resource loading information provided by the browser and sent to the resource loading monitoring device.

[0049] If the resource loading time does not reach the set time threshold, the resource loading time of the current webpage is considered normal, and it is not necessary to upload all the corresponding resource loading information. Instead, resource loading information can be reported according to a set ratio (either by first generating resource loading information based on the resource loading status provided by the browser and then selecting the resource loading information to be reported according to the set ratio, or by generating the resource loading information to be uploaded based on the resource loading status provided by the browser according to the set ratio). In one embodiment, different webpage resource sizes correspond to different set time thresholds, and the corresponding set time thresholds can be determined based on the size of the webpage resources.

[0050] In one embodiment, when generating resource loading information, the extension plugin marks the data source as the extension plugin to facilitate quick identification of resource loading information during resource loading monitoring.

[0051] S204: Obtain resource loading information generated by the extension plugin and send the resource loading information to the server, which then saves the resource loading information.

[0052] In one embodiment, after receiving resource loading information from the extension plugin, the web page aggregates the resource loading information at set time intervals (e.g., packages the resource loading information received between set time intervals) and sends the aggregated resource loading information to the server for storage. In another embodiment, after receiving the resource loading information, the resource loading information device adds sender information (e.g., username, user ID, etc.) to the resource loading information to facilitate subsequent targeted network optimization analysis based on the sender information.

[0053] The server can save the received resource loading information to a designated database. Based on this, the server provided in this solution sends the received resource loading information to a pre-defined data queue, retrieves resource loading information from the data queue, and sends the retrieved resource loading information to the pre-defined database. This solution utilizes the data queue to smooth out peak and trough periods of a large amount of resource loading information, reducing short-term traffic spikes and buffering short-term bursts of high concurrency. The data queue can be a Kafka queue, RabbitMQ queue, etc., and the database can be a ClickHouse database, MySQL database, etc.

[0054] Furthermore, the resource loading information stored in the database can be visualized to monitor webpage resource loading performance, thereby obtaining more detailed webpage resource loading information. Based on this, the database provided in this solution is also used to respond to resource loading information query requests from the data visualization end, sending the corresponding resource loading information to the data visualization end for visualization. The data visualization end is used to visualize the obtained resource loading information. It can be built on Grafana, a cross-platform, open-source data visualization web application platform. After the user configures the connected data source (database), Grafana can display corresponding data charts and warning messages in a web browser. When viewing resource loading information, the Grafana monitoring and display system can display the resource loading time and success rate in real time, and show the loading time and success rate under different dimensions (such as time and space dimensions).

[0055] Figure 3 This is a flowchart illustrating a web resource provider provided in an embodiment of this application, such as... Figure 3 As shown, taking the content review device used by content reviewers as an example of the resource loading monitoring device provided in this embodiment, an extension plugin (Chrome extension in the figure) is installed on the browser (Chrome browser) of the monitoring device. At the same time, the webpage to be monitored is specified in the resource loading monitoring device, and the resource loading monitoring device listens to the opening status of the webpage to be monitored on the browser.

[0056] like Figure 4 The provided diagram illustrates the communication sequence between a browser and its extension. After opening the monitored webpage, the browser initiates a registration event with the extension (Chrome Extension in the diagram) to register the monitored webpage within the extension and subscribe to webpage resource loading events under a specified monitoring domain. When the monitored webpage requests webpage resources from a content delivery network (CDN in the diagram) based on the specified monitoring domain, the extension detects the triggering of the webpage resource loading event, monitors the webpage resource loading event triggered by the monitored webpage, and generates resource loading information. When the content delivery network receives the network resource request from the monitored webpage, it returns a resource response to the monitored webpage and distributes the corresponding webpage resources. After the webpage resources are loaded, the extension performs a loading event completion callback for the monitored webpage and returns resource loading information to the resource loading monitoring device.

[0057] Furthermore, after receiving resource loading information, the resource loading monitoring device sends a request to the server's data queue ( Figure 3Resource loading information is sent to a Kafka queue, and the data queue retrieves this information and inserts it into the database (ClickHouse database in the diagram) for storage. Furthermore, when it's necessary to visualize the resource loading information in the database, a data visualization tool (Grafana monitoring in the diagram) can be used to query the database for the corresponding resource loading information and then visualize it.

[0058] The above describes a method that involves subscribing to web resource loading events under a specified monitoring domain through a browser extension. The extension listens for these events and generates resource loading information upon detecting them. This information is then sent to a server for storage. Staff can analyze the loading performance of web resources based on this information to optimize performance and improve loading efficiency. Monitoring web resource loading events under a specified domain via an extension reduces the need for invasive modifications to the content security audit system's source code, thus minimizing maintenance complexity and improving the quality of web resource loading monitoring. Furthermore, when the monitored webpage is opened, the extension plugin registers the monitored webpage, accurately activating the monitoring of resource loading. If the resource loading time does not reach a set threshold, resource loading information is reported according to a set ratio, reducing data consumption on storage space. A data queue is used to smooth out peak and trough periods of high resource loading information, buffering short-term bursts of high concurrency. Simultaneously, the extension plugin can be installed on all browsers used by auditors accessing the content security audit backend, enabling monitoring of webpage resource loading performance and success rate on all such browsers. This allows for timely detection of anomalies, and for failed loading resources, the specific reasons for the failure are obtained and reported to the monitoring system for real-time display via a data visualization interface. This allows users to see the resource loading status, and for some abnormal or slow-loading webpages, network lines can be analyzed and optimized using the webpage address and loading failure reason in the resource loading information.

[0059] Figure 5 This is a schematic diagram of the structure of a resource loading monitoring device provided in an embodiment of this application. (Reference) Figure 5 The resource loading monitoring device includes a domain name subscription module 51, a loading monitoring module 52, and an information storage module 53.

[0060] The domain subscription module 51 is used to subscribe to web page resource loading events under a specified monitoring domain in the browser's extension plugin; the loading listening module 52 is used to listen to the browser's web page resource loading events through the extension plugin, and the extension plugin generates resource loading information when it listens to the browser's web page resource loading events for the specified monitoring domain; the information saving module 53 is used to obtain the resource loading information generated by the extension plugin and send the resource loading information to the server, and the server saves the resource loading information.

[0061] The above describes a method that involves subscribing to web resource loading events under a specified monitoring domain through a browser extension. The extension listens for these events and generates resource loading information upon detecting them. This information is then sent to a server for storage. Staff can analyze the loading performance of web resources based on this information to optimize performance and improve loading efficiency. Monitoring web resource loading events under a specified domain via an extension reduces the need for invasive modifications to the content security audit system's source code, thus minimizing maintenance complexity and improving the quality of web resource loading monitoring.

[0062] This application also provides a resource loading monitoring device, which can integrate the resource loading monitoring apparatus provided in this application. Figure 6 This is a schematic diagram of the structure of a resource loading monitoring device provided in an embodiment of this application. (Reference) Figure 6 The resource loading monitoring device includes: an input device 63, an output device 64, a memory 62, and one or more processors 61; the memory 62 is used to store one or more programs; when one or more programs are executed by one or more processors 61, the one or more processors 61 implement the resource loading monitoring method provided in the above embodiments. The resource loading monitoring device, equipment, and computer provided above can be used to execute the resource loading monitoring method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0063] This application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the resource loading monitoring method provided in the above embodiments. Of course, the computer-executable instructions in the storage medium containing computer-executable instructions provided in this application are not limited to the resource loading monitoring method provided above, and can also perform related operations in the resource loading monitoring method provided in any embodiment of this application. The resource loading monitoring device, equipment, and storage medium provided in the above embodiments can execute the resource loading monitoring method provided in any embodiment of this application. Technical details not described in detail in the above embodiments can be found in the resource loading monitoring method provided in any embodiment of this application.

[0064] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments provided herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A resource loading monitoring method, characterized in that, include: Subscribe to webpage resource loading events for a specified monitoring domain in your browser's extension; The extension plugin listens for webpage resource loading events in the browser. When the extension plugin detects a webpage resource loading event for the specified monitoring domain, it generates resource loading information. When the extension plugin detects a webpage resource loading event for the specified monitoring domain and the resource loading time corresponding to the webpage resource loading event has not reached a set time threshold, it reports the resource loading information according to a set ratio. When the extension plugin detects a webpage resource loading event for the specified monitoring domain and the resource loading time corresponding to the webpage resource loading event reaches a set time threshold, it also reports the resource loading information. The resource loading information generated by the extension plugin is obtained and sent to the server, whereby the server saves the resource loading information.

2. The resource loading monitoring method according to claim 1, characterized in that, Before subscribing to webpage resource loading events under a specified monitoring domain in a browser extension, the method also includes: In response to the opening of the monitored webpage, register the monitored webpage with the extension plugin.

3. The resource loading monitoring method according to claim 1, characterized in that, After receiving the resource loading information, the server sends the resource loading information to a pre-defined data queue, retrieves the resource loading information from the data queue, and sends the retrieved resource loading information to a pre-defined database.

4. The resource loading monitoring method according to claim 3, characterized in that, The method further includes: In response to a resource loading information query request from the data visualization terminal, the database sends the resource loading information corresponding to the query request to the data visualization terminal so that the data visualization terminal can visualize the obtained resource loading information.

5. The resource loading monitoring method according to any one of claims 1-4, characterized in that, The resource loading information includes one or more of the following combinations when the browser loads the web page resources of the specified monitored domain: response status code, response status header, web page address responding to the web page resource loading event, web page resource size, resource loading time, loading result, and loading failure reason.

6. A resource loading monitoring device, characterized in that, It includes a domain subscription module, a loading and listening module, and an information saving module, among which: The domain subscription module is used to subscribe to webpage resource loading events under a specified monitoring domain in the browser's extension plugin; The loading monitoring module is used to monitor the browser's web page resource loading events through the extension plugin. When the extension plugin detects a web page resource loading event of the browser for the specified monitoring domain, it generates resource loading information. When the extension plugin detects a web page resource loading event of the browser for the specified monitoring domain and the resource loading time corresponding to the web page resource loading event has not reached a set time threshold, it reports the resource loading information according to a set ratio. When the extension plugin detects a web page resource loading event of the browser for the specified monitoring domain and the resource loading time corresponding to the web page resource loading event reaches a set time threshold, it also reports the resource loading information. The information storage module is used to obtain the resource loading information generated by the extension plugin, and send the resource loading information to the server, whereby the server stores the resource loading information.

7. A resource loading monitoring device, characterized in that, include: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the resource loading monitoring method as described in any one of claims 1-5.

8. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the resource loading monitoring method as described in any one of claims 1-5.

Images