Page processing method and page processing model training method

Abstract

The embodiment of the invention provides a page processing method and a page processing model training method.The page processing method comprises the steps that a to-be-processed page frame sequence is obtained, and the to-be-processed page frame sequence comprises a page image sequence corresponding to a target page in the loading process; the to-be-processed page frame sequence is input into a target page processing model, a target time interval is obtained, and the target time interval is obtained in the mode that the target page processing model positions the starting point and the ending point of the loading process of a target page in the time dimension. The efficiency and accuracy of identifying the target time interval are improved, the determined target time interval is the time interval conforming to the actual visual perception of the user, and the page loading performance can be accurately evaluated and optimized based on the target time interval to improve the user experience.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to page processing methods. It also relates to a page processing model training method, a page processing method applied to cloud-based devices, a page processing apparatus, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the rapid development of technology and web page technology, internet applications are emerging in an endless stream. Among these advancements, users have increasingly higher demands for page loading performance, as good page loading performance can improve user satisfaction and increase user dwell time.

[0003] Typically, when evaluating page load performance, the start and end times of the page load process need to be considered. However, methods that determine the start and end times of the page load process through internal page events such as DOM rendering markers ignore the actual user experience from a visual perspective, leading to a discrepancy between the determined start and end times and the user's visual perception. Therefore, identifying accurate start and end times for the page load process that align with the user's actual perception is crucial for improving page load performance and enhancing the user experience.

[0004] Based on this, this application provides a page processing method. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a page processing method. One or more embodiments of this specification also relate to a page processing model training method, a page processing method applied to cloud-side devices, a page processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, a page processing method is provided, including: Obtain the page frame sequence to be processed, wherein the page frame sequence to be processed includes the page image sequence corresponding to the target page during the loading process; The sequence of page frames to be processed is input into the target page processing model to obtain the target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

[0007] Optionally, the target time interval includes the start time of target page loading when the page element first appears, and the end time of target page loading when the page element that satisfies the page function of the target page is successfully rendered.

[0008] Optionally, after obtaining the target time interval, the method further includes: Based on the target time interval, in the sequence of page frames to be processed, the page frame corresponding to the start time of the target page loading is determined as the first page frame, and the page frame corresponding to the end time of the target page loading is determined as the second page frame.

[0009] Optionally, after obtaining the target time interval, the method further includes: Based on the target time interval, the page loading performance corresponding to the page frame sequence to be processed is evaluated.

[0010] Optionally, the target page processing model includes a spatiotemporal feature extraction network and a result prediction network; The sequence of page frames to be processed is input into the target page processing model to obtain the target time interval, including: The sequence of page frames to be processed is input into the spatiotemporal feature extraction network to obtain the spatiotemporal features corresponding to the sequence of page frames to be processed. The spatiotemporal features are input into the result prediction network to obtain the target time interval.

[0011] Optionally, the result prediction network includes a multi-scale feature extraction subnetwork and a prediction subnetwork; The spatiotemporal features are input into the result prediction network to obtain the target time interval, including: The spatiotemporal features are input into the multi-scale feature extraction subnet to obtain features at multiple scales. The multiple scale features are respectively input into the prediction subnet to obtain the time interval corresponding to each scale feature; The target time interval is determined from the plurality of time intervals.

[0012] Optionally, the result prediction network further includes a feature mapping subnetwork; The spatiotemporal features are input into the multi-scale feature extraction subnetwork to obtain multiple scale features, including: The spatiotemporal features are input into the feature mapping subnet to obtain the mapped features; The mapped features are input into the multi-scale feature extraction subnet to obtain features at multiple scales.

[0013] Optionally, the prediction subnet includes a time-series regression branch; The multiple scale features are input into the prediction subnet respectively to obtain the time interval corresponding to each scale feature, including: The multiple scale features are input into the time series regression branch respectively to obtain the time interval corresponding to each scale feature.

[0014] Optionally, the prediction subnet includes an action classification branch; Determining the target time interval from the plurality of time intervals includes: The multiple scale features are respectively input into the action classification branch to obtain the page loading sub-process corresponding to each scale feature and the confidence level corresponding to the page loading sub-process. Based on each confidence level, determine the target page loading sub-process; The time interval corresponding to the target page loading sub-process is determined as the target time interval.

[0015] According to a second aspect of the embodiments of this specification, a page processing model training method is provided, comprising: Obtain a sample page frame sequence and obtain the label time interval corresponding to the sample page frame sequence. The sample page frame sequence includes a sequence of page images corresponding to the sample page during the loading process. The label time interval is the start and end interval of the loading process of the sample page in the time dimension, which is pre-labeled. The sample page frame sequence is input into the initial page processing model to obtain the prediction time interval, wherein the prediction time interval is obtained by the initial page processing model locating the start and end points in the time dimension during the loading process of the sample page. Based on the tag time interval and the prediction time interval, the initial page processing model is trained until the model training stops, thus obtaining the target page processing model.

[0016] Optionally, the tag time interval includes the start time of sample page loading when the page element first appears, and the end time of sample page loading when the page element that satisfies the page function of the sample page is successfully rendered.

[0017] Optionally, the initial page processing model includes a spatiotemporal feature extraction network and a result prediction network. The result prediction network includes a multi-scale feature extraction subnetwork and a prediction subnetwork. The prediction subnetwork includes a temporal regression branch and an action classification branch. The sample page frame sequence is input into the initial page processing model to obtain the prediction time interval, including: The sample page frame sequence is input into the spatiotemporal feature extraction network to obtain the sample spatiotemporal features; The spatiotemporal features of the samples are input into the multi-scale feature extraction subnetwork to obtain multiple sample scale features; The multiple scale sample features are respectively input into the action classification branch to obtain the prediction time interval corresponding to each sample scale feature, and the multiple scale sample features are respectively input into the temporal regression branch to obtain the prediction page loading action corresponding to each sample scale feature. Accordingly, based on the label time and the prediction time, the initial page processing model is trained, including: The first loss is determined based on the label time and the prediction time; The second loss is determined based on the predicted page loading action and the sample page loading action; The initial page processing model is trained based on the first loss and the second loss.

[0018] According to a third aspect of the embodiments of this specification, a page processing method is provided, applied to a cloud-side device, comprising: The receiving end-side device sends a page processing request, wherein the page processing request carries a sequence of page frames to be processed, and the sequence of page frames to be processed includes a sequence of page images corresponding to the target page during the loading process; The sequence of page frames to be processed is input into the target page processing model to obtain the target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension. The target time interval is sent to the end-side device.

[0019] According to a fourth aspect of the embodiments of this specification, a page processing model training method is provided, applied to a cloud-side device, comprising: The receiving end device sends a model training request, wherein the model training request carries a sample page frame sequence and a label time interval corresponding to the sample page frame sequence. The sample page frame sequence includes a sequence of page images corresponding to the sample page during the loading process, and the label time interval is the start and end interval of the loading process of the sample page in the time dimension, which is pre-labeled. The sample page frame sequence is input into the initial page processing model to obtain the prediction time interval, wherein the prediction time interval is obtained by the initial page processing model locating the start and end points of the loading process of the sample page in the time dimension. Based on the tag time interval and the predicted time interval, train the initial page processing model until the model training stops, and obtain the target page processing model. The model parameters of the target page processing model are sent to the terminal device.

[0020] According to a fifth aspect of the embodiments of this specification, a page processing apparatus includes: The acquisition module is configured to acquire a sequence of page frames to be processed, wherein the sequence of page frames to be processed includes a sequence of page images corresponding to the target page during the loading process; The input module is configured to input the sequence of page frames to be processed into the target page processing model to obtain a target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

[0021] According to a sixth aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above method.

[0022] According to a seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0023] According to an eighth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0024] The page processing method provided in this specification can obtain a sequence of page frames to be processed, which includes a sequence of page images corresponding to the target page during the loading process; the sequence of page frames to be processed is input into a target page processing model to obtain a target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

[0025] One embodiment of this specification uses a page processing model to determine the target time interval corresponding to the page frame sequence to be processed, improving the efficiency of identifying the start and end time points of the page loading process and the accuracy of the identified target time interval. Furthermore, by identifying the start and end points of the page in the time dimension during the loading process from the page image, from a visual perspective, the determined target time interval conforms to the user's actual visual perception, thus allowing for precise optimization of page loading performance based on the target time interval to improve the user experience. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a page processing method provided in one embodiment of this specification; Figure 2 This is a schematic diagram of the structure of a page processing model provided in one embodiment of this specification; Figure 3 This is a schematic diagram of the structure of a page processing model provided in one embodiment of this specification; Figure 4 This is a schematic diagram of the structure of a page processing model provided in one embodiment of this specification; Figure 5 This is a flowchart illustrating a page processing model training method provided in one embodiment of this specification; Figure 6 This is a flowchart illustrating a page processing method provided in one embodiment of this specification; Figure 7 This is a flowchart illustrating a page processing model training method for cloud-based devices, provided in one embodiment of this specification. Figure 8 This is a schematic diagram of the structure of a page processing apparatus provided in one embodiment of this specification; Figure 9 This is a schematic diagram of the structure of a page processing model training device provided in one embodiment of this specification; Figure 10 This is a schematic diagram of the structure of a page processing device located on a cloud-side device according to one embodiment of this specification; Figure 11 This is a schematic diagram of the structure of a page processing model training device located on a cloud-side device, according to one embodiment of this specification. Figure 12 This is an architecture diagram of a page processing system provided in one embodiment of this specification; Figure 13 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0027] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0028] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0031] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0032] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0033] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0034] First Contentful Paint (FCP): refers to the point in time when the browser renders the initial DOM elements or text content.

[0035] Largest Contentful Paint (LCP): refers to the point in time when the largest image or text block within the visible area is rendered.

[0036] Time to Interactive (TTI): This refers to the point in time when a page becomes fully interactive, typically when the main thread is idle and all important resources have been loaded.

[0037] User experience (UX) refers to the feelings and reactions of users when using a product, system, or service.

[0038] VideoMAEv2: A video understanding backbone network based on the Transformer architecture. It is pre-trained on a large-scale video dataset through self-supervised learning and can effectively extract the spatiotemporal features of video frame sequences.

[0039] ActionFormer: A temporal action localization network based on the Transformer architecture, capable of accurately identifying the start and end time boundaries of a specific action (or event) from the feature sequence of a video.

[0040] Transformer: A deep learning architecture initially used for natural language processing and later widely applied in fields such as computer vision. Its core is the self-attention mechanism, which can capture long-distance dependencies in sequences.

[0041] Action Localization: Identifying the start and end times of a specific action in a video, and usually also identifying the type of action.

[0042] As described in the background section, users have increasingly higher demands for webpage loading speed and user experience. Webpage loading performance is a crucial factor in measuring webpage loading speed and influencing user satisfaction. However, traditional network-level events such as DOM events and Load events, or browser-level rendering events such as FCP and LCP, all differ from the user's actual visual experience.

[0043] For example, for FCP, the time point corresponding to the loading of a blank block on the page is used to evaluate the page loading performance. However, the blank block is just a skeleton screen or placeholder, which is not the core content that users care about or the content that users expect to see. Therefore, the time point determined by FCP to evaluate the page loading performance is inaccurate, thus affecting the user experience.

[0044] Based on this, this specification provides a page processing method that can obtain a sequence of page frames to be processed, the sequence of page frames to be processed including a sequence of page images corresponding to the target page during the loading process; input the sequence of page frames to be processed into a target page processing model to obtain a target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

[0045] The page processing method provided in this manual determines the target time interval corresponding to the page frame sequence to be processed through a page processing model, improving the efficiency of identifying the start and end time points of the page loading process and the accuracy of the identified target time interval. Furthermore, by identifying the start and end points of the page loading process in the time dimension from the page image, from a visual perspective, the determined target time interval aligns with the user's actual visual perception. This allows for precise optimization of page loading performance based on the target time interval to enhance the user experience.

[0046] This specification provides a page processing method, a page processing model training method, a page processing method applied to cloud-side devices, a page processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0047] See Figure 1 , Figure 1 A flowchart of a page processing method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0048] Step 102: Obtain the page frame sequence to be processed, wherein the page frame sequence to be processed includes the page image sequence corresponding to the target page during the loading process.

[0049] The subject executing the technical solution in this specification can be any computing device with computing capabilities, such as a client, server, etc. This specification does not impose any restrictions on this.

[0050] In one or more embodiments of this specification, the computing device can acquire a sequence of page frames to be processed. This sequence of page frames includes a sequence of page images corresponding to the target page during its loading process. Alternatively, the sequence of page frames to be processed includes a sequence of page images generated by video recording during the loading process of the target page when the target page access request is initiated.

[0051] It should be noted that the target page can be a page corresponding to the software application, such as the homepage of an e-commerce platform, the details page of any product on an e-commerce platform, the payment confirmation page, etc., or the recommendation page and playback page of a video application, etc. The target page can also be a website webpage, such as an HTML page loaded in a browser and a web application page, etc. This manual does not limit the specific type of the target page.

[0052] In practical applications, users can initiate a target page access request through a computing device. The methods for triggering a page access request, or in other words, the ways in which a user initiates a page access request, include, but are not limited to, clicking a Uniform Resource Locator (URL) link, clicking the corresponding icon of the software application, operating navigation controls within the page such as buttons and hyperlinks, voice commands, etc. This specification does not restrict the method of initiating a target page access request.

[0053] It should be noted that the page frame sequence to be processed is obtained from page videos recorded by the e-commerce platform, video application, social platform, etc., from the moment the page access request is initiated or the navigation is initiated. The page video, or page frame sequence, reflects the dynamic loading process of the page from its initial state to its final state. In practical applications, the page frame sequence can be a sequence of screen images continuously captured at a preset frame rate during the page loading process, starting from when the user initiates navigation or page access request. This page frame sequence is used to record the dynamic changes in the page's visual content. Alternatively, the page frame sequence can be a sequence of images obtained by recording from when the user initiates navigation or page access request until the page is fully loaded, that is, until all page elements are fully loaded, reflecting the continuous changes in the page's visual appearance.

[0054] The frame rate of the page frame sequence can be set according to actual needs, such as 30 frames per second, 60 frames per second, etc.

[0055] It should be noted that each page frame in the sequence of page frames to be processed corresponds to a timestamp. The timestamp for each page frame in the sequence can be obtained when acquiring the sequence, as mentioned earlier, by recording a video of screen changes. In practical applications, automated testing tools can also be used to extract each page frame, thereby obtaining the sequence of page frames to be processed and the timestamp for each page frame.

[0056] Collecting the sequence of page frames to be processed prepares data for subsequent location of the target time interval. By using automated tools to collect the timestamps corresponding to the page frames to be processed, the accuracy of the subsequently determined target time interval can be improved.

[0057] Step 104: Input the sequence of page frames to be processed into the target page processing model to obtain the target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

[0058] In one or more embodiments of this specification, the target page processing model is a pre-trained model. Specifically, the target page processing model may be obtained by pre-training an initial target page processing model based on a sample page frame sequence and the corresponding label time interval. The label time interval is the start and end time interval of the pre-labeled sample page loading process.

[0059] It should be noted that the time interval of the label, that is, the start and end interval of the loading process of the sample page in the time dimension, can be set according to actual needs. The start and end interval of the label should be able to reflect the loading process of the sample page in a way that is perceptible to the user, that is, the pre-marked start and end interval should conform to the time interval that the user's actual visual perception is consistent with.

[0060] When training the initial page processing model, by collecting time intervals that correspond to the user's actual visual perception as labels for training, the trained target page processing model can infer and predict the time intervals corresponding to the page loading process that correspond to the user's actual visual perception. In other words, the target page processing model can be used to locate the start and end points of the target page loading process in the time dimension to obtain the target time interval. This can improve the accuracy of page performance analysis through the target time interval, accurately analyze and improve the page loading performance of the page frame sequence to be processed, and improve the user experience.

[0061] In one or more embodiments of this specification, two key moments can be determined when evaluating webpage loading performance: the first is the start of page loading, which is the time when the user first perceives a page response after initiating a page access request; the second is the moment when the core content of the page is loaded, i.e., the end of page loading, which is when the main information of the page has been presented, or in other words, when the page elements that fulfill the page's functions have been successfully rendered and are available for the user to read or operate. These two key moments allow for effective evaluation of page loading performance and user experience.

[0062] Therefore, in one or more embodiments of this specification, the target time interval includes the start time of target page loading when the page element first appears, and the end time of target page loading when the page element that satisfies the page function of the target page is successfully rendered.

[0063] In one or more embodiments of this specification, the two key moments can be determined by a target page processing model. Specifically, the sequence of page frames to be processed can be input into the target page processing model, which can locate the start and end points of the target page loading process in the time dimension. These start and end points in the time dimension correspond to the target time interval. That is, the start and end points in the time dimension include the start time and the end time of the target page loading, and these two moments constitute the target time interval, thereby obtaining the target time interval.

[0064] Corresponding to the two key moments mentioned above, this target time interval includes the start time of target page loading when the first page element appears, and the end time of target page loading when the elements that fulfill the page's functions are successfully rendered. In other words, during the target page loading process, the moment when the first page element appears is the start time of target page loading, and the moment when the page elements that fulfill the page's functions are rendered is the end time of target page loading. As mentioned earlier, for page loading, users cannot perceive or pay attention to when skeleton screens or placeholders appear during the page loading process. The core content that users care about, or rather, the content they do not expect to see, is when page elements appear during the page loading process, especially elements related to the page's functions. Therefore, the target time interval in this specification is a time interval that corresponds to the user's actual visual perception. This time interval, which corresponds to the user's actual visual perception, starts from the first appearance of the page element and ends when the rendering of the page elements that fulfill the page's functions is completed.

[0065] The page elements that fulfill the functionality of the target page can be determined based on the specific type of the target page. For example, if the target page is a payment confirmation page on an e-commerce platform, then the page element fulfilling the target page's functionality would be a payment control. If the target page is a playback page in a video application, then the page element fulfilling the target page's functionality would be a playback control. If the target page is a document page loaded by a browser, then the page elements fulfilling the target page's functionality could be a specified number of document text elements. In short, the target page provides page functionality, and page elements are the elements that provide that functionality; the specific page elements that fulfill the target page's functionality can be determined based on the actual situation.

[0066] The moment when a page element first appears is the moment when a page element first appears on the target page, that is, the moment when the target page changes from a blank screen to the moment when a page element appears. This page element can include controls, characters, banners, etc.

[0067] It should be understood that the target time interval, which consists of the start time and the end time of the target page loading, is a part of the time interval corresponding to the sequence of page frames to be processed, or a subset of the time interval corresponding to the sequence of page frames to be processed.

[0068] Based on the above page processing method, a page frame sequence to be processed can be obtained, which includes a page image sequence corresponding to the target page during the loading process; the page frame sequence to be processed is input into the target page processing model to obtain the target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

[0069] The aforementioned page processing method determines the target time interval corresponding to the page frame sequence to be processed through a page processing model, improving the efficiency of identifying the start and end time points of the page loading process and the accuracy of the identified target time interval. Furthermore, by identifying the start and end points of the page loading process in the time dimension from the page image, from a visual perspective, the determined target time interval aligns with the user's actual visual perception. This allows for precise optimization of page loading performance based on the target time interval, thereby enhancing the user experience.

[0070] In practical applications, after obtaining the target time interval, page loading performance can be evaluated based on it. Specifically, a page loading performance report can be generated and visualized based on the target time interval. Trend analysis and warnings can also be performed based on the target time interval. For example, the baseline start and end pages expected by the user can be obtained to compare page elements of the first and second page frames corresponding to the target time interval, thereby analyzing whether the determined first and second page frames meet the user's expectations. Another example is obtaining the user's expected baseline loading time difference, comparing the time difference corresponding to the target time interval (i.e., the time difference between the start and end times of the target page loading) with the baseline loading time difference, thereby analyzing whether the determined target time interval meets the user's expectations, and so on. Furthermore, the stability of loading between the first and second page frames can be analyzed to improve the user experience. In summary, in one or more embodiments of this specification, how to evaluate page loading performance based on a target time interval to improve the user experience can be determined based on the actual situation.

[0071] In one or more embodiments of this specification, after obtaining the target time interval, the computing device can determine the first page frame and the second page frame corresponding to the target time interval from the sequence of page frames to be processed. Specifically, the page frame corresponding to the start time of target page loading can be determined as the first page frame, and the page frame corresponding to the end time of target page loading can be determined as the second page frame. Here, the first page frame refers to the page frame where the page element first appears, and the second page frame refers to the page frame where the page element that satisfies the page function of the target page is successfully rendered.

[0072] In one or more embodiments described herein, after obtaining the target time interval corresponding to the target page loading process, a first page frame and a second page frame can be determined from the sequence of page frames to be processed based on the target time interval. Specifically, the page frame corresponding to the start time of target page loading can be determined as the first page frame, and the page frame corresponding to the end time of target page loading can be determined as the second page frame from the sequence of page frames to be processed.

[0073] It should be understood that the start time of the target page loading may not necessarily be a page frame with the same start time in the sequence of page frames to be processed. Page frames whose difference from the start time of the target page loading is within a preset range can be used as the first page frame. Similarly, the end time of the target page loading may not necessarily be a page frame with the same end time in the sequence of page frames to be processed. Page frames whose difference from the end time of the target page loading is within a preset range can also be used as the second page frame.

[0074] In other words, in one or more embodiments of this specification, based on the timestamp of each page frame in the sequence of page frames to be processed, page frames whose difference from the start time of loading the target page is within a first preset difference can be determined as first page frames, and page frames whose difference from the end time of loading the target page is within a second preset difference can be determined as second page frames. The first and second preset differences can be the same or different, and can be set based on actual needs.

[0075] In practical applications, when determining the first and second page frames in the sequence of page frames to be processed based on the target time interval, the nearest matching method or linear interpolation algorithm can be used, etc.

[0076] In one or more embodiments of this specification, page loading performance can be evaluated based on the first page frame and the second page frame.

[0077] Corresponding to the aforementioned performance evaluation based on the target time interval, in practical applications, a page loading performance report can be generated and visualized based on the first and second page frames. Trend analysis and warnings can also be performed based on the first and second page frames. For example, the baseline loading start page and baseline loading end page that the user expects to see can be obtained for comparing page elements between the first and second page frames, thereby analyzing whether the determined first and second page frames meet the user's expectations. Furthermore, the stability of loading between the first and second page frames can be analyzed, thereby improving the user experience. In summary, in one or more embodiments of this specification, how to evaluate page loading performance based on the first and second page frames to improve the user experience can be determined based on the actual situation.

[0078] In one or more embodiments of this specification, the page processing model may include a spatiotemporal feature extraction network and a result prediction network.

[0079] In step 104 above, the sequence of page frames to be processed is input into the target page processing model to obtain the target time interval, including: The sequence of page frames to be processed is input into the spatiotemporal feature extraction network to obtain the spatiotemporal features corresponding to the sequence of page frames to be processed. The spatiotemporal features are input into the result prediction network to obtain the target time interval.

[0080] In one or more embodiments of this specification, before inputting the sequence of page frames to be processed into the target page processing model, video data preprocessing may be performed on the sequence of page frames to be processed. Specifically, the size of each page frame in the sequence of page frames to be processed may be normalized so that the size of each page frame conforms to the input size of the target page processing model, such as 224×224 pixels. Of course, color space conversion and pixel value normalization may also be performed to meet the input format requirements of the target page processing model.

[0081] The spatiotemporal feature extraction network extracts spatiotemporal features to characterize the dynamic changes of page content over time. The result prediction network can be used to locate the start time of page loading when a page element first appears during the page loading process, and to locate the end time of page loading when page elements that satisfy the page functions of the target page are successfully rendered, i.e., to locate the corresponding time interval of the page loading process.

[0082] In one or more embodiments of this specification, the spatiotemporal feature extraction network may be VideoMAEv2. VideoMAEv2 is a Transformer-based video model. It is obtained through self-supervised pre-training on a video dataset such as Kinetics710 and possesses spatiotemporal feature representation capabilities, effectively capturing motion, texture, and complex semantic information in the video (page frame sequence). The working principle of VideoMAEv2 is to decompose the input page frame sequence into several non-overlapping spatiotemporal block sequences, process these sequences using a Transformer encoder, and utilize a self-attention mechanism to capture long-distance spatiotemporal dependencies between spatiotemporal blocks, thereby outputting a high-dimensional, semantically rich spatiotemporal feature sequence. Each feature vector in the spatiotemporal feature sequence represents the visual information of a certain region within a specific time period in the page frame sequence. In one or more embodiments of this specification, VideoMAEv2 can extract core visual features corresponding to user visual perception from the page frame sequence, such as changes in page elements, the stability of the page layout, and the readability of text and images on the page.

[0083] In one or more embodiments of this specification, the processing of input data by VideoMAEv2 can be specifically represented by Formula 1.

[0084] Formula 1 Where F represents the spatiotemporal features output by the VideoMAEv2 network, This indicates the processing of the spatiotemporal feature extraction network. This indicates the sequence of page frames to be processed.

[0085] The aforementioned spatiotemporal feature extraction network in the target page processing model can extract spatiotemporal features that reflect changes in the user's actual visual perception, thereby improving the accuracy of the subsequently determined target time interval.

[0086] Furthermore, in one or more embodiments of this specification, the result prediction network includes a multi-scale feature extraction subnetwork and a prediction subnetwork; The spatiotemporal features are input into the result prediction network to obtain the target time interval, including: The spatiotemporal features are input into the multi-scale feature extraction subnet to obtain features at multiple scales. The multiple scale features are respectively input into the prediction subnet to obtain the time interval corresponding to each scale feature; The target time interval is determined from the plurality of time intervals.

[0087] like Figure 2 As shown, Figure 2This is a schematic diagram of the structure of a page processing model provided in one embodiment of this specification. As can be seen, the page processing model includes a spatiotemporal feature extraction network and a result prediction network. The result prediction network includes a multi-scale feature extraction subnetwork and a prediction subnetwork.

[0088] In one or more embodiments of this specification, when determining the target time interval from the plurality of time intervals, non-maximum suppression (NMS) or fractional threshold filtering algorithms may be used, and this specification does not impose specific limitations on this.

[0089] By setting up a multi-scale feature extraction subnet, the accuracy of the determined target time interval can be improved.

[0090] Furthermore, in one or more embodiments of this specification, the result prediction network further includes a feature mapping subnetwork; The spatiotemporal features are input into the multi-scale feature extraction subnetwork to obtain multiple scale features, including: The spatiotemporal features are input into the feature mapping subnet to obtain the mapped features; The mapped features are input into the multi-scale feature extraction subnet to obtain features at multiple scales.

[0091] like Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of a page processing model provided in one embodiment of this specification. As can be seen, the page processing model includes a spatiotemporal feature extraction network and a result prediction network. The result prediction network includes a feature mapping subnetwork, a multi-scale feature extraction subnetwork, and a prediction subnetwork.

[0092] In practical applications, the processed page frame sequence can be input into a spatiotemporal feature extraction network to obtain spatiotemporal features. These spatiotemporal features can then be input into a feature mapping subnetwork to obtain mapped features, and these mapped features can be input into a multi-scale feature extraction subnetwork to obtain multiple scale features. Next, these multiple scale features can be input into a prediction subnetwork to obtain the time interval corresponding to each scale feature, thereby allowing the determination of the target time interval from multiple time intervals.

[0093] In one or more embodiments of this specification, the result prediction network, including the feature mapping subnetwork, the multi-scale feature extraction subnetwork, and the prediction subnetwork, can employ the ActionFormer temporal action localization network. ActionFormer can be connected to the aforementioned VideoMAEv2. ActionFormer is a high-efficiency Transformer-based temporal action localization network that can directly predict the start and end times of actions or events from temporal feature sequences. In this specification, the predicted action or time is the page loading process, which includes multiple page loading subprocesses, each corresponding to a time interval.

[0094] In one or more embodiments of this specification, inputting spatiotemporal features into ActionFormer yields several time intervals, from which a target time interval can be determined. The acquisition of several time intervals based on ActionFormer can be expressed by Formula 2, as shown below: Formula 2 in, It consists of multiple time intervals. This is for handling ActionFormer.

[0095] In one or more embodiments of this specification, ActionFormer receives the spatiotemporal features output by VideoMAEv2 as input, and then obtains the mapped features through the feature mapping subnet, which can be specifically represented by Formula 3.

[0096] Formula 3 in, F represents the spatiotemporal features output by VideoMAEv2, Conv1D(.) represents a one-dimensional convolution operation, and ReLU(.) is the activation function. The activation function can introduce nonlinearity to enhance the feature representation capability of the page processing model.

[0097] In one or more embodiments of this specification, the multi-scale feature extraction subnetwork can be a Feature Pyramid Network (FPN). The FPN can predict action boundaries at different scales, thereby obtaining features at multiple scales. In this specification, the mapped features can be downsampled using an FPN comprising six feature layers, as specifically represented by Equation 4.

[0098] Formula 4 in, This represents the feature generated in the s-th feature layer of the FPN. Indicates FPN, It is the mapping feature output by the aforementioned feature mapping subnet.

[0099] In one or more embodiments of this specification, after acquiring multiple scale features, the time interval corresponding to each scale feature can be obtained through a prediction subnetwork. In ActionFormer, the prediction subnetwork can be a lightweight convolutional neural network (CNN).

[0100] Furthermore, in one or more embodiments of this specification, the prediction subnet includes a temporal regression branch; The multiple scale features are input into the prediction subnet respectively to obtain the time interval corresponding to each scale feature, including: The multiple scale features are input into the time series regression branch respectively to obtain the time interval corresponding to each scale feature.

[0101] In one or more embodiments of this specification, the prediction subnet includes an action classification branch; Determining the target time interval from the plurality of time intervals includes: The multiple scale features are respectively input into the action classification branch to obtain the page loading sub-process corresponding to each scale feature and the confidence level corresponding to the page loading sub-process. Based on each confidence level, determine the target page loading sub-process; The time interval corresponding to the target page loading sub-process is determined as the target time interval.

[0102] like Figure 4 The diagram shown is a schematic representation of a page processing model provided in one embodiment of this specification. As can be seen, the page processing model includes a spatiotemporal feature extraction network and a result prediction network. The result prediction network includes a feature mapping subnetwork, a multi-scale feature extraction subnetwork, and a prediction subnetwork. The prediction subnetwork includes a temporal regression branch and an action classification branch.

[0103] In one or more embodiments of this specification, after obtaining multiple scale features, for each scale feature, the scale feature can be input into a temporal regression branch and an action classification branch to obtain the page loading sub-process corresponding to each scale feature output by the action classification branch and the confidence level corresponding to the page loading sub-process, and to obtain the time interval corresponding to each scale feature output by the temporal regression branch. That is, the action classification branch is used to predict the target page loading sub-process in the sequence of page frames to be processed. This target page loading sub-process is part of the target page loading process. Each target page loading sub-process corresponds to a time interval and outputs the confidence level corresponding to that target page loading sub-process. The temporal regression branch is used to predict the target time interval corresponding to the target page loading sub-process; that is, the loading process corresponding to the target time interval is the predicted target page loading sub-process.

[0104] In one or more embodiments of this specification, the prediction subnet in ActionFormer can independently predict two types of information: predicting the page loading sub-process through the action classification branch and predicting the target time interval through the temporal regression branch. This can be specifically represented by Formula 5.

[0105] Formula 5 in, This represents the confidence level of the predicted target page loading sub-process under scale feature s, where scale feature s is the feature generated by the s-th feature layer of FPN. It is the target time interval corresponding to the target page loading sub-process predicted under the scale feature s. This represents a prediction subnetwork that includes both time-series regression and action classification branches. This represents the feature generated in the s-th feature layer of the FPN, as output by the aforementioned multi-scale feature extraction subnet.

[0106] It should be noted that the temporal regression branch and action classification branch in the prediction subnet are strictly aligned in time and space. Therefore, when the temporal regression branch outputs the target page loading sub-process, the target time interval output in parallel by the action classification branch can be aligned with the target page loading sub-process, meaning the target time interval is the target time interval corresponding to the target page loading sub-process. In other words, the page processing model in this specification can determine the target page loading sub-process and the corresponding target time interval in the page frame sequence to be processed based on the page frame sequence to be processed. In one or more embodiments of this specification, to improve the accuracy of the determined target time interval in aligning with user visual perception, the start time of target page loading within the target time interval is defined as the moment when the page element first appears, and the end time of target page loading within the target time interval is defined as the moment when the page element that fulfills the page function of the target page is successfully rendered.

[0107] Furthermore, as mentioned above, in one or more embodiments of this specification, a target time interval can be selected from several time intervals using NMS or a fractional threshold filtering algorithm. Correspondingly, after the prediction subnet outputs the page loading sub-process corresponding to each scale feature and the time interval of the page loading sub-process corresponding to each scale feature, the target time interval corresponding to the target page loading sub-process can also be determined using NMS or a fractional threshold filtering algorithm.

[0108] In one or more embodiments of this specification, based on a score threshold filtering algorithm, a confidence threshold can be set. This allows for the determination of a target page loading sub-process based on each confidence level and the confidence threshold, and the determination of the time interval corresponding to the target page loading sub-process as the target time interval. Specifically, page loading sub-processes with a confidence level not less than the confidence threshold can be determined as target page loading sub-processes, thereby determining the time interval corresponding to the target page loading sub-process as the target time interval.

[0109] In one or more embodiments of this specification, when the target time interval is obtained by NMS, it can be represented by the following Formula Six.

[0110] Formula Six in, This indicates the start time of page loading in the Kth time interval. This indicates the end time of page loading for the Kth time interval. NMS(.) represents the NMS algorithm. It is the confidence level of the predicted target page loading sub-process under the scale feature s. It is the target time interval corresponding to the target page loading sub-process predicted under the scale feature s.

[0111] Based on the above, by constructing a page processing model structure and setting up a spatiotemporal feature extraction network, a feature mapping subnetwork, a multi-scale feature extraction subnetwork, a temporal regression branch, and an action classification branch, the accuracy and richness of features extracted from page frame sequences by the page processing model can be improved. This enhances the feature representation capability of the page processing model and the accuracy of the page loading sub-process and its corresponding target time interval in the model's location. Consequently, it provides an accurate target time interval for page performance analysis, improving the accuracy of page performance analysis and ultimately enhancing the user experience.

[0112] This manual also provides a method for training a page processing model, such as Figure 5 As shown, Figure 5 This is a flowchart of a page processing model training method provided in one embodiment of the present specification, which specifically includes the following steps.

[0113] Step 502: Obtain the sample page frame sequence and obtain the label time interval corresponding to the sample page frame sequence. The sample page frame sequence includes the page image sequence corresponding to the sample page during the loading process, and the label time interval is the start and end interval of the loading process of the sample page in the time dimension, which is pre-labeled.

[0114] In one or more embodiments of this specification, the sample page frame sequence may be a historical page frame sequence obtained from a database, or it may be a page frame sequence collected based on an automated testing platform by accessing pages and simulating a real network environment. This specification does not impose specific restrictions on the source of the sample page frame sequence, as long as it includes the page image sequence corresponding to the page loaded when the sample page access request is initiated, or in other words, as long as the sample page sequence includes the page sequence corresponding to the sample page during the loading process.

[0115] As mentioned earlier, the time interval of the label, which is the start and end interval of the loading process of the sample page in the time dimension, can be set according to actual needs. The start and end interval of the label should reflect that the loading process of the sample page is perceptible to the user, that is, the pre-marked start and end interval should conform to the time interval that the user's actual visual perception is consistent with.

[0116] In one or more embodiments of this specification, in order to provide a clearer start and end interval, the tag time interval includes the start time of sample page loading when the page element first appears, and the end time of sample page loading when the page element that satisfies the page function of the sample page is successfully rendered.

[0117] In practical applications, the start and end times of sample page loading in the sample page frame sequence can be pre-marked manually to obtain the labeled time intervals. The start time is the moment when a page element first appears during the sample page loading process, and the end time is the moment when a page element that fulfills the page's functionality is successfully rendered.

[0118] It should be noted that the definitions of the start and end times of page loading are consistent with those described in the aforementioned page processing methods, and will not be repeated here.

[0119] Step 504: Input the sample page frame sequence into the initial page processing model to obtain the prediction time interval, wherein the prediction time interval is obtained by the initial page processing model locating the start and end points in the time dimension during the loading process of the sample page.

[0120] The initial page processing model is an untrained page processing model, while the target page processing model is a trained page processing model. A page processing model is a model that can locate the start and end points of the page loading process in the time dimension based on the page frame sequence, generating a time interval.

[0121] Step 506: Train the initial page processing model according to the tag time interval and the prediction time interval until the model training stops, and obtain the target page processing model.

[0122] In one or more embodiments of this specification, an initial page processing model may be trained with the goal of minimizing the difference between the label time interval and the prediction time interval.

[0123] In one or more embodiments of this specification, the model training stopping conditions include, but are not limited to, the determined loss being less than a preset loss threshold, the number of iterations reaching a preset number, the number of sample page frame sequences used reaching a preset number, etc.

[0124] Additionally, as mentioned above, the initial page processing model includes a spatiotemporal feature extraction network and a result prediction network. The result prediction network includes a multi-scale feature extraction subnetwork and a prediction subnetwork. The prediction subnetwork includes a temporal regression branch and an action classification branch. In one or more embodiments of this specification, the sample page frame sequence is input into the initial page processing model to obtain the prediction time interval, including: The sample page frame sequence is input into the spatiotemporal feature extraction network to obtain the sample spatiotemporal features; The spatiotemporal features of the samples are input into the multi-scale feature extraction subnetwork to obtain multiple sample scale features; The multiple scale sample features are respectively input into the action classification branch to obtain the prediction time interval corresponding to each sample scale feature, and the multiple scale sample features are respectively input into the temporal regression branch to obtain the prediction page loading action corresponding to each sample scale feature. Accordingly, based on the label time and the prediction time, the initial page processing model is trained, including: The first loss is determined based on the label time and the prediction time; The second loss is determined based on the predicted page loading action and the sample page loading action; The initial page processing model is trained based on the first loss and the second loss.

[0125] In one or more embodiments of this specification, the first loss may be L1 Loss, and the second loss may be FocalLoss. Of course, other loss functions may also be used, depending on the actual needs.

[0126] In one or more embodiments of this specification, the first loss may be represented by Formula 7.

[0127] Formula 7 in, That's the first loss. It is the number of sample page loading sub-processes in the sample page frame sequence. These represent the start and end times of page loading in the Kth time interval corresponding to the sample page loading sub-process, respectively. These are the start and end times of the tag sample page loading within the tag time interval corresponding to the sample page loading sub-process.

[0128] In one or more embodiments of this specification, when training the initial page processing model based on the first loss and the second loss, a total loss can be calculated based on the first loss and the second loss, and the initial page processing model can be trained based on the total loss. When calculating the total loss, the sum of the first loss and the second loss can be used as the total loss, or a weighted sum of the first loss and the second loss can be used as the total loss; this specification does not impose specific limitations on this.

[0129] In one or more embodiments of this specification, the total loss may be expressed using the following Formula 8.

[0130] Formula 8 in, That's the first loss. It is the weight of the first loss. This is the second loss. As the weight of the second loss, That is the total loss.

[0131] Furthermore, it should be understood that if the page processing model also includes other network layers such as the feature mapping subnet, it can still be trained according to the above model training method. However, the data processing process in the page processing model needs to be processed through the feature mapping subnet, which will not be elaborated here.

[0132] Based on the above page processing model training method, the trained target page processing model can locate the target time interval corresponding to the page loading sub-process in the page frame sequence, thereby improving the efficiency and accuracy of locating the time interval corresponding to the page loading sub-process in the page frame sequence.

[0133] In addition, in one or more embodiments of this specification, after evaluating the webpage loading performance based on the first page frame and the second page frame or the target time interval, and optimizing the application or website hosted on the page, the page processing model can be further optimized and retrained based on the actual application data of the optimized webpage, so as to improve the long-term accuracy and adaptability of the page processing model.

[0134] Based on the aforementioned page processing methods and page processing model training methods, we can determine the time intervals that directly reflect when users begin to perceive page loading and when they believe the core content of the page is ready. This provides a more guiding objective for page performance analysis and optimization. Furthermore, the trained target page processing model possesses strong robustness and generalization ability. Under diverse webpage loading scenarios, this target page processing model can still adapt to various complex webpage loading modes, including skeleton screens, lazy loading, and animations, and is applicable to various types of websites without requiring specific rule configurations for each website, thus significantly reducing usage and maintenance costs.

[0135] Furthermore, webpage loading performance can be evaluated, diagnosed, and optimized based on the first and second page frames and the target time interval. By using the first and second page frames and the target time interval, developers can more clearly understand where the page loading bottlenecks are, thus enabling better optimization of webpage loading.

[0136] This specification also provides a flowchart of a page processing method applied to cloud-side devices, such as... Figure 6 As shown, Figure 6 This is a flowchart of a page processing method provided in one embodiment of the present specification, applied to a cloud-side device, and specifically includes the following steps.

[0137] Step 602: Receive a page processing request sent by the receiving end device, wherein the page processing request carries a sequence of page frames to be processed, and the sequence of page frames to be processed includes a sequence of page images corresponding to the target page during the loading process.

[0138] Step 604: Input the sequence of page frames to be processed into the target page processing model to obtain the target time interval, wherein the target time interval is obtained by the target page processing model by locating the start and end points of the loading process of the target page in the time dimension.

[0139] Step 606: Send the target time interval to the end-side device.

[0140] The page processing method for cloud-side devices provided in this embodiment can obtain a sequence of page frames to be processed, which includes a sequence of page images corresponding to the target page during the loading process; the sequence of page frames to be processed is input into a target page processing model to obtain a target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

[0141] By determining the target time interval corresponding to the page frame sequence to be processed through a page processing model, the efficiency of identifying the start and end time points of the page loading process and the accuracy of the identified target time interval are improved. Furthermore, by identifying the start and end points of the page loading process in the time dimension from the page image, from a visual perspective, the determined target time interval aligns with the user's actual visual perception. This allows for precise optimization of page loading performance based on the target time interval, thereby enhancing the user experience.

[0142] It should be noted that the technical solution of the page processing method applied to cloud-side devices is based on the same concept as the technical solution of the page processing method described above. For details not described in detail in the technical solution of the page processing method applied to cloud-side devices, please refer to the description of the technical solution of the page processing method described above.

[0143] This specification also provides a flowchart of a page processing model training method applied to cloud-based devices, such as... Figure 7 As shown, Figure 7 This document provides a flowchart of a page processing model training method for cloud-based devices, which is an embodiment of the present specification and specifically includes the following steps.

[0144] Step 702: Receive a model training request sent by the receiving end device, wherein the model training request carries a sample page frame sequence and a label time interval corresponding to the sample page frame sequence, the sample page frame sequence includes a page image sequence corresponding to the sample page during the loading process, and the label time interval is the start and end interval of the loading process of the sample page in the time dimension, which is pre-labeled.

[0145] Step 704: Input the sample page frame sequence into the initial page processing model to obtain the prediction time interval, wherein the prediction time interval is obtained by the initial page processing model locating the start and end points of the loading process of the sample page in the time dimension.

[0146] Step 706: Train the initial page processing model according to the tag time interval and the prediction time interval until the model training stops, and obtain the target page processing model.

[0147] Step 708: Send the model parameters of the target page processing model to the terminal device.

[0148] The page processing model training method for cloud-side devices provided in this embodiment enables the trained target page processing model to locate the target time interval corresponding to the page loading process from the page frame sequence, thereby improving the efficiency and accuracy of locating the target time interval corresponding to the page loading process in the page frame sequence, and the target time interval conforms to visual perception.

[0149] It should be noted that the technical solution of the page processing model training method applied to cloud-side devices is based on the same concept as the technical solution of the page processing model training method described above. For details not described in detail in the technical solution of the page processing model training method applied to cloud-side devices, please refer to the description of the technical solution of the page processing model training method described above.

[0150] Corresponding to the above method embodiments, this specification also provides embodiments of a page processing apparatus. Figure 8 A schematic diagram of the structure of a page processing apparatus according to one embodiment of this specification is shown. Figure 8 As shown, the device includes: The first acquisition module 802 is configured to acquire a sequence of page frames to be processed, wherein the sequence of page frames to be processed includes a sequence of page images corresponding to the target page during the loading process; The first input module 804 is configured to input the sequence of page frames to be processed into the target page processing model to obtain a target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

[0151] Optionally, the target time interval includes the start time of target page loading when the page element first appears, and the end time of target page loading when the page element that satisfies the page function of the target page is successfully rendered.

[0152] Optionally, the first input module 804 is further configured to determine, based on the target time interval, the page frame corresponding to the start time of the target page loading as the first page frame, and the page frame corresponding to the end time of the target page loading as the second page frame in the sequence of page frames to be processed.

[0153] Optionally, the apparatus further includes an evaluation module; The evaluation module is configured to evaluate the page loading performance corresponding to the page frame sequence to be processed based on the target time interval.

[0154] Optionally, the target page processing model includes a spatiotemporal feature extraction network and a result prediction network; The first input module 804 is further configured to input the page frame sequence to be processed into the spatiotemporal feature extraction network to obtain the spatiotemporal features corresponding to the page frame sequence to be processed; and input the spatiotemporal features into the result prediction network to obtain the target time interval.

[0155] Optionally, the result prediction network includes a multi-scale feature extraction subnetwork and a prediction subnetwork; The first input module 804 is further configured to input the spatiotemporal features into the multi-scale feature extraction subnetwork to obtain multiple scale features; input the multiple scale features into the prediction subnetwork respectively to obtain the time interval corresponding to each scale feature; and determine the target time interval from the multiple time intervals.

[0156] Optionally, the result prediction network further includes a feature mapping subnetwork; The first input module 804 is further configured to input the spatiotemporal features into the feature mapping subnet to obtain mapped features; and to input the mapped features into the multi-scale feature extraction subnet to obtain multiple scale features.

[0157] Optionally, the prediction subnet includes a time-series regression branch; The first input module 804 is further configured to input the multiple scale features into the time-series regression branch respectively to obtain the time interval corresponding to each scale feature.

[0158] Optionally, the prediction subnet includes an action classification branch; The first input module 804 is further configured to input the multiple scale features into the action classification branch respectively to obtain the page loading sub-process corresponding to each scale feature and the confidence level corresponding to the page loading sub-process; determine the target page loading sub-process based on each confidence level; and determine the time interval corresponding to the target page loading sub-process as the target time interval.

[0159] Based on the above-mentioned page processing device, a sequence of page frames to be processed can be obtained, which includes a sequence of page images corresponding to the target page during the loading process; the sequence of page frames to be processed is input into the target page processing model to obtain a target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

[0160] The aforementioned page processing device determines the target time interval corresponding to the page frame sequence to be processed through a page processing model, improving the efficiency of identifying the start and end time points of the page loading process and the accuracy of the identified target time interval. Furthermore, by identifying the start and end points of the page loading process in the time dimension from the page image, from a visual perspective, the determined target time interval aligns with the user's actual visual perception. This allows for precise evaluation and optimization of page loading performance based on the target time interval, thereby enhancing the user experience.

[0161] The above is an illustrative scheme of a page processing apparatus according to this embodiment. It should be noted that the technical solution of this page processing apparatus and the technical solution of the page processing method described above belong to the same concept. For details not described in detail in the technical solution of the page processing apparatus, please refer to the description of the technical solution of the page processing method described above.

[0162] Corresponding to the above method embodiments, this specification also provides embodiments of a page processing model training device. Figure 9 A schematic diagram of a page processing model training device according to one embodiment of this specification is shown. Figure 9 As shown, the device includes: The second acquisition module 902 is configured to acquire a sample page frame sequence and acquire the tag time interval corresponding to the sample page frame sequence, wherein the sample page frame sequence includes a page image sequence corresponding to the sample page during the loading process, and the tag time interval is the start and end interval of the loading process of the sample page in the time dimension as pre-labeled. The second input module 904 is configured to input the sample page frame sequence into the initial page processing model to obtain a prediction time interval, wherein the prediction time interval is obtained by the initial page processing model locating the start and end points in the time dimension during the loading process of the sample page. The first training module 906 is configured to train the initial page processing model based on the label time interval and the prediction time interval until the model training stops, thereby obtaining the target page processing model.

[0163] Optionally, the tag time interval includes the start time of sample page loading when the page element first appears, and the end time of sample page loading when the page element that satisfies the page function of the sample page is successfully rendered.

[0164] Optionally, the initial page processing model includes a spatiotemporal feature extraction network and a result prediction network. The result prediction network includes a multi-scale feature extraction subnetwork and a prediction subnetwork. The prediction subnetwork includes a temporal regression branch and an action classification branch. The second input module 904 is further configured to: input the sample page frame sequence into the spatiotemporal feature extraction network to obtain sample spatiotemporal features; input the sample spatiotemporal features into the multi-scale feature extraction subnetwork to obtain multiple sample scale features; input the multiple scale sample features into the action classification branch to obtain the prediction time interval corresponding to each sample scale feature; and input the multiple scale sample features into the temporal regression branch to obtain the prediction page loading action corresponding to each sample scale feature. The first training module 906 is further configured to: determine a first loss based on the label time and the prediction time; determine a second loss based on the predicted page loading action and the sample page loading action; and train the initial page processing model based on the first loss and the second loss.

[0165] Based on the above-mentioned page processing model training device, the trained target page processing model can locate the target time interval corresponding to the page loading sub-process in the page frame sequence, thereby improving the efficiency and accuracy of locating the time interval corresponding to the page loading sub-process in the page frame sequence.

[0166] The above is a schematic scheme of a page processing model training device according to this embodiment. It should be noted that the technical solution of this page processing device and the technical solution of the page processing model training method described above belong to the same concept. For details not described in detail in the technical solution of the page processing model training device, please refer to the description of the technical solution of the page processing model training method described above.

[0167] Corresponding to the above method embodiments, this specification also provides embodiments of a page processing device located on a cloud-side device. Figure 10 A schematic diagram of a page processing apparatus located on a cloud-side device according to one embodiment of this specification is shown. Figure 10 As shown, the device includes: The first receiving module 1002 is configured to receive a page processing request sent by the end-side device, wherein the page processing request carries a page frame sequence to be processed, and the page frame sequence to be processed includes a page image sequence corresponding to the target page during the loading process. The third input module 1004 is configured to input the page frame sequence to be processed into the target page processing model to obtain the target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension. The first sending module 1006 is configured to send the target time interval to the end-side device.

[0168] Based on the page processing device located on the cloud side, a sequence of page frames to be processed can be obtained, which includes a sequence of page images corresponding to the target page during the loading process. The sequence of page frames to be processed is input into the target page processing model to obtain a target time interval, wherein the target time interval is obtained by the target page processing model by locating the start and end points of the loading process of the target page in the time dimension.

[0169] The aforementioned page processing device located on the cloud side determines the target time interval corresponding to the sequence of page frames to be processed through a page processing model, improving the efficiency of identifying the start and end times of the page loading process and the accuracy of the identified target time interval. Furthermore, by identifying the start and end points of the page loading process in the time dimension from the page image, from a visual perspective, the determined target time interval aligns with the user's actual visual perception. This allows for precise evaluation and optimization of page loading performance based on the target time interval, thereby enhancing the user experience.

[0170] The above is an illustrative scheme of a page processing device located on a cloud-side device according to this embodiment. It should be noted that the technical solution of this page processing device located on a cloud-side device and the technical solution of the page processing method applied to a cloud-side device described above belong to the same concept. For details not described in detail in the technical solution of the page processing device located on a cloud-side device, please refer to the description of the technical solution of the page processing method located on a cloud-side device described above.

[0171] Corresponding to the above method embodiments, this specification also provides an embodiment of a page processing model training device located on a cloud-side device. Figure 11 This specification illustrates a schematic diagram of a page processing model training device located on a cloud-side device, according to one embodiment of the present specification. Figure 11 As shown, the device includes: The second receiving module 1102 is configured to receive a model training request sent by the end-side device, wherein the model training request carries a sample page frame sequence and a label time interval corresponding to the sample page frame sequence, the sample page frame sequence includes a page image sequence corresponding to the sample page during the loading process, and the label time interval is a pre-labeled start and end interval of the loading process of the sample page in the time dimension. The fourth input module 1104 is configured to input the sample page frame sequence into the initial page processing model to obtain a prediction time interval, wherein the prediction time interval is obtained by the initial page processing model locating the start and end points of the loading process of the sample page in the time dimension. The second training module 1106 is configured to train the initial page processing model according to the label time interval and the prediction time interval until the model training stop condition is met, and obtain the target page processing model. The second sending module 1108 is configured to send the model parameters of the target page processing model to the end device.

[0172] Based on the page processing model training device located on the cloud side, the trained target page processing model can locate the target time interval corresponding to the page loading sub-process in the page frame sequence, thereby improving the efficiency and accuracy of locating the time interval corresponding to the page loading sub-process in the page frame sequence.

[0173] The above is an illustrative scheme of a page processing model training device located on a cloud-side device according to this embodiment. It should be noted that the technical solution of this page processing model training device located on a cloud-side device and the technical solution of the page processing model training method applied to a cloud-side device described above belong to the same concept. For details not described in detail in the technical solution of the page processing model training device located on a cloud-side device, please refer to the description of the technical solution of the page processing model training method located on a cloud-side device described above.

[0174] See Figure 12 , Figure 12 This specification illustrates an architecture diagram of a page processing system according to an embodiment of the present specification. The page processing system may include a client 100 and a server 200. Client 100 is used to send a sequence of page frames to be processed to server 200, wherein the sequence of page frames to be processed includes a sequence of page images corresponding to the target page during the loading process; Server 200 is used to receive a sequence of page frames to be processed; input the sequence of page frames to be processed into a target page processing model to obtain a target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension; and send the target time interval to client 100. Client 100 is also used to receive the target time interval sent by server 200.

[0175] The solution implemented in this specification allows for the determination of the target time interval corresponding to the page frame sequence to be processed through a page processing model. This improves the efficiency of identifying the start and end times of the page loading process and enhances the accuracy of the identified target time interval. Furthermore, by identifying the start and end points of the page in the time dimension during the loading process from the page image, and from a visual perspective, the determined target time interval aligns with the user's actual visual perception. Consequently, page loading performance can be precisely optimized based on the target time interval to improve the user experience.

[0176] The page processing system may include multiple clients 100 and a server 200. Clients 100 can be referred to as edge devices, and server 200 can be referred to as cloud devices. Multiple clients 100 can establish communication connections through server 200. In the page processing scenario, server 200 is used to provide page processing services between multiple clients 100. Each client 100 can act as a sender or receiver, communicating through server 200.

[0177] Users can interact with server 200 through client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In page processing scenarios, users can publish data streams to server 200 through client 100, and server 200 can determine the target time interval based on the data stream and push the target time interval to other clients that have established communication.

[0178] In this system, client 100 and server 200 establish a connection via a network. The network provides the medium for communication between client 100 and server 200. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. Data transmitted by client 100 may need to undergo encoding, transcoding, compression, or other processing before being published to server 200.

[0179] Client 100 can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. Client 100 can be developed based on the software development kit (SDK) of the corresponding service provided by server 200, such as a real-time communication (RTC) SDK. Client 100 can be deployed on a computing device and depends on the device or certain apps on the device to run. The computing device may have a display screen and support information browsing, such as a personal mobile terminal like a mobile phone, tablet, or personal computer. Various other types of applications can also be configured on the computing device, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0180] Server 200 may include servers providing various services, such as servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It should be noted that server 200 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0181] It is worth noting that the page processing methods and page processing model training methods provided in the embodiments of this specification are generally executed by the server. However, in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the page processing methods and page processing model training methods provided in the embodiments of this specification. In other embodiments, the page processing methods and page processing model training methods provided in the embodiments of this specification may also be executed jointly by the client and the server.

[0182] Figure 13A structural block diagram of a computing device 1300 according to an embodiment of this application is shown. The components of the computing device 1300 include, but are not limited to, a memory 1310 and a processor 1320. The processor 1320 is connected to the memory 1310 via a bus 1330, and a database 1350 is used to store data.

[0183] The computing device 1300 also includes an access device 1340, which enables the computing device 1300 to communicate via one or more networks 1360. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1340 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0184] In one embodiment of this application, the aforementioned components of the computing device 1300 and Figure 13 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 13 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0185] The computing device 1300 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1300 can also be a mobile or stationary server.

[0186] The processor 1320 is used to execute the following computer program / instruction, which, when executed by the processor, implements the steps of the above-mentioned page processing method and page processing model training method.

[0187] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solutions of the page processing method and the page processing model training method described above. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solutions of the page processing method and the page processing model training method described above.

[0188] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described page processing method and page processing model training method.

[0189] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computer-readable storage medium embodiments are relatively simple in description because they are fundamentally similar to the page processing method and page processing model training method embodiments; relevant parts can be referred to in the descriptions of the page processing method and page processing model training method embodiments.

[0190] An embodiment of this specification also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described page processing method and page processing model training method.

[0191] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solutions of the page processing method and the page processing model training method described above. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solutions of the page processing method and the page processing model training method described above.

[0192] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0193] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0194] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0195] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0196] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A page processing method, comprising: Obtain the page frame sequence to be processed, wherein the page frame sequence to be processed includes the page image sequence corresponding to the target page during the loading process; The sequence of page frames to be processed is input into the target page processing model to obtain the target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

2. The method as described in claim 1, wherein the target time interval includes the start time of target page loading when the first page element appears, and the end time of target page loading when the page element that satisfies the page function of the target page is successfully rendered.

3. The method as described in claim 2, further comprising, after obtaining the target time interval: Based on the target time interval, in the sequence of page frames to be processed, the page frame corresponding to the start time of the target page loading is determined as the first page frame, and the page frame corresponding to the end time of the target page loading is determined as the second page frame.

4. A method for training a page processing model, comprising: Obtain a sample page frame sequence and obtain the label time interval corresponding to the sample page frame sequence. The sample page frame sequence includes a sequence of page images corresponding to the sample page during the loading process. The label time interval is the start and end interval of the loading process of the sample page in the time dimension, which is pre-labeled. The sample page frame sequence is input into the initial page processing model to obtain the prediction time interval, wherein the prediction time interval is obtained by the initial page processing model locating the start and end points in the time dimension during the loading process of the sample page. Based on the tag time interval and the prediction time interval, the initial page processing model is trained until the model training stops, thus obtaining the target page processing model.

5. A page processing method, applied to a cloud-side device, comprising: The receiving end-side device sends a page processing request, wherein the page processing request carries a sequence of page frames to be processed, and the sequence of page frames to be processed includes a sequence of page images corresponding to the target page during the loading process; The sequence of page frames to be processed is input into the target page processing model to obtain the target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension. The target time interval is sent to the end-side device.

6. A page processing model training method, applied to cloud-side devices, comprising: The receiving end device sends a model training request, wherein the model training request carries a sample page frame sequence and a label time interval corresponding to the sample page frame sequence. The sample page frame sequence includes a sequence of page images corresponding to the sample page during the loading process, and the label time interval is the start and end interval of the loading process of the sample page in the time dimension, which is pre-labeled. The sample page frame sequence is input into the initial page processing model to obtain the prediction time interval, wherein the prediction time interval is obtained by the initial page processing model locating the start and end points of the loading process of the sample page in the time dimension. Based on the tag time interval and the predicted time interval, train the initial page processing model until the model training stops, and obtain the target page processing model. The model parameters of the target page processing model are sent to the terminal device.

7. A page processing apparatus, comprising: The acquisition module is configured to acquire a sequence of page frames to be processed, wherein the sequence of page frames to be processed includes a sequence of page images corresponding to the target page during the loading process; The input module is configured to input the sequence of page frames to be processed into the target page processing model to obtain a target time interval, wherein the target time interval is obtained by the target page processing model locating the start and end points of the loading process of the target page in the time dimension.

8. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6.

Images