Page loading duration determination method and device and storage medium

Abstract

The invention provides a page loading time length determination method and device and a storage medium, and the method comprises the steps: carrying out the framing processing of a page loading video, and obtaining a plurality of video frames; inputting the plurality of video frames into a state classification model to obtain page loading states corresponding to the plurality of video frames; classifying the continuous video frames in the same page loading state in the plurality of video frames to obtain a plurality of page loading stages; and according to the video frames corresponding to the plurality of page loading stages, determining page loading durations corresponding to the plurality of page loading stages. The development cost and the computing resource cost of page loading duration determination are reduced, and the page loading duration determination efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a method, device and storage medium for determining page loading time. Background Art

[0002] The page loading fluency test mainly analyzes whether the page interaction process is smooth by evaluating the time consumed in each link or stage of the page loading process, that is, evaluating the page loading fluency by the page loading time.

[0003] At present, the time taken to load a page is often determined by the tracking point statistics method, thereby determining the smoothness of page loading. However, since each page loading scenario involves different page loading conditions, etc., it is necessary to customize the tracking point statistics method for each scenario and perform separate programming implementation, which results in high development costs and computing resource costs. Summary of the invention

[0004] The present application provides a method, device and storage medium for determining page loading time, which can reduce the development cost and computing resource cost of determining page loading time and improve the efficiency of determining page loading time.

[0005] In a first aspect, the present application provides a method for determining a page loading duration, the method comprising: performing frame processing on a page loading video to obtain multiple video frames; inputting the multiple video frames into a state classification model to obtain page loading states corresponding to the multiple video frames; classifying video frames with consecutive identical page loading states among the multiple video frames to obtain multiple page loading stages; and determining the page loading duration corresponding to each of the multiple page loading stages based on the video frames corresponding to each of the multiple page loading stages.

[0006] In a second aspect, the present application provides a device for determining a page loading duration, the device comprising: a frame sampling module, used to perform frame processing on a page loading video to obtain a plurality of video frames; a state classification module, used to input the plurality of video frames into a state classification model to obtain a page loading state corresponding to each of the plurality of video frames; a state classification module, used to classify video frames with consecutive identical page loading states among the plurality of video frames to obtain a plurality of page loading stages; a duration determination module, used to determine the page loading duration corresponding to each of the plurality of page loading stages based on the video frames corresponding to each of the plurality of page loading stages.

[0007] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the method in the first aspect or its various implementations.

[0008] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the method in the first aspect or its various implementations.

[0009] In a fifth aspect, the present application provides a computer program product, comprising computer program instructions, which enable a computer to execute the method in the first aspect or its various implementations.

[0010] In a sixth aspect, the present application provides a computer program, which enables a computer to execute the method in the first aspect or its various implementations.

[0011] Other contents and effects of the technical solution of this application will be explained in subsequent specific embodiments and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for describing the embodiments are briefly introduced below.

[0013] Figure 1 A flowchart of a method for determining page loading time provided in an embodiment of the present application;

[0014] Figure 2 A schematic diagram of a method for determining page loading time provided in an embodiment of the present application;

[0015] Figure 3 A schematic diagram of another method for determining page loading time provided in an embodiment of the present application;

[0016] Figure 4 A schematic diagram of another method for determining page loading time provided in an embodiment of the present application;

[0017] Figure 5 A schematic diagram of a device 500 for determining page loading duration provided in an embodiment of the present application;

[0018] Figure 6 A schematic block diagram of an electronic device 600 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0020] It should be noted that the information, data (including but not limited to data for analysis, data for storage, data for display, etc., such as page loading videos, sample page loading videos, page loading scenes) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the page loading video involved in this application and the operations and operations performed on the page loading video are all obtained with full authorization.

[0021] In one embodiment, the technical solution of the present application can be used in scenarios such as determining page loading time and testing page loading fluency. Specifically, the pages involved in the present application can be pages in applications, mini-programs, or web pages, and the present application does not impose any restrictions on this.

[0022] In one embodiment, the solution provided by the present application can be executed by any electronic device with data processing capabilities. For example, the electronic device can be a server, specifically an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. For another example, the electronic device can be a terminal device, specifically a tablet computer, a laptop computer, or a desktop computer. For another example, the electronic device can be a combination of a server and a terminal device, wherein the server and the terminal device in the combination can communicate wirelessly or wired, and the present application does not impose specific restrictions on the electronic device.

[0023] Figure 1 This is a flow chart of a method for determining page loading time provided in an embodiment of the present application. The method can be executed by the above electronic device, but is not limited thereto. Figure 1 As shown, the method may include the following steps:

[0024] S110: performing frame processing on the page loaded video to obtain multiple video frames;

[0025] S120: Inputting the multiple video frames into the state classification model to obtain the page loading states corresponding to the multiple video frames;

[0026] S130: Classifying the video frames in the same page loading state among the multiple video frames to obtain multiple page loading stages;

[0027] S140: Determine page loading durations corresponding to each of the multiple page loading stages according to the video frames corresponding to each of the multiple page loading stages.

[0028] In one embodiment, before S110, the electronic device may first determine a video recording script; thereafter, in response to the start of a page loading process corresponding to the page loading video, the electronic device may record the page loading process corresponding to the page loading process through the video recording script to obtain a page loading video.

[0029] For example, a video recording script may define a fixed operation path, an operation for triggering the start of recording, an operation for triggering the end of recording, and recording parameters, etc. Among them, the fixed operation path may be the screen display path corresponding to the page loading video; the operation for triggering the start of recording may be the start operation of the page loading process; the operation for triggering the end of recording may be the end operation of the page loading process; the recording parameters may include the resolution, frame rate, and video format of the page loading video to be recorded.

[0030] Exemplarily, an electronic device may integrate a video recording script and establish a communication connection with a playback device that plays a page loading video; thereafter, in response to the playback device starting a page loading process corresponding to the page loading video, the page loading process is used to display the page corresponding to the page loading video, that is, play the screen corresponding to the page loading video; the electronic device may start a recording operation through the video recording script, record the page loading process corresponding to the page loading process, and in response to the playback device ending the page loading process, end the recording operation through the video recording script to obtain a page loading video, and the parameters such as resolution, frame rate and video format corresponding to the page loading video are consistent with the recording parameters defined by the video recording script.

[0031] For example, a page loading video can be a video recording the page loading situation during the application startup process, which can include all page changes from triggering the application startup to the complete presentation of the application's main interface. Correspondingly, a page loading process can refer to the process from triggering the application startup to the complete presentation of the application's main interface.

[0032] In one embodiment, the electronic device can perform frame processing on the page loading video based on a set frame rate through a written script or preset video processing software to obtain multiple video frames.

[0033] Among them, the electronic device can first determine the loading time accuracy, and determine that the set frame rate is positively correlated with the loading time accuracy. That is to say, when a higher loading time accuracy is required, a higher set frame rate can be used to obtain a larger number of video frames, thereby meeting the demand for higher loading time accuracy; or, when lower storage and processing costs are required, a lower set frame rate can be used to obtain a suitable number of video frames, thereby improving data processing efficiency and saving loading time statistics costs.

[0034] For example, assuming that the page loads a video for 1 second, if the loading time accuracy is 33.33 milliseconds, that is, the time interval between two video frames is 33.33 milliseconds, then it can be determined that the set frame rate is 30FPS, that is, 30 frames per second; if the loading time accuracy is 1 millisecond, that is, the time interval between two video frames is 1 millisecond, then it can be determined that the set frame rate is 1000FPS, that is, 1000 frames per second.

[0035] In one embodiment, before inputting the multiple video frames into the state classification model, the electronic device may also pre-process the multiple video frames through a digital image processing scheme, for example, normalizing the sizes of the multiple video frames (for example, normalizing the sizes of the video frames to 224*224), so as to ensure that the sizes of the video frames (for example, size or resolution, etc.) meet the requirements of the state classification model for the video frames, and avoid the influence of video frames of different sizes on the accuracy of state classification.

[0036] The digital image processing scheme may be a scaling algorithm, such as bilinear interpolation, bicubic interpolation, but is not limited thereto.

[0037] The present application will introduce the page loading state, page loading stage and state classification model in the following embodiments, wherein the content and effect corresponding to the training method of the state classification model can be referenced with the content and effect corresponding to the application process of the state classification model (for example, the page loading state in S120 can be determined by using the method for determining the actual loading state in the training method), and the present application will not elaborate on this.

[0038] It should be noted that the page loading state may refer to the loading state of the video frame in the page loading video (i.e., the video page in the page loading video). Specifically, the page loading state includes two types: stable loading state and unstable loading state. Among them, the stable loading state refers to the state where the corresponding page has been rendered, and the page content in the corresponding video frame is relatively complete; the unstable loading state refers to the state where the corresponding page has not been rendered (i.e., the state in the page rendering process), and the page content in the corresponding video frame is incomplete.

[0039] The consecutive video frames in the same page loading state among the multiple video frames correspond to the same page loading stage.

[0040] Exemplarily, assuming that the page loading video includes page 1, the complete content of page 1 includes text 1 and image 1, after the multi-page loading video is framed, page 1 corresponds to 5 video frames: video frame 1, video frame 2, video frame 3, video frame 4, video frame 5; wherein, video frame 1 displays the first part of the content of text 1, video frame 2 displays the second part of the content of text 1, video frame 3 displays the entire content of text 1, video frame 4 displays text 1 and image 1, and video frame 5 displays text 1 and image 1; if the video loading status corresponding to video frame 1 and video frame 2 is a stable loading status, the video loading status corresponding to video frame 3 is an unstable loading status, and the video loading status corresponding to video frame 4 and video frame 5 is a stable loading status, then it can be determined that video frame 1 and video frame 2 correspond to the same page loading stage, video frame 3 corresponds to the same page loading stage, video frame 4 and video frame 5 correspond to the same page loading stage, and 3 page loading stages are obtained.

[0041] In one embodiment, the electronic device can perform frame processing on the sample page loading video to obtain multiple sample video frames; calculate the inter-frame similarity of each pair of adjacent sample video frames in the multiple sample video frames; determine the actual page loading status corresponding to each of the multiple sample video frames based on the inter-frame similarity of each pair of adjacent sample video frames; and train a state classification model based on the multiple sample video frames and the multiple actual page loading statuses.

[0042] Exemplarily, the above-mentioned calculation of the inter-frame similarity of each pair of adjacent sample video frames in multiple sample video frames may include: calculating the inter-frame difference index of each pair of adjacent sample video frames; determining the inter-frame similarity of each pair of adjacent sample video frames according to the inter-frame difference index of each pair of adjacent sample video frames; wherein the inter-frame difference index includes at least one of the following: Structural Similarity Index (SSIM), Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR).

[0043] Among them, the inter-frame difference index can be used to indicate the degree of difference between frames. For example, the structural similarity index can consider the similarity of the brightness, contrast and structure of the image. The value range is between -1 and 1. The closer the value is to 1, the smaller the difference between the two video frames. The mean square error can calculate the average of the squares of the difference in pixel values ​​between the two video frames. The smaller the value, the smaller the difference between the two images. The peak signal-to-noise ratio indicates the ratio of the maximum possible power of the image signal to the noise power. The higher the value, the better the image quality and the smaller the difference between the two images.

[0044] For example, if the inter-frame difference index includes one of the above items, the inter-frame difference index can be directly determined as the inter-frame similarity; if the inter-frame difference index includes multiple items, the maximum value, minimum value, median value or average value of the multiple items can be determined as the inter-frame similarity.

[0045] Exemplarily, the above-mentioned determination of the actual page loading status corresponding to each of the plurality of sample video frames according to the inter-frame similarity of each pair of adjacent sample video frames may be implemented in any of the following ways:

[0046] Method 1: determine the first inter-frame similarity that is greater than or equal to the first similarity threshold value among the inter-frame similarities of each pair of adjacent sample video frames; determine at least one pair of adjacent sample video frames corresponding to the first inter-frame similarity among multiple sample video frames to obtain a target sample video frame; determine the actual page loading state of the target sample video frame, which is an actual stable loading state; determine the actual page loading state of other sample video frames except the target sample video frame among multiple sample video frames, which is an actual unstable loading state.

[0047] For example, suppose there are 5 video frames: video frame 1, video frame 2, video frame 3, video frame 4, and video frame 5, among which the inter-frame similarities between video frame 1 and video frame 2, video frame 2 and video frame 3, video frame 3 and video frame 4, and video frame 4 and video frame 5 are 90%, 70%, 70%, and 95% respectively, and the first similarity threshold is 90%; then it can be determined that at least one pair of adjacent sample video frames corresponding to the first inter-frame similarity includes: video frame 1 and video frame 2, video frame 4 and video frame 5; thereby determining that the target sample video frames include: video frame 1, video frame 2, video frame 4, and video frame 5, and determining that the actual page loading status of video frame 1, video frame 2, video frame 4, and video frame 5 is an actual stable loading status, and determining that the actual page loading status of video frame 3 is an actual unstable loading status.

[0048] Alternatively, a second inter-frame similarity that is less than a first similarity threshold value among the inter-frame similarities of each pair of adjacent sample video frames may be determined; at least one pair of adjacent sample video frames corresponding to the second inter-frame similarity among multiple sample video frames may be determined to obtain a second sample video frame; the actual page loading state of the target sample video frame may be determined to be an actually unstable loading state; the actual page loading state of other sample video frames among multiple sample video frames except the second sample video frame may be determined to be an actually stable loading state.

[0049] Alternatively, determine a third inter-frame similarity greater than or equal to the first similarity threshold among the inter-frame similarities of each pair of adjacent sample video frames, and determine a fourth inter-frame similarity less than the first similarity threshold among the inter-frame similarities of each pair of adjacent sample video frames; determine at least one pair of adjacent sample video frames corresponding to the third inter-frame similarity among multiple sample video frames to obtain a third sample video frame; determine at least one pair of adjacent sample video frames corresponding to the fourth inter-frame similarity among multiple sample video frames to obtain a fourth sample video frame; determine that the actual page loading state of the third sample video frame is an actual stable loading state, and determine that the actual page loading state of the fourth sample video frame is an actual unstable loading state.

[0050] Method 2: for a first specific sample video frame among the multiple sample video frames, if the inter-frame similarity between the first specific sample video frame and the second specific sample video frame among the multiple sample video frames is greater than or equal to a first inter-frame similarity threshold, then the actual page loading state of the first specific sample video frame is determined to be an actual stable loading state; if the inter-frame similarity between the first specific sample video frame and the second specific sample video frame is less than the first inter-frame similarity threshold, then the actual page loading state of the first specific sample video frame is determined to be an actual unstable loading state; wherein, if the first specific sample video frame is the first sample video frame among the multiple sample video frames, then the second specific video frame is the second sample video frame among the multiple sample video frames; if the first specific sample video frame is The frame is the last sample video frame among the multiple sample video frames, then the second specific video frame is the second to last sample video frame among the multiple sample video frames; for any single sample video frame except the first specific sample video frame among the multiple sample video frames, if the inter-frame similarities between the single sample video frame and the previous sample video frame and the next sample video frame of the single sample video frame are respectively greater than or equal to the first inter-frame similarity threshold, then the actual page loading state of the single sample video frame is determined to be the actual stable loading state; if at least one of the inter-frame similarities between the single sample video frame and the previous sample video frame and the next sample video frame of the single sample video frame is less than the first inter-frame similarity threshold, then the actual page loading state of the single sample video frame is determined to be the actual unstable loading state.

[0051] Method three, combining multiple sample video frames into a video frame set in order of sampling time from early to late; determining the actual page loading state of the first sample video frame in the video frame set as the actual first loading state; for the video frame set, determining the inter-frame similarity between the second sample video frame in the video frame set and the first sample video frame in the video frame set as the set inter-frame similarity, and comparing the set inter-frame similarity with the first inter-frame similarity threshold; if the set inter-frame similarity is greater than or equal to the first inter-frame similarity threshold, determining that the actual page loading state of the second sample video frame is the same as the actual page loading state of the first sample video frame; if the set inter-frame similarity is less than the first inter-frame similarity threshold, determining that the actual page loading state of the second sample video frame is different from the actual page loading state of the first sample video frame; removing the first sample video frame in the video frame set to obtain an updated video frame set, and based on the updated video frame set, continuing to compare the set inter-frame similarity with the first inter-frame similarity threshold until the actual page loading states corresponding to the multiple sample video frames are obtained.

[0052] For example, the actual page loading state different from the actual first loading state is the actual second loading state. The actual first loading state may be an actual stable loading state or an actual unstable loading state; the actual second loading state may be an actual unstable loading state or an actual stable loading state.

[0053] In one embodiment, when training a state classification model, the input of the state classification model may be multiple sample video frames and multiple actual page loading states (participating in training in the form of labels), and the output may be the predicted page loading states corresponding to each of the multiple sample video frames; the electronic device may train the state classification model based on the difference between the multiple actual page loading states and the multiple predicted page loading states. For example, when the above difference is less than a specific threshold or when the number of training rounds reaches a preset number, it may be determined that the training of the state classification model is completed.

[0054] Exemplarily, the state classification model can be a classification neural network model, specifically a model based on Resnet50, which is not limited in this application.

[0055] Exemplarily, the state classification model may include: (1) an input layer, which is used to receive sample video frames and actual page loading states. The actual page loading states may be data in numerical, categorical or other forms, or may be data obtained by preprocessing the actual page loading states according to encoding methods such as one-hot encoding and label encoding. (2) a feature extraction layer, which is used to extract features in video frames using components such as convolutional neural networks and recurrent neural networks. The features may be spatial features (such as edges, textures, colors, etc. in video frames) and temporal features (such as page loading speed, loading order, etc.). (3) a fusion layer, which is used to fuse features in video frames. Different features may be combined by concatenation, addition, dot product, etc. to obtain feature representations of video frames. (4) a classification layer, which is implemented by one or more fully connected layers and a softmax layer, and is used to classify according to the feature representations of video frames and calculate the probability distribution of each category (including multiple categories corresponding to the predicted page loading states, for example, predicted stable loading states and predicted unstable loading states). (5) an output layer, which outputs the category with the highest probability according to the probability distribution.

[0056] In one embodiment, the electronic device may first determine the page loading scenario corresponding to the sample page loading video; then obtain other sample page loading videos except the sample page loading video under the page loading scenario; thereafter, a state classification model may be trained based on other sample page loading videos; wherein, there is a difference in the content displayed on at least one corresponding page between the other sample page loading videos and the sample page loading video.

[0057] For example, assuming that the page loading scenario is the startup scenario of an application, the advertisements played each time in this scenario may be different, that is, the corresponding multiple sample page loading videos have at least one corresponding page (specifically, the page corresponding to the advertisement) with different displayed content. Therefore, for the same page loading scenario, the recording can be repeated multiple times to obtain multiple sample page loading videos with at least one corresponding page with different content, which can enrich the training set, ensure classification accuracy, and improve the generalization ability of the model.

[0058] In one embodiment, in combination with the above content, a processing unit corresponding to the feature of "classifying video frames with the same consecutive page loading state in multiple video frames to obtain multiple page loading stages" can be added in the processing process of the state classification model, so that the output of the state classification model can be multiple page loading stages.

[0059] In one embodiment, the above-mentioned determination of the page loading duration corresponding to each of the multiple page loading stages based on the video frames corresponding to each of the multiple page loading stages may include: determining the frame number difference between the initial video frame and the end video frame corresponding to each of the multiple page loading stages; determining the time interval between each video frame; calculating the product of each frame number difference and the time interval to obtain the page loading duration corresponding to each of the multiple page loading stages.

[0060] Exemplarily, the time interval between each video frame may be determined by a frame rate (eg, the set frame rate in the above embodiment). For example, the reciprocal of the set frame rate may be determined as the time interval.

[0061] In one embodiment, the electronic device can adopt the method in the above embodiment to classify each page loading stage through the second similarity threshold, that is, to reclassify the video frames corresponding to the page loading stage, and determine multiple page loading stages and their loading times under the page loading stage to meet the page loading time statistics at different granularities.

[0062] The technical solution of this application is introduced in the following way by means of a schematic diagram:

[0063] In one embodiment, Figure 2As shown, the electronic device can first generate a training set for training a state classification model. Specifically, the video of the specified scene loading process can be automatically recorded through the user interface (UI) video recording script of the application to obtain a sample page loading video; then, the sample page loading video is framed, and each frame process picture is obtained in time sequence to obtain multiple sample video frames; then, the multiple sample video frames can be divided into a series of "state stable" (i.e. stable loading state) and "state change" (i.e. unstable loading state) picture sets by a digital image processing method: the SSIM, MSE and PSNR values ​​between two adjacent video frames are calculated to determine the pixel difference between the two video frames. When the pixel difference between the two video frames is less than or equal to the difference threshold, it is determined to be a stable state, and the video frame set of continuous stable state can be determined to be the same stable stage (i.e. the same page loading stage); when the pixel difference between the two video frames is greater than the difference threshold, it is determined to be an unstable state, i.e. a state change, and the video frame set of continuous unstable state can be determined to be the same change stage (i.e. the same page loading stage). Thus, the actual page loading states corresponding to the multiple sample video frames can be determined as state 1, ..., state k, where k is a positive integer; then, the multiple sample video frames can be automatically labeled: labeled as Label1, Label2, ..., Labelk in the order of sampling, to obtain the classification labels of the sample video frames (the classification labels are used to mark the actual page loading states), and the mapping relationship between each sample video frame and the classification label is determined, and a training data set is constructed by combining multiple sample video frames.

[0064] In one embodiment, Figure 3 As shown, in the daily production and testing process, the electronic device can use the above script to record the page loading process video of the corresponding scene to obtain the page loading video; then, the recorded video to be tested, that is, the page recording video, is divided into frames, and each frame process picture is obtained in time sequence: the a1th frame, ..., the anth frame, and multiple video frames are obtained; then, each frame picture, that is, each video frame, is input into the classification model for classification calculation to obtain the page loading state of each of the multiple video frames; then, the pictures with the same classification results, that is, the video frames with the same page loading state, are classified into the same stage, and multiple page loading stages are obtained: stage 1, ..., stage k; finally, according to the product of the frame number difference between the start frame and the end frame of each stage and the fixed time interval between each frame, the time distribution of each stage is calculated. For example, the stage time consumption of stage 1, that is, the page loading time length is T1, ..., and the stage time consumption of stage k, that is, the page loading time length is Tk.

[0065] In one embodiment, Figure 4As shown, in combination with the above content, the technical solution of the present application may include a model training part and a model application part. Among them, the recorded sample page loading video can be divided into frames, and the inter-frame similarity can be calculated. The actual page loading state of each sample video frame can be determined according to the size of the inter-frame similarity and the similarity threshold, and the sample video frames can be marked accordingly, and then the state classification model can be trained according to the marking results and the sample video frames. Afterwards, the video frames obtained by framing can be directly input into the state classification model to obtain the page loading state and page loading stage of each video frame, and the time consumption of each stage can be determined according to the number of video frames corresponding to the page loading stage, and the time consumption distribution result can be obtained.

[0066] Through the technical solution of the present application, the electronic device can automatically record the page loading process, determine the loading state of each video frame through the state classification model, and determine the continuous same loading state as a stage. Finally, the duration is determined according to the number of video frames corresponding to each stage. The time of each link of the page loading can be accurately measured and recorded to evaluate the time distribution to determine whether the interaction process is smooth. It is also highly reusable and can not only solve the problems of high development cost, high invasiveness or labor cost of the buried point collection solution or the manual screen recording solution, but also is not affected by the differences in client systems and versions, and can better ensure the consistency of classification results under different systems or versions.

[0067] It should be noted that all the above technical solutions can be combined in any way to form optional embodiments of the present application, which will not be described one by one here.

[0068] Figure 5 A schematic diagram of a device 500 for determining page loading duration provided in an embodiment of the present application.

[0069] like Figure 5 As shown, the page loading duration determination device 500 includes: a frame sampling module 501, a state classification module 502, a state categorization module 503, a duration determination module 504, a frame processing module 505, an inter-frame processing module 506, a state determination module 507, a first training module 508, a scene determination module 509, a sample acquisition module 510, a second training module 511, a script determination module 512, and a video recording module 513.

[0070] Exemplarily, a frame sampling module 501 is used to perform frame processing on a page loading video to obtain a plurality of video frames; a state classification module 502 is used to input the plurality of video frames into a state classification model to obtain a page loading state corresponding to each of the plurality of video frames; a state classification module 503 is used to classify video frames of consecutive same page loading states among the plurality of video frames to obtain a plurality of page loading stages; a duration determination module 504 is used to determine the page loading duration corresponding to each of the plurality of page loading stages according to the video frames corresponding to each of the plurality of page loading stages.

[0071] Exemplarily, the frame processing module 505 is used to perform frame processing on the sample page loading video to obtain multiple sample video frames; the inter-frame processing module 506 is used to calculate the inter-frame similarity of each pair of adjacent sample video frames in the multiple sample video frames; the state determination module 507 is used to determine the actual page loading state corresponding to each of the multiple sample video frames according to the inter-frame similarity of each pair of adjacent sample video frames; the first training module 508 is used to train a state classification model according to the multiple sample video frames and the multiple actual page loading states.

[0072] Exemplarily, the inter-frame processing module 506 is specifically used to: calculate the inter-frame difference index of each pair of adjacent sample video frames; determine the inter-frame similarity of each pair of adjacent sample video frames based on the inter-frame difference index of each pair of adjacent sample video frames; wherein the inter-frame difference index includes at least one of the following: structural similarity index SSIM, mean square error MSE, peak signal-to-noise ratio PSNR.

[0073] Exemplarily, the state determination module 507 is specifically used to: determine a first inter-frame similarity greater than or equal to a first similarity threshold among the inter-frame similarities of each pair of adjacent sample video frames; determine at least one pair of adjacent sample video frames corresponding to the first inter-frame similarity among multiple sample video frames to obtain a target sample video frame; determine the actual page loading state of the target sample video frame, which is an actual stable loading state; determine the actual page loading state of other sample video frames among multiple sample video frames except the target sample video frame, which is an actual unstable loading state.

[0074] Exemplarily, the state determination module 507 is specifically used to: combine multiple sample video frames into a video frame set in the order of sampling time from early to late; determine that the actual page loading state of the first sample video frame in the video frame set is the actual first loading state; for the video frame set, determine the inter-frame similarity between the second sample video frame in the video frame set and the first sample video frame in the video frame set as the set inter-frame similarity, and compare the set inter-frame similarity with the first inter-frame similarity threshold; if the set inter-frame similarity is greater than or equal to the first inter-frame similarity threshold, then determine that the actual page loading state of the second sample video frame is the same as the actual page loading state of the first sample video frame; if the set inter-frame similarity is less than the first inter-frame similarity threshold, then determine that the actual page loading state of the second sample video frame is different from the actual page loading state of the first sample video frame; remove the first sample video frame in the video frame set to obtain an updated video frame set, and based on the updated video frame set, continue to compare the set inter-frame similarity with the first inter-frame similarity threshold until the actual page loading states corresponding to each of the multiple sample video frames are obtained.

[0075] Exemplarily, the state determination module 507 is specifically used to: for a first specific sample video frame among multiple sample video frames, if the inter-frame similarity between the first specific sample video frame and the second specific sample video frame among the multiple sample video frames is greater than or equal to a first inter-frame similarity threshold, then determine that the actual page loading state of the first specific sample video frame is an actual stable loading state; if the inter-frame similarity between the first specific sample video frame and the second specific sample video frame is less than the first inter-frame similarity threshold, then determine that the actual page loading state of the first specific sample video frame is an actual unstable loading state; wherein, if the first specific sample video frame is the first sample video frame among the multiple sample video frames, the second specific video frame is the second sample video frame among the multiple sample video frames; if the first specific sample video frame is the last sample video frame among multiple sample video frames, then the second specific video frame is the second to last sample video frame among the multiple sample video frames; for any single sample video frame except the first specific sample video frame among the multiple sample video frames, if the inter-frame similarities between the single sample video frame and the previous sample video frame and the next sample video frame of the single sample video frame are respectively greater than or equal to the first inter-frame similarity threshold, then the actual page loading state of the single sample video frame is determined to be an actual stable loading state; if at least one of the inter-frame similarities between the single sample video frame and the previous sample video frame and the next sample video frame of the single sample video frame is less than the first inter-frame similarity threshold, then the actual page loading state of the single sample video frame is determined to be an actual unstable loading state.

[0076] Exemplarily, the scene determination module 509 is used to determine the page loading scene corresponding to the sample page loading video; the sample acquisition module 510 is used to obtain other sample page loading videos except the sample page loading video under the page loading scene; the second training module 511 is used to train the state classification model according to other sample page loading videos; wherein, there is a difference in the content displayed on at least one corresponding page between the other sample page loading videos and the sample page loading video.

[0077] Exemplarily, the script determination module 512 is used to determine the video recording script; the video recording module 513 is used to respond to the start of the page loading process corresponding to the page loading video, record the page loading process corresponding to the page loading process through the video recording script, and obtain the page loading video.

[0078] Exemplarily, the duration determination module 504 is specifically used to: determine the frame difference between the initial video frame and the end video frame corresponding to each of the multiple page loading stages; determine the time interval between each video frame; calculate the product of each frame difference and the time interval to obtain the page loading duration corresponding to each of the multiple page loading stages.

[0079] It should be understood that the embodiments of the device are similar to the embodiments of the above method, and the content and effects thereof can refer to the content and effects of the above method for determining the page loading time, which will not be described in detail in this application. Figure 5 The device 500 shown can execute the above method embodiments, and the above and other operations and / or functions of each module in the device 500 are respectively for implementing the corresponding processes in the above methods, which will not be repeated here for the sake of brevity.

[0080] The above describes the device 500 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0081] Figure 6 A schematic block diagram of an electronic device 600 provided in an embodiment of the present application.

[0082] like Figure 6 As shown, the electronic device 600 may include: a memory 610 and a processor 620, wherein the memory 610 is used to store a computer program and transmit the program code to the processor 620. In other words, the processor 620 may call and run the computer program from the memory 610 to implement the method in the embodiment of the present application.

[0083] For example, the processor 620 may be configured to execute the above method embodiments according to instructions in the computer program.

[0084] In some embodiments of the present application, the processor 620 may include but is not limited to:

[0085] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0086] In some embodiments of the present application, the memory 610 includes, but is not limited to, volatile memory and / or non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SynchLink DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

[0087] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 610 and executed by the processor 620 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0088] like Figure 6 As shown, the electronic device may further include: a transceiver 630 , which may be connected to the processor 620 or the memory 610 .

[0089] The processor 620 may control the transceiver 630 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 630 may include a transmitter and a receiver. The transceiver 630 may further include an antenna, and the number of antennas may be one or more.

[0090] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0091] The present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0092] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instruction is loaded and executed on a computer, the computer can be made to perform the corresponding flow in each method in the embodiment of the present application in whole or in part, and generate the functions that can be realized by each method in the embodiment of the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instruction can be transmitted from a website site, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrations. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc. Those skilled in the art may appreciate that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein may be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0093] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the system, device or module can be electrical, mechanical or other forms.

[0094] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

Claims

1. A method for determining page loading time, characterized in that: include: Perform frame processing on the page loading video to obtain multiple video frames; Inputting the plurality of video frames into a state classification model to obtain page loading states corresponding to the plurality of video frames; Classifying the video frames in the same page loading state among the multiple video frames to obtain multiple page loading stages; The page loading durations corresponding to the multiple page loading stages are determined according to the video frames corresponding to the multiple page loading stages.

2. The method according to claim 1, characterized in that Before inputting the plurality of video frames into the state classification model to obtain the page loading states corresponding to the plurality of video frames, the method further includes: Perform frame processing on the sample page loading video to obtain multiple sample video frames; Calculating the inter-frame similarity between each pair of adjacent sample video frames in the plurality of sample video frames; Determining actual page loading states corresponding to each of the plurality of sample video frames according to the inter-frame similarity of each pair of adjacent sample video frames; The state classification model is trained according to the multiple sample video frames and the multiple actual page loading states.

3. The method according to claim 2, characterized in that The calculating the inter-frame similarity of each pair of adjacent sample video frames in the plurality of sample video frames comprises: Calculating an inter-frame difference index of each pair of adjacent sample video frames; Determining the inter-frame similarity of each pair of adjacent sample video frames according to the inter-frame difference index of each pair of adjacent sample video frames; The inter-frame difference index includes at least one of the following: structural similarity index SSIM, mean square error MSE, and peak signal-to-noise ratio PSNR.

4. The method according to claim 2, characterized in that: The determining, according to the inter-frame similarity of each pair of adjacent sample video frames, the actual page loading states corresponding to each of the plurality of sample video frames comprises: Determine a first inter-frame similarity among the inter-frame similarities of each pair of adjacent sample video frames that is greater than or equal to a first similarity threshold; Determine at least one pair of adjacent sample video frames corresponding to the first inter-frame similarity among the multiple sample video frames to obtain a target sample video frame; Determining that the actual page loading state of the target sample video frame is an actual stable loading state; It is determined that the actual page loading state of other sample video frames among the multiple sample video frames except the target sample video frame is an actual unstable loading state.

5. The method according to claim 2, characterized in that: The determining, according to the inter-frame similarity of each pair of adjacent sample video frames, the actual page loading states corresponding to each of the plurality of sample video frames comprises: Combining the plurality of sample video frames into a video frame set in the order of sampling time from earliest to latest; Determining that the actual page loading state of the first sample video frame in the video frame set is the actual first loading state; For the video frame set, determining the inter-frame similarity between the second sample video frame in the video frame set and the first sample video frame in the video frame set as a set inter-frame similarity, and comparing the set inter-frame similarity with a first inter-frame similarity threshold; If the set inter-frame similarity is greater than or equal to the first inter-frame similarity threshold, it is determined that the actual page loading state of the second sample video frame is the same as the actual page loading state of the first sample video frame; if the set inter-frame similarity is less than the first inter-frame similarity threshold, it is determined that the actual page loading state of the second sample video frame is different from the actual page loading state of the first sample video frame; The first sample video frame in the video frame set is removed to obtain the updated video frame set, and based on the updated video frame set, the step of comparing the set inter-frame similarity with the first inter-frame similarity threshold is continued until the actual page loading status corresponding to each of the multiple sample video frames is obtained.

6. The method according to claim 2, characterized in that The determining, according to the inter-frame similarity of each pair of adjacent sample video frames, the actual page loading states corresponding to each of the plurality of sample video frames comprises: For a first specific sample video frame among the multiple sample video frames, if the inter-frame similarity between the first specific sample video frame and a second specific sample video frame among the multiple sample video frames is greater than or equal to a first inter-frame similarity threshold, then determining that the actual page loading state of the first specific sample video frame is an actual stable loading state; if the inter-frame similarity between the first specific sample video frame and the second specific sample video frame is less than the first inter-frame similarity threshold, then determining that the actual page loading state of the first specific sample video frame is an actual unstable loading state; Wherein, if the first specific sample video frame is the first sample video frame among the multiple sample video frames, the second specific video frame is the second sample video frame among the multiple sample video frames; if the first specific sample video frame is the last sample video frame among the multiple sample video frames, the second specific video frame is the second to last sample video frame among the multiple sample video frames; For any single sample video frame among the multiple sample video frames except the first specific sample video frame, if the inter-frame similarities between the single sample video frame and the previous sample video frame and the next sample video frame of the single sample video frame are respectively greater than or equal to the first inter-frame similarity threshold, then the actual page loading state of the single sample video frame is determined to be the actual stable loading state; if at least one of the inter-frame similarities between the single sample video frame and the previous sample video frame and the next sample video frame of the single sample video frame is less than the first inter-frame similarity threshold, then the actual page loading state of the single sample video frame is determined to be the actual unstable loading state.

7. A device for determining page loading time, characterized in that: include: The frame sampling module is used to perform frame processing on the page loading video to obtain multiple video frames; A state classification module, used for inputting the plurality of video frames into a state classification model to obtain page loading states corresponding to the plurality of video frames; A state classification module, used for classifying the video frames with the same page loading state in succession among the plurality of video frames to obtain a plurality of page loading stages; The duration determination module is used to determine the page loading duration corresponding to each of the multiple page loading stages according to the video frames corresponding to each of the multiple page loading stages.

8. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising instructions, characterized in that When the computer program product runs on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 6.

Images