Method and related device for obtaining page switching response time
By classifying the videos of the terminal software page switching process, the page switching response time is automatically determined, which solves the problems of statistical inaccuracy and time-consuming in the prior art, and improves efficiency and accuracy.
Patent Information
- Application Number
- CN202010695835.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-17
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-07-17
AI Technical Summary
In the prior art, the method of obtaining the response time of software page switching has the problem that statistical results are inaccurate and time-consuming, especially in the statistical process of non-independent research and development of software.
By obtaining the video of the terminal software page switching process, converting it into multi-frame target images according to the preset frequency, and classifying these images in states, and dividing them into stable state images and switching state images. The video is divided according to these images and the switching is determined. The time period of the phase is used as the page switching response time. Optional methods include image classification using pre-trained object detection models, template matching algorithms, feature matching algorithms, and classification neural network models.
It realizes automatic processing of page switching response time, significantly reduces acquisition time, improves processing efficiency and accuracy, and is faster and more reliable than manual statistics.
Smart Images

Figure CN111859019B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of software technology, and in particular to a method for acquiring page switching response time and related equipment. Background Art
[0002] In the field of communications, monitoring the length of the page switching response time of the software is very important for analyzing the software performance. In the prior art, for self-developed software, the time consumption of various switches can only be counted by stubbing in the code. The statistical time consumption is closely related to the location of the stubbing in the code, and there are often situations where the statistical results are inconsistent with people's subjective feelings. For software that is not self-developed, the time consumption of various switches can only be counted by manual stopwatch or manual frame counting, which has the problem of inaccurate statistical results and is very time-consuming. Therefore, it is urgent to solve the above problems. Summary of the invention
[0003] The embodiment of the present invention provides a method for obtaining a page switching response time and related devices, which can reduce the time required for obtaining the page switching response time and improve the processing efficiency and accuracy of obtaining the page switching response time.
[0004] In a first aspect, an embodiment of the present invention provides a method for obtaining a page switching response time, comprising:
[0005] Acquire a target video, where the target video is a video including a software page switching process of the terminal;
[0006] Converting the target video into multiple frames of target images according to a preset frequency;
[0007] Classifying the states of the multiple frames of target images into stable state images before or after page switching and switching state images during page switching;
[0008] The target video is divided into stages according to the stable state image and the switching state image, and time periods when the target video is in the switching stage and the stable stage are obtained respectively. The length of the time period in the switching stage is determined as the software page switching response time.
[0009] Optionally, the software is a video live broadcast software, and the state classification of the multiple frames of target images includes:
[0010] Detecting whether there is a live broadcaster in the first target image and the second target image in the multiple frames of target images according to a pre-trained target detection model, wherein the second target image is an image of a subsequent frame of the first target image, and when there is no live broadcaster in the first target image or the second target image, determining that the first target image or the second target image is a switching state image;
[0011] When there are live broadcasters in both the first target image and the second target image, the image similarity between the first target image and the second target image is obtained. When the image similarity is less than or equal to a similarity threshold, the first target image is a stable state image and the second target image is a switching state image; when the image similarity is greater than the similarity threshold, the first target image and the second target image are stable state images.
[0012] Optionally, performing state classification on the multiple frames of target images includes:
[0013] Detecting whether the multiple frames of target images include a preset specific icon, where the specific icon is an icon on the software page when in a stable state;
[0014] determining the target image in which the specific icon is detected as a stable state image;
[0015] The target image in which the specific icon is not detected is determined as a switching state image.
[0016] Optionally, performing state classification on the multiple frames of target images includes:
[0017] Using a template matching algorithm to detect whether the multiple frames of target images include a preset specific icon, wherein the specific icon is an icon on the software page when in a stable state;
[0018] Determine the target image of the specific icon detected by the template matching algorithm as a stable state image;
[0019] If the specific icon is not detected from the target image using the template matching algorithm, the feature matching algorithm is used to detect whether the target image includes a preset specific icon, and the target image in which the specific icon is detected using the feature matching algorithm is determined as a stable state image, and the target image in which the specific icon is not detected using the feature matching algorithm is determined as a switching state image.
[0020] Optionally, performing state classification on the multiple frames of target images includes:
[0021] The target image is classified according to a pre-trained classification neural network model to determine whether the target image is a stable state image or a switching state image, and the training data of the classification neural network model includes software page images in a stable stage and a switching stage.
[0022] In a second aspect, an embodiment of the present invention provides a device for obtaining a page switching response time, including:
[0023] An acquisition module, used to acquire a target video, wherein the target video is a video including a software page switching process of a terminal;
[0024] A conversion module, used for converting the target video into multiple frames of target images according to a preset frequency;
[0025] A classification module, used for classifying the states of the multiple frames of target images into stable state images before or after page switching, and switching state images during page switching;
[0026] A determination module is used to divide the target video into stages according to the stable state image and the switching state image, obtain the time periods when the target video is in the switching stage and the stable stage respectively, and determine the time period length of the switching stage as the software page switching response time.
[0027] Optionally, the software is a live video broadcasting software, and the classification module includes:
[0028] The first submodule is used to detect whether there is a live broadcaster in the first target image and the second target image in the multiple frames of target images according to a pre-trained target detection model, the second target image is a frame image after the first target image, and when there is no live broadcaster in the first target image or the second target image, determine that the first target image or the second target image is a switching state image;
[0029] The second submodule is used to obtain the image similarity between the first target image and the second target image when there are live broadcasters in both the first target image and the second target image. When the image similarity is less than or equal to a similarity threshold, the first target image is a stable state image and the second target image is a switching state image; when the image similarity is greater than the similarity threshold, the first target image and the second target image are stable state images.
[0030] Optionally, the classification module includes:
[0031] The third submodule is used to detect whether the multiple frames of target images include a preset specific icon, where the specific icon is an icon on the software page when in a stable state; determine the target image in which the specific icon is detected as a stable state image; and determine the target image in which the specific icon is not detected as a switching state image.
[0032] In a third aspect, an embodiment of the present invention provides a device for obtaining a page switching response time, including: a processor and a memory;
[0033] The processor is connected to a memory, wherein the memory is used to store program codes, and the processor is used to call the program codes to execute the method for obtaining page switching response time as described in the first aspect.
[0034] In a fourth aspect, an embodiment of the present invention provides a computer storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the method for obtaining a page switching response time as described in the first aspect is executed.
[0035] In an embodiment of the present invention, after acquiring a target video, the target video is a video including a software page switching process of a terminal; the target video is converted into a plurality of target image frames at a preset frequency; the plurality of target image frames are then classified into stable state images before or after a page switch, and switching state images during a page switch; the target video is divided into stages according to the stable state images and the switching state images, and time periods in which the target video is in a switching stage and a stable stage are obtained respectively, and the length of the time period in the switching stage is determined as the software page switching response time; it can be seen that, by utilizing an embodiment of the present invention, the page switching response time can be automatically acquired, which effectively reduces the time required for acquiring the page switching response time compared to manual acquisition, and improves the processing efficiency and accuracy of acquiring the page switching response time. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.
[0037] Figure 1a , Figure 1b , Figure 1c is a scenario schematic diagram of a method for obtaining page switching response time provided by an embodiment of the present invention;
[0038] Figure 2 It is a flow chart of a method for obtaining page switching response time provided by an embodiment of the present invention;
[0039] Figure 3 It is a structural schematic diagram of a device for obtaining page switching response time provided by an embodiment of the present invention;
[0040] Figure 4 It is a structural schematic diagram of a device for obtaining page switching response time provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0041] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0042] It should be understood that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.
[0043] Reference to "embodiments" in the present invention means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present invention may be combined with other embodiments.
[0044] In this application, software includes computer software or mobile software, and the scenarios of software page switching on the terminal include page switching when the software is started (page changes from not starting the software to the end of software startup), page jumps based on user operations during software use, such as a jump after the user clicks on a URL on the software page, or in a live broadcast application (audio live broadcast application or video live broadcast application), the user slides to switch live broadcast rooms, etc. Among them, terminals include smart devices such as computers, laptops, mobile phones, and smart watches. The page switching response time obtained by statistically analyzing the response time of software page switching can be used to evaluate software performance and assist users in improving software performance.
[0045] Based on the problem in the prior art that the time required for page switching response time statistics is too long and the accuracy is low, the present application proposes a method for obtaining page switching response time, which can effectively reduce the time required for obtaining page switching response time and improve the acquisition efficiency and accuracy.
[0046] See also Figure 1a , Figure 1b and Figure 1c , Figure 1a , Figure 1b , Figure 1c is a schematic diagram of a scenario of a method for obtaining a page switching response time provided by an embodiment of the present invention; Figure 1aAs shown, the terminal takes a mobile phone as an example, and the processing terminal 101 is used to execute the method for obtaining the page switching response time. The processing terminal 101 can be a computer or a notebook, etc. Taking obtaining the software page switching response time of the software installed in the mobile phone as an example, the software takes the video live broadcast software as an example.
[0047] First, after user 102 starts screen recording on the mobile phone, he enters the video live broadcast software and switches from user A's live broadcast room to user B's live broadcast room, and then switches to user C's live broadcast room. The mobile phone obtains the recorded video of this switching process. Then the mobile phone transmits the recorded video to the processing terminal 101 for processing, and the following can be obtained: Figure 1b and Figure 1c The processing result report in HTML format is shown. After the processing terminal 101 obtains the recorded video, it first converts the recorded video into multiple frames of images according to the preset frequency. For example, if the preset frequency is 50ms / frame, then 20 frames of images are converted per second of the video. The multiple frames of images obtained above are classified into stable state images before or after the page switch, and switching state images during the page switch. The recorded video is then divided into stages according to the stable state images and the switching state images, and the time periods of the recorded video in the switching stage and the stable stage are obtained respectively. The length of the time period of each switching stage is determined as the software page switching response time. Among them, Figure 1b and Figure 1c In the example of converting a recorded video into 183 frames, the processing result report includes the image frames corresponding to each stable stage and switching stage (for manual inspection), as well as the duration of each stage, and a linear waveform diagram with stage transformation (such as Figure 1c ), and the software page switching response time corresponding to each page switching (such as Figure 1c The time costs shown in 1.033334 and 0.966667, Figure 1b and Figure 1c The time unit is s). Alternatively, Figure 1c As shown, the result processing report can also display the storage path of the recorded video in the processing terminal 101, the total number of image frames corresponding to the recorded video (such as 183 frames), and the offset between each frame of the image, that is, the size of the preset frequency.
[0048] It can be seen that the method of the embodiment of the present invention can be used to automatically process the recorded video to obtain the software page switching response time corresponding to each software page switching. Compared with the manual acquisition method, the switching response time is more accurate and efficient in obtaining.
[0049] See also Figure 2 , is a flow chart of a method for obtaining a page switching response time provided by an embodiment of the present invention; the method for obtaining a page switching response time includes:
[0050] 201. Obtain a target video, where the target video is a video including a software page switching process of a terminal;
[0051] Specifically, the method for obtaining the target video may be to start a screen recording function of the terminal, and record a video when a page switch occurs, wherein the number of page switches in the target video is at least one.
[0052] 202. Convert the target video into multiple frames of target images according to a preset frequency;
[0053] Specifically, the preset frequency can be set according to actual needs. For example, if the target video is 5 seconds and the preset frequency is 50 ms / frame, the target video can be converted into a target image of 100 frames.
[0054] 203. Classify the states of the multiple frames of target images into stable state images before or after page switching and switching state images during page switching;
[0055] Specifically, all the target images are classified into states, the state of each frame of the target image is determined, and the target image is classified into a stable state image or a switching state image. When the target image is before or after a page switch, the target image is a stable state image; and when the target image is during a page switch, the target image is a switching state image.
[0056] 204. Divide the target video into stages according to the stable state image and the switching state image, obtain time periods when the target video is in the switching stage and the stable stage respectively, and determine the length of the time period in the switching stage as the software page switching response time.
[0057] Specifically, according to the target image after state classification, the target video can be divided into stages, divided into video segments in the switching stage and the stable stage, wherein the time length of the video segment in the switching stage is the software page switching response time. Taking a 5-second target video as an example, assuming that 0-2.2 seconds, 2.8-3.1 seconds, and 3.5-5 seconds are the stable stage, and 2.2-2.8 seconds and 3.1-3.5 seconds are the switching stage, then 0.6 seconds and 0.4 seconds are the software page switching response time, and two page switches occurred in the target video.
[0058] It can be seen that using Figure 2 The method can automatically process and obtain the page switching response time. Compared with manual acquisition, it effectively reduces the time required for obtaining the page switching response time, improves the processing efficiency and accuracy of obtaining the page switching response time, and saves labor costs.
[0059] In a possible embodiment, in step 202, the target video needs to be framed to obtain a target image, and each frame of the target image records a frame number and a timestamp. For multiple target images, a frame number is assigned to each frame of the target image in the order from the start of video recording to the end of recording, and the timestamp is the time point corresponding to the target image in the target video.
[0060] In a possible embodiment, step 203 includes:
[0061] Detecting whether the multiple frames of target images include a preset specific icon, where the specific icon is an icon on the software page when in a stable state;
[0062] determining the target image in which the specific icon is detected as a stable state image;
[0063] The target image in which the specific icon is not detected is determined as a switching state image.
[0064] Specifically, this state classification method is applicable to all software page switching scenarios. Before obtaining the software page switching response time of a certain software, it is necessary to obtain in advance the icons existing on the page when the software page is in a stable state as the specific icons of the software. The number of specific icons for each software is more than one. The specific icons are fixed and non-replaceable elements on the page, such as the function icons in the page, such as the follow icon, chat icon, gift icon, flower icon, etc., taking the live broadcast software as an example.
[0065] When the state of the target image is classified, the target detection can be performed on each frame of the target image according to the pre-acquired specific icon. When the specific icon is detected in the target image, the target image is a stable state image, otherwise, the target image is a switching state image. In particular, in order to further improve the state classification accuracy of the target image, the specific icon of a certain software includes a function icon at the top of the page and two or more function icons at the bottom of the page (that is, the specific icon includes at least one function icon at the top of the page and at least one function icon at the bottom of the page). When a certain target image detects two or more function icons at the top and bottom at the same time, the target image is a stable state image, otherwise, the target image is a switching state image. Taking the live broadcast software as an example, when the attention icon at the top of the page and the chat icon at the bottom of the page can be detected at the same time in a certain target image, the target image is a stable state image, and when the attention icon and the chat icon cannot be detected, or when only the attention icon or the chat icon can be detected, the target image is a switching state image. Similarly, the specific icon of a certain software can also be a function icon including a function icon at the left side of the page and two or more function icons at the right side of the page (that is, the specific icon includes at least one function icon at the left side of the page and at least one function icon at the right side of the page).
[0066] Furthermore, in order to improve the accuracy of target detection, detecting whether multiple frames of target images include a preset specific icon includes:
[0067] Using a template matching algorithm to detect whether multiple frames of target images include a preset specific icon;
[0068] Determine the target image in which the specific icon is detected by the template matching algorithm as a stable state image;
[0069] If no specific icon is detected from the target image using the template matching algorithm, the feature matching algorithm is used to detect whether the target image includes a preset specific icon, and the target image in which the specific icon is detected using the feature matching algorithm is determined as a stable state image, and the target image in which the specific icon is not detected using the feature matching algorithm is determined as a switching state image.
[0070] Specifically, according to the specific icon, the target image is detected by using the template matching algorithm and the feature matching algorithm in turn to determine whether the specific icon exists in the target image, which can effectively improve the accuracy of target detection. When any algorithm detects that there is a specific icon in the target image, the target image is a stable state image; and when neither algorithm detects that there is a specific icon in the target image, the target image is a switching state image.
[0071] In a possible embodiment, step 203 includes:
[0072] The target image is classified according to a pre-trained classification neural network model to determine whether the target image is a stable state image or a switching state image. The training data of the classification neural network model includes software page images in a stable stage and a switching stage.
[0073] Specifically, similarly, this state classification method is applicable to all software page switching scenarios. Before obtaining the software page switching response time of a certain software, it is necessary to obtain page images in a stable state and a switching state as training data in advance, and train the neural network model to obtain a classification neural network model. More specifically, in order to improve the training efficiency, when obtaining the training data, after obtaining multiple page images, the page images are pre-classified according to the pre-trained SVM linear classifier, and then the classification results are manually reviewed to classify the page images into two categories of stable state and switching state to obtain the final training data. The training data obtained by this method is more efficient than relying entirely on manual classification, saving time in preparing training data. Then the training data is used to train the model to obtain a classification neural network model. By using the classification neural network model to predict each frame of the target image, the target image can be determined as a stable state image or a switching state image.
[0074] In particular, in order to improve the accuracy of the state classification of the target image, the prediction results of the specific icon detection and classification neural network model can be combined, and through manual review, the target image can be finally determined to be a stable state image or a switching state image, thereby improving the accuracy of the subsequently obtained page switching response time.
[0075] In a possible embodiment, when the software is a live video broadcasting software, step 203 includes:
[0076] 2031. Detecting whether there is a live broadcaster in a first target image and a second target image in a plurality of target image frames according to a pre-trained target detection model, wherein the second target image is an image of a subsequent frame of the first target image, and when there is no live broadcaster in the first target image or the second target image, determining that the first target image or the second target image is a switching state image;
[0077] Specifically, for the page switching scenario of the video live broadcast software, the live broadcaster can be a person or an animal; every two target images are compared as a group, for example, (the first frame, the second frame) is a group, and (the second frame, the third frame) is a group. The target image without the live broadcaster in the live broadcast screen must be a switching state image. When the live broadcaster cannot be detected in one or both of the first target image and the second target image, the target image is a switching state image.
[0078] 2032. When there are live broadcasters in both the first target image and the second target image, obtain the image similarity between the first target image and the second target image. When the image similarity is less than or equal to the similarity threshold, the first target image is a stable state image and the second target image is a switching state image. When the image similarity is greater than the similarity threshold, the first target image and the second target image are stable state images.
[0079] Specifically, the specific value of the similarity threshold can be set as needed, for example, 99.8%. When the image similarity of a group of images is greater than the similarity threshold, it is considered that the group of images is in a stable stage. When both the first target image and the second target image have live broadcasters, the image similarity of this group of images needs to be calculated. When the image similarity is greater than the similarity threshold, the first target image and the second target image are stable state images; and when the image similarity is less than or equal to the similarity threshold, the first target image is a stable state image, and the second target image is a switching state image.
[0080] It should be noted that the reference Figure 1b In the switching stage from anchor A to anchor B, in the existing video live broadcast software, the way to switch the live broadcast room is to slide up from the bottom of the software page. As the sliding process changes, a "mask" will gradually appear at the bottom of the software page (such as Figure 1b), and below the mask, the identification image of anchor B (such as anchor B's live avatar) is generally loaded. Due to the existence of the "mask", even if anchor B's face appears in the last few frames of the switching phase, these frames cannot detect the face of the live broadcaster, and they will be directly determined as switching state images. Simply put, in step 2032, due to the technical characteristics of existing video live broadcast software, there will not be a situation where both the first target image and the second target image have live broadcasters, the image similarity between the first target image and the second target image is less than or equal to the similarity threshold, the first target image is a switching state image, and the second target image is a stable state image.
[0081] All switching state images can be determined by using the above steps 2031 and 2032, and the remaining target images are stable state images. Then, the software page switching response time can be determined based on the switching state images and the stable state images. In this embodiment, a new method for obtaining the software page switching response time of a video live broadcast software is provided.
[0082] For calculating the image similarity between the first target image and the second target image, an embodiment of the present invention further provides a calculation method, including:
[0083] SSIM (structural similarity) is a full-reference image quality evaluation indicator that measures image similarity from three aspects: brightness, contrast, and structure. The SSIM value range is [0,1]. The larger the value, the smaller the image distortion. The SSIM value between the first target image and the second target image can be calculated, and the similarity threshold is the SSIM threshold accordingly.
[0084] In practical applications, a sliding window can be used to divide an image into blocks. Let the total number of blocks be N. Considering the influence of window shape on the blocks, Gaussian weighting is used to calculate the mean, variance, and covariance of each window, and then the structural similarity SSIM of the corresponding block is calculated. Finally, the average value is used as the structural similarity measure of the two images, that is, the average structural similarity MSSIM. The structural similarity of the two images is evaluated by comparison based on the MSSIM and SSIM thresholds, that is, the image similarity.
[0085] In addition, a more accurate method for evaluating image similarity is provided, including:
[0086] Calculating PSNR and mutual information values between the first target image and the second target image;
[0087] When the PSNR is greater than the PSNR threshold and the mutual information value is greater than the mutual information threshold, it indicates that the image similarity between the first target image and the second target image is greater than the similarity threshold.
[0088] Among them, Peak Signal to Noise Ratio (PSNR) is an objective standard for measuring image distortion. The larger the PSNR value between two images, the more similar the two images are. Mutual Information (MI) is an important concept in information theory, which describes the correlation between two systems, or the amount of information they contain. The mutual information of two images reflects the degree of mutual inclusion of information between them through their entropy and joint entropy. When the joint entropy of the two images is smaller, that is, the mutual information value is larger, the similarity of the two images is higher.
[0089] Furthermore, after the target image is classified into different states using the above method, different state marks are added to the stable state image and the switching state image according to the state classification result to distinguish the target images in different states. For example, the state mark of the stable state image is True. The state mark of the switching state image is False. An information item may be added to the attribute information of the image to record the state mark. In step 204, the target video may be divided into stages according to the state mark, and the ones with the same state mark are grouped into one stage. For example, among the 1-15 frames of target images, the 1-6 frames are True and the 7-15 frames are False. Then, the time taken for each stage may be calculated according to the timestamp of the target image, and the size of the software page switching response time may be obtained. For example, the timestamp of the target image of the 15th frame minus the timestamp of the target image of the 7th frame is the software page switching response time. A processing result report may be generated according to the software page switching response time. For the convenience of transmission, the processing result report may be a report in HTML format. The effect of the HTML report can be referred to. Figure 1b and Figure 1c The report will show thumbnails of all image frames at each stage for easy manual inspection, and will also show statistics of each stage and time consumption.
[0090] Based on the description of the above method embodiment for obtaining the page switching response time, the embodiment of the present invention further discloses a device for obtaining the page switching response time. Figure 3 , Figure 3 1 is a schematic diagram of a structure of a device for obtaining a page switching response time provided by an embodiment of the present invention. The device for obtaining a page switching response time includes:
[0091] An acquisition module 301 is used to acquire a target video, where the target video is a video including a software page switching process of a terminal;
[0092] A conversion module 302, used to convert the target video into multiple frames of target images according to a preset frequency;
[0093] The classification module 303 is used to classify the states of the multiple frames of target images into stable state images before or after page switching, and switching state images during page switching;
[0094] The determination module 304 is used to divide the target video into stages according to the stable state image and the switching state image, obtain the time periods of the target video in the switching stage and the stable stage respectively, and determine the time period length of the switching stage as the software page switching response time.
[0095] In a possible embodiment, the software is a video live broadcast software, and the classification module 303 includes:
[0096] The first submodule is used to detect whether there is a live broadcaster in the first target image and the second target image in the multiple frames of target images according to the pre-trained target detection model, and the second target image is a frame image after the first target image. When there is no live broadcaster in the first target image or the second target image, it is determined that the first target image or the second target image is a switching state image;
[0097] The second submodule is used to obtain the image similarity between the first target image and the second target image when there are live broadcasters in both the first target image and the second target image. When the image similarity is less than or equal to the similarity threshold, the first target image is a stable state image and the second target image is a switching state image; when the image similarity is greater than the similarity threshold, the first target image and the second target image are stable state images.
[0098] In a possible embodiment, the classification module 303 includes:
[0099] The third submodule is used to detect whether the multi-frame target image includes a preset specific icon, where the specific icon is an icon on the software page when it is in a stable state; the target image in which the specific icon is detected is determined as a stable state image, and the target image in which the specific icon is not detected is determined as a switching state image.
[0100] The step of detecting whether the multiple frames of target images include a preset specific icon specifically includes:
[0101] Using a template matching algorithm to detect whether multiple frames of target images include a preset specific icon;
[0102] Determine the target image in which the specific icon is detected by the template matching algorithm as a stable state image;
[0103] If no specific icon is detected from the target image using the template matching algorithm, the feature matching algorithm is used to detect whether the target image includes a preset specific icon, and the target image in which the specific icon is detected using the feature matching algorithm is determined as a stable state image, and the target image in which the specific icon is not detected using the feature matching algorithm is determined as a switching state image.
[0104] In a possible embodiment, the classification module 303 includes:
[0105] The fourth submodule is used to classify the target image according to the pre-trained classification neural network model to determine whether the target image is a stable state image or a switching state image. The training data of the classification neural network model includes software page images in the stable stage and the switching stage.
[0106] It is worth pointing out that, among them, the specific functional implementation method of the device for obtaining the page switching response time can refer to the description of the method for obtaining the page switching response time mentioned above, and will not be repeated here. The various units or modules in the device for obtaining the page switching response time can be respectively or completely merged into one or several other units or modules to constitute, or one (some) of the units or modules can also be divided into multiple functionally smaller units or modules to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units or modules are divided based on logical functions. In actual applications, the functions of one unit (or module) can also be implemented by multiple units (or modules), or the functions of multiple units (or modules) can be implemented by one unit (or module).
[0107] Based on the description of the above method embodiment and device embodiment, an embodiment of the present invention further provides a device for obtaining a page switching response time.
[0108] See also Figure 4 , is a schematic diagram of the structure of a device for obtaining a page switching response time provided by an embodiment of the present invention. Figure 4As shown, the above-mentioned device for obtaining the page switching response time can be applied to the device 400 for obtaining the page switching response time, and the device 400 for obtaining the page switching response time may include: a processor 401, a network interface 404 and a memory 405. In addition, the device 400 for obtaining the page switching response time may also include: a user interface 403, and at least one communication bus 402. Among them, the communication bus 402 is used to realize the connection and communication between these components. Among them, the user interface 403 may include a display screen (Display), a keyboard (Keyboard), and the user interface 403 may also include a standard wired interface and a wireless interface. The network interface 404 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 405 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 405 may also be optionally at least one storage device located away from the aforementioned processor 401. As Figure 4 As shown, the memory 405 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.
[0109] exist Figure 4 In the device 400 for obtaining the page switching response time, the network interface 404 can provide a network communication function; the user interface 403 is mainly used to provide an input interface for the user; and the processor 401 can be used to call the device control application stored in the memory 405 to achieve:
[0110] Obtaining a target video, where the target video is a video including a software page switching process of the terminal;
[0111] Convert the target video into multiple frames of target images according to a preset frequency;
[0112] Classifying the states of the multiple frames of target images into stable state images before or after page switching and switching state images during page switching;
[0113] The target video is divided into stages according to the stable state image and the switching state image, and the time periods of the target video in the switching stage and the stable stage are obtained respectively. The length of the time period in the switching stage is determined as the software page switching response time.
[0114] In one embodiment, when the software is a live video broadcasting software, the processor 401 specifically performs the following steps when performing state classification on multiple frames of target images:
[0115] According to the pre-trained target detection model, it is detected whether there is a live broadcaster in the first target image and the second target image in the multiple frames of target images, the second target image is an image of the next frame of the first target image, and when there is no live broadcaster in the first target image or the second target image, it is determined that the first target image or the second target image is a switching state image;
[0116] When there are live broadcasters in both the first target image and the second target image, the image similarity between the first target image and the second target image is obtained. When the image similarity is less than or equal to the similarity threshold, the first target image is a stable state image and the second target image is a switching state image; when the image similarity is greater than the similarity threshold, the first target image and the second target image are stable state images.
[0117] In one embodiment, when the processor 401 performs state classification on multiple frames of target images, the processor 401 specifically performs the following steps:
[0118] Detecting whether the multiple frames of target images include a preset specific icon, where the specific icon is an icon on the software page when in a stable state;
[0119] determining the target image in which the specific icon is detected as a stable state image;
[0120] The target image in which the specific icon is not detected is determined as a switching state image.
[0121] In one embodiment, when the processor 401 detects whether multiple frames of target images include a preset specific icon, the processor 401 specifically performs the following steps:
[0122] Using a template matching algorithm to detect whether multiple frames of target images include a preset specific icon;
[0123] Determine the target image in which the specific icon is detected by the template matching algorithm as a stable state image;
[0124] If no specific icon is detected from the target image using the template matching algorithm, the feature matching algorithm is used to detect whether the target image includes a preset specific icon, and the target image in which the specific icon is detected using the feature matching algorithm is determined as a stable state image, and the target image in which the specific icon is not detected using the feature matching algorithm is determined as a switching state image.
[0125] In one embodiment, when the processor 401 performs state classification on multiple frames of target images, the processor 401 specifically performs the following steps:
[0126] The target image is classified according to a pre-trained classification neural network model to determine whether the target image is a stable state image or a switching state image. The training data of the classification neural network model includes software page images in a stable stage and a switching stage.
[0127] It should be understood that the device 400 for obtaining the page switching response time described in the embodiment of the present invention can execute the method for obtaining the page switching response time described above, and can also execute the description of the device for obtaining the page switching response time described above, which will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated.
[0128] In addition, it should be pointed out here that: an embodiment of the present invention also provides a computer storage medium, and the computer storage medium stores a computer program executed by the device for obtaining the page switching response time mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the description of the method for obtaining the page switching response time mentioned above, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer storage medium embodiment involved in the present invention, please refer to the description of the method embodiment of the present invention.
[0129] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0130] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.
Claims
1. A method for obtaining page switching response time, characterized in that: include: Acquire a target video, wherein the target video is a video including a software page switching process of a terminal, and the software is a video live broadcast software; Converting the target video into multiple frames of target images according to a preset frequency; The multiple frames of target images are classified into stable state images before or after page switching, and switching state images during page switching, wherein the multiple frames of target images are classified into the following state: detecting whether there is a live broadcaster in the first target image and the second target image in the multiple frames of target images according to a pre-trained target detection model, wherein the second target image is a frame image after the first target image, and when there is no live broadcaster in the first target image or the second target image, determining that the first target image or the second target image is a switching state image; when there is a live broadcaster in both the first target image and the second target image, obtaining the image similarity between the first target image and the second target image, and when the image similarity is less than or equal to a similarity threshold, the first target image is a stable state image, and the second target image is a switching state image, and when the image similarity is greater than the similarity threshold, the first target image and the second target image are stable state images; The target video is divided into stages according to the stable state image and the switching state image, and time periods when the target video is in the switching stage and the stable stage are obtained respectively. The length of the time period in the switching stage is determined as the software page switching response time.
2. The method according to claim 1, characterized in that The multiple frames of target images are classified into different states, including: Detecting whether the multiple frames of target images include a preset specific icon, where the specific icon is an icon on the software page when in a stable state; determining the target image in which the specific icon is detected as a stable state image; The target image in which the specific icon is not detected is determined as a switching state image.
3. The method according to claim 1, characterized in that The multiple frames of target images are classified into different states, including: Using a template matching algorithm to detect whether the multiple frames of target images include a preset specific icon, wherein the specific icon is an icon on the software page when in a stable state; Determine the target image of the specific icon detected by the template matching algorithm as a stable state image; If the specific icon is not detected from the target image using the template matching algorithm, the feature matching algorithm is used to detect whether the target image includes a preset specific icon, and the target image in which the specific icon is detected using the feature matching algorithm is determined as a stable state image, and the target image in which the specific icon is not detected using the feature matching algorithm is determined as a switching state image.
4. The method according to claim 1, characterized in that: The multiple frames of target images are classified into different states, including: The target image is classified according to a pre-trained classification neural network model to determine whether the target image is a stable state image or a switching state image, and the training data of the classification neural network model includes software page images in a stable stage and a switching stage.
5. A device for obtaining page switching response time, characterized in that: include: An acquisition module, used to acquire a target video, wherein the target video is a video including a software page switching process of a terminal, and the software is a video live broadcast software; A conversion module, used for converting the target video into multiple frames of target images according to a preset frequency; A classification module, used for classifying the states of the multiple frames of target images into stable state images before or after page switching, and switching state images during page switching; A determination module, configured to divide the target video into stages according to the stable state image and the switching state image, obtain time periods when the target video is in a switching stage and a stable stage respectively, and determine the length of the time period in the switching stage as the software page switching response time; Wherein, the classification module includes: The first submodule is used to detect whether there is a live broadcaster in the first target image and the second target image in the multiple frames of target images according to a pre-trained target detection model, the second target image is a frame image after the first target image, and when there is no live broadcaster in the first target image or the second target image, determine that the first target image or the second target image is a switching state image; The second submodule is used to obtain the image similarity between the first target image and the second target image when there are live broadcasters in both the first target image and the second target image. When the image similarity is less than or equal to a similarity threshold, the first target image is a stable state image and the second target image is a switching state image; when the image similarity is greater than the similarity threshold, the first target image and the second target image are stable state images.
6. The device according to claim 5, characterized in that The classification module comprises: The third submodule is used to detect whether the multiple frames of target images include a preset specific icon, where the specific icon is an icon on the software page when in a stable state; determine the target image in which the specific icon is detected as a stable state image; and determine the target image in which the specific icon is not detected as a switching state image.
7. A device for obtaining page switching response time, characterized in that: include: Processor and memory; The processor is connected to a memory, wherein the memory is used to store program codes, and the processor is used to call the program codes to execute the method for obtaining page switching response time according to any one of claims 1 to 4.
8. A computer storage medium, characterized in that: The computer storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the method for obtaining a page switching response time according to any one of claims 1 to 4 is executed.
