Application startup node localization

Abstract

Embodiments of the present specification disclose an application startup node localization method and apparatus, a storage medium, and an electronic device. The application startup node localization method comprises: acquiring a video file of an application startup process; determining a reference frame from the video file, the reference frame comprising a monitoring area and a similarity detection algorithm corresponding to the monitoring area; on the basis of the reference frame, searching the video file for hit frames to obtain a hit frame set; and determining key frames from the hit frame set, the key frames corresponding to nodes in the application startup process.

Description

Application startup node positioning Technical Field

[0001] The embodiments of this specification belong to the field of terminal technology, and particularly relate to a method, device, storage medium, and electronic device for locating an application startup node. Background Art

[0002] With the rapid development of smartphones, more and more applications have been developed to meet the diverse needs of users, such as social networking, shopping, navigation, multimedia entertainment, etc. Application performance, as a key factor in user experience, has a profound impact on user stickiness and user retention.

[0003] An application's startup speed is a crucial component of its performance and forms the user's first impression of the application. Therefore, accurately understanding the application's startup process is crucial for reducing startup time. However, accurately locating each stage of the application startup process remains a significant challenge. Therefore, a more accurate solution is needed to identify the intermediate stages of application loading. Summary of the Invention

[0004] The embodiments of this specification provide a method, device, storage medium, and electronic device for locating an application startup node, and the technical solutions are as follows.

[0005] In a first aspect, an embodiment of the present specification provides a method for locating an application startup node, comprising: obtaining a video file of the application startup process; determining a reference frame from the video file, wherein the reference frame includes a monitoring area and a similarity detection algorithm corresponding to the monitoring area; searching for a hit frame in the video file based on the reference frame to obtain a hit frame set; determining a key frame in the hit frame set, wherein the key frame corresponds to a node in the application startup process.

[0006] In the second aspect, an embodiment of the present specification provides an application startup node positioning device, including: a video acquisition unit, configured to acquire a video file of the application startup process; a reference frame determination unit, configured to determine a reference frame from the video file, the reference frame including a monitoring area and a similarity detection algorithm corresponding to the monitoring area; a hit frame determination unit, configured to search for a hit frame in the video file based on the reference frame to obtain a hit frame set; a key frame determination unit, configured to determine a key frame in the hit frame set, the key frame corresponding to a node in the application startup process.

[0007] In a third aspect, an embodiment of this specification provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.

[0008] In a fourth aspect, an embodiment of this specification provides an electronic device comprising: one or more processors, and a memory associated with the one or more processors, wherein the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the steps of the method described in the first aspect are executed.

[0009] The beneficial effects brought about by the technical solutions provided by one or more embodiments of this specification include at least the following: first, by analyzing the video files of the application startup process, the key frames corresponding to the application startup nodes are determined, making the analysis of the application startup process more accurate and reliable; second, detection is performed based on the monitoring area of ​​the reference frame, and then the key video frames corresponding to each startup node are screened out using the similarity threshold, thereby realizing the positioning of each key node in the application startup process, with good robustness, high accuracy and flexible application scenarios; third, a variety of different similarity detection algorithms are adopted, which can meet a richer range of detection scenarios while improving the calculation accuracy; fourth, it supports a variety of configuration methods, and can achieve a fairly high accuracy without the need for massive data training models. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of this specification, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0011] FIG1 is an application scenario diagram of an embodiment of this specification.

[0012] FIG2 is a flow chart of a method for locating an application startup node according to an embodiment of the present disclosure.

[0013] FIG3 is a flow chart of a process for determining a hit frame set in the application startup node positioning method provided in an embodiment of this specification.

[0014] FIG4 is a graph showing a trend of regional similarity changes according to an embodiment of the present disclosure.

[0015] FIG5 is a schematic block diagram of an application startup node location device provided in an embodiment of this specification.

[0016] FIG6 is a schematic block diagram of another application startup node location device provided in an embodiment of this specification.

[0017] FIG7 is a schematic block diagram of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of this specification will be described clearly and completely below in conjunction with the drawings in the embodiments of this specification.

[0019] The terms "first", "second", "third", etc. in the description and claims of this specification and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. Depending on the context, the word "if" as applied herein can be interpreted as "at..." or "when..." or "in response to determining" or "in response to detecting".

[0020] Application startup performance, a key element of user experience, has a profound impact on user retention and user retention. Therefore, analyzing an application's actual loading process is crucial. Application startup can be categorized into two scenarios: cold startup and hot startup. A cold startup occurs when an application is launched after being killed by the system; a hot startup occurs when an application is still running in the background and has not been killed by the system. A cold startup encompasses the entire application startup process, and therefore, industry analysis of application startup processes primarily focuses on cold startup scenarios.

[0021] Prior art primarily uses tracking to locate application launches and exits. This involves tracking corresponding program events and performing statistical analysis on the application launch time. However, due to the complex loading sequence of applications, simple tracking can only superficially reflect the application launch duration and cannot accurately reflect the user's actual loading process.

[0022] In view of this, the embodiments of this specification provide a new method and apparatus for locating application startup nodes, which can locate each key node in the application startup process, making the analysis of the application startup process more accurate and reliable.

[0023] As shown in Figure 1, this is an application scenario diagram of an embodiment of this specification. The test platform in the figure can be deployed on terminal devices (such as smart phones, tablet computers, desktops, laptops, notebook computers, ultra-mobile personal computers (UMPCs), handheld computers, PC devices, personal digital assistants (PDAs), virtual reality devices, etc.) for users to use.

[0024] After starting the test task, first obtain the video file of the application startup process, then decompose the video file into an image frame sequence, select a reference frame from the image frame sequence, and configure the monitoring area and similarity detection algorithm in the reference frame. The hit frame is determined based on the image similarity between the video image frame and the reference frame. Finally, the target key frame corresponding to the application startup node is filtered out from the hit frame, thereby achieving accurate positioning of the application startup process node and providing data support for accurate analysis of the application startup process.

[0025] The method for locating the application launch node described above in the embodiments of this specification first requires the user to select a reference frame, configure a monitoring area within the reference frame, and a similarity monitoring algorithm corresponding to the monitoring area. The video image frame and the reference frame are then compared for similarity to obtain a hit frame, ultimately determining the key frame corresponding to the application launch node. This allows the user to select reference frames and monitoring areas in a targeted manner, making similarity detection more targeted. Configuring similarity detection algorithms for different monitoring areas allows for more accurate similarity calculations.

[0026] Figure 2 is a flow chart of a method for locating an application startup node provided in an embodiment of this specification. The method can be executed by a general device, such as an electronic device, or a client device or a server device equipped with a client or a server. The method may include the following steps.

[0027] Step 202: Obtain a video file of the application startup process.

[0028] Step 204: Determine a reference frame from the video file, where the reference frame includes a monitoring area and a similarity detection algorithm corresponding to the monitoring area.

[0029] Step 206: Searching for hit frames in the video file based on the reference frame to obtain a hit frame set.

[0030] Step 208: Determine a key frame in the hit frame set, where the key frame corresponds to a node in the application startup process.

[0031] The application startup process is as follows: (1) The user clicks the application icon or starts the application through other means, and then the operating system loads the application's executable file into the memory. The operating system then calls the application's entry function to perform initialization operations. At this stage, the application will perform some basic settings, such as creating windows, initializing data structures, etc.; (2) Cache loading: The application may need to load some resources, such as images, audio, configuration files, etc. During the startup process, the application will read these resource files and load them into the memory for subsequent use; (3) Request network: After completing the cache loading, the application may perform some specific tasks at startup, such as communicating with the server, refreshing the page, upgrading components, etc.; (4) Startup completion: The interface refresh is completed and waits for user operation. Based on this, in the embodiment of this specification, the cold start process of the application is divided into four key nodes: the startup start node, the cache loading completion node, the network request node, and the startup completion node. The above nodes are the nodes that need to be located in the embodiment of this specification.

[0032] Each step in FIG. 2 is described below with reference to specific examples and embodiments.

[0033] In step 202, the video file is a screen recording file of the application startup process. The screen recording file can be obtained through the device's built-in screen recording function or by filming the device with an external recording device. The user can determine the recording time according to actual conditions or use a script to record the screen for a fixed length of time. After the screen recording is completed, the user saves the screen recording file and then sends it to the test platform for subsequent operations. The specific method and specific form of obtaining the video file are not specifically limited in the embodiments of this specification.

[0034] In step 204, after the user obtains the screen recording file through the test platform, the video file is first read, and then the video file is decomposed into a video frame sequence according to the timestamp. The above operation can be implemented through the OpenCV (Open Source Computer Vision Library) visual analysis library. After obtaining the video frame sequence, the user can determine the reference frame according to actual needs, and determine the monitoring area through the graphical configuration interface. The monitoring area is preferably an area that will change significantly during the application startup process and can remain stable after the application startup is completed. Therefore, the number and position of the monitoring area are not fixed, and the user can flexibly adjust it according to the actual application. After determining the monitoring area, based on the picture characteristics of different monitoring areas, the user further determines the corresponding similarity detection algorithm for each monitoring area for subsequent similarity detection.

[0035] In a specific embodiment of this specification, a variety of similarity detection algorithms may be provided for user selection, such as one or more of a PSNR detection algorithm, an SSIM detection algorithm, a Canny detection algorithm, a black and white screen detection algorithm, and a template feature matching algorithm.

[0036] Peak Signal-to-Noise Ratio (PSNR) measures the difference between two images, such as the difference between a compressed image and the original image, to evaluate the quality of the compressed image; and the difference between a restored image and the ground truth image, to evaluate the performance of the restoration algorithm. The minimum PSNR value is 0. A larger PSNR indicates a smaller difference between the two images. PSNR is simple to calculate and has a clear physical meaning.

[0037] SSIM (structural similarity index) is a metric that measures the similarity between two images. SSIM measures the similarity between two images based on brightness contrast, contrast contrast, and structural contrast. Humans are insensitive to pixel brightness and color but are highly sensitive to the location of edges and textures. SSIM mimics human perception by focusing primarily on edge and texture similarity. SSIM ≤ 1: The larger the SSIM, the more similar the two images are.

[0038] The Canny detection algorithm is a method for detecting edges using a multi-level edge detection algorithm. The changing trend of edge points can reflect the stage changes of the video. In order to facilitate the quantification of edge points, the last frame can be used as the reference frame when using edge points. The edge points of all frames are divided by the edge points of the last frame to obtain the ratio of edge points. Its implementation steps are as follows: (1) Convert the RGB image to a grayscale image, that is, convert the color image into a black and white image; (2) Gaussian blur the image to eliminate noise; (3) Calculate the image gradient and calculate the image edge amplitude and angle based on the gradient; (4) Non-maximum suppression is used to find the local maximum value of the pixel point, that is, the edge point; (5) Double threshold edge connection processing is used to filter out the noise of the edge point; (6) Binarize the image output result to obtain the coordinates of all edge points on the image.

[0039] Template matching is one method for finding a specific target within an image. The principle is simple: a template is moved pixel by pixel across the image to be detected, from left to right and from top to bottom. The degree of match between the template image and the overlapping sub-image is calculated. The greater the degree of match, the greater the likelihood that the two are identical. Some applications require a pull-to-refresh feature, which causes the entire page to move downward. By the time the refreshed area returns to its original position, the page has already loaded. In this case, the template matching algorithm can be used to directly match the image to the target area, accurately identifying the desired end frame.

[0040] In a specific embodiment of this specification, a method for detecting black and white screens is also proposed as an effective supplement to the above-mentioned algorithm. When detecting scenes such as video playback, the content within the selected area is constantly changing, making it difficult to detect the end frame using similarity algorithms such as PSNR and SSIM. Furthermore, when playing a video, the entire screen may be covered by a solid color area. In this case, there is no way to detect edge points within the selected area, and it is impossible to use the changing trends of edge points to determine whether to start playback. Generally, before a page is officially loaded, there will be a large number of black or white screens besides the loading bar. If white or black screens can be effectively identified, such scenes can be accurately detected. HSV (Hue, Saturation, Value) is an image color model. The color parameters in this model are hue (H), saturation (S), and value (V). Converting an RGB image to HSV format and calculating the mean of the V channel can determine the image's average brightness. The lower the brightness, the darker the image, and the higher the brightness, the whiter the image. Therefore, the current color of an image can be distinguished based on its brightness. However, it is not easy to distinguish the following type of non-white screen pictures based solely on the brightness and darkness of the picture, but it is not difficult to see that the color distribution in the following picture is very uneven. In order to distinguish such a situation, a main color extraction algorithm is introduced in the embodiment of this specification to assist in judging whether the picture is a black and white screen. Clustering refers to the process of dividing a collection of physical or abstract objects into multiple classes consisting of similar objects. As one of the important algorithms for data clustering, the KMeans algorithm can be used to identify and extract the main tones in the image and their proportions. When the proportion of the main tones of the picture is not large enough, it can also be considered that the image is not a white screen or a black screen. Based on the above principles, the black and white screen detection algorithm specifically includes the following steps: determining the average brightness of the monitoring area based on the HSV model; extracting and sorting the colors of the monitoring area based on the Kmeans algorithm; determining whether the monitoring area is a black and white image based on the average brightness and main colors of the monitoring area.

[0041] The above similarity detection algorithms have different advantages and characteristics. Users can configure appropriate algorithms for the monitoring area according to actual needs to achieve better detection results. One monitoring area can be configured with one or more similarity detection algorithms.

[0042] In step 206, after completing the above configuration, the test platform automatically performs the test operation. The test platform compares the video frame sequence with the reference frame, identifies the video frame that meets the specified conditions as a hit frame, and adds the hit frame to the hit frame set. A hit frame is an image frame that meets the specified conditions and contains the key frame to be located.

[0043] In step 208, the hit frame set includes all video frames that meet the specific conditions. Finally, the key video frame corresponding to the node in the application startup process is determined in the hit frame set. The key video frame can be output in various ways. For example, the key frame's attribute information such as the sequence number and timestamp can be exported to a table for storage, or the key frame can be directly exported as an image with the sequence number, timestamp and other attribute information attached for storage. In the embodiments of this specification, the specific output method of the key frame is not further limited.

[0044] As shown in FIG3 , the method further includes the following steps.

[0045] Step 302: Determine a similarity threshold corresponding to the similarity detection algorithm.

[0046] Step 304: Calculate the regional similarity between each frame image in the video file and the reference frame based on the monitoring area and the similarity detection algorithm.

[0047] Step 306: Determine the image frame whose region similarity meets the similarity threshold as a hit frame.

[0048] Step 308: Determine the hit frame set based on the frame sequence number of the hit frame.

[0049] Each step in FIG3 is described below with reference to specific examples and embodiments.

[0050] After configuring the monitoring area and the similarity detection algorithm corresponding to the monitoring area, it is also necessary to determine the similarity threshold corresponding to the similarity detection algorithm to form a combination relationship of monitoring area-similarity detection algorithm-similarity threshold. The similarity threshold can be obtained based on experiments or data statistics. The method for determining the similarity threshold is not specifically limited in the embodiments of this specification.

[0051] In a specific embodiment of the present specification, for the PSNR algorithm, the similarity threshold can be set to 0.8; for the SSIM algorithm, the similarity threshold can be set to 40; for the Canny detection algorithm, the similarity threshold can be set to 0.5; for the template matching algorithm, the similarity threshold can be set to 0.5; for the black and white screen detection algorithm, when the brightness of the image is less than 0.01f or the brightness of the image is less than 0.05f and the proportion of the main color of the image is greater than the set threshold, it is considered to be a black screen; when the brightness of the image is greater than 0.99f or the brightness of the image is greater than 0.95f and the proportion of the main color of the image is greater than the set threshold, it is considered to be a white screen.

[0052] During the test, the user calculates the regional similarity between the video frames in the video frame sequence and the reference frame based on the monitoring area to obtain the similarity result, and then compares the similarity result with the similarity threshold. The video frame that meets the similarity threshold can be determined as a hit frame. In order to improve the accuracy and reliability of the hit frame, in a specific embodiment of this specification, multiple monitoring areas can be selected for similarity detection. The similarity detection algorithm of each monitoring area can be the same or different. The user configures it according to the actual situation. When there are multiple monitoring areas, the video frame whose similarity in each monitoring area meets the corresponding similarity threshold can be determined as a hit frame. After the hit frame is determined, the serial number of the hit frame is added to the hit frame set. The serial number of the hit frame is its serial number in the video frame sequence. In the above embodiment, the key frames corresponding to the application startup nodes are determined by analyzing the video files of the application startup process, so that the analysis of the application startup process is more accurate and reliable; secondly, detection is performed based on the monitoring area of ​​the reference frame, and then the key video frames corresponding to each startup node are screened out using the similarity threshold, thereby realizing the positioning of each key node in the application startup process, with good robustness, high accuracy and flexible application scenarios; a variety of different similarity detection algorithms are adopted, which can meet a richer range of detection scenarios while improving the calculation accuracy.

[0053] The hit frame set obtained through the above embodiment is a candidate set of key frames. The principle of determining key frames from the hit frame set is explained below with reference to Figure 4. Figure 4 shows a trend diagram of regional similarity changes during application startup. It should be noted that Figure 4 is merely an illustrative diagram of the embodiment of this specification and does not limit the embodiments of this specification in any way. Figure 4 shows that the changing trends of regional similarity differ significantly at different stages of application startup. Upon startup, the application first loads the executable file and initializes it, then loads the data cache. During the cache loading process, the image of the monitored area changes significantly, resulting in a downward trend in regional similarity. As the cache loading process progresses, the regional similarity drops to a certain level before rebounding. After cache loading is complete, the regional similarity reaches a high value. After cache loading is complete, the application requests data from the network and refreshes the application interface based on the network data. During this process, the image of the monitored area changes significantly again, resulting in a significant decrease in regional similarity. As the interface refresh process progresses, the regional similarity drops to a certain level before rebounding. After the interface refresh is complete, the regional similarity returns to a high value and remains stable for a period of time. The network request process may occur multiple times, but the changing trends of regional similarity remain the same. Based on the above principle, the different stages of application startup can be identified based on the similarity changing trends: application startup - cache loading - network request - application startup completion.

[0054] In combination with the above principles, in an embodiment of the present specification, after determining the hit frame set from the video frame sequence based on the regional similarity threshold, the hit frame set is further divided into three subsets based on the continuity of the frame sequence numbers in the hit frame set: a starting subset, a steady-state subset, and an ending subset. The three subsets correspond to different stages of application startup, and the corresponding key frames can be determined from the above three subsets.

[0055] In a specific embodiment of the present specification, taking scientificity and accuracy into consideration, the frame with the largest sequence number in the starting subset is used as the key frame corresponding to the starting starting node; the frame with the smallest sequence number in the steady-state subset is used as the key frame corresponding to the cache loading completion node; the frame with the largest sequence number in the steady-state subset is used as the key frame corresponding to the requesting network node; and the frame with the smallest sequence number in the ending subset is used as the key frame corresponding to the starting completion node.

[0056] In this way, the key nodes of application startup can be determined from the audio recording of the application startup process, thereby providing accurate and scientific data support for analyzing the application startup process. The method in the embodiments of this specification allows users to configure multiple monitoring areas and detection algorithms, thereby achieving high accuracy without requiring a large amount of data to train the model.

[0057] In a specific embodiment of this specification, considering that there is a significant difference between the interface when the application starts and the interface after the application is started, if only one reference frame is used during the test, it will lead to a large deviation in the determination of specific nodes. Therefore, in order to improve the accuracy of node positioning, the first reference frame is selected to determine the startup starting node, and the second reference frame is selected to determine the other three nodes.

[0058] In a specific embodiment of this specification, the first frame of a video file is selected as the first reference frame, and the last frame of the video file is selected as the second reference frame. For the startup start node, after the first frame is selected as the reference frame, the similarity calculation is performed on all subsequent frames with the first frame. For the cache load completion node, the request network node, and the startup completion node, after the last frame is selected as the reference frame, the similarity calculation is performed on all frames with the last frame.

[0059] Please refer to FIG5 , which illustrates an application startup node location device provided by an embodiment of this specification. As shown in FIG5 , the application startup node location device 500 includes a video acquisition unit 502 , a reference frame determination unit 504 , a hit frame determination unit 506 , and a key frame determination unit 508 . The primary functions of each component are as follows.

[0060] The video acquisition unit 502 is configured to acquire a video file of the application startup process.

[0061] The reference frame determining unit 504 is configured to determine a reference frame from the video file, where the reference frame includes a monitoring area and a similarity detection algorithm corresponding to the monitoring area.

[0062] The hit frame determining unit 506 is configured to search for hit frames in the video file based on the reference frame to obtain a hit frame set.

[0063] The key frame determining unit 508 is configured to determine a key frame in the hit frame set, where the key frame corresponds to a node in the application startup process.

[0064] The application startup process is as follows: (1) The user clicks the application icon or starts the application through other means; (2) The operating system loads the application's executable file into the memory, and then the operating system calls the application's entry function to perform initialization operations. At this stage, the application will perform some basic settings, such as creating windows, initializing data structures, etc.; (3) Cache loading: The application may need to load some resources, such as images, audio, configuration files, etc. During the startup process, the application will read these resource files and load them into the memory for subsequent use; (4) User interface display: Once the application is initialized, it will start to create and display the user interface; (5) Request network: The application may perform some specific tasks at startup, such as communicating with the server, refreshing the page, upgrading components, etc. Based on this, in the embodiment of this specification, the cold start process of the application is divided into four key nodes: the startup start node, the cache loading completion node, the network request node, and the startup completion node. The above nodes are the nodes that need to be located in the embodiment of this specification.

[0065] 6 shows a hit frame determination unit 506 provided in an embodiment of this specification, which includes a threshold determination unit 602, a similarity calculation unit 604, a hit frame search unit 606, and a set generation unit 608. The main functions of each component unit are as follows.

[0066] The threshold determination unit 602 is configured to determine a similarity threshold corresponding to the similarity detection algorithm.

[0067] The similarity calculation unit 604 is configured to calculate the regional similarity between each frame image in the video file and the reference frame based on the monitoring area and the similarity detection algorithm.

[0068] The hit frame search unit 606 is configured to determine the image frame whose region similarity meets the similarity threshold as a hit frame.

[0069] The set generating unit 608 is configured to determine the hit frame set based on the frame sequence number of the hit frame.

[0070] After configuring the monitoring area and the similarity detection algorithm corresponding to the monitoring area, it is also necessary to determine the similarity threshold corresponding to the similarity detection algorithm to form a combination relationship of monitoring area-similarity detection algorithm-similarity threshold. The similarity threshold can be obtained based on experiments or data statistics. The method for determining the similarity threshold is not specifically limited in the embodiments of this specification.

[0071] During the test, the user calculates the regional similarity between the video frames in the video frame sequence and the reference frame based on the monitoring area to obtain the similarity result, and then compares the similarity result with the similarity threshold. The video frame that meets the similarity threshold can be determined as a hit frame. In order to improve the accuracy and reliability of the hit frame, in a specific embodiment of this specification, multiple monitoring areas can be selected for similarity detection. The similarity detection algorithm of each monitoring area can be the same or different. The user configures it according to the actual situation. When there are multiple monitoring areas, the video frame whose similarity in each monitoring area meets the corresponding similarity threshold can be determined as a hit frame. After the hit frame is determined, the serial number of the hit frame is added to the hit frame set. The serial number of the hit frame is its serial number in the video frame sequence. In the above embodiment, the key frames corresponding to the application startup nodes are determined by analyzing the video files of the application startup process, so that the analysis of the application startup process is more accurate and reliable; secondly, detection is performed based on the monitoring area of ​​the reference frame, and then the key video frames corresponding to each startup node are screened out using the similarity threshold, thereby realizing the positioning of each key node in the application startup process, with good robustness, high accuracy and flexible application scenarios; a variety of different similarity detection algorithms are adopted, which can meet a richer range of detection scenarios while improving the calculation accuracy.

[0072] In an embodiment of the present specification, after determining the hit frame set from the video frame sequence based on the regional similarity threshold, the key frame determination unit also includes a subset determination unit, which is configured to divide the hit frame set into three subsets based on the continuity of the frame sequence numbers in the hit frame set: a starting subset, a steady-state subset, and an ending subset. The three subsets correspond to different stages of application startup, respectively, and the corresponding key frames can be determined from the above three subsets.

[0073] In a specific embodiment of the present specification, taking into account scientificity and accuracy, the key frame search unit is configured to use the frame with the largest sequence number in the starting subset as the key frame corresponding to the starting starting node; use the frame with the smallest sequence number in the steady-state subset as the key frame corresponding to the cache loading completion node; use the frame with the largest sequence number in the steady-state subset as the key frame corresponding to the requesting network node; and use the frame with the smallest sequence number in the ending subset as the key frame corresponding to the starting completion node.

[0074] In this way, the key nodes of application startup can be determined from the audio recording of the application startup process, thereby providing accurate and scientific data support for analyzing the application startup process. The method in the embodiments of this specification allows users to configure multiple monitoring areas and detection algorithms, thereby achieving high accuracy without requiring a large amount of data to train the model.

[0075] In an embodiment of the present specification, a black and white screen calculation unit is also included, which is configured to determine the average brightness of the monitoring area based on the HSV model; extract and sort the colors of the monitoring area based on the Kmeans algorithm; and determine whether the monitoring area is a black and white image based on the average brightness and main color of the monitoring area.

[0076] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0077] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0078] In addition, embodiments of this specification also provide another computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the steps of any of the methods described in the aforementioned embodiments. If the components of the aforementioned apparatus are implemented as software functional units and sold or used as independent products, they may be stored in the computer-readable storage medium.

[0079] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiments of this specification is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0080] An embodiment of this specification also provides an electronic device, comprising: one or more processors, and a memory associated with the one or more processors, wherein the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the steps of the method described in any one of the aforementioned method embodiments are executed.

[0081] This specification also provides a computer program product, including a computer program, which implements the steps of any one of the methods in the aforementioned method embodiments when executed by a processor.

[0082] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When executed, the program can include the processes of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. The technical features of this embodiment and the implementation scheme can be combined in any manner unless they conflict.

[0083] 7 exemplarily shows the architecture of an electronic device 700, which may include a processor 710, a disk drive 720, an input / output interface 730, a network interface 740, and a memory 750. The processor 710, disk drive 720, input / output interface 730, network interface 740, and memory 750 may be communicatively connected via a communication bus.

[0084] Among them, the processor 710 can be implemented by a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to implement the technical solutions provided in this application.

[0085] The memory 750 can be implemented in the form of ROM (Read Only Memory), RAM (Read Access Memory), static memory, dynamic storage device, etc. The memory 750 can store an operating system 751 for controlling the operation of the electronic device 700 and a basic input and output system (BIOS) 752 for controlling the first-level operations of the electronic device 700. In addition, a web browser 753, a data storage management system 754, and an application startup node positioning device 755 can also be stored. In short, when the technical solution provided by this application is implemented through software or firmware, the relevant program code is stored in the memory 750 and is called and executed by the processor 710.

[0086] The input / output interface 730 is used to connect an input / output module to implement information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0087] The network interface 740 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0088] The bus comprises a pathway that transmits information between various components of the device (eg, processor 710, disk drive 720, input / output interface 730, network interface 740, and memory 750).

[0089] It should be noted that although the above device only shows the processor 710, the disk drive 720, the input / output interface 730, the network interface 740, the memory 750, the bus, etc., in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the above device may only include the components necessary to implement the method of the present application, and does not necessarily include all the components shown in the figure.

[0090] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Without departing from the design spirit of this specification, various modifications and improvements made to the technical solutions of this specification by ordinary technicians in this field should fall within the scope of protection determined by the claims of this specification.

Claims

1. Application startup node positioning method, including: Get the video file of the application startup process; determining a reference frame from the video file, the reference frame including a monitoring area and a similarity detection algorithm corresponding to the monitoring area; Searching for a hit frame in the video file based on the reference frame to obtain a hit frame set; A key frame is determined in the hit frame set, where the key frame corresponds to a node in the application startup process.

2. The method according to claim 1, wherein searching for hit frames in the video file based on the reference frame to obtain a hit frame set comprises: Determining a similarity threshold corresponding to the similarity detection algorithm; Calculating the regional similarity between each frame image in the video file and the reference frame based on the monitoring area and the similarity detection algorithm; Determine the image frame whose region similarity meets the similarity threshold as a hit frame; The hit frame set is determined based on the frame sequence number of the hit frame.

3. The method according to claim 2, wherein determining a key frame in the hit frame set comprises: Dividing the hit frame set into three subsets based on the continuity of frame sequence numbers in the hit frame set, the three subsets comprising a start subset, a steady-state subset, and an end subset; The key frames are determined from the three subsets.

4. The method according to claim 3, comprising: The nodes in the application startup process include a startup start node, and the frame with the largest sequence number in the startup subset is used as the key frame corresponding to the startup start node.

5. The method according to claim 3, comprising: The nodes in the application startup process include a cache loading completion node, and the frame with the smallest sequence number in the steady-state subset is used as the key frame corresponding to the cache loading completion node.

6. The method according to claim 3, comprising: The nodes in the application startup process include a requesting network node, and the frame with the largest sequence number in the steady-state subset is used as a key frame corresponding to the requesting network node.

7. The method according to claim 3, comprising: The nodes in the application startup process include a startup completion node, and the frame with the smallest sequence number in the end subset is used as the key frame corresponding to the startup completion node.

8. The method according to claim 1, wherein the similarity detection algorithm comprises: One or more of the following detection algorithms: PSNR detection algorithm, SSIM detection algorithm, Canny detection algorithm, black and white screen detection algorithm, and template feature matching algorithm.

9. The method according to claim 8, wherein the black and white screen detection algorithm comprises: Determine the average brightness of the monitoring area based on the HSV model; Extracting and sorting the colors of the monitoring area based on the Kmeans algorithm; It is determined whether the monitoring area is a black and white image based on the average brightness and main color of the monitoring area.

10. The method according to claim 1, comprising: The reference frame includes a plurality of monitoring areas; Different monitoring areas correspond to the same or different similarity detection algorithms.

11. The method according to claim 2, comprising: When there are multiple monitoring areas, an image frame whose similarity in each area meets a corresponding similarity threshold is determined as a hit frame.

12. Application startup node positioning device, including: A video acquisition unit, configured to acquire a video file of the application startup process; a reference frame determining unit configured to determine a reference frame from the video file, the reference frame including a monitoring area and a similarity detection algorithm corresponding to the monitoring area; a hit frame determining unit, configured to search for a hit frame in the video file based on the reference frame to obtain a hit frame set; The key frame determining unit is configured to determine a key frame in the hit frame set, where the key frame corresponds to a node in the application startup process.

13. The apparatus according to claim 12, wherein the hit frame determining unit comprises: a threshold determination unit, configured to determine a similarity threshold corresponding to the similarity detection algorithm; a similarity calculation unit configured to calculate a region similarity between each frame image in the video file and the reference frame based on the monitoring region and the similarity detection algorithm; a hit frame search unit, configured to determine an image frame whose region similarity satisfies a similarity threshold as a hit frame; The set generating unit is configured to determine the hit frame set based on the frame sequence number of the hit frame.

14. The apparatus according to claim 13, wherein the key frame determining unit comprises: a subset determining unit configured to divide the hit frame set into three subsets based on continuity of frame sequence numbers in the hit frame set, the three subsets comprising a start subset, a steady-state subset, and an end subset; The key frame search unit is configured to determine the key frame from the three subsets.

15. The apparatus according to claim 14, comprising: The nodes in the application startup process include a startup start node, and the frame with the largest sequence number in the startup subset is used as the key frame corresponding to the startup start node.

16. The apparatus according to claim 14, wherein the key frame search unit is configured to: The nodes in the application startup process include a cache loading completion node, and the frame with the smallest sequence number in the steady-state subset is used as the key frame corresponding to the cache loading completion node.

17. The apparatus according to claim 14, comprising: The nodes in the application startup process include a requesting network node, and the frame with the largest sequence number in the steady-state subset is used as a key frame corresponding to the requesting network node.

18. The apparatus according to claim 14, comprising: The nodes in the application startup process include a startup completion node, and the frame with the smallest sequence number in the end subset is used as the key frame corresponding to the startup completion node.

19. The apparatus according to claim 12, wherein the similarity detection algorithm comprises: One or more of the following detection algorithms: PSNR detection algorithm, SSIM detection algorithm, Canny detection algorithm, black and white screen detection algorithm, and template feature matching algorithm.

20. The apparatus according to claim 19, comprising: a black and white screen calculation unit, configured to determine the average brightness of the monitoring area based on an HSV model; The colors of the monitoring area are extracted and sorted based on the Kmeans algorithm; and whether the monitoring area is a black and white image is determined based on the average brightness and main color of the monitoring area.

21. The apparatus according to claim 12, comprising: The reference frame includes a plurality of monitoring areas; Different monitoring areas correspond to the same or different similarity detection algorithms.

22. The apparatus according to claim 13, comprising: When there are multiple monitoring areas, an image frame whose similarity in each area meets a corresponding similarity threshold is determined as a hit frame.

23. A computer storable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 11.

24. Electronic equipment, including: one or more processors, and A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the steps of the method according to any one of claims 1 to 11 are executed.

Images