Method for identifying starting time of application software in video

Abstract

The invention provides a method for identifying the starting time of application software in a video, and the method comprises the following operation steps: S1, automatic screen recording: carrying out the screen recording from the static state of a mobile phone desktop to the beginning of clicking the application software; s2, acquiring starting time of application starting; s3, acquiring the time when the home page is obtained; and S4, performing subsequent treatment. According to the method, manual intervention is not needed, a machine is used for automatically counting the starting time length of the application software and the time length of complete loading to the home page, the video of the application software from click starting to complete loading to the home page can be automatically recorded, video pictures are extracted, key information data are extracted by using an OCR technology, corresponding analysis algorithms are formulated for different platforms, and the analysis efficiency is improved. According to the method, the starting duration and the home page loading duration of the application are analyzed, whether advertisements are contained or not during starting can be automatically recognized, the accuracy of counting the starting time of the application software is high, manpower is not needed, and the labor cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of identifying the startup time of application software, and particularly to a method for identifying the startup time of application software in a video. Background Art

[0002] The startup time of application software refers to the time elapsed from when a user triggers a startup operation (such as clicking on a desktop icon, running a program through the command line, etc.) until the application software is fully loaded and available for user interaction. With the popularization of smart devices, the number of users using smart devices is increasing, resulting in various application software installed on smart devices being frequently used. Therefore, there are relatively high requirements for the startup time of application software. The startup time interval of application software covers multiple stages, including but not limited to: system response, where the operating system receives the user's startup request and begins to schedule resources to load the application software; process creation, where the operating system creates a new process or thread for the application software and allocates necessary system resources such as memory and processor time; dependency loading, where the application software may depend on other library files, dynamic link libraries (DLLs), services, or components, and these dependencies need to be found and loaded into memory before the main functions of the application software are loaded; initialization, where the application software executes its initialization code, which may include setting global variables, configuring system resources, loading configuration files, etc.; user interface rendering, if the application software has a graphical user interface (GUI), then the startup screen or main interface needs to be rendered, which usually involves interaction with the graphics hardware and the operating system; background tasks, where the application software may perform some startup tasks in the background, such as checking for updates, connecting to a server, loading data, etc.; ready state, finally, the application software reaches the ready state and the user can interact with it.

[0003] The existing methods for measuring the startup time of application software are generally manual statistics. When performing manual statistics, it is necessary to turn on the screen recording function of the mobile phone and analyze frame by frame from the moment the application software is clicked until the application software jumps to the home page or the home page is fully loaded. Considering various reasons such as different mobile phone performances and network factors, one statistics is not accurate, and it is necessary to record and statistics multiple times using different mobile phones, which is a large workload and wastes too much human resources. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of low accuracy and high labor cost in statistically calculating the startup time of application software in the prior art, and to provide a method for identifying the startup time of application software in a video.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] The present invention provides a method for identifying the startup time of application software in a video, and the method includes the following operating steps:

[0007] S1. Automatically record the screen, starting from the stationary state of the mobile phone desktop until clicking on the application software to start recording the screen.

[0008] S2. Obtain the application startup start time, split the video of the automatic screen recording into pictures, record the time stamps corresponding to each frame of the picture, compare the similarity of the pictures of the front and back frames, and determine whether it exceeds the set threshold. If it does not exceed, re - compare. If it exceeds, determine that the application starts.

[0009] S3. Obtain the time to reach the home page. According to the result of the application startup start in step S2, it is divided into two cases. The first is the case with advertisements, then perform OCR comparison and similarity comparison on the pictures, and calculate the specific time from the application startup to the home page. The second is the case without advertisements, perform OCR comparison on the pictures, and calculate the specific time from the application startup to the home page.

[0010] S4. Follow - up processing, perform follow - up processing and summarization on the calculation results of S3, and generate a report.

[0011] In this technical solution, without manual intervention, the machine automatically counts the startup duration of the application software and the duration until the home page is fully loaded. It can automatically record the video of the application software from clicking to start until the home page is fully loaded, extract video pictures, use OCR technology to extract key information data, formulate corresponding analysis algorithms for different platforms, analyze the startup duration of the application and the home page loading duration, and can automatically identify whether there are advertisements at startup. The statistical accuracy of the application software startup time is high and does not require manual labor, reducing the labor cost.

[0012] Preferably, in step S1, the application software includes Android applications and IOS applications.

[0013] In this technical solution, the application software is divided into applications in the Android system and the IOS system. The Android system and the IOS system are the two most popular mobile operating systems in today's smartphone and tablet computer markets.

[0014] Preferably, the specific operation process of step S2 is as follows:

[0015] A1. Video reception, receive the video generated by the automatic screen recording in S1.

[0016] A2. Video frame extraction, split the video received in A1 into pictures, and record the time stamps corresponding to each frame of the picture.

[0017] A3. Compare the similarity of the front and back frame pictures, start from the first frame to compare the similarity of the front and back frame pictures.

[0018] A4. Determine whether it exceeds the threshold. If it exceeds the set threshold twice consecutively in A3, it is determined that the application startup begins, and step S3 is executed. If it does not exceed the set threshold, step A3 is executed.

[0019] In this technical solution, the application startup start time is obtained by using the operation steps of S2.

[0020] Preferably, in the A4 step, if it exceeds the set threshold twice consecutively, it is determined that the application startup begins, and the picture similarity between the front and back frames is compared starting from the first frame.

[0021] Preferably, the picture similarity comparison uses the SSIM similarity algorithm.

[0022] Preferably, in the case of an advertisement in the S3 step, OCR recognition is performed on the picture. If an advertisement or a keyword pattern for jumping to a third-party application is recognized, it is intercepted from bottom to top according to the height of the bottom navigation bar of the application software, and only the height of the bottom navigation bar is retained. OCR recognition is performed on this part of the picture again. If the keyword pattern of the bottom navigation bar of the application is recognized, it indicates that the application has been started to the home page, and the timestamp of this picture frame is obtained, and the specific time from the startup of the application to the home page is obtained by subtracting the startup start timestamp.

[0023] In this technical solution, it is determined how to obtain the specific time from the startup of the application to the home page in the case of an advertisement.

[0024] Preferably, in the case of no advertisement in the S3 step, OCR recognition is performed on the picture. If the keyword pattern of the bottom navigation bar of the application is recognized, it indicates that the application has been started to the home page, and the timestamp of this picture frame is obtained, and the specific time from the startup of the application to the home page is obtained by subtracting the startup start timestamp.

[0025] In this technical solution, it is determined the specific time from the startup of the application to the home page in the case of no advertisement.

[0026] Preferably, the keyword pattern can be set differently according to the startup advertisements of different application software.

[0027] In this technical solution, the definition of the keyword pattern is determined.

[0028] Preferably, in the S3 step, the smaller the picture size, the faster the OCR recognition speed and the picture similarity comparison speed.

[0029] Preferably, the specific operation process of the S4 step is as follows:

[0030] B1. Video log, obtain the video log of step S3;

[0031] B2. Classify and store video logs, and store the video logs in step B1.

[0032] B3. Summarize and analyze, and summarize and analyze the video logs in step B2.

[0033] B4. Generate a report, and generate a report according to the results in step B3.

[0034] In this technical solution, there is no need for manual intervention. The computer is used for recognition throughout the process and automatically reported to the log collection platform for summarizing and analyzing the startup completion time or loading completion time of each application software.

[0035] On the basis of conforming to the common knowledge in the field, the above preferred conditions can be combined arbitrarily to obtain various preferred embodiments of the present invention.

[0036] The positive and progressive effects of the present invention are as follows:

[0037] The present invention does not require manual intervention. It uses a machine to automatically count the startup duration of application software and the duration from the start of clicking to the full loading of the home page. It can automatically record the video of the application software from clicking to start to the full loading of the home page, extract video pictures, use OCR technology to extract key information data, formulate corresponding analysis algorithms for different platforms, analyze the startup duration and home page loading duration of the application, and can automatically identify whether there are advertisements at startup. The accuracy of counting the startup time of application software is high and no manual work is required, reducing the labor cost. Brief Description of the Drawings

[0038] Figure 1 It is a schematic flowchart of the method for identifying the startup time of application software in the video of the embodiment of the present invention.

[0039] Figure 2 is Figure 1 The complete interaction flowchart of the method for identifying the startup time of application software in the video shown. Detailed Embodiments

[0040] The present invention will be further described below by way of examples, but the present invention is not limited to the scope of the described examples.

[0041] Figures 1 to 2 The following shows the structural schematic diagram of the embodiment of the method for identifying the startup time of application software in the video of the present invention. The method for identifying the startup time of application software in the video includes the following operating steps:

[0042] S1. Automatically record the screen, and start recording the screen from the static state of the mobile phone desktop until starting to click on the application software.

[0043] S2. Obtain the start time of application startup. Split the automatically recorded screen video into frames of pictures, record the time stamps corresponding to each frame of picture, compare the similarity of pictures between the front and back frames, and determine whether it exceeds the set threshold. If it does not exceed, compare again. If it exceeds, determine the start of application startup;

[0044] S3. Obtain the time to reach the home page. According to the result of the start of application startup in step S2, there are two cases. The first is the case with advertisements, then perform OCR comparison and similarity comparison on the pictures, and calculate the specific time from application startup to the home page. The second is the case without advertisements, perform OCR comparison on the pictures, and calculate the specific time from application startup to the home page;

[0045] OCR (Optical Character Recognition) is a technology that can convert the text in images, scanned documents or photos into machine-encoded text;

[0046] The working principle of OCR technology usually involves the following steps:

[0047] Step 1. Image preprocessing. First, preprocess the input image to improve the recognition accuracy of OCR. This includes operations such as denoising, binarization, rotation correction, and image enhancement. The purpose of preprocessing is to make the text in the image clearer and easier to recognize;

[0048] Step 2. Character segmentation. In the preprocessed image, the OCR algorithm needs to accurately segment individual characters. This step is crucial for subsequent character recognition. The accuracy of character segmentation directly affects the recognition effect of OCR;

[0049] Step 3. Feature extraction. For each segmented character, the OCR algorithm extracts its features. These features may include information such as the contour, shape, size, and strokes of the character. The purpose of feature extraction is to convert the character into a representation that can be understood by a computer;

[0050] Step 4. Character recognition. After extracting the features, the OCR algorithm uses machine learning or deep learning models to classify and recognize the characters. These models have been trained to recognize various characters and their variants. The accuracy of character recognition depends on the training degree of the model and the quality of character feature extraction;

[0051] Step 5. Post-processing. After character recognition, the OCR algorithm may also need to perform post-processing steps, such as correcting incorrect characters, adjusting the text layout, and merging characters with split errors. The purpose of post-processing is to further improve the recognition accuracy of OCR and the readability of the output text.

[0052] S4. Subsequent processing: perform subsequent processing and summarization on the calculation results of S3, and generate a report.

[0053] The present invention can automatically analyze various application software on iOS and Android platforms from startup to loading the screen recording video, convert the video into pictures, calculate the target of the recognition area, and use OCR technology to only recognize the text in the recognition area, reducing useless recognition data and improving the recognition efficiency. By formulating partial features of applications on different platforms, calculate the time from the startup of the application software to jumping to the home page and the completion of the home page loading. Without manual intervention, the whole process uses computer recognition and automatically reports to the log collection platform for summarizing and analyzing the startup completion or loading completion time of each application software.

[0054] In this technical solution, without manual intervention, the machine is used to automatically count the startup duration of the application software and the duration until the home page is fully loaded. It can automatically record the video of the application software from clicking to start until the home page is fully loaded, extract video pictures, use OCR technology to extract key information data, formulate corresponding analysis algorithms for different platforms, analyze the startup duration and home page loading duration of the application, and can automatically identify whether there is an advertisement at startup. The statistical accuracy of the application software startup time is high and no manual work is required, reducing the labor cost.

[0055] In the S1 step, the application software includes Android applications and IOS applications.

[0056] In this technical solution, the application software is divided into applications in the Android system and the IOS system. The Android system and the IOS system are the two most popular mobile operating systems in today's smartphone and tablet computer markets.

[0057] In the S1 step, the automatic screen recording starts from the static state of the mobile phone desktop before starting to click on the application software and ends when entering the home page of the application software and the home page is fully loaded.

[0058] The specific operation process of the S2 step is as follows:

[0059] A1. Video reception: receive the video generated by the automatic screen recording in S1;

[0060] A2. Video frame extraction: split the video received in A1 into pictures and record the time stamp corresponding to each picture;

[0061] A3. Compare the similarity of the front and rear frame pictures: start from the first frame and compare the similarity of the front and rear frame pictures;

[0062] A4. Judge whether it exceeds the threshold: if it continuously exceeds the set threshold twice in A3, it is judged that the application startup starts, and the S3 step is executed; if it does not exceed the set threshold, the A3 step is executed.

[0063] In this technical solution, the application startup start time is obtained by using the S2 operation step.

[0064] When in use, the video of automatic screen recording is split into frames to form pictures, and the time stamp corresponding to each frame picture is recorded. The height of 90% of the screen is intercepted from bottom to top to avoid the influence of system time change on the picture similarity result. The first step: start from the first frame to compare the picture similarity of the front and back frames. If it exceeds the set threshold twice in a row, it is determined that the application starts. Set the picture that exceeds the threshold for the first time as the startup start and obtain the time stamp of this frame.

[0065] In the A4 step, if it exceeds the set threshold twice in a row, it is determined that the application starts, and the picture similarity of the front and back frames is compared starting from the first frame.

[0066] The picture similarity comparison uses the SSIM similarity algorithm.

[0067] The SSIM similarity algorithm is an index for measuring the similarity of two images. It takes into account the brightness, contrast and structural information of the images, so it is closer to the visual perception of the human eye;

[0068] The SSIM algorithm compares from three dimensions of the brightness, contrast and structure of the image to evaluate the similarity of the two images.

[0069] In the case of an advertisement in the S3 step, OCR recognition is performed on the picture. If keywords indicating an advertisement or jumping to a third-party application are recognized, then it is intercepted from bottom to top according to the height of the bottom navigation bar of the application software, and only the height of the bottom navigation bar is retained. OCR recognition is performed on this part of the picture again. If keywords of the bottom navigation bar of the application are recognized, it indicates that the application has started to the home page. Obtain the time stamp of this picture frame, and subtract it from the startup start time stamp to obtain the specific time from the startup of the application to the home page.

[0070] In this technical solution, it is determined how to obtain the specific time from the startup of the application to the home page in the case of an advertisement.

[0071] In the case of no advertisement in the S3 step, OCR recognition is performed on the picture. If keywords of the bottom navigation bar of the application are recognized, it indicates that the application has started to the home page. Obtain the time stamp of this picture frame, and subtract it from the startup start time stamp to obtain the specific time from the startup of the application to the home page.

[0072] In this technical solution, it is determined the specific time from the startup of the application to the home page in the case of no advertisement.

[0073] The keyword patterns can be set differently according to the startup advertisements of different application software.

[0074] In this technical solution, the definition of the keyword pattern is determined.

[0075] In the S3 step, the smaller the picture size, the faster the OCR recognition speed and the picture similarity comparison speed.

[0076] After obtaining that the application software jumps to the home page, take the last frame of the recorded video, and intercept a part of the picture according to different application software (this part needs to avoid the animated area and intercept the area with more content), and compare the SSIM similarity with the pictures in the same area of each frame after jumping to the home page. If it exceeds a certain threshold continuously for 3 times, it is determined that the home page loading is completed. Subtract the timestamp corresponding to the picture when it first exceeds the threshold from the startup timestamp, and then the total time from the startup of the application to the completion of the home page loading can be obtained.

[0077] The specific operation process of the S4 step is as follows:

[0078] B1. Video log, obtain the video log of the S3 step;

[0079] B2. Classify and store the video log, and store the video log in the B1 step;

[0080] B3. Summarize and analyze, summarize and analyze the video log in the B2 step;

[0081] B4. Generate a report, generate a report according to the result in the B3 step.

[0082] In this technical solution, no manual intervention is required. The computer recognition is used throughout the process and automatically reported to the log collection platform for summarizing and analyzing the startup completion or loading completion time of each application software.

[0083] Although the specific implementation manners of the present invention have been described above, those skilled in the art should understand that this is only an example. The protection scope of the present invention is defined by the appended claims. Without departing from the principles and essence of the present invention, those skilled in the art can make various changes or modifications to these implementation manners, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A method for identifying the startup time of application software in a video, characterized in that: The method comprises the following steps: S1, automatic screen recording, starting from the static state of the mobile phone desktop to before clicking the application software; S2. Obtain the start time of application startup, split the automatically recorded video into pictures, record the timestamp corresponding to each frame, compare the picture similarity of the previous and next frames, and determine whether it exceeds the set threshold. If not, re-compare. If so, determine that the application startup has started; S3, the time of getting the home page is divided into two cases according to the result of the application startup in step S2. The first case is the case with advertisements, in which OCR comparison and similarity comparison are performed on the pictures to calculate the specific time when the application is started to the home page. The second case is the case without advertisements, in which OCR comparison is performed on the pictures to calculate the specific time when the application is started to the home page. S4: Subsequent processing: summarize the calculation results of S3 and generate a report.

2. The method for identifying the startup time of application software in a video according to claim 1, characterized in that: The application software in step S1 includes Android applications and IOS applications.

3. The method for identifying the startup time of application software in a video according to claim 1, characterized in that: The specific operation process of the S2 step is: A1, video reception, receiving the video generated by the automatic screen recording in S1; A2: Extract video frames, split the video received in A1 into frames, and record the timestamp corresponding to each frame; A3, comparing the similarity of the previous and next frames, starting from the first frame, the previous and next frames are compared; A4: Determine whether the threshold is exceeded. If the threshold is exceeded twice in A3, it is determined that the application is started and step S3 is executed. If the threshold is not exceeded, step A3 is executed.

4. The method for identifying the startup time of application software in a video according to claim 1, characterized in that: In the step A4, if the set threshold is exceeded twice in succession, it is determined that the application is started, and the image similarity of the previous and next frames is compared starting from the first frame.

5. The method for identifying the startup time of application software in a video according to claim 4, characterized in that: The image similarity comparison adopts the SSIM similarity algorithm.

6. The method for identifying the startup time of application software in a video according to claim 1, characterized in that: In the case where there are advertisements in the S3 step, OCR recognition is performed on the image. If an advertisement or a keyword pattern that jumps to a third-party application is recognized, the image is cut from bottom to top according to the height of the bottom navigation bar of the application software, and only the height of the bottom navigation bar is retained. OCR recognition is performed on this part of the image again. If the keyword pattern of the bottom navigation bar of the application is recognized, it indicates that the application has been started to the home page. The timestamp of the image frame is obtained, and the specific time from the startup to the home page is obtained by subtracting it from the startup start timestamp.

7. The method for identifying the startup time of application software in a video according to claim 1, characterized in that: In the absence of advertisements in step S3, OCR recognition is performed on the image. If the keyword sample of the bottom navigation bar of the application is recognized, it indicates that the application has been launched to the home page. The timestamp of the image frame is obtained and subtracted from the startup start timestamp to obtain the specific time from the startup of the application to the home page.

8. The method for identifying the startup time of application software in a video according to claim 6, characterized in that: The keyword patterns may be set differently according to the startup advertisements of different application software.

9. The method for identifying the startup time of application software in a video according to claim 1, characterized in that: In the step S3, the smaller the image size is, the faster the OCR recognition speed and the image similarity comparison speed are.

10. The method for identifying the startup time of application software in a video according to claim 1, characterized in that: The specific operation process of the S4 step is: B1, video log, obtain the video log of step S3; B2, classify and store the video logs, and store the video logs in step B1; B3, summary analysis, summary analysis of the video logs in step B2; B4. Generate a report based on the results in step B3.

Images