Video frame determination method, apparatus, and program load time test method

Abstract

The application provides a video frame determination method, device and program loading time test method. The video frame determination method gradually updates a first reference frame or a second reference frame based on the similarity between each frame of a to-be-processed video and the initial first reference frame, utilizes the idea of dichotomy, gradually reduces the video frames between the updated first reference frame and the second reference frame, gradually approaches the region where the target reference frame is located, and until the frame number between the current first reference frame and the second reference frame is less than a preset frame number difference, the target reference frame can be selected in the region with high efficiency. The direction corresponding to the loading start frame or the loading end frame is selected, and the loading start frame or the loading end frame can be found near the target reference frame. The method greatly improves the efficiency of finding the target frame, and does not need to modify a large number of parameters for different scenes, and has strong universality.

Description

Technical Field

[0001] This application relates to the field of software testing technology, and in particular to a video frame determination method, apparatus, program loading time testing method, storage medium, and computer equipment. Background Technology

[0002] When users click on the icons of various applications (Apps) installed on their devices, these interactions trigger corresponding events, resulting in page transitions. If these page transitions are time-consuming, the application may appear sluggish, leading to a poor user experience. To improve this experience, application loading speeds need to be measured, and the results used for targeted optimization to shorten the loading process.

[0003] Traditional techniques often require screen recording of the terminal, followed by manual frame-by-frame review to determine the start and end frames of the loading event. This method is inefficient. Summary of the Invention

[0004] The purpose of this application is to at least solve one of the aforementioned technical defects, particularly the technical defect of low efficiency in determining the start frame or end frame of loading in the prior art.

[0005] Firstly, this application provides a video frame determination method, including:

[0006] Determine the initial first reference frame and the second reference frame in the video to be processed; the video to be processed is a recorded video loaded by the test program, and the initial first reference frame is the first frame of the video to be processed and the second reference frame is the last frame of the video to be processed, or the initial first reference frame is the last frame and the second reference frame is the first frame.

[0007] Calculate the similarity between each frame of the video to be processed and the initial first reference frame;

[0008] Determine the intermediate frame and calculate the first similarity difference and the second similarity difference; the intermediate frame is the midpoint between the current first reference frame and the second reference frame, the first similarity difference is the difference between the similarity of the intermediate frame and the similarity of the first reference frame, and the second similarity difference is the difference between the similarity of the intermediate frame and the similarity of the second reference frame.

[0009] When the first similarity difference is less than the second similarity difference, the first reference frame is updated to an intermediate frame; when the second similarity difference is less than the first similarity difference, the second reference frame is updated to an intermediate frame.

[0010] Return to the steps of determining the intermediate frame and calculating the first similarity difference and the second similarity difference, until the frame number difference between the current first reference frame and the second reference frame is less than the preset frame number difference;

[0011] Select the frame with the largest similarity fluctuation between the current first reference frame and the second reference frame as the target reference frame;

[0012] The target frame is determined between the initial first reference frame and the target reference frame; when the initial first reference frame is the first frame, the target frame is the loading start frame; when the initial first reference frame is the last frame, the target frame is the loading end frame.

[0013] In one embodiment, calculating the similarity between each frame of the video to be processed and an initial first reference frame includes:

[0014] Binarize each frame of the video to be processed;

[0015] For any frame of the binarized video to be processed, determine whether the pixels at the same position in this frame have the same pixel value as the first reference frame, and determine the target pixel count based on the determination result; the target pixel count is the number of pixels with the same pixel value.

[0016] The similarity between the frame and the first reference frame is determined by the ratio between the number of target pixels and the total number of pixels.

[0017] In one embodiment, determining the target frame between the initial first reference frame and the target reference frame includes:

[0018] The video frames between the initial first reference frame and the target reference frame are selected as candidate frames;

[0019] For any candidate frame, calculate the difference between the pixel values ​​of the candidate frame and the initial first reference frame at each same position, and amplify the differences of pixel values ​​that are greater than the first threshold. Based on the sum of the squares of the differences of all pixel values, obtain the difference between the candidate frame and the first reference frame.

[0020] Calculate the difference in the degree of difference between each adjacent candidate frame to obtain the degree of difference fluctuation sequence;

[0021] Identify multiple abrupt change points in the differential fluctuation sequence;

[0022] The candidate frame that is closest to the target reference frame among the candidate frames corresponding to each mutation point is taken as the target frame.

[0023] In one embodiment, amplifying the difference between pixel values ​​that are greater than a second threshold includes:

[0024] The difference between pixel values ​​that are greater than the first threshold and less than or equal to the second threshold is squared and amplified.

[0025] Cube amplify the difference between pixel values ​​that are greater than the second threshold.

[0026] In one embodiment, before determining multiple abrupt change points in the difference fluctuation sequence, the method further includes:

[0027] Points in the difference fluctuation sequence that are less than the difference fluctuation threshold are smoothed.

[0028] In one embodiment, abrupt change points include extreme points, abrupt increase points located at the boundary, and / or abrupt decrease points located at the boundary.

[0029] Secondly, embodiments of this application also provide a method for testing program loading time, including:

[0030] Record the interactive interface displaying the test program for a preset duration, and trigger the loading event of the test program during the recording process to obtain the video to be processed;

[0031] The video to be processed is processed using a preset recognition algorithm to obtain the loading start frame and loading end frame; the recognition algorithm is the video frame determination method in any of the above embodiments.

[0032] The loading time of the loading event loaded by the test program is determined based on the frame rate of the video to be processed and the frame difference between the starting frame and the ending frame.

[0033] Thirdly, embodiments of this application also provide a video frame determination device, comprising:

[0034] The initialization module is used to determine the initial first reference frame and the second reference frame in the video to be processed. The video to be processed is a recorded video loaded by the test program. The initial first reference frame is the first frame of the video to be processed and the second reference frame is the last frame of the video to be processed, or the initial first reference frame is the last frame and the second reference frame is the first frame.

[0035] The first calculation module is used to calculate the similarity between each frame of the video to be processed and the initial first reference frame;

[0036] The second calculation module is used to determine the intermediate frame and calculate the first similarity difference and the second similarity difference; the intermediate frame is the midpoint between the current first reference frame and the second reference frame, the first similarity difference is the absolute value of the difference between the similarity of the intermediate frame and the similarity of the first reference frame, and the second similarity difference is the absolute value of the difference between the similarity of the intermediate frame and the similarity of the second reference frame.

[0037] The update module is used to update the first reference frame to an intermediate frame when the first similarity difference is less than the second similarity difference, and to update the second reference frame to an intermediate frame when the second similarity difference is less than the first similarity difference.

[0038] The return module is used to return the steps of determining the intermediate frame and calculating the first similarity difference and the second similarity difference until the frame number difference between the current first reference frame and the second reference frame is less than the preset frame number difference;

[0039] The target reference frame determination module is used to select the frame with the largest similarity fluctuation between the current first reference frame and the second reference frame as the target reference frame.

[0040] The target frame determination module is used to determine the target frame between the initial first reference frame and the target reference frame; when the initial first reference frame is the first frame, the target frame is the loading start frame; when the initial first reference frame is the last frame, the target frame is the loading end frame.

[0041] Fourthly, embodiments of this application also provide a computer device, including one or more processors and a memory, the memory storing computer-readable instructions, which, when executed by one or more processors, perform the steps of the video frame determination method or the program loading time test method in any of the above embodiments.

[0042] Fifthly, embodiments of this application also provide a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the video frame determination method or the program loading time test method in any of the above embodiments.

[0043] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0044] Through any of the above embodiments, based on the similarity between each frame of the video to be processed and the initial first reference frame, the first or second reference frame is gradually updated using the idea of ​​binary search. This gradually reduces the number of video frames between the updated first and second reference frames, thus gradually approaching the region where the target reference frame is located. This continues until the number of frames between the current first and second reference frames is less than a preset frame difference, at which point the target reference frame can be selected with high efficiency within that region. By selecting the direction corresponding to the loading start frame or loading end frame, the loading start frame or loading end frame can be found near the target reference frame. This method greatly improves the efficiency of finding the target frame and does not require extensive parameter modifications for different scenarios, making it highly versatile. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating the video frame determination method provided in this application embodiment;

[0047] Figure 2 This is a schematic diagram illustrating similarity changes in one embodiment of this application;

[0048] Figure 3 This is a schematic diagram illustrating similarity changes in another embodiment of this application;

[0049] Figure 4 This is a schematic diagram illustrating the changes in a differential fluctuation sequence in one embodiment of this application;

[0050] Figure 5 This is a schematic diagram illustrating the changes in the differential fluctuation sequence in another embodiment of this application;

[0051] Figure 6 This is a schematic diagram illustrating the changes in the differential fluctuation sequence in yet another embodiment of this application;

[0052] Figure 7 To Figure 6 A schematic diagram illustrating the changes in a differential fluctuation sequence after smoothing.

[0053] Figure 8 A flowchart illustrating the program loading time testing method provided in this application embodiment;

[0054] Figure 9 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] Firstly, this application provides a method for determining video frames; please refer to [link to relevant documentation]. Figure 1 This includes steps S102 to S114.

[0057] S102, determine the initial first reference frame and second reference frame in the video to be processed.

[0058] The video to be processed is a recorded video of the test program loading. Loading the test program can be from the terminal's desktop to the test program's main interface, or it can be the process of loading from one interface to another after clicking a control within the test program. The recorded video is generated by recording a preset duration on the terminal with the test program installed during the test. During recording, the terminal will be interacted with to trigger the test program to load. The frame at which the test program begins loading is called the loading start frame, which is the frame where the screen displayed on the terminal begins to change. The frame at which the test program ends loading is called the loading end frame, which is the frame where the screen displayed on the terminal stops changing. If the purpose is to determine the position of the loading start frame in the video to be processed, the initial first reference frame is the first frame of the video to be processed, and the initial second reference frame is the last frame of the video to be processed. If the purpose is to determine the position of the loading end frame in the video to be processed, the initial first reference frame is the last frame of the video to be processed, and the initial second reference frame is the first frame of the video to be processed.

[0059] S104, calculate the similarity between each frame of the video to be processed and the initial first reference frame.

[0060] It is understandable that the similarity between each frame of the video to be processed and the initial first reference frame will be related to whether loading has started or finished. Specifically, such as... Figure 2 As shown, taking the initial first reference frame as the first frame of the video to be processed as an example, during the loading process of the test program, the frames before the loading start frame are basically consistent with the first frame of the video to be processed, that is, they have a high and stable similarity. The similarity between the loading start frame and the loading end frame and the first frame of the video to be processed gradually decreases. Frames after the loading end frame, because the change has ended, differ significantly from the first frame of the video to be processed, that is, they have a low and stable similarity. Figure 3 As shown, taking the initial first reference frame as the last frame of the video to be processed as an example, during the loading process of the test program, the frames before the loading start frame differ significantly from the last frame of the video to be processed, indicating low and stable similarity. The similarity between the loading start frame and the loading end frame gradually increases with the last frame of the video to be processed. Frames after the loading end frame, having completed the change, are essentially identical to the last frame of the video to be processed, indicating high and stable similarity. There are many ways to determine the similarity between two images, which can be selected according to the actual situation, such as the various similarity comparison functions integrated in the OpenCV library.

[0061] During the research, it was found that there are similarity fluctuations at the loading start frame and loading end frame. However, for test programs with GIFs before and after loading, the similarity between the corresponding frames of the GIFs also fluctuates, making it difficult to pinpoint the loading start frame and loading end frame. However, because loading involves interface switching, the similarity fluctuations before and after loading are much larger than those caused by GIFs. Therefore, the frame with the largest similarity fluctuation must be between the loading start frame and loading end frame. Furthermore, since loading time is not very long, after identifying the frame with the largest similarity fluctuation, the loading start frame and loading end frame can be found in its vicinity. That is, using the frame with the largest similarity fluctuation as the target reference frame, and filtering directly among the video frames near the target reference frame, the efficiency of determining the loading start frame and loading end frame can be greatly improved. To accelerate the finding of the target reference frame, this application uses a binary search approach to find the target reference frame from all frames of the video to be processed.

[0062] S106, determine the intermediate frame and calculate the first similarity difference and the second similarity difference.

[0063] The intermediate frame is the midpoint between the current first reference frame and the current second reference frame. That is, the number of frames in the intermediate frame is half the sum of the number of frames in the current first reference frame and the number of frames in the current second reference frame. The first similarity difference is the absolute value of the difference between the similarity of the intermediate frame and the similarity of the first reference frame, and the second similarity difference is the absolute value of the difference between the similarity of the intermediate frame and the similarity of the second reference frame.

[0064] S108, when the first similarity difference is less than the second similarity difference, the first reference frame is updated to an intermediate frame, and when the second similarity difference is less than the first similarity difference, the second reference frame is updated to an intermediate frame.

[0065] Understandable, based on Figure 2 and Figure 3 It can be observed that the similarity is relatively stable before and after loading begins, transitioning from one stable segment to the next only after reaching the target reference frame. Based on this, a binary search method can be used to gradually narrow the range defined by the first and second reference frames. When the first similarity difference is less than the second similarity difference, it means that the intermediate frame and the first reference frame are both in the same stable segment, with the intermediate frame being closer to the target reference frame. Therefore, the current first reference frame is updated to the intermediate frame, further narrowing the range defined by the first and second reference frames. Similarly, when the second similarity difference is less than the first similarity difference, it means that the intermediate frame and the second reference frame are both in the same stable segment, with the intermediate frame being closer to the target reference frame. Therefore, the current second reference frame is updated to the intermediate frame, further narrowing the range defined by the first and second reference frames.

[0066] S110: Determine whether the frame difference between the current first reference frame and the second reference frame is less than a preset frame difference. If not, return to step S106; if yes, proceed to step S112.

[0067] It is understandable that the smaller the frame difference between the first and second reference frames, the easier it is to find the target reference frame. However, if the frame difference is too small, the target reference frame may be overlooked. Therefore, the preset frame difference can be selected according to the actual situation, ensuring that the range is small enough without ignoring the target reference frame. When the frame difference is not less than the preset frame difference, it means that the currently defined range is still too large, and we can return to step S106 to further narrow down the defined range according to the updated first and second reference frames. Conversely, it means that the range is small enough, and we can start determining the position of the target reference frame.

[0068] by Figure 2 Taking the data used for plotting as an example, the initial first reference frame is the first frame of the video to be processed, and the initial second reference frame is the last frame of the video to be processed (frame 199), with a preset frame difference of 15 frames. The intermediate frame is now frame 100. The first similarity difference is |0.232-1| = 0.768, and the second similarity difference is |0.232-0.233| = 0.001. Since the second similarity difference is less than the first similarity difference, the current second reference frame is updated to frame 100. At this point, the frame difference between the current first reference frame (frame 1) and the second reference frame (frame 100) is 99 frames, which should be further reduced. The newly determined intermediate frame is frame 51. The first similarity difference is |0.83-1| = 0.17, and the second similarity difference is |0.83-0.232| = 0.598. Since the first similarity difference is less than the second similarity difference, the current first reference frame is updated to frame 51. At this point, the frame difference between the current first reference frame (frame 51) and the second reference frame (frame 100) is 49 frames, and should be further reduced. The newly determined intermediate frame is frame 76. The first similarity difference is |0.823 - 0.83| = 0.007, and the second similarity difference is |0.823 - 0.232| = 0.591. Since the first similarity difference is less than the second similarity difference, the current first reference frame is updated to frame 76. At this point, the frame difference between the current first reference frame (frame 76) and the second reference frame (frame 100) is 24 frames, and should be further reduced. The newly determined intermediate frame is frame 88. The first similarity difference is |0.833 - 0.823| = 0.010, and the second similarity difference is |0.833 - 0.232| = 0.601. Since the first similarity difference is less than the second similarity difference, the current first reference frame is updated to frame 88. At this point, the frame difference between the current first reference frame (frame 88) and the second reference frame (frame 100) is 12 frames, so the target reference frame can be determined between frame 88 and frame 100.

[0069] S112, Select the frame with the largest similarity fluctuation between the current first reference frame and the second reference frame as the target reference frame.

[0070] Since there are not many frames between the first reference frame and the second reference frame, we can subtract the similarity of adjacent frames to obtain the similarity fluctuation value of each frame, and then determine the frame with the largest similarity fluctuation from the similarity fluctuation values ​​as the target reference frame.

[0071] S114, Determine the target frame between the initial first reference frame and the target reference frame.

[0072] When the initial first reference frame is the first frame, the target frame is the loading start frame; when the initial first reference frame is the last frame, the target frame is the loading end frame. In chronological order, the loading start frame precedes the target reference frame determined by using the first frame as the initial first reference frame. The loading end frame follows the target reference frame determined by using the last frame as the initial first reference frame. By searching in the corresponding direction from the target reference frame, the target frame can be found near it.

[0073] Based on the video frame determination method in this embodiment, the method uses a binary search approach, based on the similarity between each frame of the video to be processed and the initial first reference frame, to gradually update the first or second reference frame. This reduces the number of video frames between the updated first and second reference frames, gradually approaching the region where the target reference frame is located, until the number of frames between the current first and second reference frames is less than a preset frame difference. In this case, the target reference frame can be selected efficiently within that region. By selecting the direction corresponding to the loading start frame or loading end frame, the loading start frame or loading end frame can be found near the target reference frame. This method significantly improves the efficiency of finding the target frame and does not require extensive parameter modifications for different scenarios, making it highly versatile.

[0074] In one embodiment, calculating the similarity between each frame of the video to be processed and an initial first reference frame includes:

[0075] 1) Binarize each frame of the video to be processed.

[0076] 2) For any frame of the binarized video to be processed, determine whether the pixels at the same position in this frame have the same pixel value as those in the first reference frame, and determine the target pixel count based on the determination result. The target pixel count is the number of pixels with the same pixel value.

[0077] 3) Determine the similarity between the target pixel and the first reference frame based on the ratio between the number of target pixels and the total number of pixels.

[0078] It's understandable that after binarization, each pixel has a value of 0 or 1. Before binarization, two similar pixels will have the same pixel value. Therefore, the number of target pixels reflects the number of similar pixels in two frames. By comparing each pixel of the two binarized frames and counting the number of target pixels, as well as the ratio of the number of target pixels to the total number of pixels, the higher the result, the higher the similarity between the frame and the initial first reference frame. For example, if a video frame has 1000 pixels and the number of target pixels is 996, then 996 / 1000 = 0.996 can be directly used as the similarity between the frame and the first reference frame.

[0079] In one embodiment, determining the target frame between the initial first reference frame and the target reference frame includes:

[0080] 1) Use the video frames between the initial first reference frame and the target reference frame as candidate frames.

[0081] 2) For any candidate frame, calculate the difference between the pixel value of each pixel at the same position in the candidate frame and the initial first reference frame, and amplify the difference of the pixel values ​​that are greater than the first threshold. Based on the sum of the squares of the differences of all pixel values, obtain the difference between the candidate frame and the first reference frame.

[0082] It is understandable that, since the similarity fluctuation between the loading start frame and the loading end frame may not be significant, to accurately find the target frame from the candidate frames, the similarity between frames can be evaluated from another perspective, namely, the difference in this step. For pixels at the same location, if the difference in pixel values ​​is less than a first threshold, it means that the two pixels are relatively similar. For pixels whose difference in pixel values ​​is greater than the first threshold, these two pixels have obvious differences, and magnifying them can highlight the location of the loading start frame or loading end frame. Optionally, the specific magnification method can be to square the difference in pixel values ​​greater than the first threshold and less than or equal to the second threshold; or to cube the difference in pixel values ​​greater than the second threshold. That is, to further amplify the already obvious differences, and the larger the original difference, the greater the magnification factor. In a specific embodiment, the first threshold is 5 and the second threshold is 10.

[0083] 3) Calculate the difference between adjacent candidate frames to obtain the difference fluctuation sequence.

[0084] That is, the difference between two adjacent candidate frames is used to obtain the difference fluctuation value, and the difference fluctuation sequence includes all difference fluctuation values.

[0085] 4) Identify the mutation points in the differential fluctuation sequence.

[0086] Specifically, a mutation point is an extreme point of the difference fluctuation value. Alternatively, if there are no extreme points, and the difference fluctuation value at the boundary suddenly decreases or suddenly increases, then these sudden increases or decreases are mutation points.

[0087] 5) The candidate frame that is closest to the target reference frame among the candidate frames corresponding to the mutation point is taken as the target frame.

[0088] Because the image changes gradually from a stable image during loading in the test program, the difference will fluctuate, resulting in abrupt changes. If only one abrupt change exists, that abrupt change is the target frame. Figure 4 Taking the differential fluctuation sequence shown as an example, the initial first reference frame selected in this figure is the last frame of the video to be processed (frame 199). At this time, the target frame is the loading end frame, and the determined target reference frame is frame 90. At this time, there is no extreme point between the target reference frame and the initial first reference frame, only a drop point (dropping from frame 90 to frame 91). The video frame (frame 90) corresponding to this drop point is taken as the loading end frame.

[0089] Based on the image change patterns during loading, there may be multiple abrupt changes in the candidate frames between the target reference frame and the initial first reference frame. For example... Figure 5 Taking the differential fluctuation sequence shown as an example, the initial first reference frame selected in this figure is the last frame of the video to be processed (frame 199). At this time, the target frame is the loading end frame, and the determined target reference frame is frame 88. Then, there are multiple extreme points between the target reference frame and the initial first reference frame. Therefore, there is no need to consider sudden drop points and sudden increase points. The extreme point corresponding to frame 90 is closest to frame 88, so frame 90 is taken as the loading end frame. The multiple extreme values ​​in the interval from frame 150 to frame 170 are caused by GIF changes on the interface after the loading time is completed. Since the target reference frame has been found, these extreme points far away from the target reference frame can be excluded as possible from being the target frame, thereby filtering out the influence of GIFs on this solution.

[0090] In one embodiment, before determining multiple abrupt change points in the difference fluctuation sequence, the method further includes: smoothing points in the difference fluctuation sequence that are less than a difference fluctuation threshold. It is understood that due to potential errors during image processing, some small difference fluctuations may be mistakenly identified as abrupt change points. Figure 6Taking the differential fluctuation sequence shown as an example, the initial first reference frame selected in this figure is the first frame of the video to be processed. At this time, the target frame is the loading start frame, and the determined target reference frame is the 90th frame. There are multiple extreme points between the target reference frame and the initial first reference frame. However, the extreme points between frames 52 and 67 are relatively small and are caused by errors. In this embodiment, these points smaller than the differential fluctuation threshold are smoothed, i.e., kept equal to the stable value, to filter out the fluctuations caused by these errors. The smoothed image is shown below. Figure 7 As shown, the extreme point corresponding to frame 48 after smoothing is closest to frame 90, so frame 48 is taken as the loading start frame. Alternatively, besides using smoothing, the target mutation point can be selected from multiple mutation points that are greater than the difference fluctuation threshold and are closest to the target loading frame, and the video frame corresponding to the target mutation point can be taken as the target frame.

[0091] Secondly, this application also provides a method for testing program loading time. Please refer to [link to relevant documentation]. Figure 8 This includes steps S802 to S806.

[0092] S802 records the interactive interface displaying the test program for a preset duration, and triggers the test program's loading event during the recording process to obtain the video to be processed. The preset duration should be sufficient for the test program to complete the loading event.

[0093] S804, the video to be processed is processed using a preset recognition algorithm to obtain the loading start frame and loading end frame. The recognition algorithm is the video frame determination method in any of the above embodiments.

[0094] S806 determines the loading time of the loading event loaded by the test program based on the frame rate of the video to be processed and the frame difference between the starting frame and the ending frame.

[0095] As we can understand it, the frame rate is the number of video frames displayed per second. The duration between the start and end frames can be calculated based on the frame rate and the frame count difference. For example, if the frame rate is 60Hz and the frame count difference is 42 frames, then 42 / 60 = 0.7s is the loading time of the test program.

[0096] Thirdly, embodiments of this application also provide a video frame determination device, including an initialization module, a first calculation module, a second calculation module, an update module, a return module, a target reference frame determination module, and a target frame determination module.

[0097] The initialization module determines the initial first and second reference frames in the video to be processed. The video to be processed is a recorded video loaded by the test program. The initial first reference frame is the first frame of the video to be processed, and the second reference frame is the last frame of the video to be processed, or the initial first reference frame is the last frame and the second reference frame is the first frame. The first calculation module calculates the similarity between each frame of the video to be processed and the initial first reference frame. The second calculation module determines the intermediate frame and calculates the first similarity difference and the second similarity difference. The intermediate frame is the midpoint between the current first and second reference frames. The first similarity difference is the absolute value of the difference between the similarity of the intermediate frame and the similarity of the first reference frame, and the second similarity difference is the absolute value of the difference between the similarity of the intermediate frame and the similarity of the second reference frame. The update module updates the first reference frame to the intermediate frame when the first similarity difference is less than the second similarity difference, and updates the second reference frame to the intermediate frame when the second similarity difference is less than the first similarity difference. The return module is used to return the determined intermediate frame and calculate the first similarity difference and the second similarity difference, until the frame number difference between the current first reference frame and the second reference frame is less than a preset frame number difference. The target reference frame determination module is used to select the frame with the largest similarity fluctuation between the current first reference frame and the second reference frame as the target reference frame. The target frame determination module is used to determine the target frame between the initial first reference frame and the target reference frame. When the initial first reference frame is the first frame, the target frame is the loading start frame; when the initial first reference frame is the last frame, the target frame is the loading end frame.

[0098] The solution provided by this video frame determination device is similar to the solution described in the above-mentioned video frame determination method. Therefore, for specific limitations of the video frame determination device, please refer to the limitations of the video frame determination method above, and will not be repeated here.

[0099] Fourthly, embodiments of this application also provide a computer device, including one or more processors and a memory, the memory storing computer-readable instructions, which, when executed by one or more processors, perform the steps of the video frame determination method or the program loading time test method in any of the above embodiments.

[0100] Fifthly, embodiments of this application also provide a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the video frame determination method or the program loading time test method in any of the above embodiments.

[0101] Indicatively, such as Figure 9 As shown, Figure 9 This is a schematic diagram of the internal structure of a computer device 900 provided in an embodiment of this application. The computer device 900 can be provided as a server. (Refer to...) Figure 9 The computer device 900 includes a processing component 902, which further includes one or more processors, and memory resources represented by memory 901 for storing instructions, such as application programs, that can be executed by the processing component 902. The application programs stored in memory 901 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 902 is configured to execute instructions to perform the text recognition method of any of the above embodiments.

[0102] The computer device 900 may also include a power supply component 903 configured to perform power management of the computer device 900, a wired or wireless network interface 904 configured to connect the computer device 900 to a network, and an input / output (I / O) interface 905. The computer device 900 may operate on an operating system stored in memory 901, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0103] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0104] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0105] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0106] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of video frame determination, the method comprising: include: Determine the initial first reference frame and the second reference frame in the video to be processed; the video to be processed is a recorded video loaded by the test program, and the initial first reference frame is the first frame of the video to be processed and the second reference frame is the last frame of the video to be processed, or the initial first reference frame is the last frame and the second reference frame is the first frame. Calculate the similarity between each frame of the video to be processed and the initial first reference frame; Determine an intermediate frame and calculate a first similarity difference and a second similarity difference; the intermediate frame is the midpoint between the current first reference frame and the second reference frame, the first similarity difference is the absolute value of the difference between the similarity of the intermediate frame and the similarity of the first reference frame, and the second similarity difference is the absolute value of the difference between the similarity of the intermediate frame and the similarity of the second reference frame. When the first similarity difference is less than the second similarity difference, the first reference frame is updated to the intermediate frame; when the second similarity difference is less than the first similarity difference, the second reference frame is updated to the intermediate frame. Return to the steps of determining the intermediate frame and calculating the first similarity difference and the second similarity difference, until the frame number difference between the current first reference frame and the second reference frame is less than the preset frame number difference; Select the frame with the largest similarity fluctuation between the current first reference frame and the second reference frame as the target reference frame; A target frame is determined between the initial first reference frame and the target reference frame; when the initial first reference frame is the first frame, the target frame is the loading start frame; when the initial first reference frame is the last frame, the target frame is the loading end frame. Wherein, determining the target frame between the initial first reference frame and the target reference frame includes: The video frames between the initial first reference frame and the target reference frame are selected as candidate frames; For any candidate frame, calculate the difference between the pixel values ​​of the candidate frame and the pixel values ​​of the initial first reference frame at each same position, and amplify the differences of the pixel values ​​that are greater than a first threshold. Based on the sum of the squares of the differences of all the pixel values, obtain the difference between the candidate frame and the first reference frame. Calculate the difference in the degree of difference between each adjacent candidate frame to obtain the degree of difference fluctuation sequence; Identify multiple mutation points in the difference fluctuation sequence; The candidate frame that is closest to the target reference frame among the candidate frames corresponding to each mutation point is taken as the target frame.

2. The method of claim 1, wherein, The calculation of the similarity between each frame of the video to be processed and the initial first reference frame includes: Binarize each frame of the video to be processed; For any frame of the video to be processed after binarization, it is determined whether the pixels at the same position in the frame and the first reference frame have the same pixel value, and the number of target pixels is determined based on the determination result; the number of target pixels is the number of pixels with the same pixel value. The similarity between the frame and the first reference frame is determined based on the ratio between the number of target pixels and the total number of pixels.

3. The method of claim 1, wherein, Amplifying the difference between the pixel values ​​that are greater than the first threshold includes: The difference between the pixel values ​​that are greater than the first threshold and less than or equal to the second threshold is squared and amplified. The difference between the pixel values ​​that are greater than the second threshold is cubically amplified.

4. The method of claim 1, wherein, Before determining the multiple mutation points in the differential fluctuation sequence, the method further includes: Points in the difference fluctuation sequence that are less than the difference fluctuation threshold are smoothed.

5. The method of claim 1, wherein, The mutation points include extreme points, abrupt increases at the boundary, and / or abrupt decreases at the boundary.

6. A program load time testing method characterized by, include: Record the interactive interface displaying the test program for a preset duration, and trigger the loading event of the test program during the recording process to obtain the video to be processed; The video to be processed is processed using a preset recognition algorithm to obtain the loading start frame and loading end frame; The recognition algorithm is the video frame determination method according to any one of claims 1-5; The loading time of the loading event loaded by the test program is determined based on the frame rate of the video to be processed and the frame difference between the loading start frame and the loading end frame.

7. A video frame determination apparatus characterized by comprising: include: An initialization module is used to determine the initial first reference frame and the second reference frame in the video to be processed; the video to be processed is a recorded video loaded by the test program, and the initial first reference frame is the first frame of the video to be processed and the second reference frame is the last frame of the video to be processed, or the initial first reference frame is the last frame and the second reference frame is the first frame. The first calculation module is used to calculate the similarity between each frame of the video to be processed and the initial first reference frame; The second calculation module is used to determine the intermediate frame and calculate the first similarity difference and the second similarity difference; the intermediate frame is the midpoint between the current first reference frame and the second reference frame, the first similarity difference is the difference between the similarity of the intermediate frame and the similarity of the first reference frame, and the second similarity difference is the difference between the similarity of the intermediate frame and the similarity of the second reference frame. The update module is used to update the first reference frame to the intermediate frame when the first similarity difference is less than the second similarity difference, and to update the second reference frame to the intermediate frame when the second similarity difference is less than the first similarity difference. The return module is used to return the steps of determining the intermediate frame and calculating the first similarity difference and the second similarity difference until the frame number difference between the current first reference frame and the second reference frame is less than the preset frame number difference; The target reference frame determination module is used to select the frame with the largest similarity fluctuation between the current first reference frame and the second reference frame as the target reference frame; The target frame determination module is used to determine a target frame between the initial first reference frame and the target reference frame; when the initial first reference frame is the first frame, the target frame is the loading start frame; when the initial first reference frame is the last frame, the target frame is the loading end frame. Wherein, determining the target frame between the initial first reference frame and the target reference frame includes: The video frames between the initial first reference frame and the target reference frame are selected as candidate frames; For any candidate frame, calculate the difference between the pixel values ​​of the candidate frame and the pixel values ​​of the initial first reference frame at each same position, and amplify the differences of the pixel values ​​that are greater than a first threshold. Based on the sum of the squares of the differences of all the pixel values, obtain the difference between the candidate frame and the first reference frame. Calculate the difference in the degree of difference between each adjacent candidate frame to obtain the degree of difference fluctuation sequence; Identify multiple mutation points in the difference fluctuation sequence; The candidate frame that is closest to the target reference frame among the candidate frames corresponding to each mutation point is taken as the target frame.

8. A computer device, comprising: The device includes one or more processors and a memory storing computer-readable instructions that, when executed by the one or more processors, perform the steps of the video frame determination method as described in any one of claims 1 to 5 or the program loading time testing method as described in claim 6.

9. A storage medium, characterized by The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the video frame determination method as described in any one of claims 1 to 5 or the program loading time testing method as described in claim 6.

Images