Interface switching duration testing method, device, and computer storage medium

Abstract

A method, apparatus, and computer storage medium for testing the interface switching duration mainly include triggering an interface sliding operation and an interface switching operation on the interface to be tested according to a test instruction, and triggering a video acquisition operation according to the test instruction to obtain a first target video of the interface sliding operation and a second target video of the interface switching operation, obtaining at least two sliding start frames of the interface sliding operation according to the first target video, obtaining the sliding frame interval time of the interface sliding operation based on two adjacent sliding start frames, obtaining the switching start frame and the switching end frame of the interface switching operation according to the second target video, and obtaining the switching duration of the interface switching operation based on the switching start frame and the switching end frame. Thus, the present application can improve the accuracy of testing the interface switching duration and shorten the time of the test operation.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of automotive electronics, and in particular, to a method, a device, and a computer storage medium for testing the interface switching duration applied to an automotive intelligent cockpit. Background Art

[0002] The intelligent cockpit interface (touch interface) has been widely used in the automotive industry. Currently, the main method for testing the switching time of the intelligent cockpit interface is as follows:

[0003] The tester performs various gesture operations on the touch screen to be tested, such as click gesture operations or swipe gesture operations, etc., to trigger the touch screen to perform the corresponding interface switching. At the same time, the tester uses a mobile phone camera to record the video of the touch screen from the start to the end of the interface switching process and keeps it. Then, the tester analyzes the video on a computer and calculates the start time and end time of the screen change by frame-by-frame comparison, and calculates the duration of the current interface switching.

[0004] However, in the performance test of current in-vehicle intelligent cockpit products, the key index for the interface dynamic migration time needs to be less than 150 ms, or the animation playback frame rate needs to reach 30 to 60 fps, and the error requirement needs to be less than 10%. However, since the maximum frame rate of camera recording can only reach 240 fps, the relevant analysis results obtained from the video recorded by the mobile phone camera cannot meet the above accuracy requirements; and there are also factors such as different touch areas and forces of the operator's fingers, resulting in a system error greater than 50 ms.

[0005] In view of this, the prior art has problems such as low test accuracy, and the tester often needs to compare hundreds or thousands of frames of pictures with the naked eye to obtain test data, resulting in a long test time and the test results are extremely vulnerable to the subjective judgment of the tester, resulting in a low test accuracy. Summary of the Invention

[0006] In view of this, one of the technical problems to be solved by the embodiments of the present invention is to provide a method, a device, and a computer storage medium for testing the interface switching duration, which can improve the test efficiency and accuracy.

[0007] According to a first aspect of the present invention, there is provided a method for testing the interface switching duration, which includes: triggering an interface sliding operation and an interface switching operation on the interface to be tested according to a test instruction, and triggering a video acquisition operation according to the test instruction to obtain a first target video of the interface sliding operation and a second target video of the interface switching operation; obtaining at least two sliding start frames of the interface sliding operation according to the first target video, and obtaining a sliding frame interval time for the interface to be tested to perform the interface sliding operation based on two adjacent sliding start frames; and obtaining a switching start frame and a switching end frame of the interface switching operation according to the second target video, and obtaining a switching duration for the interface to be tested to perform the interface switching operation based on the switching start frame and the switching end frame.

[0008] According to a second aspect of the present invention, there is provided a computer storage medium storing instructions for executing the steps of the method for testing the interface switching duration according to the first aspect.

[0009] According to a third aspect of the present invention, there is provided an apparatus for testing the interface switching duration, which includes: an interface control module for triggering an interface switching operation and an interface sliding operation on the interface to be tested according to a test instruction; a video acquisition module for triggering a video acquisition operation according to the test instruction to obtain a first target video of the interface switching operation and a second target video of the interface sliding operation; a first analysis module for obtaining at least two sliding start frames of the interface sliding operation according to the first target video, and obtaining a sliding frame interval time for the interface to be tested to perform the interface sliding operation based on two adjacent sliding start frames; and a second analysis module for obtaining a switching start frame and a switching end frame of the interface switching operation according to the second target video, and obtaining a switching duration for the interface to be tested to perform the interface switching operation based on the switching start frame and the switching end frame.

[0010] As can be seen from the above technical solutions, the method, apparatus and computer storage medium for testing the interface switching duration provided by the embodiments of the present invention can trigger an interface switching operation and an interface sliding operation on the interface to be tested according to a test instruction, and simultaneously trigger a video acquisition operation according to the test instruction to obtain a first target video including image data of the interface switching operation and a second target video including image data of the interface sliding operation, and then automatically analyze the sliding frame interval time for the interface to be tested to perform the interface sliding operation according to the first target video, and automatically analyze the switching duration for the interface to be tested to perform the interface switching operation according to the second target video. Thereby, the present application can shorten the test operation time and reduce the error of human judgment by providing fully automated test operations, so as to improve the stability and reliability of the test. Description of the Drawings

[0011] Some specific embodiments of the embodiments of the present invention will be described in detail hereinafter with reference to the accompanying drawings in an exemplary but not restrictive manner. The same reference numerals in the drawings denote the same or similar components or parts. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0012] Figure 1 It is a schematic flowchart shown for the interface switching duration testing method according to the first embodiment of the present invention;

[0013] Figure 2 It is a schematic diagram of the hardware architecture configuration applicable to the interface switching duration testing method of each embodiment of the present invention;

[0014] Figure 3 It is a schematic flowchart shown for the interface switching duration testing method according to the second embodiment of the present invention;

[0015] Figure 4 It is a schematic flowchart shown for the interface switching duration testing method according to the third embodiment of the present invention;

[0016] Figure 5 It is a schematic flowchart shown for the interface switching duration testing method according to the fourth embodiment of the present invention;

[0017] Figure 6 It is a schematic flowchart shown for the interface switching duration testing method according to the fifth embodiment of the present invention;

[0018] Figure 7 It is a schematic flowchart shown for the interface switching duration testing method according to the sixth embodiment of the present invention;

[0019] Figure 8 It is a schematic diagram of the architecture shown for the interface switching duration testing device according to the eighth embodiment of the present invention. Detailed Description of the Specific Embodiments

[0020] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art shall fall within the protection scope of the embodiments of the present invention.

[0021] In the prior art, most of the interface switching duration testing technologies are executed through manual operations, resulting in the problem of unstable test results due to many human interference factors.

[0022] In view of this, the present invention provides a fully automated test technology for the switching duration of an interface to be tested (touch interface), which can solve the problem of low test efficiency in the prior art. The following will further illustrate the specific implementation of each embodiment of the present invention with reference to the accompanying drawings of the embodiments of the present invention.

[0023] The First Embodiment

[0024] Figure 1 The interface switching duration test method of the first embodiment of the present invention is shown, which is mainly used to test the interface switching time of an automotive intelligent cockpit controller and its display module. As shown in the figure, the interface switching duration test method of this embodiment mainly includes the following steps:

[0025] Step S11, trigger the interface sliding operation and the interface switching operation of the interface to be tested according to the test instruction, and trigger the video acquisition operation according to the test instruction to obtain the first target video of the interface sliding operation and the second target video of the interface switching operation.

[0026] Optionally, a first simulated touch instruction and a second simulated touch instruction can be generated according to the test instruction, so that the interface to be tested responds to the first simulated touch instruction to perform the interface switching operation, and responds to the second simulated touch instruction to perform the interface sliding operation.

[0027] Please refer to Figure 2 , in this embodiment, a test instruction including multiple test items can be preset. When the test instruction is triggered and executed, for example, by opening the upper computer program in the computer and clicking the test button to trigger the execution of the test instruction, and the first simulated touch instruction or the second simulated touch instruction is generated corresponding to each test item included in the test instruction to simulate the corresponding touch gesture operation. In this embodiment, the first simulated touch instruction is used to simulate the gesture operations of the click type, such as finger pressing operation, finger tapping operation, etc.; the second simulated touch instruction is used to simulate the gesture operations of the sliding type, such as finger sliding operation, etc. Subsequently, the computer sends an ADB command to the interface to be tested to make the interface to be tested respond to the first simulated touch instruction to perform the interface sliding operation, and respond to the second simulated touch instruction to perform the interface switching operation. At the same time, when the computer sends a serial port command, the signal control board outputs a 15mv low voltage to trigger the video acquisition operation of the imaging device, so as to obtain the first target video of the interface sliding operation and the second target video of the interface switching operation.

[0028] In this embodiment, the signal control board can control the opening and closing of the imaging device through the upper computer program. One end of the signal control board is connected to the computer through a usb, and the other end is connected to the imaging device through a BNC interface, and supports the serial communication RS485 protocol, and can send the trigger signal to the imaging device within 1ms when the upper computer program starts the test.

[0029] In this embodiment, the imaging device is a high-speed camera, for example, a high-performance camera with a resolution of 1280×860 (1000fps), a maximum memory of 16G (high-speed Flash), and an exposure time of 1us, which can ensure that the time for image acquisition is controlled at the millisecond level. Preferably, a fixing device including a dark box and a bracket can be set for the imaging device to reduce the influence of external light and product jitter on the test results.

[0030] After the shooting is completed, the first target video and the second target video are transmitted to a computer to, for example, provide the upper computer program in the computer to execute the image analysis task.

[0031] Step S12: Obtain at least two sliding start frames of the interface sliding operation according to the first target video, and based on two adjacent sliding start frames, obtain the sliding frame interval time for the interface to perform the interface sliding operation to be measured.

[0032] In this embodiment, the first image frames in the first target video can be compared by using the image pixel matching degree method to determine at least two sliding start frames in the first target video, and according to the time difference between two adjacent sliding start frames, analyze the sliding frame interval time for the interface to perform the interface sliding operation to be measured.

[0033] Step S13: Obtain the switching start frame and the switching end frame of the interface switching operation according to the second target video, and based on the switching start frame and the switching end frame, obtain the switching duration for the interface to perform the interface switching operation to be measured.

[0034] In this embodiment, the switching start frame and the switching end frame of the interface switching operation in the second target video can be determined by using the binary image connected region analysis method and the histogram similarity calculation method, and according to the time difference between the switching start frame and the switching end frame, analyze the switching duration for the interface to perform the interface switching operation to be measured.

[0035] In summary, for the interface switching duration test method of the embodiment of the present application, the interface sliding operation or the interface switching operation can be triggered for the interface to be measured according to the test instruction, and the video acquisition operation can be triggered simultaneously according to the test instruction to obtain the first target video of the interface sliding operation and the second target video of the interface switching operation. By determining at least two sliding start frames in the first target video to obtain the sliding frame interval time of the interface sliding operation, and by determining the switching start frame and the switching end frame in the second target video to obtain the switching duration of the interface switching operation. Accordingly, the present application can realize fully automated interface test operations, can shorten the test operation time, and can reduce the errors caused by human operations, improving the accuracy of the test results.

[0036] Second Embodiment

[0037] Figure 3 This is a flowchart of the interface switching duration testing method according to the second embodiment of the present invention, which shows the specific implementation process of the above step S12. As shown in the figure, the interface switching duration testing method of this embodiment mainly includes:

[0038] Step S31: obtaining a plurality of first image frames arranged in sequence according to a first target video.

[0039] Optionally, the first target video of M seconds may be segmented into M*1000 first image frames according to the segmentation accuracy of one frame per millisecond. It should be noted that other segmentation accuracies may be used according to actual test standards, and this application does not limit this.

[0040] Step S32 : comparing the difference between two adjacent first image frames to determine a plurality of changed image frames and a plurality of unchanged image frames.

[0041] Optionally, the difference between two adjacent first image frames may be compared by using pixel matching to determine each first image frame as a changed image frame or an unchanged image frame.

[0042] Step S33: determining at least two sliding start frames in the first target video according to each changed image frame and each unchanged image frame.

[0043] Optionally, the arrangement order of each changing image frame and each unchanged image frame may be determined according to the arrangement order of each first image frame in the first target video and the determination results of each changing image frame and each unchanged image frame in each step S32, and according to the arrangement order of each changing image frame and each unchanged image frame, if the number of continuously arranged changing image frames in the first target video exceeds a third preset threshold (for example, 2 frames), the first changing image frame in the continuously arranged multiple changing image frames is determined as the sliding start frame, and so on, to determine at least two sliding start frames in the first target video (to be described in the following Figure 4 described in detail in ).

[0044] Step S34: performing difference calculation on two time stamps corresponding to two adjacent sliding start frames to obtain the sliding frame interval time of the interface sliding operation performed on the interface to be tested.

[0045] For example, a difference calculation may be performed based on respective timestamps of two adjacent sliding start frames in the first target video, so as to obtain the sliding frame interval time of the interface to be tested performing the interface sliding operation.

[0046] Third embodiment

[0047] Figure 4It is a flowchart of the interface switching duration test method according to the third embodiment of the present application, which shows the specific implementation process of the above steps S32 to S33. As shown in the figure, the interface switching duration test method of this embodiment mainly includes:

[0048] Step S401: Obtain a first image frame in sequence as the current first image frame according to multiple first image frames arranged in sequence in the first target video.

[0049] In this embodiment, each first image frame can be obtained in sequence starting from the second first image frame in the first target video.

[0050] Step S402: Calculate the differences between each pixel point in the current first image frame and the corresponding pixel points in the previous first image frame adjacent thereto according to the average pixel values corresponding to each pixel point in each first image frame, and obtain the difference values corresponding to each pixel point in the current first image frame.

[0051] Optionally, the sizes of each first image frame can be uniformly adjusted first. For example, each first image frame is uniformly compressed to a size of 128*128 (rectangular pixel points of 128*128), and then each first image frame is intercepted according to the coordinate range of (x1y1, x2y2) to obtain the detected region images corresponding to each first image frame, and each pixel point in each detected region image is compared one by one to obtain the difference values corresponding to each pixel point in each first image frame.

[0052] In this embodiment, the difference calculation can be performed on the three-channel pixel average values (i.e., b, g, r pixel average values) of each pixel point in each first image frame and the three-channel pixel average values of the corresponding pixel points in the previous first image frame adjacent thereto, so as to obtain the difference values corresponding to each pixel point in the first image frame.

[0053] Step S403: Determine each pixel point with a difference value greater than the first preset threshold in the current first image frame as each differential pixel point according to the first preset threshold.

[0054] In this embodiment, the first preset threshold can be set to 7. That is to say, if the difference between the three-channel pixel average values of a certain pixel point in two first image frames is greater than 7, it means that there is a difference between this pixel point in the adjacent two first image frames, and then it is marked as a differential pixel point.

[0055] Step S404: Judge whether the total number of differential pixel points is greater than the second preset threshold. If it is greater, perform step S405; otherwise, perform step S409.

[0056] Optionally, the second preset threshold can be 5.

[0057] Step S405: Determine the current first image frame as the changed image frame, and proceed to step 406.

[0058] In this embodiment, if the total number of different pixel points included in the current first image frame is greater than 5, then the current first image frame is determined as the changed image frame, which indicates that an obvious change has occurred between the current first image frame and the previous first image frame.

[0059] Step S406: Update the marker value, and assign the current marker value to the current first image frame.

[0060] In this embodiment, the initial value of the marker value i is 0, and the marker value is updated using i = i + 1, and then the updated marker value is assigned to the current first image frame.

[0061] Step S407: Determine whether the marker value is greater than the third preset threshold. If not, proceed to step S411; if so, proceed to step S408.

[0062] Optionally, the third preset threshold can be set to 2, but it is not limited thereto, and other thresholds can also be used.

[0063] Step S408: Determine the first image frame with a marker value of 1 as the sliding start frame, and then proceed to step S411.

[0064] Specifically, when the number of consecutive changed image frames exceeds 2, that is, when it is detected that three consecutive first image frames have changed, then the first image frame at the head of the consecutive arranged first image frames is determined as the sliding start frame.

[0065] Step S409: Determine the current first image frame as an unchanged image frame, and proceed to step 410.

[0066] In this embodiment, if the total number of different pixel points included in the current first image frame is not greater than 5, then the current first image frame is determined as an unchanged image frame, which indicates that no obvious change has occurred between the current first image frame and the previous first image frame.

[0067] Step S410: Clear the marker value, and proceed to S411.

[0068] Specifically, when it is analyzed that the current first image frame is an unchanged image frame, the marker value is cleared to re - locate the sliding start frame.

[0069] Step S411: Determine whether there is an unjudged first image frame in the first target video. If so, return to execute step S401; if not, end this process.

[0070] Thus, the second and third embodiments of the present application divide the first target video into multiple first image frames, and use the three-channel pixel average value comparison method to obtain the determination result of whether each first image frame is a changing image frame or an unchanged image frame, and then determine at least two sliding start frames in the first target video according to the continuous distribution of the changing image frames, so as to calculate the sliding frame interval time of the interface sliding operation performed by the interface to be tested according to the time difference between the two adjacent sliding start frames. The above-mentioned technical means provided by the present application can effectively improve the accuracy of the frame interval time analysis results of the interface sliding operation.

[0071] Fourth embodiment

[0072] Figure 5 The flowchart of the interface switching duration test method according to the fourth embodiment of the present application shows the specific implementation process of the above step S13. As shown in the figure, the interface switching duration test method of this embodiment mainly includes:

[0073] Step S51, according to the second target video, obtain multiple second image frames arranged in order, and use the first second image frame as the starting image frame, the last second image frame as the ending image frame, and each second image frame between the starting image frame and the ending image frame as the intermediate image frames.

[0074] Optionally, the second target video of M seconds may be cut into M*1000 second image frames according to the segmentation accuracy of one frame per millisecond. It should be noted that other segmentation accuracies may be used according to actual test standards, and this application does not limit this.

[0075] Optionally, this step may further include performing image compression preprocessing on each second image frame based on a preset size to obtain each second image frame that meets the preset size. For example, each second image frame may be compressed to a size of 64×64.

[0076] In step S52 , according to the arrangement order of the intermediate image frames, the difference between the intermediate image frames and the initial image frame is sequentially compared to determine the reference image frame.

[0077] Optionally, this step may further include:

[0078] Step S521, according to the arrangement order of the intermediate image frames, sequentially acquiring one intermediate image frame as the current image frame;

[0079] Step S522, calculating the difference between the current image frame and the starting image frame, performing difference calculation on the average value of each pixel corresponding to each pixel point in the current image frame and the average value of each pixel corresponding to each pixel point in the starting image frame, and obtaining each difference value corresponding to each pixel point in the current image frame;

[0080] Step S523: Determine each pixel point in the current image frame with a difference degree value greater than the fourth preset threshold as each difference pixel point according to the fourth preset threshold;

[0081] Step S524: Count the total number of difference pixel points in the current image frame. If the total number is greater than the fifth preset threshold, determine the current image frame as the reference image frame. If the total number is less than the fifth preset threshold, sequentially obtain the next intermediate image frame as the current image frame, and repeat the step of calculating the difference degree between the current image frame and the starting image frame (i.e., Step S522).

[0082] Optionally, the above fourth preset threshold can be set to 7, and the fifth preset threshold can be set to 5.

[0083] In this embodiment, the determination method of the reference image frame is basically the same as the determination method of the changed image frame in the foregoing embodiment. For the detailed description of Steps S521 to S524, reference can be made to Figure 4 Steps S401 to S405 therein, which will not be elaborated here.

[0084] Step S53: Determine the annotation box according to the reference image frame and the starting image frame.

[0085] Optionally, the annotation box to be detected can be determined according to the difference of the three-channel pixel values of each pixel point between the reference image frame and the starting image frame (to be elaborated later in Figure 6 ).

[0086] Step S54: Based on the annotation box, compare each intermediate image frame with the starting image frame to obtain the switching starting frame, and compare each intermediate image frame with the ending image frame to obtain the switching ending frame.

[0087] In this embodiment, based on the annotation box, in the arrangement order of each intermediate image frame, starting from the first intermediate image frame, the standard box regions in each intermediate image frame are sequentially compared with the annotation box region in the starting image frame for difference until a frame of intermediate image frame that has changed significantly compared to the starting image frame is found and determined as the switching starting frame.

[0088] In this embodiment, the switching ending frame can be determined in the same way as the above switching starting frame. Specifically, in the reverse arrangement order of each intermediate image frame, starting from the last intermediate image frame, the standard box regions in each intermediate image frame are sequentially compared with the annotation box region in the ending image frame for difference until a frame of intermediate image frame that has changed significantly compared to the ending image frame is found and determined as the switching ending frame.

[0089] Step S55: Calculate the difference between the timestamps of the switching start frame and the switching end frame respectively to obtain the switching duration of the interface switching operation to be measured for the interface under test.

[0090] For example, the difference calculation can be performed based on the timestamps of the switching start frame and the switching end frame in the second target video, so as to obtain the switching duration of the interface switching operation to be measured for the interface under test.

[0091] The Fifth Embodiment

[0092] Figure 6 FIG. is a flowchart showing the interface switching duration test method according to the fifth embodiment of the present application, which shows the specific implementation process of the above step S53. As shown in the figure, the interface switching duration test method of this embodiment mainly includes:

[0093] Step S61: Perform a subtraction operation on the reference image frame and the start image frame according to the three-channel pixel values corresponding to each pixel point in the reference image frame and the three-channel pixel values corresponding to each pixel point in the start image frame to obtain an intermediate processed image of the subtraction operation.

[0094] Specifically, the three-channel pixel values of all pixel points in the reference image frame (for example, the second image frame of the 4th frame in the second target video) and the three-channel pixel values of all pixel points in the start image frame (that is, the second image frame of the 1st frame in the second target video) can be obtained, and a subtraction operation is performed on each pixel point to generate a corresponding intermediate processed image.

[0095] Step S62: Perform grayscale processing and binary processing on the intermediate processed image to determine the sub-connected regions in the intermediate processed image.

[0096] Specifically, grayscale processing and binary processing are performed on the generated intermediate processed image to obtain the sub-connected regions of the intermediate processed image. For example, relevant functions for finding sub-connected regions (such as: function cv2.connectedComponentsWithStats) are called in the opencv library to obtain them.

[0097] Step S63: Determine a bounding box surrounding the sub-connected region according to the coordinate information of the sub-connected region.

[0098] For example, according to the coordinate information of the sub-connected region, the upper left corner position of the sub-connected region and the length and width of the sub-connected region can be determined, and a bounding box surrounding the sub-connected region is generated accordingly.

[0099] In this embodiment, the shape of the bounding box is, for example, a regular shape such as a rectangle or a circle, and can also be set to any irregular shape. The present application does not limit this.

[0100] The Sixth Embodiment

[0101] Figure 7 It is a flowchart shown by the interface switching duration test method according to the sixth embodiment of the present application, which shows the specific implementation process of the above step S54. As shown in the figure, the interface switching duration test method of this embodiment mainly includes:

[0102] Step S71: Based on the coordinate position of the annotation box in the intermediate processed image, intercept each second image frame to obtain each detection area corresponding to each second image frame.

[0103] Specifically, according to the coordinate position of the annotation box in the intermediate image, intercept the corresponding area in each second image frame to obtain each detection area corresponding to each second image frame.

[0104] Step S721: Obtain an intermediate image frame in sequence according to the arrangement order of each intermediate image frame.

[0105] Specifically, according to the arrangement order of each second image frame serving as the intermediate image frame in the second target video, starting from the intermediate image frame closest to the starting image frame, obtain a second image frame serving as the intermediate image frame in sequence.

[0106] Step S722: Compare the detection area of the intermediate image frame with the detection area of the starting image frame to obtain the first similarity value of the intermediate image frame.

[0107] In this embodiment, the histogram similarity algorithm can be used to calculate each first similarity value.

[0108] Specifically, according to the pixel values of each pixel point in the detection area of the intermediate image frame and the pixel values of each pixel point in the detection area of the starting image frame, the histograms corresponding to the intermediate image frame and the starting image frame can be obtained respectively. Then, calculate the similarity between the histogram of the intermediate image frame and the histogram of the starting image frame to obtain the first similarity value of the intermediate image frame.

[0109] Step S723: Determine whether the first similarity value of the intermediate image frame exceeds the preset similarity threshold. If so, perform step S724; if not, return to step S721 to obtain the next intermediate image frame in sequence and perform analysis.

[0110] Step S724: Determine the intermediate image frame as the switching start frame.

[0111] Step S731: Obtain an intermediate image frame in sequence according to the reverse arrangement order of each intermediate image frame.

[0112] Specifically, according to the reverse arrangement order of the second image frames serving as intermediate image frames in the second target video, starting from the intermediate image frame closest to the end image frame, second image frames serving as intermediate image frames are acquired one by one in sequence.

[0113] Step S732: compare the area to be detected in the intermediate image frame with the area to be detected in the final image frame to obtain a second similarity value of the intermediate image frame.

[0114] In this embodiment, a histogram similarity algorithm may be used to calculate each second similarity value.

[0115] Specifically, based on the pixel values ​​of each pixel point in the area to be detected in the intermediate image frame and the pixel values ​​of each pixel point in the area to be detected in the starting image frame, the histograms corresponding to the intermediate image frame and the ending image frame can be obtained, and then the similarity of the histogram of the intermediate image frame and the histogram of the ending image frame is calculated to obtain the second similarity value of the intermediate image frame.

[0116] Step S733, determining whether the second similarity value of the intermediate image frame exceeds a preset similarity threshold, if so, proceeding to step S734, if not, returning to step S731 to sequentially obtain the next intermediate image frame and perform analysis.

[0117] Step S734: determine the intermediate image frame as the switching end frame.

[0118] In summary, the fourth to sixth embodiments of the present application divide the second target video into multiple second image frames, and first use the three-channel pixel average value comparison method to roughly obtain the second image frame that changes after the interface to be tested performs an interface switching operation as a reference image frame, and then perform a subtraction operation based on the reference image frame and the starting image frame, and then grayscale and binarize it to determine the sub-connected area with the largest change, and generate a labeling box based on the sub-connected area, and then compare each intermediate image frame with the starting image frame based on the labeling box to obtain the switching start frame, and compare each intermediate image frame with the ending image frame to obtain the switching end frame, so as to obtain the switching duration of the interface to be tested to perform the interface switching operation. The above-mentioned technical means provided by the present application can effectively improve the accuracy of the interface switching duration analysis results.

[0119] Seventh embodiment

[0120] A seventh embodiment of the present invention provides a computer storage medium, in which instructions for executing each step of the interface switching duration test method in the first to sixth embodiments are stored.

[0121] Eighth embodiment

[0122] Figure 8shows the main architecture of the interface switching duration testing device according to the eighth embodiment of the present invention, as Figure 8 shown, the interface switching duration testing device 800 according to the embodiment of the present invention mainly includes: an interface control module 810, a video acquisition module 820, a first analysis module 830, and a second analysis module 840.

[0123] The interface control module 810 is configured to trigger an interface switching operation and an interface sliding operation of the interface to be tested according to a test instruction.

[0124] Optionally, a first simulated touch instruction and a second simulated touch instruction are generated according to the test instruction, so that the interface to be tested responds to the first simulated touch instruction to perform the interface switching operation, and responds to the second simulated touch instruction to perform the interface sliding operation.

[0125] The video acquisition module 820 is configured to trigger a video acquisition operation according to the test instruction, so as to obtain a first target video of the interface switching operation and a second target video of the interface sliding operation.

[0126] The first analysis module 830 is configured to obtain at least two sliding start frames of the interface sliding operation according to the first target video, and obtain a sliding frame interval time for the interface to be tested to perform the interface sliding operation based on two adjacent sliding start frames.

[0127] Optionally, the first analysis module 830 includes obtaining a plurality of first image frames arranged in order according to the first target video; comparing the difference degrees between two adjacent first image frames to determine a plurality of changed image frames and a plurality of unchanged image frames; and determining at least two sliding start frames in the first target video according to each changed image frame and each unchanged image frame.

[0128] Optionally, the first analysis module 830 includes calculating difference degrees between each pixel point corresponding to each pixel average value in each first image frame and each pixel point in the previous first image frame adjacent thereto, to obtain difference degree values corresponding to each pixel point in each first image frame; determining each pixel point with a difference degree value greater than a first preset threshold in the first image frame as each differential pixel point according to the first preset threshold; counting the total number of differential pixel points in the first image frame, and if the total number is greater than a second preset threshold, determining the first image frame as the changed image frame, and if the total number is less than the second preset threshold, determining the first image frame as the unchanged image frame.

[0129] Optionally, the first analysis module 830 includes performing a difference calculation on the three-channel pixel average value of each pixel point in each of the first image frames and the three-channel pixel average value of each pixel point in the first image frame of the previous adjacent frame to obtain the difference values ​​corresponding to each pixel point in the first image frame.

[0130] Optionally, the first analysis module 830 includes the first preset threshold being 7 and the second preset threshold being 5.

[0131] Optionally, the first analysis module 830 includes determining an arrangement order of each changing image frame and each unchanged image frame according to an arrangement order of each of the first image frames in the first target video and determination results of each of the changing image frames and each of the unchanged image frames; and according to the arrangement order of each of the changing image frames and each of the unchanged image frames, if the number of the changing image frames arranged continuously in the first target video exceeds a third preset threshold, determining the first changing image frame among the multiple changing image frames arranged continuously as the sliding start frame.

[0132] Optionally, the first analysis module 830 includes performing difference calculation on two timestamps corresponding to two adjacent sliding start frames to obtain the interval time of each sliding frame when the interface to be tested performs the interface sliding operation.

[0133] The second analysis module 840 is used to obtain the switching start frame and the switching end frame of the interface switching operation according to the second target video, and obtain the switching duration of the interface to be tested to perform the interface switching operation based on the switching start frame and the switching end frame.

[0134] Optionally, the second analysis module 840 includes obtaining a plurality of second image frames arranged in order according to the second target video, and taking the second image frame at the first position as the starting image frame, the second image frame at the last position as the ending image frame, and each of the second image frames between the starting image frame and the ending image frame as an intermediate image frame; comparing the difference between each of the intermediate image frames and the starting image frame in turn according to the arrangement order of each of the intermediate image frames, and determining one of the intermediate image frames as a reference image frame; determining a labeling box according to the reference image frame and the starting image frame; and based on the labeling box, comparing each of the intermediate image frames with the starting image frame to obtain the switching starting frame, and comparing each of the intermediate image frames with the ending image frame to obtain the switching ending frame.

[0135] Optionally, the second analysis module 840 includes performing picture compression preprocessing on each of the second image frames based on a preset size to obtain each of the second image frames that meets the preset size.

[0136] Optionally, the second analysis module 840 includes successively obtaining one of the intermediate image frames as the current image frame according to the arrangement order of the intermediate image frames; calculating the degree of difference between the current image frame and the starting image frame, so as to perform difference calculation on the average pixel values corresponding to each pixel point in the current image frame and the average pixel values corresponding to each pixel point in the starting image frame, and obtaining the difference degree values corresponding to each pixel point in the current image frame; according to a fourth preset threshold, determining each pixel point in the current image frame with a difference degree value greater than the fourth preset threshold as each differential pixel point; counting the total number of the differential pixel points in the current image frame, if the total number is greater than a fifth preset threshold, determining the current image frame as the reference image frame, if the total number is less than the fifth preset threshold, successively obtaining the next intermediate image frame as the current image frame, and repeating the step of calculating the degree of difference between the current image frame and the starting image frame.

[0137] Optionally, the second analysis module 840 includes that the fourth preset threshold is 7 and the fifth preset threshold is 5.

[0138] Optionally, the second analysis module 840 includes performing a subtraction operation on the reference image frame and the starting image frame according to the three-channel pixel values corresponding to each pixel point in the reference image frame and the three-channel pixel values corresponding to each pixel point in the starting image frame, to obtain an intermediate processed image of the subtraction operation; performing grayscale processing and binarization processing on the intermediate processed image to determine sub-connected regions in the intermediate processed image; and determining the annotation box surrounding the sub-connected regions according to the coordinate information, length information, and width information of the sub-connected regions.

[0139] Optionally, the second analysis module 840 includes intercepting each of the second image frames according to the coordinate position of the annotation box in the intermediate processed image to obtain each detection region corresponding to each of the second image frames; successively obtaining one of the intermediate image frames according to the arrangement order of the intermediate image frames, and comparing the detection region of the intermediate image frame with the detection region of the starting image frame to obtain a first similarity value of the intermediate image frame, until an intermediate image frame with a first similarity value exceeding a preset similarity threshold is obtained, and determining the intermediate image frame as the switching start frame; and successively obtaining one of the intermediate image frames according to the reverse arrangement order of the intermediate image frames, and comparing the detection region of the intermediate image frame with the detection region of the ending image frame to obtain a second similarity value of the intermediate image frame, until an intermediate image frame with a second similarity value exceeding the preset similarity threshold is obtained, and determining the intermediate image frame as the switching end frame.

[0140] Optionally, the second analysis module 840 includes calculating each of the first similarity values ​​or calculating each of the second similarity values ​​using a histogram similarity algorithm.

[0141] In addition, the interface switching duration testing device 800 of each embodiment of the present invention can also be used to implement other steps in the aforementioned interface switching duration testing method embodiments, and has the beneficial effects of the corresponding method step embodiments, which will not be repeated here.

[0142] In summary, the interface switching duration test method, device and computer storage medium provided in each embodiment of the present application can simultaneously trigger the interface to be tested to perform an interface sliding operation or an interface switching operation and trigger the execution of a video acquisition operation when the test instruction is executed, so as to obtain a first target video of the interface sliding operation and a second target video of the interface switching operation, and analyze the first target video and the second target video to obtain the sliding frame interval time of the interface to be tested to perform the interface sliding operation and the switching duration of the interface switching operation. Accordingly, the present application can complete the interface switching duration test operation in a fully automated manner without manual intervention, which can not only shorten the test time, but also avoid the problem of low accuracy of the test results due to human operation errors.

[0143] Furthermore, the embodiment of the present application uses a pixel comparison method to analyze at least two sliding start frames in the first target video, and then obtains the sliding frame interval time of the interface sliding operation performed by the interface to be tested based on the time difference between the two adjacent sliding start frames; for the analysis of the interface switching duration, the present application uses binary graph connected area analysis and histogram similarity comparison to obtain it. By using the above-mentioned technical means, the accuracy of the test results can be improved.

[0144] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present invention can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.

[0145] The method according to an embodiment of the present invention can be implemented in hardware, firmware, or be implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or be implemented as computer code that is originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded through a network and will be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (such as RAM, ROM, flash memory, etc.) that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, the processor, or the hardware, it implements the interface switching duration test described herein. In addition, when a general-purpose computer accesses the code for implementing the interface switching duration test shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the interface switching duration test shown herein.

[0146] Those of ordinary skill in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional person can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present invention.

[0147] The above embodiments are only used to illustrate the embodiments of the present invention, rather than to limit the embodiments of the present invention. Those of ordinary skill in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present invention. The patent protection scope of the embodiments of the present invention shall be defined by the claims.

Claims

1. A method for testing the interface switching duration, characterized in that The method includes: Triggering the interface under test to perform an interface sliding operation and an interface switching operation according to a test instruction, and triggering a video acquisition operation according to the test instruction to obtain a first target video of the interface sliding operation and a second target video of the interface switching operation; Obtaining at least two sliding start frames of the interface sliding operation according to the first target video, and obtaining a sliding frame interval time for the interface under test to perform the interface sliding operation based on two adjacent sliding start frames; and Obtaining a switching start frame and a switching end frame of the interface switching operation according to the second target video, and obtaining a switching duration for the interface under test to perform the interface switching operation based on the switching start frame and the switching end frame; The obtaining the switching start frame and the switching end frame of the interface switching operation according to the second target video includes: obtaining a plurality of second image frames arranged in order according to the second target video, taking the first second image frame as a start image frame, taking the last second image frame as an end image frame, and taking each second image frame between the start image frame and the end image frame as an intermediate image frame; sequentially comparing the difference degrees between each intermediate image frame and the start image frame according to the arrangement order of each intermediate image frame to determine a reference image frame; determining a bounding box according to the reference image frame and the start image frame; and comparing each intermediate image frame with the start image frame based on the bounding box to obtain the switching start frame, and comparing each intermediate image frame with the end image frame to obtain the switching end frame; Among them, the sequentially comparing the difference degrees between each intermediate image frame and the start image frame according to the arrangement order of each intermediate image frame to determine a reference image frame includes: sequentially obtaining one intermediate image frame as a current image frame according to the arrangement order of each intermediate image frame; calculating the difference degree between the current image frame and the start image frame, that is, performing a difference calculation on the pixel average values corresponding to each pixel point in the current image frame and the pixel average values corresponding to each pixel point in the start image frame, and obtaining the difference degree values corresponding to each pixel point in the current image frame; according to a fourth preset threshold, determining each pixel point with a difference degree value greater than the fourth preset threshold in the current image frame as each differential pixel point; counting the total number of differential pixel points in the current image frame, if the total number is greater than a fifth preset threshold, then determining the current image frame as the reference image frame, if the total number is less than the fifth preset threshold, then sequentially obtaining the next intermediate image frame as the current image frame, and repeating the step of calculating the difference degree between the current image frame and the start image frame; The step of determining a labeling frame based on the reference image frame and the starting image frame includes: performing a subtraction operation on the reference image frame and the starting image frame based on three-channel pixel values ​​corresponding to each pixel point in the reference image frame and three-channel pixel values ​​corresponding to each pixel point in the starting image frame to obtain an intermediate processed image processed by the subtraction operation; performing grayscale processing and binarization processing on the intermediate processed image to determine a sub-connected area in the intermediate processed image; and determining the labeling frame surrounding the sub-connected area based on coordinate information of the sub-connected area.

2. The interface switching duration testing method according to claim 1, wherein The triggering of the interface to be tested to perform an interface switching operation and an interface sliding operation according to the test instruction includes: A first simulated touch instruction and a second simulated touch instruction are generated according to the test instruction, so that the interface to be tested performs the interface switching operation in response to the first simulated touch instruction, and performs the interface sliding operation in response to the second simulated touch instruction.

3. The interface switching duration testing method according to claim 1, wherein The obtaining, according to the first target video, at least two sliding start frames of the interface sliding operation comprises: According to the first target video, obtaining a plurality of first image frames arranged in sequence; Comparing the difference between two adjacent first image frames to determine a plurality of changed image frames and a plurality of unchanged image frames; At least two sliding start frames in the first target video are determined according to each of the changed image frames and each of the unchanged image frames.

4. The interface switching duration testing method according to claim 3, wherein The comparing the difference between two adjacent first image frames to determine a plurality of changed image frames and a plurality of unchanged image frames comprises: Calculate the difference between each pixel in each first image frame and each pixel in the first image frame of the previous adjacent frame according to the average values ​​of each pixel corresponding to each pixel in each first image frame, and obtain the difference value corresponding to each pixel in each first image frame; According to a first preset threshold, determining the pixels in the first image frame whose difference value is greater than the first preset threshold as difference pixels; The total number of the difference pixels in the first image frame is counted, and if the total number is greater than a second preset threshold, the first image frame is determined as the changed image frame; if the total number is less than the second preset threshold, the first image frame is determined as the unchanged image frame.

5. The interface switching duration testing method according to claim 4, wherein The step of calculating the difference between each pixel point in each first image frame and each pixel point in the first image frame of the previous adjacent frame according to the average values ​​of each pixel corresponding to each pixel point in each first image frame, and obtaining the difference value corresponding to each pixel point in the first image frame comprises: A difference calculation is performed on the three-channel pixel average value of each pixel point in each of the first image frames and the three-channel pixel average value of each pixel point in the first image frame of the previous adjacent frame to obtain the difference values ​​corresponding to each pixel point in the first image frame.

6. The interface switching duration testing method according to claim 5, wherein The first preset threshold is 7, and the second preset threshold is 5.

7. The interface switching duration testing method according to claim 3, wherein Determining the at least two sliding start frames according to the respective changed image frames and the respective unchanged image frames includes: Determining the arrangement order of the respective changed image frames and the respective unchanged image frames according to the arrangement order of the respective first image frames in the first target video and the determination results of the respective changed image frames and the respective unchanged image frames; and According to the arrangement order of the respective changed image frames and the respective unchanged image frames, when the number of consecutive changed image frames in the first target video exceeds a third preset threshold, determining the first changed image frame among the multiple consecutive changed image frames as the sliding start frame.

8. The interface switching duration testing method according to claim 7, wherein Obtaining the sliding frame interval time for the interface sliding operation of the interface to be tested based on two adjacent sliding start frames includes: Performing a difference calculation on two timestamps corresponding to two adjacent sliding start frames to obtain the respective sliding frame interval times for the interface sliding operation of the interface to be tested.

9. The interface switching duration testing method according to claim 1, characterized in that After obtaining multiple second image frames arranged in order according to the second target video, the method further includes: Performing picture compression preprocessing on each of the second image frames based on a preset size to obtain each of the second image frames that meet the preset size.

10. The interface switching duration testing method according to claim 1, wherein The fourth preset threshold is 7, and the fifth preset threshold is 5.

11. The interface switching duration testing method according to claim 1, wherein Comparing each of the intermediate image frames with the start image frame based on the annotation box to obtain the switching start frame, and comparing each of the intermediate image frames with the end image frame to obtain the switching end frame includes: Based on the coordinate position of the annotation box in the intermediate processed image, intercepting each of the second image frames to obtain respective detection regions corresponding to each of the second image frames; Sequentially obtaining one of the intermediate image frames according to the arrangement order of the respective intermediate image frames, respectively calculating the histograms of the detection regions of the intermediate image frame and the detection region of the start image frame, and obtaining a first similarity value of the intermediate image frame according to the histogram of the intermediate image frame and the histogram of the start image frame, until obtaining the intermediate image frame whose first similarity value exceeds a preset similarity threshold, and determining the intermediate image frame as the switching start frame; and Sequentially obtaining one of the intermediate image frames according to the reverse arrangement order of the respective intermediate image frames, respectively calculating the histograms of the detection regions of the intermediate image frame and the detection region of the end image frame, and obtaining a second similarity value of the intermediate image frame according to the histogram of the intermediate image frame and the histogram of the end image frame, until obtaining the intermediate image frame whose second similarity value exceeds a preset similarity threshold, and determining the intermediate image frame as the switching end frame.

12. The interface switching duration testing method according to claim 11, wherein The method further includes: Calculating each of the first similarity values or calculating each of the second similarity values by using a histogram similarity algorithm.

13. A computer storage medium, characterized in that, The computer storage medium stores computer instructions for executing the interface switching duration test method according to any one of claims 1 to 12.

14. An interface switching duration testing device, characterized in that, The device includes: An interface control module, configured to trigger an interface switching operation and an interface sliding operation of a to-be-tested interface according to a test instruction; A video acquisition module, configured to trigger a video acquisition operation according to the test instruction to obtain a first target video of the interface sliding operation and a second target video of the interface switching operation; A first analysis module, configured to obtain at least two sliding start frames of the interface sliding operation according to the first target video, and obtain a sliding frame interval time for the to-be-tested interface to perform the interface sliding operation based on two adjacent sliding start frames; and, A second analysis module, configured to obtain a switching start frame and a switching end frame of the interface switching operation according to the second target video, and obtain a switching duration for the to-be-tested interface to perform the interface switching operation based on the switching start frame and the switching end frame; The second analysis module is further configured to: obtain a plurality of second image frames arranged in sequence according to the second target video, use the first second image frame as a start image frame, use the last second image frame as an end image frame, and use each of the second image frames between the start image frame and the end image frame as an intermediate image frame; compare the difference degrees between each of the intermediate image frames and the start image frame in sequence according to the arrangement order of the intermediate image frames to determine a reference image frame; determine a bounding box according to the reference image frame and the start image frame; and compare each of the intermediate image frames with the start image frame based on the bounding box to obtain the switching start frame, and compare each of the intermediate image frames with the end image frame to obtain the switching end frame; The second analysis module is further configured to: obtain one of the intermediate image frames as a current image frame in sequence according to the arrangement order of the intermediate image frames; calculate the difference degree between the current image frame and the start image frame, that is, perform a difference calculation on the average pixel values corresponding to each pixel point in the current image frame and the average pixel values corresponding to each pixel point in the start image frame to obtain the difference degree values corresponding to each pixel point in the current image frame; determine each pixel point with a difference degree value greater than a fourth preset threshold in the current image frame as each difference pixel point according to a fourth preset threshold; count the total number of the difference pixel points in the current image frame, if the total number is greater than a fifth preset threshold, determine the current image frame as the reference image frame, if the total number is less than the fifth preset threshold, sequentially obtain the next intermediate image frame as the current image frame, and repeat the step of calculating the difference degree between the current image frame and the start image frame; The second analysis module is further configured to: perform a subtraction operation on the reference image frame and the starting image frame according to the three-channel pixel values corresponding to each pixel point in the reference image frame and the three-channel pixel values corresponding to each pixel point in the starting image frame, to obtain an intermediate processed image of the subtraction operation; perform grayscale processing and binarization processing on the intermediate processed image to determine sub-connected regions in the intermediate processed image; and determine the annotation box surrounding the sub-connected regions according to the coordinate information of the sub-connected regions.

15. The interface switching duration testing device according to claim 14, wherein The first analysis module includes: obtaining a plurality of first image frames arranged in sequence according to the first target video; comparing the difference degrees between adjacent two of the first image frames to determine a plurality of changed image frames and a plurality of unchanged image frames; determining at least two of the sliding starting frames in the first target video according to each of the changed image frames and each of the unchanged image frames.

16. The interface switching duration testing device according to claim 15, wherein The first analysis module includes: determining the arrangement orders of each of the changed image frames and each of the unchanged image frames according to the arrangement order of each of the first image frames in the first target video and the determination results of each of the changed image frames and each of the unchanged image frames; and according to the arrangement orders of each of the changed image frames and each of the unchanged image frames, if the number of consecutive changed image frames in the first target video exceeds a third preset threshold, determining the first changed image frame among the plurality of consecutive changed image frames as the sliding starting frame.

Images