Response time testing method and device, equipment, medium and program product

Abstract

The invention provides a response time testing method and device, equipment, a medium and a program product. The method comprises the following steps: controlling a tested device to carry out screen recording, and then operating a UI interface of the tested device; after the UI successfully responds, video recording is stopped, and a recorded video is obtained; determining an identifier of an action triggering frame and an identifier of a response frame from the recorded video; and determining the response time of UI interaction based on the identifier of the action triggering frame, the identifier of the response frame and the video frame rate of the recorded video.

Description

Technical Field

[0001] This application relates to the field of automated testing technology, specifically to a method, apparatus, device, medium, and program product for testing response time. Background Technology

[0002] In the electronic information age, the response speed of the user interface (UI) directly impacts the user experience. Furthermore, UI response time testing typically involves quantifying the time interval between user actions and response feedback to assess the real-time performance and smoothness of the device under test. Therefore, accurately measuring UI interaction response time is of great significance. Summary of the Invention

[0003] This application provides a method and apparatus for testing response time, an electronic device, a storage medium, and a computer program product, which helps to improve the accuracy of testing the response time of UI interactions.

[0004] In a first aspect, embodiments of this application provide a method for testing response time. The method includes: after controlling the device under test to record the screen, operating the UI interface of the device under test; after the UI interface successfully responds, stopping the video recording to obtain the recorded video; determining the identifiers of the action trigger frame and the response frame from the recorded video; and determining the response time of the UI interaction based on the identifiers of the action trigger frame, the identifiers of the response frame, and the video frame rate of the recorded video.

[0005] It is understood that in the response time testing method provided in this application embodiment, after controlling the device under test to record the screen, the UI interface of the device under test is operated; and after the UI interface successfully responds, the video recording is stopped, resulting in a recorded video. Thus, the recorded video includes both action trigger frames and response frames. Next, the identifiers of the action trigger frames and response frames are determined from the recorded video; and based on the identifiers of the action trigger frames, the identifiers of the response frames, and the video frame rate of the recorded video, the response time of the UI interaction is determined. In this way, by improving the accuracy of the determined identifiers of the action trigger frames and response frames, the accuracy of the tested UI interaction response time can be improved.

[0006] In some embodiments, determining the identifier of the action trigger frame and the identifier of the response frame from the recorded video includes: sequentially comparing a pre-stored initial image with video frames in the recorded video to obtain a first similarity value; the initial image refers to the image before operating the UI interface of the device under test; determining the identifier corresponding to the previous frame of the video frame with a first similarity value less than a first similarity threshold as the identifier of the action trigger frame; sequentially comparing a pre-stored response image with video frames after the action trigger frame in the recorded video to obtain a second similarity value; and determining the identifier corresponding to the video frame with a second similarity value greater than a second similarity threshold as the identifier of the response frame.

[0007] It is understood that in the response time testing method provided in this application embodiment, a first similarity value is obtained by comparing the pre-stored initial image with the video frames in the recorded video frame by frame; the identifier corresponding to the previous frame of the video frame with the first similarity value less than the first similarity threshold is determined as the identifier of the action trigger frame; then, the pre-stored response image is compared with the video frames after the action trigger frame frame by frame to obtain a second similarity value, and the identifier corresponding to the video frame with the second similarity value greater than the second similarity threshold is determined as the identifier of the response frame. In this way, the identifiers of the action trigger frames and the response frames can be identified more accurately, thereby improving the accuracy of the UI interaction response time test by improving the accuracy of the identified action trigger frame identifiers and response frame identifiers.

[0008] In some embodiments, the method further includes: cropping the initial image, the response image, and the video frame, and then determining the identifiers of the action trigger frame and the response frame from the recorded video.

[0009] It is understood that in the response time testing method provided in this application embodiment, the initial image, response image, and video frames are cropped before determining the identifiers of the action trigger frame and the response frame from the recorded video. This helps to reduce the amount of image data to be processed and reduce processing time, and can avoid the influence of interference factors (such as animations) that may exist in the complete image on the judgment result, thereby identifying the identifiers of the action trigger frame and the response frame more quickly and accurately.

[0010] In some embodiments, operating the UI interface of the device under test includes: clicking the UI interface of the device under test; or operating the UI interface of the device under test by voice; or operating the UI interface of the device under test by eye contact; or operating the UI interface of the device under test by facial expression; or operating the UI interface of the device under test by gesture.

[0011] It is understood that the response time testing method provided in this application embodiment supports operating the UI interface of the device under test through clicks, voice, eye contact, facial expressions, or gestures. In other words, it supports simulation of multiple user interaction methods. This allows the response time testing method provided in this application embodiment to be applicable to different types of UI interface testing scenarios, thus enhancing the versatility and adaptability of the response time testing method.

[0012] In some embodiments, determining the response time of a UI interaction based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video includes: determining a single response time of the UI interaction based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video; and determining the average of multiple single response times as the response time of the UI interaction.

[0013] It is understood that in the response time testing method provided in this application embodiment, the response time of a single UI interaction is determined based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video; the average of multiple single response times is determined as the response time of the UI interaction. Thus, by averaging multiple tests, random errors that may exist in a single test can be effectively eliminated, thereby improving the stability and reliability of the determined UI interaction response time.

[0014] In some embodiments, the method further includes: stopping video recording and restoring the UI to its initial state if the UI does not respond within a preset time period.

[0015] It is understood that in the response time testing method provided in this application embodiment, a preset duration is set. If, after operating on the UI interface, the duration exceeds the preset duration and the UI interface still fails to respond successfully, an abnormal situation is considered to have occurred. At this time, video recording is stopped, and the UI interface is restored to its initial state. This helps to avoid the problem of subsequent testing being affected by prolonged test blocking, thereby ensuring the continuity and efficiency of the testing process.

[0016] Secondly, embodiments of this application provide a response time testing device, which includes: an operation module configured to operate the UI interface of the device under test after controlling the device under test to perform screen recording; a stop module configured to stop video recording after the UI interface successfully responds, thereby obtaining a recorded video; a first determination module configured to determine the identifier of the action trigger frame and the identifier of the response frame from the recorded video; and a second determination module configured to determine the response time of the UI interaction based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video.

[0017] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor executes the program, it implements the response time testing method described in the first aspect.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the response time testing method described in the first aspect.

[0019] Fifthly, embodiments of this application provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the response time testing method described in the first aspect. Attached Figure Description

[0020] Figure 1 A schematic diagram of the implementation process of a response time testing method provided in this application embodiment. Figure 1 ; Figure 2 A schematic diagram of the connection between an electronic device and a device under test provided in this application embodiment. Figure 1 ; Figure 3 A schematic diagram of the connection between an electronic device and a device under test provided in this application embodiment. Figure 2 ; Figure 4 This application provides an embodiment of an implementation process for determining action trigger frames and response frames. Figure 1 ; Figure 5 A schematic diagram of the implementation process of a response time testing method provided in this application embodiment. Figure 2 ; Figure 6 This application provides an embodiment of an implementation process for determining action trigger frames and response frames. Figure 2 ; Figure 7 A schematic diagram of the implementation process of a response time testing method provided in this application embodiment. Figure 3 ; Figure 8 A schematic diagram of a response time testing device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described below in conjunction with the accompanying drawings. The embodiments described below are only some embodiments of this application, not all embodiments. Therefore, the described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] In the following description, references to “some embodiments” or “other embodiments” describe a subset of all possible embodiments. However, it is understood that “some embodiments” or “other embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0024] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0026] In a related technology, a method and system for accurately testing touchscreen interaction response time is provided. This method involves using a pressure-sensor stylus to collect touch signals and recording the start time of the stylus touch on the screen; recording screen images using a high-frame-rate camera and simultaneously acquiring timestamped video frame data; comparing the images and recording the corresponding end times to calculate the touch response time. However, this method involves complex environments and expensive equipment.

[0027] In another related technology, a method, apparatus, device, and computer storage medium for testing response time are provided. This method uses a series of video frames, searching for the response frame from the end to the beginning, and then comparing the images backward to find the perceived frame. The time difference between this frame and the first frame is then calculated to determine the response time. However, this method starts capturing video after the action is triggered and assumes the first frame of the video is the moment the action is triggered. It does not consider the time elapsed between starting video recording and the actual start of recording, inherently introducing errors. This is unreasonable for millisecond-level response time testing.

[0028] In another related technology, an automated testing method, apparatus, device, and medium for remote control response time are provided. A mobile phone is controlled via a signal transmitter to send a control signal to a target vehicle, and the time of this transmission is taken as the action trigger time. The signal transmitter then receives the signal from the vehicle, and this moment is taken as the test end time. The response time of the remotely controlled vehicle is then calculated. However, the challenge of how the receiver determines whether it has received the vehicle's feedback signal within a short time is not addressed. If the feedback is in the form of a signal or message, it can be obtained in a short time. However, if the feedback signal is a UI response, it is difficult to determine whether the page has responded successfully within tens of milliseconds. Conventional methods such as judging by element attributes or image comparison require hundreds of milliseconds or even more. This poses a significant challenge to UI response time testing, which requires high precision. Furthermore, this method is only applicable to remotely controlled vehicle scenarios, but all testing methods require actual vehicles. Its application scope is relatively limited and it is not suitable for UI interaction response time testing scenarios unrelated to vehicles. In addition, this method requires the use of robotic arms, high-speed cameras, etc., resulting in a complex environment and high equipment costs.

[0029] In another related technology, UI interaction response time is determined through manual testing. This involves manually recording the entire testing process as video, then manually extracting frames from the video to identify the frame that triggered the operation and the frame where the UI successfully responded, ultimately calculating the UI interaction response time. Alternatively, a high frame rate camera can be used to record the screen, while an external pressure-sensitive device records the touch press and release times. Similarly, the recording is broken down into frames and image recognized to find the start frame of the operation and the frame where the UI successfully responded, calculating the response time. However, both of these methods have the following problems: manual testing requires manual calculation of response time, which is inefficient, time-consuming, and labor-intensive; the method using a high frame rate camera and an external pressure-sensitive device is expensive; and the removal of the external pressure-sensitive device after clicking may obstruct part of the camera's field of view. In some fast-response situations, incorrect response time results may be obtained.

[0030] In view of this, this application provides a method for testing response time, which is applied to electronic devices. Figure 1 A schematic diagram of the implementation process of a response time testing method provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the method includes steps 101 to 104: Step 101: After controlling the device under test to record the screen, operate the UI interface of the device under test; Step 102: After the UI interface responds successfully, stop the video recording and obtain the recorded video; Step 103: Identify the identifiers of the action trigger frames and response frames from the recorded video; Step 104: Determine the response time of the UI interaction based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video.

[0031] It is understood that in the response time testing method provided in this application embodiment, after controlling the device under test to record the screen, the UI interface of the device under test is operated; and after the UI interface successfully responds, the video recording is stopped, resulting in a recorded video. Thus, the recorded video includes both action trigger frames and response frames. Next, the identifiers of the action trigger frames and response frames are determined from the recorded video; and based on the identifiers of the action trigger frames, the identifiers of the response frames, and the video frame rate of the recorded video, the response time of the UI interaction is determined. In this way, by improving the accuracy of the determined identifiers of the action trigger frames and response frames, the accuracy of the tested UI interaction response time can be improved.

[0032] The following sections will describe further optional implementation methods for each of the above steps, as well as related terms.

[0033] In step 101, after controlling the device under test to record the screen, the UI interface of the device under test is operated.

[0034] It should be understood that the UI interface is not limited in this embodiment. The UI interface refers to the graphical interface through which the user interacts with the application. The device under test is not limited in this embodiment. The device under test refers to the terminal device that needs to undergo UI interaction response time testing. The device under test is running the application or system interface to be tested.

[0035] In some embodiments, the device under test includes: smartphones, tablets, handheld computers, personal digital assistants (PDAs), wearable devices, in-vehicle infotainment systems, dashboards, rear-seat entertainment systems, other smart electronic devices connected to the vehicle (such as smartphones, smartwatches, etc.), or head-up displays in the vehicle.

[0036] It should be understood that the operating system of the device under test is not limited in this application embodiment. In some embodiments, the operating system of the device under test is Android. Figure 2 A schematic diagram of the connection between an electronic device and a device under test provided in this application embodiment. Figure 1 ,like Figure 2 As shown, the device under test (DUT) and electronic devices (such as Windows computers) running Android systems are connected via Android Debug Bridge (ADB).

[0037] In other embodiments, the operating system of the device under test is an iOS system. Figure 3 A schematic diagram of the connection between an electronic device and a device under test provided in this application embodiment. Figure 2 ,like Figure 3 As shown, for iOS devices, the device under test (DUT) and electronic devices (such as Windows computers or Appium clients) are connected via the Appium server (iOS computer); the iOS DUT and the Appium server are connected via Universal Serial Bus (USB) or via a network. Appium is an automated testing tool.

[0038] It should be understood that in the embodiments of this application, the electronic device is a test device. In some embodiments, the electronic device is the host computer of the device under test.

[0039] In this embodiment, screen recording refers to the process of recording the screen content of the device under test using software tools and saving it as a video file. In some embodiments, the UI interface of the device under test is recorded using a screen copying tool (scrcpy) or screen recording software.

[0040] It is understood that in this embodiment, the UI interface of the device under test is recorded using scrcpy or screen recording software, rather than using a high frame rate camera. This helps reduce the hardware cost of determining the response time of UI interactions.

[0041] In some embodiments, operating the UI interface of the device under test includes: clicking the UI interface of the device under test; or operating the UI interface of the device under test by voice; or operating the UI interface of the device under test by eye contact; or operating the UI interface of the device under test by facial expression; or operating the UI interface of the device under test by gesture.

[0042] It is understood that the response time testing method provided in this application embodiment supports operating the UI interface of the device under test through clicks, voice, eye contact, facial expressions, or gestures. In other words, it supports simulation of multiple user interaction methods. This allows the response time testing method provided in this application embodiment to be applicable to different types of UI interface testing scenarios, thus enhancing the versatility and adaptability of the response time testing method.

[0043] It should be understood that, in the embodiments of this application, a click operation refers to the act of triggering elements (such as buttons, icons, etc.) on a UI interface through a touch screen, an external mouse, or other input devices. For example, in one possible implementation, the UI interface of the device under test is clicked automatically.

[0044] In this embodiment, voice operation refers to the method by which a user interacts with the device under test (DUT) via voice commands. Voice operation relies on technologies such as speech recognition and natural language processing, and is suitable for scenarios where users have limited or no hands. In automated testing, pre-recorded voice commands can be played to simulate a user issuing voice commands, thereby testing whether the DUT can correctly respond and execute the corresponding operations. For example, in one possible implementation, a voice command to "open settings" is issued. After receiving the voice command, the DUT parses the specific command content using a speech recognition model and executes the corresponding operation.

[0045] It should be understood that, in the embodiments of this application, eye-tracking operation is an interaction method based on eye-tracking technology. A camera captures the user's eye movement trajectory, determines the user's gaze position, and considers the position indicated by the user's eye movement trajectory as the operation target. For example, in one possible implementation, a user's gaze is simulated on a button; the device under test interprets the simulated gaze as a click operation and responds to the operation. In automated testing, specific eye movement trajectories can be simulated to verify whether the device under test can correctly recognize and respond to gaze operations.

[0046] In this embodiment, facial expression control refers to a user controlling the device's behavior through changes in facial expressions. Facial expression control relies on facial recognition and emotion computing technologies, enabling a contactless interactive experience. For example, in one possible implementation, a smile indicates confirmation, and a frown indicates cancellation. In automated testing, specific facial expression images can be synthesized to simulate changes in user expressions, verifying whether the device under test can accurately recognize and respond to these changes.

[0047] It should be understood that, in the embodiments of this application, gesture operation refers to an interaction method in which a user interacts with the device under test (DUT) through hand movements. Gesture operation relies on hardware support such as depth cameras and inertial sensors, and uses computer vision algorithms to recognize gesture intentions. For example, in one possible implementation, a simulated upward waving gesture is used; the DUT interprets the simulated upward waving gesture as a page-turning operation and executes the corresponding operation. In automated testing, the ability of the DUT to correctly recognize and respond to gesture operations can be verified by playing a preset gesture video or using a robotic arm to simulate gesture movements.

[0048] In step 102, after the UI interface responds successfully, video recording is stopped, and the recorded video is obtained.

[0049] It should be understood that, in this embodiment, a successful UI response means that the device under test, after receiving an operation on the UI, completes the corresponding interface update or state change, such as page redirection after a button is pressed or data loading completion. Stopping video recording means terminating the screen recording process after detecting that the UI has successfully responded to the operation.

[0050] For example, in one possible implementation, an automated script can determine whether the UI has successfully responded to an operation on the UI interface, thereby triggering a stop recording operation. It should be understood that timely stopping video recording can avoid unnecessary redundant video data, improve testing efficiency, and reduce storage burden.

[0051] In some embodiments, after the UI interface successfully responds within a preset time, video recording stops, and the recorded video is obtained.

[0052] In some embodiments, the method further includes: stopping video recording and restoring the UI to its initial state if the UI does not respond within a preset time period.

[0053] It is understood that in the response time testing method provided in this application embodiment, a preset duration is set. If, after operating on the UI interface, the duration exceeds the preset duration and the UI interface still fails to respond successfully, an abnormal situation is considered to have occurred. At this time, video recording is stopped, and the UI interface is restored to its initial state. This helps to avoid the problem of subsequent testing being affected by prolonged test blocking, thereby ensuring the continuity and efficiency of the testing process.

[0054] It should be understood that the preset duration is not limited in the embodiments of this application. The preset duration refers to a time threshold set during the test to determine whether the device under test has made an effective response to the operation. In some embodiments, the preset duration can be set according to the specific test scenario and test requirements. For example, in one possible implementation, the preset duration is set to 3 seconds or 5 seconds.

[0055] In this embodiment, the initial state is not limited. The initial state refers to the original state of the UI interface before operation. For example, in one possible implementation, in a button click test, the initial state may be that the button in the UI interface is not pressed, the background color is the default color, and there is no animation or prompt information. At the start of the test, the device under test records this initial state, and if the UI interface does not respond to the click operation within a preset time, the video recording stops, and the UI interface is restored to the recorded initial state.

[0056] In some embodiments, if the UI interface does not respond within a preset time period, the video recording is stopped and the UI interface is restored to its initial state, including: if the UI interface does not respond within a preset time period, first stopping the video recording and then restoring the UI interface to its initial state; or, if the UI interface does not respond within a preset time period, stopping the video recording and restoring the UI interface to its initial state.

[0057] It is understood that in the response time testing method provided in this application embodiment, if the UI interface does not respond within a preset time, video recording can be stopped first, and then the UI interface can be restored to its initial state; or, video recording can be stopped while the UI interface is restored to its initial state. This helps ensure that the test environment returns to its initial state, facilitating the next round of testing in a stable and consistent environment, thereby improving the reliability and repeatability of the test.

[0058] Furthermore, in some embodiments, the method further includes recording the test results if the UI interface does not respond within a preset time period. This helps to eliminate abnormal situations and improve the intelligence and stability of the test without affecting the overall test plan.

[0059] In step 103, the identifiers of the action trigger frame and the response frame are determined from the recorded video.

[0060] It should be understood that, in this embodiment, the action trigger frame is not limited. The action trigger frame refers to the last frame of the UI interface after an operation is completed and before the UI interface undergoes a state change; that is, the frame where the operation has just been completed. The action trigger frame is used to mark the time point when the operation on the UI interface ends. In this embodiment, the response frame is not limited. The response frame refers to the first frame of the UI interface after it successfully responds to an operation on the UI interface; that is, the frame where the UI interface successfully responds. The response frame is used to mark the time point when the device under test has completed its response.

[0061] In some embodiments, Figure 4 This application provides an embodiment of an implementation process for determining action trigger frames and response frames. Figure 1 ,like Figure 4 As shown, the identifiers of the action trigger frames and response frames can be determined from the recorded video through the following steps 401 to 404: Step 401: Compare the pre-stored initial image with the video frames in the recorded video in turn to obtain the first similarity value; the initial image refers to the image before operating the UI interface of the device under test; Step 402: The identifier corresponding to the previous frame of the video frame whose first similarity value is less than the first similarity threshold is determined as the identifier of the action trigger frame; Step 403: Compare the pre-stored response image with the video frames after the action trigger frame in the recorded video in turn to obtain the second similarity value; Step 404: The identifier corresponding to the video frame whose second similarity value is greater than the second similarity threshold is determined as the identifier of the response frame.

[0062] It is understood that in the response time testing method provided in this application embodiment, a first similarity value is obtained by comparing the pre-stored initial image with the video frames in the recorded video frame by frame; the identifier corresponding to the previous frame of the video frame with the first similarity value less than the first similarity threshold is determined as the identifier of the action trigger frame; then, the pre-stored response image is compared with the video frames after the action trigger frame frame by frame to obtain a second similarity value, and the identifier corresponding to the video frame with the second similarity value greater than the second similarity threshold is determined as the identifier of the response frame. In this way, the identifiers of the action trigger frames and the response frames can be identified more accurately, thereby improving the accuracy of the UI interaction response time test by improving the accuracy of the identified action trigger frame identifiers and response frame identifiers.

[0063] It should be understood that the initial image is not limited in the embodiments of this application. The initial image refers to the image of a specific area on the interface of the device under test before the UI interface responds to any operation on the UI interface. The initial image is used to determine whether an operation on the device under test is triggered. For example, in one possible implementation, the initial state of the button area before the button is clicked is the initial image. The initial image can be pre-captured and stored in the electronic device by the tester.

[0064] In this embodiment, the response image is not limited. A response image refers to the image of a specific area on the interface of the device under test after a successful UI response. The response image is used to determine whether the UI has successfully responded. For example, in one possible implementation, after clicking a button, if the clicked button is responded to, then the image displayed on the UI after responding to that button is the response image. The response image is pre-captured and stored in the electronic device by the tester.

[0065] It should be understood that, in the embodiments of this application, video frames are not limited. A video frame refers to a series of single images arranged in chronological order in a recorded video, with each frame representing a screenshot at a specific moment.

[0066] In some embodiments, the pre-stored initial image is compared sequentially with video frames in the recorded video to obtain a first similarity value, including: a template matching mechanism based on normalized relevance matching (i.e., TM_CCORR_NORMED) to compare the pre-stored initial image with video frames in the recorded video sequentially to obtain a first similarity value; wherein, the TM_CCORR_NORMED template matching mechanism is a template matching mechanism in OpenCV's matchTemplate method.

[0067] In some embodiments, the pre-stored response image is compared sequentially with video frames after the action trigger frame in the recorded video to obtain a second similarity value, including: based on the TM_CCORR_NORMED template matching mechanism, the pre-stored response image is compared sequentially with video frames after the action trigger frame in the recorded video to obtain a second similarity value.

[0068] It should be understood that, in this embodiment, a higher first similarity value indicates a higher similarity between the initial image and the corresponding video frame. When the first similarity value is greater than the first similarity threshold, it means that the video frame corresponding to the first similarity value is very close to the initial image, indicating that the UI interface has not changed at this time; while when the first similarity value is less than the first similarity threshold, it means that the UI interface has changed, that is, the operation may have been responded to. Therefore, the frame preceding the video frame whose first similarity value is first lower than the first similarity threshold is determined as the action trigger frame.

[0069] In this embodiment, a higher second similarity value indicates a higher similarity between the response image and the corresponding video frame. When the second similarity value is greater than the second similarity threshold, it indicates that the video frame corresponding to the second similarity value is very close to the response image, signifying that the UI interface has successfully responded. Therefore, video frames whose second similarity value first exceeds the second similarity threshold are determined as response frames.

[0070] It should be understood that, in the embodiments of this application, the first similarity threshold and the second similarity threshold are not limited. In some embodiments, the first similarity threshold and the second similarity threshold are preset. It should be understood that the first similarity threshold and the second similarity threshold must be set reasonably. If the first similarity threshold is too high, it may lead to the inability to identify the real action trigger frame; if the first similarity threshold is too low, it may misjudge non-triggered video frames as action trigger frames. By setting a reasonable second similarity threshold, misjudgments caused by slight image jitter or noise can be avoided, thereby improving the reliability of the test.

[0071] In some embodiments, the method further includes: cropping the initial image, the response image, and the video frame, and then determining the identifiers of the action trigger frame and the response frame from the recorded video.

[0072] It is understood that in the response time testing method provided in this application embodiment, the initial image, response image, and video frames are cropped before determining the identifiers of the action trigger frame and the response frame from the recorded video. This helps to reduce the amount of image data to be processed and reduce processing time, and can avoid the influence of interference factors (such as animations) that may exist in the complete image on the judgment result, thereby identifying the identifiers of the action trigger frame and the response frame more quickly and accurately.

[0073] It should be understood that, in the embodiments of this application, cropping refers to the operation of extracting regions from the initial image, response image, and video frame. The cropping operation extracts the part of the image that is relevant to the test target and removes irrelevant background information to improve the efficiency and accuracy of subsequent comparisons. For example, in one possible implementation, only a small area may have changed in the UI interface. Therefore, a rectangular frame is used to crop this small area, and this small area is compared as an independent sub-image, thus eliminating the need to process the entire image or video frame.

[0074] For example, in one possible implementation, the acquired recorded video is first split into individual frames (an example of video frames). Then, the action trigger frame is located: starting with the first frame, an image of the initial comparison area size is cropped from the video frame and compared with the initial image (an example of the initial image) to obtain its similarity (i.e., the first similarity value). If the first similarity value is greater than an initial threshold (i.e., the first similarity threshold), it indicates that the video frame is still in the state before the action is triggered. At this point, the next frame is taken, and the above steps are repeated until the first similarity value of a certain frame is less than the first similarity threshold. This indicates that the UI has changed, meaning that the operation has occurred. Therefore, it can be inferred that the previous frame was the moment the operation was completed. The previous frame is recorded as the action trigger frame, and its frame number is recorded (an example of an identifier).

[0075] Finding Successful Response Frames: Starting from the action trigger frame, crop a frame from the video frame to the size of the response comparison area. Immediately compare this cropped frame with the successful response image (an example of a response image) to obtain their similarity (i.e., the second similarity value). If this second similarity value is less than the response threshold (i.e., the second similarity threshold), it means that the video frame is still in a state of unsuccessful response. At this point, take the next frame and repeat the above steps until the second similarity value obtained by comparing the response comparison area of ​​a certain frame with the successful response image is greater than the second similarity threshold. This indicates that the video frame has successfully responded to the operation at that moment. Record this video frame as a successful response frame (i.e., a response frame) and record its sequence number.

[0076] In step 104, the response time of the UI interaction is determined based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video.

[0077] It should be understood that the identifier is not limited in the embodiments of this application. The identifier refers to the number of each video frame on the timeline, used to indicate the position of the video frame in the entire recorded video. The video frame rate refers to the number of video frames played per unit of time, usually measured in frames per second (FPS). The formula for calculating the response time is: (response frame identifier - action trigger frame identifier) ​​× (1 / video frame rate) × 1000. For example, in one possible implementation, an automated script is used to read the identifiers of the action trigger frame and the response frame, and combined with the frame rate information of the recorded video, calculates the response time of the UI interaction, thereby achieving full automation of the testing process.

[0078] In some embodiments, determining the response time of a UI interaction based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video includes: determining a single response time of the UI interaction based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video; and determining the average of multiple single response times as the response time of the UI interaction.

[0079] It is understood that in the response time testing method provided in this application embodiment, the response time of a single UI interaction is determined based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video; the average of multiple single response times is determined as the response time of the UI interaction. Thus, by averaging multiple tests, random errors that may exist in a single test can be effectively eliminated, thereby improving the stability and reliability of the determined UI interaction response time.

[0080] Furthermore, in some embodiments, after the UI interface successfully responds, video recording is stopped to obtain the recorded video, including: after the UI interface successfully responds, restoring the UI interface to its initial state and operating the UI interface of the device under test; iteratively executing the above steps until the number of successful responses is greater than or equal to N, then stopping video recording to obtain the recorded video.

[0081] It should be understood that N is not limited in the embodiments of this application. N is a preset value, representing the number of times the test is repeated.

[0082] The following describes an exemplary application of the embodiments of this application in a real-world application scenario.

[0083] This application provides an embodiment of a precise test for the response time of a UI interface interaction (i.e., an example of a response time testing method). By considering reducing the cost of UI interaction response time testing, balancing testing efficiency, and improving testing accuracy and precision, a complete automated UI interaction response time testing method (i.e., an example of a response time testing method) is proposed. This method mainly involves automatically simulating human operation of the UI interface, and automatically recording video before the automated simulation. The video is then analyzed, video frames are cropped, and the moment the action is triggered and the moment the UI successfully responds are found through image comparison, thus calculating the UI interaction response time.

[0084] To prepare the test environment, the host computer (i.e., an example of an electronic device) should be configured as follows: (1) Save an initial image (i.e., an example of an initial image); where the initial image refers to the UI page of a certain area of ​​the device under test before the operation is to be performed.

[0085] (2) Set the initial comparison area; where the initial comparison area refers to the area to be compared with the initial image.

[0086] (3) Set an initial threshold (i.e., an example of the first similarity threshold); where the initial threshold is the boundary value of the similarity with the initial image, and the value is between 0 and 1. When comparing a video frame with the initial image, if the obtained first similarity value is greater than or equal to the initial threshold, the video frame is considered to be in the initial image state.

[0087] (4) Save a response success image (i.e., an example of a response image); where a response success image refers to the UI page of a certain area of ​​the device under test after the device under test successfully responds to the operation on the device under test.

[0088] (5) Set the response comparison area; where the response comparison area is the area to be compared with the response success image.

[0089] (6) Set a response threshold (i.e., an example of a second similarity threshold); where the response threshold is the boundary value of the similarity with the successful response image, and the value is between 0 and 1. The video frame is compared with the successful response image. If the obtained second similarity value is greater than or equal to the response threshold, it is considered that the frame is in the successful response image state.

[0090] (7) Set the response timeout (an example of a preset duration); where the response timeout refers to the maximum acceptable response time during the test. Because various abnormal situations may occur during the test, the response may take a very long time. Setting the response timeout is to avoid waiting for a certain test to finish for a long time, which may block the commencement of subsequent tests.

[0091] (8) The host computer is connected to the device under test via methods such as physical wiring harnesses or networks. For example... Figure 2 , Figure 3 As shown in the diagram. The host computer refers to the computer connected to the device under test (DUT), used for automated control of the DUT, video analysis, finding action trigger frames, responding to success frames (an example of a response frame), calculating response time, and outputting the response time test results.

[0092] Initial and successful response images can be obtained by first acquiring the UI interface image using tools such as uiautomator2, ADB, and Appium, and then cropping it. ADB refers to the Android Debug Bridge, used for communication and debugging with Android devices. It allows for file transfer, screen recording, etc. uiautomator2 is an Android application automation testing tool; Appium is an open-source, cross-platform automation testing tool that supports Android and iOS systems and multiple programming languages.

[0093] Figure 5 A schematic diagram of the implementation process of a response time testing method provided in this application embodiment. Figure 2 ,like Figure 5 As shown, the method includes the following steps 501 to 503: Step 501, video unit acquisition; Step 502, video data processing; Step 503, Response time calculation.

[0094] For step 501, it can be implemented as follows: First, put the device under test (DUT) into its initial state, then automatically record the screen of the DUT's interface (e.g., using the scrcpy tool or screen recording software). Next, automatically click on the DUT's UI. Then, continuously check the DUT's UI for successful responses. If a successful response occurs within the response timeout period, stop recording. Conversely, if the response timeout period expires and the UI still doesn't respond, stop recording and restore the device's environment. Because the UI response in this operation far exceeded the expected response time, an abnormal situation has occurred. To prevent prolonged blocking and impact on subsequent testing, the response time test is terminated directly, and the abnormal response result is recorded. Finally, export the video (an example of video recording) to the host computer. Android devices can export via ADB. scrcpy is used to project the Android device screen to the computer in real time and supports reverse control of the device from the computer, making it suitable for Android device screen recording.

[0095] For step 502, it can be implemented as follows: Figure 6 This application provides an embodiment of an implementation process for determining action trigger frames and response frames. Figure 2 ,like Figure 6 As shown, the acquired recorded video is first split into individual frames (an example of a video frame). Then, the action trigger frame is located: starting from the first frame, an image of the initial comparison area size is cropped from the video frame and compared with the initial image (an example of an initial image) to obtain its similarity (i.e., the first similarity value). If this first similarity value is greater than the initial threshold (i.e., the first similarity threshold), it indicates that the video frame is still in the state before the action is triggered. At this point, the next frame is taken, and the above steps are repeated until the first similarity value of a certain frame is less than the first similarity threshold. This indicates that the UI interface has changed, meaning that we are now at the moment after the operation. Therefore, it can be deduced that the previous frame was the moment the operation was completed. The previous frame is recorded as the action trigger frame, and its frame number is recorded (an example of an identifier).

[0096] Finding Successful Response Frames: Starting from the action trigger frame, crop a frame from the video frame to the size of the response comparison area. Immediately compare this cropped frame with the successful response image (an example of a response image) to obtain their similarity (i.e., the second similarity value). If this second similarity value is less than the response threshold (i.e., the second similarity threshold), it means that the video frame is still in a state of unsuccessful response. At this point, take the next frame and repeat the above steps until the second similarity value obtained by comparing the response comparison area of ​​a certain frame with the successful response image is greater than the second similarity threshold. This indicates that the video frame has successfully responded to the operation at that moment. Record this video frame as a successful response frame (i.e., a response frame) and record its sequence number.

[0097] The image comparison algorithm described above can employ the TM_CCORR_NORMED template matching mechanism in OpenCV's matchTemplate method. It calculates the template matching for each position in the initial image / response success image I. In the initial comparison region / response comparison region map T and The correlation within the template region is then normalized to a value between 0 and 1, where 1 represents a perfect match and 0 represents a no-match. Each element in the normalized similarity map represents the degree of similarity between that location in the image and the template. The maximum value in the similarity map is the similarity value (i.e., the first or second similarity value) between the initial image / response success image and the initial comparison region / response comparison region image of the frame.

[0098] For step 503, it can be implemented as follows: Assuming the sequence number of the action trigger frame is n, the sequence number of the successful response frame is m, and the video frame rate is r, then the formula for calculating the response time t (in milliseconds) of the action is: (1) Figure 7 A schematic diagram of the implementation process of a response time testing method provided in this application embodiment. Figure 3 ,like Figure 7 As shown, the method includes steps 701 to 712: Step 701: The device under test is in the initial diagram interface; Step 702: Start recording video; Step 703: Automated clicking on the UI interface; Step 704: Was the response successful? If yes, proceed to step 705; otherwise, proceed to step 710. Step 705: Stop video recording; Step 706, Export video; Step 707: Locate the action trigger frame; Step 708: Locate the UI response success frame; Step 709: Calculate and output the response time; Step 710: Has the timeout period been reached? If yes, proceed to step 711; otherwise, proceed to step 704. Step 711: Stop video recording; Step 712, output response time timeout.

[0099] Based on the above implementation steps, a fully automated process is achieved, from interface video recording to operation clicks and response time calculation. It also considers handling abnormal responses. In this embodiment, only the device under test and a computer (an example of an electronic device) are required, significantly reducing the cost of UI response time testing. Furthermore, by cropping video frames and then using them for image comparison, action trigger frames and successful response frames can be identified more quickly and accurately, greatly reducing the time consumed by conventional video data processing. This is because a complete interface image (an example of a video frame) may contain animated GIFs, and comparing a complete interface image with the initial image often results in consistently low similarity, making it difficult to determine which frame is the action trigger frame and which is the successful response frame. Additionally, comparing complete interface images is time-consuming, and each frame before finding the successful response frame is compared, resulting in a large amount of data and increasing the overall video data processing time. Cropping images reduces the area for comparison, significantly reducing the time spent on individual image comparisons, which cumulatively greatly reduces the overall video data processing time.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application, and all such modifications or substitutions should be covered within the protection scope of this application.

[0101] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that these steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps; or steps from different embodiments may be combined into a new technical solution. Based on the foregoing embodiments, this application provides an apparatus comprising the included modules and the units included in each module, which can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in implementation, the processor can be an AI acceleration engine (such as an NPU), a graphics processing unit (GPU), a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA), etc.

[0102] Figure 8 A schematic diagram of a response time testing device provided in an embodiment of this application is shown below. Figure 8 As shown, the response time testing device 80 includes: an operation module 801, a stop module 802, a first determination module 803, and a second determination module 804; wherein, The operation module 801 is configured to operate the UI interface of the device under test after controlling the device under test to record the screen. Stop module 802 is configured to stop video recording after a successful response from the UI interface, and then obtain the recorded video. The first determining module 803 is configured to determine the identifier of the action trigger frame and the identifier of the response frame from the recorded video; The second determining module 804 is configured to determine the response time of UI interaction based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video.

[0103] In some embodiments, the first determining module 803 is configured to compare a pre-stored initial image with video frames in a recorded video sequentially to obtain a first similarity value; the initial image refers to the image before the operation of the UI interface of the device under test; the identifier corresponding to the previous frame of a video frame with a first similarity value less than a first similarity threshold is determined as the identifier of the action trigger frame; the pre-stored response image is compared with video frames after the action trigger frame in the recorded video sequentially to obtain a second similarity value; the identifier corresponding to a video frame with a second similarity value greater than a second similarity threshold is determined as the identifier of the response frame.

[0104] In some embodiments, the first determining module 803 is further configured to determine the identifier of the action trigger frame and the identifier of the response frame from the recorded video after cropping the initial image, the response image and the video frame.

[0105] In some embodiments, the operation module 801 is configured to click on the UI interface of the device under test; or, operate the UI interface of the device under test by voice; or, operate the UI interface of the device under test by eye contact; or, operate the UI interface of the device under test by facial expression; or, operate the UI interface of the device under test by gesture.

[0106] In some embodiments, the second determining module 804 is configured to determine the response time of a UI interaction based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video; and to determine the average of multiple response times as the response time of the UI interaction.

[0107] In some embodiments, the response time testing apparatus 80 further includes a recovery module; wherein the recovery module is configured to stop video recording and restore the UI to its initial state if the UI does not respond within a preset time.

[0108] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0109] It should be noted that the module division in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or have two or more units integrated into one unit. The integrated units can be implemented in hardware, as software functional units, or a combination of software and hardware.

[0110] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0111] This application provides an electronic device. Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 9 As shown, the electronic device 90 includes a memory 901 and a processor 902. The memory 901 stores a computer program that can run on the processor 902. When the processor 902 executes the program, it implements the steps in the method provided in the above embodiments.

[0112] It should be noted that the memory 901 is configured to store instructions and applications executable by the processor 902, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) in the processor 902 and various modules in the electronic device 90. It can be implemented by flash memory or random access memory (RAM).

[0113] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0114] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0115] It should be noted that the descriptions of the above storage medium and electronic device embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the storage medium and electronic device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0116] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0117] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0118] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or electronic device. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of electronic devices or modules can be electrical, mechanical, or other forms.

[0120] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0121] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0122] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0123] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium.

[0124] Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0125] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0126] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0127] The features disclosed in the several method or electronic device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or electronic device embodiments.

[0128] The above are merely embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for testing response time, characterized in that, The method includes: After controlling the device under test to record the screen, operate the UI interface of the device under test; After the UI interface responds successfully, video recording stops and the recorded video is obtained; Identify the identifiers of the action trigger frames and response frames from the recorded video; The response time of the UI interaction is determined based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video.

2. The method for testing response time according to claim 1, characterized in that, Determining the identifiers of the action trigger frames and response frames from the recorded video includes: The pre-stored initial image is compared sequentially with the video frames in the recorded video to obtain a first similarity value; the initial image refers to the image before the operation of the UI interface of the device under test; the identifier corresponding to the previous frame of the video frame whose first similarity value is less than the first similarity threshold is determined as the identifier of the action trigger frame; The pre-stored response image is compared sequentially with the video frames following the action trigger frame in the recorded video to obtain a second similarity value; the identifier corresponding to the video frame whose second similarity value is greater than the second similarity threshold is determined as the identifier of the response frame.

3. The method for testing response time according to claim 2, characterized in that, The method further includes: After cropping the initial image, the response image, and the video frame, the identifiers of the action trigger frame and the response frame are determined from the recorded video.

4. The method for testing response time according to claim 1, characterized in that, The UI interface for operating the device under test includes: Click on the UI interface of the device under test; or... Operate the UI interface of the device under test via voice; or... Operate the UI interface of the device under test through eye contact; or... The user interface of the device under test can be operated via facial expressions; or... The UI interface of the device under test can be operated through gestures.

5. The method for testing response time according to any one of claims 1 to 4, characterized in that, Determining the UI interaction response time based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video includes: Based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video, the response time of one UI interaction is determined; The average of the multiple single response times is determined as the response time of the UI interaction.

6. The method for testing response time according to any one of claims 1 to 4, characterized in that, The method further includes: If the UI does not respond within a preset time, video recording will stop and the UI will be restored to its initial state.

7. A response time testing device, characterized in that, The device includes: The operation module is configured to operate the UI interface of the device under test after controlling the device under test to perform screen recording. The stop module is configured to stop video recording and obtain the recorded video after the UI interface responds successfully. The first determining module is configured to determine the identifier of the action trigger frame and the identifier of the response frame from the recorded video; The second determining module is configured to determine the response time of the UI interaction based on the identifier of the action trigger frame, the identifier of the response frame, and the video frame rate of the recorded video.

8. An electronic device, characterized in that, The electronic device includes: a memory and a controller; the memory stores a computer program that can run on a processor, and the processor executes the program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the method described in any one of claims 1 to 6.

Images