Response time determination method and device, storage medium and electronic equipment

Abstract

The invention discloses a response time determination method and device, a storage medium and electronic equipment. The method comprises the following steps: connecting target equipment through an equipment interface; in response to the video recording instruction, video recording is carried out on an interface of an application program running on target equipment to obtain a target video, and the interface of the application program responds to the application operation thread; extracting video frames in the target video; and determining a timestamp corresponding to the video frame, and determining response time of the application program to the application operation thread based on the timestamp. Through the method and the device, the problem of relatively low efficiency of testing the response time of the application program in related technologies is solved.

Description

Background Art

[0002] The existing technology has the following deficiencies when testing the response time of an application:

[0003] 1) The screenshots are obtained by recording videos with high-speed cameras. Each high-speed camera can only record one DHU device at a time, and the test equipment cost is high. In addition, the screenshots obtained by the camera each time need to be manually imported into the computer for analysis. The test process requires manual intervention, and the execution efficiency is low.

[0004] 2) Manually analyze the screenshots to determine the timestamps corresponding to the start frame and the end frame, and then calculate the response time result. The execution process is complicated and consumes more testing manpower.

[0005] 3) If there are animated images on the APP page or the page change area is very small, the test results obtained by comparing the grayscale value changes between screenshots through a script will have large errors.

[0006] Based on the above shortcomings, the efficiency of testing the response time of the application is further reduced.

[0007] With regard to the problem of low efficiency in testing the response time of an application program in related technologies, no effective solution has been proposed so far. Summary of the invention

[0008] The main purpose of the present application is to provide a method, device, storage medium and electronic device for determining response time, so as to solve the problem of low efficiency in testing the response time of an application program in the related art.

[0009] In order to achieve the above-mentioned purpose, according to the first aspect of the present application, a method for determining response time is provided. The method comprises: connecting a target device through a device interface; in response to a video recording instruction, recording a video of an interface of an application running on the target device to obtain a target video, wherein the interface of the application responds to an application operation thread; extracting a video frame from the target video; determining a timestamp corresponding to the video frame, and determining the response time of the application to the application operation thread based on the timestamp.

[0010] Furthermore, determining the timestamp corresponding to the video frame includes: selecting a static area in the video frame as a target video frame; obtaining an image grayscale value of the target video frame, and determining the timestamp according to the image grayscale value.

[0011] Furthermore, there are multiple video frames, and the image grayscale value of the target video frame is obtained, and the timestamp is determined according to the image grayscale value, including: determining multiple target video frames corresponding to the multiple video frames, wherein the video frames correspond to the target video frames one-to-one; determining the time sequence of the multiple target video frames; comparing the image grayscale values ​​of two adjacent target video frames in time sequence to obtain multiple grayscale difference values, and determining the timestamp according to the multiple grayscale difference values.

[0012] Furthermore, the time sequence is the sequence from the head frame to the tail frame, and the timestamp includes: a start frame timestamp, comparing the image grayscale values ​​of two adjacent target video frames in time sequence to obtain multiple grayscale differences, and determining the timestamp based on the multiple grayscale differences, including: comparing the image grayscale values ​​of two adjacent target video frames in time sequence to obtain multiple positive-sequence grayscale differences; if a target positive-sequence grayscale difference greater than a preset threshold is detected for the first time among the multiple positive-sequence grayscale differences, then obtaining two target video frames that generate the target positive-sequence grayscale difference; selecting the later target video frame in the two target video frames as the start video frame, wherein the start video frame undergoes a preset degree of image change compared to the previous adjacent target video frame; and determining the start frame timestamp of the start video frame.

[0013] Furthermore, the time sequence is the sequence from the tail frame to the head frame, and the timestamp includes: an end frame timestamp, comparing the image grayscale values ​​of two adjacent target video frames in time sequence to obtain multiple grayscale differences, and determining the timestamp based on the multiple grayscale differences, including: comparing the image grayscale values ​​of two adjacent target video frames in the sequence from the tail frame to the head frame to obtain multiple reverse grayscale differences; if a target reverse grayscale difference greater than a preset threshold is detected among the multiple reverse grayscale differences for the first time, then obtaining two target video frames that generate the target reverse grayscale difference; selecting the later target video frame in the two target video frames as the end video frame, wherein the end video frame undergoes a preset degree of image change compared to the previous adjacent target video frame; and determining the end frame timestamp based on the end video frame.

[0014] Further, determining the response time of the application to the application operation thread based on the timestamp includes: calculating the timestamp difference between the end frame timestamp and the start frame timestamp; and determining the response time of the application to the application operation thread according to the timestamp difference.

[0015] Furthermore, there are multiple target devices, and in response to a video recording instruction, the interface of the application running on the target device is recorded to obtain a target video, including: obtaining the device number of each target device respectively; triggering multiple video recording instructions according to the multiple device numbers respectively, and recording the interface of the application running on the multiple target devices to obtain multiple target videos, wherein the interface of the application on each target device responds to the corresponding application operation thread.

[0016] Furthermore, in response to a video recording instruction, the interface of the application running on the target device is recorded to obtain a target video, including: triggering the video recording instruction according to a preset script; and recording the interface based on the video recording instruction to obtain the target video.

[0017] Furthermore, extracting video frames from the target video includes: calling an audio and video processing tool according to the picture extraction instruction; and extracting video frames from the target video based on the audio and video processing tool.

[0018] In order to achieve the above-mentioned purpose, according to the second aspect of the present application, a device for determining response time is provided. The device comprises: a connection unit, used to connect to a target device through a device interface; a recording unit, used to respond to a video recording instruction, record the interface of an application running on the target device, and obtain a target video, wherein the interface of the application responds to an application operation thread; an extraction unit, used to extract a video frame in the target video; and a determination unit, used to determine a timestamp corresponding to the video frame, and determine the response time of the application to the application operation thread based on the timestamp.

[0019] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the method for determining the response time of any one of the above items is implemented.

[0020] According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for determining the response time according to any one of the above items is implemented.

[0021] Through this application, the following steps are adopted: connecting the target device through the device interface; in response to the video recording instruction, recording the interface of the application running on the target device to obtain the target video, wherein the interface of the application responds to the application operation thread; extracting the video frame in the target video; determining the timestamp corresponding to the video frame, and determining the response time of the application to the application operation thread based on the timestamp. Through this application, the problem of low efficiency in testing the response time of the application in the related art is solved. By automatically responding to the video recording instruction, the video frame in the target video is extracted, and the timestamp corresponding to the video frame is determined. Thereby achieving the effect of improving the test efficiency of the response time of the application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 is a flow chart of a method for determining a response time according to an embodiment of the present application;

[0024] Figure 2 It is a schematic diagram of the system architecture of the method for determining the response time provided in an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of a device for determining a response time according to an embodiment of the present application;

[0026] Figure 4 It is a schematic diagram of the network architecture of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0031] DHU system: also known as Desktop Head Unit, is a software development tool used to simulate and test automotive infotainment systems (in-vehicle entertainment systems).

[0032] adb command: a command line tool used to communicate with Android devices. It can be used to install applications, debug and test applications, view device information, and more.

[0033] FFmpeg: It consists of a set of powerful libraries and command-line tools that support a variety of audio and video formats and codecs. By using FFmpeg, developers can implement audio and video encoding, decoding, conversion, editing, merging, filtering and other operations. It has a wide range of applications, such as video editing, media players, streaming media servers, etc.

[0034] According to an embodiment of the present application, a method for determining a response time is provided.

[0035] Figure 1 FIG. 1 is a flow chart of a method for determining a response time according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0036] Step S101, connecting to a target device via a device interface.

[0037] The target device may be a device running a desktop vehicle DHU system, which may also be referred to as a DHU device. For example, by connecting the DHU device via a USB interface of a host, the host can realize automatic operation of the device.

[0038] Step S102, in response to the video recording instruction, recording the interface of the application program running on the target device to obtain a target video, wherein the interface of the application program responds to the application operation thread.

[0039] Among them, the application operation thread can refer to the startup process of the application of the DHU system, or it can be the page loading process of the application. This application is not limited to this. This application automatically records the video of the application interface by responding to video recording instructions, thereby improving the subsequent testing efficiency of the response time of the application operation thread.

[0040] Specifically, in response to a video recording instruction, the interface of an application running on a target device is recorded to obtain a target video, which can be obtained by the following steps: triggering the video recording instruction according to a preset script; and recording the interface based on the video recording instruction to obtain the target video.

[0041] Exemplarily, when the application operation thread refers to the startup process of the application of the DHU system, the preset script can be an adb command code, which triggers the video recording instruction according to the adb command. On the front-end interface, the response process of the application interface to the startup process is recorded to obtain a response video, which is the target video in this application. This application realizes automatic recording of the application interface in the form of response instructions, thereby improving the subsequent testing efficiency of the response time of the application operation thread.

[0042] In some embodiments, there are multiple target devices, and in response to a video recording instruction, a video recording is performed on the interface of an application running on the target device to obtain a target video. This can be achieved by the following steps: obtaining the device number of each target device respectively; triggering multiple video recording instructions according to the multiple device numbers respectively, and recording the interface of the application running on the multiple target devices to obtain multiple target videos, wherein the interface of the application on each target device responds to the corresponding application operation thread.

[0043] Exemplarily, multiple target devices can be connected through multiple USB interfaces of a host. If the host is connected to three target devices, a video recording instruction can be selectively triggered based on the name of the target device. If the applications running on the three target devices respond to the startup process, the video recording instruction can be a simultaneous video recording of the startup process on the interface of the application on the three target devices, or a video recording of the interface of the application on one of the three target devices. The present application does not limit this. By starting multiple video recording instructions by specifying the device number, the limitation that only one DHU device can be manually captured at a time is overcome, which greatly saves testing manpower and improves testing efficiency.

[0044] Step S103: extracting video frames from the target video.

[0045] This application requires screenshot analysis of the target video. The specific implementation method can be to extract video frames from the target video. By analyzing and processing the video frames, the subsequent test efficiency of the response time of the application operation thread is improved.

[0046] Specifically, extracting the video frames in the target video can be achieved through the following steps: calling an audio and video processing tool according to an image extraction instruction; and extracting the video frames in the target video based on the audio and video processing tool.

[0047] Exemplarily, according to the adb instruction for extracting pictures, the FFmpeg tool is called to extract video frames in the target video. For example, when the target video is the response video of the application interface to the startup process, the video frames are also the pictures that constitute the response video. The script execution process does not require human intervention, which improves the efficiency of subsequent testing of the response time of the application operation thread, and the time interval between each screenshot is obtained is within 10ms, making the test results more accurate.

[0048] Step S104: determining a timestamp corresponding to the video frame, and determining a response time of the application program to the application operation thread based on the timestamp.

[0049] Among them, the timestamp of this application includes a start timestamp and an end timestamp, which are mainly used to test the response time of the application operation thread. That is, this application determines the start timestamp and the end timestamp of the response video, and then realizes the test research on the response time. Through the automatically calculated timestamp, the accurate test of the response time of the application operation thread can be realized.

[0050] Specifically, the timestamp corresponding to the video frame can be determined by the following steps: selecting a static area in the video frame as a target video frame; obtaining an image grayscale value of the target video frame, and determining the timestamp according to the image grayscale value.

[0051] Among them, the static area can be a relatively stable part of the image, that is, an area that remains unchanged or changes very little over a period of time. These areas usually include backgrounds, fixed objects or motionless elements, such as text elements. By selecting the static area in the video frame as the target video frame and studying the image grayscale value of the target video frame, the test accuracy of the response time is improved.

[0052] In some embodiments, there are multiple video frames, and the image grayscale value of the target video frame is obtained, and the timestamp is determined based on the image grayscale value. This can be achieved by the following steps: determining multiple target video frames corresponding to the multiple video frames, wherein the video frames correspond to the target video frames one-to-one; determining the time sequence of the multiple target video frames; comparing the image grayscale values ​​of two adjacent target video frames in time sequence to obtain multiple grayscale difference values, and determining the timestamp based on the multiple grayscale difference values.

[0053] Exemplarily, there is a video frame 1 corresponding to the target video frame 1, a video frame 2 corresponding to the target video frame 2, and a video frame 3 corresponding to the target video frame 3. When the target video frame 1 is used as the head frame, the target video frame 2 is used as the middle video frame, and the target video frame 3 is used as the tail frame, the time sequence from the head frame to the tail frame is determined to be the positive generation sequence, and the time sequence from the tail frame to the head frame is determined to be the reverse generation sequence. For example, in the case of the positive generation sequence, the image grayscale value difference between the target video frame 1 and the target video frame 2 is first determined to obtain the grayscale difference a, and then the image grayscale value difference between the target video frame 2 and the target video frame 3 is determined to obtain the grayscale difference b. When the timestamp cannot be determined based on a, the timestamp corresponding to the video frame is determined based on b. For another example, in the case of reverse generation order, the image grayscale value difference between the target video frame 2 and the target video frame 3 is first determined to obtain the grayscale difference b, and then the image grayscale value difference between the target video frame 1 and the target video frame 2 is determined to obtain the grayscale difference a. When the timestamp cannot be determined based on b, the timestamp corresponding to the video frame is determined based on a. The present application further determines the timestamp in the present application by comparing different grayscale differences through different timestamp orders, which improves the test accuracy of the response time while also saving data processing time.

[0054] In some embodiments, in order to improve the efficiency of determining the start frame timestamp, the present application determines the start frame timestamp of the timestamp when the time sequence is the order from the head frame to the tail frame. Specifically, the image grayscale values ​​of two adjacent target video frames are compared in time sequence to obtain multiple grayscale differences, and the timestamp is determined based on the multiple grayscale differences. The timestamp can be obtained through the following steps: the image grayscale values ​​of two adjacent target video frames are compared in time sequence from the head frame to the tail frame to obtain multiple positive-sequence grayscale differences; if a target positive-sequence grayscale difference greater than a preset threshold is detected among the multiple positive-sequence grayscale differences for the first time, two target video frames that generate the target positive-sequence grayscale difference are obtained; the target video frame with a later order among the two target video frames is selected as the start video frame, wherein the start video frame undergoes a preset degree of image change compared with the previous adjacent target video frame; and the start frame timestamp of the start video frame is determined.

[0055] Exemplarily, there is a video frame 1 corresponding to the target video frame 1, video frame 2 corresponding to the target video frame 2, video frame 3 corresponding to the target video frame 3, video frame 4 corresponding to the target video frame 4, and video frame 5 corresponding to the target video frame 5. If the target video frame 1 is the head video frame, the target video frame 2, the target video frame 3, and the target video frame 4 are respectively the middle video frames, and the target video frame 5 is the tail video frame, it is determined that the time sequence of the target video frame 1, the target video frame 2, the target video frame 3, the target video frame 4, and the target video frame 5 is the positive sequence generation sequence, that is, in the order from the head frame to the tail frame, the image grayscale value difference between the target video frame 1 and the target video frame 2 is obtained to obtain the positive sequence grayscale difference a, and the image grayscale value difference between the target video frame 2 and the target video frame 3 is obtained to obtain the positive sequence grayscale difference b. The image grayscale value difference between the target video frame 3 and the target video frame 4 is obtained to obtain the positive-sequence grayscale difference c. The image grayscale value difference between the target video frame 4 and the target video frame 5 is obtained to obtain the positive-sequence grayscale difference d. If, in the order of monitoring priority a>b>c>d, a is first monitored to be greater than the preset threshold, then a is used as the target positive-sequence grayscale difference, and the target video frame 1 and the target video frame 2 for generating a are obtained. The target video frame 2 is selected as the starting video frame, and the timestamp corresponding to the target video frame 2 is obtained as the starting frame timestamp of the response time. This application selects a static area in a video frame as the target video frame, and obtains the starting frame timestamp by studying the image grayscale value of the target video frame, thereby ensuring the test accuracy of the response time while improving the test efficiency of the response time.

[0056] That is, the present application compares the image grayscale values ​​of two adjacent target video frames in the order of the head frame to the tail frame, which can quickly capture the changes in the starting video frame, greatly improve the efficiency of determining the starting timestamp, and thus improve the test accuracy of the response time.

[0057] In some embodiments, in order to improve the efficiency of determining the end frame timestamp, the present application determines the end frame timestamp of the timestamp when the time sequence is the order of the tail frame to the head frame. Specifically, the image grayscale values ​​of two adjacent target video frames are compared in time sequence to obtain multiple grayscale differences, and the timestamp is determined based on the multiple grayscale differences. The timestamp can be obtained through the following steps: the image grayscale values ​​of two adjacent target video frames are compared in time sequence to obtain multiple reverse grayscale differences; if a target reverse grayscale difference greater than a preset threshold is detected among the multiple reverse grayscale differences for the first time, two target video frames that generate the target reverse grayscale difference are obtained; the target video frame that is later in order among the two target video frames is selected as the end video frame, wherein the end video frame undergoes a preset degree of image change compared to the previous adjacent target video frame; and the end frame timestamp is determined based on the end video frame.

[0058] Exemplarily, there is a video frame 1 corresponding to the target video frame 1, video frame 2 corresponding to the target video frame 2, video frame 3 corresponding to the target video frame 3, video frame 4 corresponding to the target video frame 4, and video frame 5 corresponding to the target video frame 5. If the target video frame 1 is the head video frame, the target video frame 2, the target video frame 3, and the target video frame 4 are respectively the middle video frames, and the target video frame 5 is the tail video frame, it is determined that the time sequence of the target video frame 5, the target video frame 4, the target video frame 3, the target video frame 2, and the target video frame 1 is the reverse generation sequence, that is, in the order from the tail frame to the head frame, the image grayscale value difference between the target video frame 5 and the target video frame 4 is obtained to obtain the reverse grayscale difference a, and the image grayscale value difference between the target video frame 4 and the target video frame 3 is obtained to obtain the reverse grayscale difference b. The image grayscale value difference between the target video frame 3 and the target video frame 2 is obtained to obtain the reverse grayscale difference c, and the image grayscale value difference between the target video frame 2 and the target video frame 1 is obtained to obtain the reverse grayscale difference d. If, in the order of monitoring priority a>b>c>d, b is first monitored to be greater than the preset threshold, then b is used as the target reverse grayscale difference, and the target video frame 4 and the target video frame 3 for generating b are obtained. The target video frame 3 is selected as the end video frame, and the timestamp corresponding to the target video frame 3 is obtained as the end frame timestamp of the response time. The present application selects a static area in a video frame as the target video frame, and obtains the end frame timestamp by studying the image grayscale value of the target video frame, thereby ensuring the test accuracy of the response time and improving the test efficiency of the response time.

[0059] That is, the present application compares the image grayscale values ​​of two adjacent target video frames in the order of the tail frame to the head frame, which can quickly capture the changes in the end video frame, greatly improve the efficiency of determining the end timestamp, and thus improve the test accuracy of the response time.

[0060] Specifically, the response time of the application to the application operation thread can be determined based on the timestamp by the following steps: calculating the timestamp difference between the end frame timestamp and the start frame timestamp; determining the response time of the application to the application operation thread based on the timestamp difference.

[0061] Exemplarily, the end frame timestamp is a, the start frame timestamp is b, the timestamp difference is ba, and the response time calculated by ba is used as the test result. This application effectively improves the test efficiency of the response time by replacing the manual analysis of screenshots and calculation of test results.

[0062] Optionally, Figure 2 is a schematic diagram of the system architecture of the method for determining the response time provided in the embodiment of the present application, such as Figure 2As shown, the host tests the DHU and the host through a USB connection. Start the test script and read the test parameter configuration file, such as test indicators, test case ID, test project name, and test version number. Then start the fixed-length thread pool and run the screenshot acquisition thread for each DHU device at the same time according to the corresponding device number. When there are too many DHU devices that trigger the thread at the same time, the fixed-length thread pool can be used to limit the number of concurrent threads to queue and wait, thereby ensuring the stable operation of the host. Running the screenshot acquisition thread includes: first recording the video through the adb command, and then using ffmpeg to decompose the video into corresponding video frame screenshots. Run the APP operation thread and use uiautomator2 to realize the automation of response time use cases such as APP startup, APP page click, and APP return to the DHU homepage. After the APP operation is completed, the screenshot acquisition thread and the APP operation thread are terminated, and the screenshots of each DHU device are saved in the corresponding folder, and the screenshots of each DHU device are analyzed: the grayscale value between each two screenshots is compared through the script, and the effective screenshot comparison area is selected, and the interference of the dynamic image area and the area that has not changed during the whole process is excluded, and the starting frame screenshot and the ending frame screenshot are determined, and each screenshot name is the corresponding timestamp. Among them, the test task initialization includes: obtaining the test DHU screen size, and obtaining the test case standard value according to the test case parameter configuration file, etc.

[0063] Finally, the response time test result is calculated by the start frame timestamp and the end frame timestamp, and compared with the corresponding standard value to determine whether the test case result has passed. In addition, the test result data will be stored in the database to facilitate subsequent search of historical test data.

[0064] The method for determining the response time provided in the embodiment of the present application connects the target device through the device interface; in response to the video recording instruction, the interface of the application running on the target device is recorded to obtain the target video, wherein the interface of the application responds to the application operation thread; the video frame in the target video is extracted; the timestamp corresponding to the video frame is determined, and the response time of the application to the application operation thread is determined based on the timestamp. Through this application, the problem of low efficiency in testing the response time of the application in the related art is solved. By automatically responding to the video recording instruction, the video frame in the target video is extracted, and the timestamp corresponding to the video frame is determined. Thereby, the effect of improving the test efficiency of the response time of the application is achieved.

[0065] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0066] The embodiment of the present application also provides a device for determining a response time. It should be noted that the device for determining a response time in the embodiment of the present application can be used to execute the method for determining a response time provided in the embodiment of the present application. The device for determining a response time provided in the embodiment of the present application is introduced below.

[0067] Figure 3 Schematic diagram of a device for determining a response time according to an embodiment of the present application. Figure 3 As shown, the device includes: a connecting unit 301, a recording unit 302, an extracting unit 303, and a determining unit 304.

[0068] Specifically, the connection unit 301 is used to connect to the target device through the device interface;

[0069] The recording unit 302 is used to record the interface of the application running on the target device in response to the video recording instruction to obtain the target video, wherein the interface of the application responds to the application operation thread;

[0070] An extraction unit 303, used to extract video frames from a target video;

[0071] The determining unit 304 is configured to determine a timestamp corresponding to the video frame, and determine a response time of the application program to the application operation thread based on the timestamp.

[0072] Optionally, in the response time determination device provided in an embodiment of the present application, the determination unit includes: a selection subunit, used to select a static area in a video frame as a target video frame; a determination subunit, used to obtain an image grayscale value of the target video frame, and determine a timestamp based on the image grayscale value.

[0073] Optionally, in the response time determination device provided in the embodiment of the present application, when there are multiple video frames, the determination subunit includes: a first determination module, used to determine multiple target video frames corresponding to the multiple video frames, wherein the video frames correspond to the target video frames one-to-one; a second determination module, used to determine the time sequence of the multiple target video frames; and a third determination module, used to compare the image grayscale values ​​of two adjacent target video frames in time sequence to obtain multiple grayscale difference values, and determine the timestamp based on the multiple grayscale difference values.

[0074] Optionally, in the response time determination device provided in the embodiment of the present application, the time sequence is the sequence from the head frame to the tail frame, and the timestamp includes: a start frame timestamp, and a third determination module includes: a first comparison submodule, which is used to compare the image grayscale values ​​of two adjacent target video frames in the order from the head frame to the tail frame to obtain multiple positive-sequence grayscale differences; a first monitoring submodule, which is used to obtain two target video frames that generate the target positive-sequence grayscale difference if a target positive-sequence grayscale difference greater than a preset threshold is detected for the first time among the multiple positive-sequence grayscale differences; a first selection submodule, which is used to select a target video frame that is later in sequence among the two target video frames as a start video frame, wherein the start video frame undergoes a preset degree of image change compared with the previous adjacent target video frame; and a first determination submodule, which is used to determine the start frame timestamp of the start video frame.

[0075] Optionally, in the response time determination device provided in the embodiment of the present application, the time sequence is the sequence from the tail frame to the head frame, and the timestamp includes: an end frame timestamp, and a third determination module includes: a second comparison submodule, used to compare the image grayscale values ​​of two adjacent target video frames in the sequence from the tail frame to the head frame to obtain multiple reverse grayscale differences; a second monitoring submodule, used to obtain two target video frames that generate the target reverse grayscale difference if a target reverse grayscale difference greater than a preset threshold is detected among the multiple reverse grayscale differences for the first time; a second selection submodule, used to select the target video frame with a later sequence among the two target video frames as the end video frame, wherein the end video frame undergoes a preset degree of image change compared with the previous adjacent target video frame; and a second determination submodule, used to determine the end frame timestamp according to the end video frame.

[0076] Optionally, in the response time determination device provided in an embodiment of the present application, the determination unit includes: a calculation subunit, used to calculate the timestamp difference between the end frame timestamp and the start frame timestamp; and a determination subunit, used to determine the response time of the application to the application operation thread based on the timestamp difference.

[0077] Optionally, in the response time determination device provided in the embodiment of the present application, there are multiple target devices, and the recording unit includes: an acquisition subunit, used to obtain the device number of each target device respectively; a first recording subunit, used to trigger multiple video recording instructions according to the multiple device numbers, to record the interfaces of applications running on the multiple target devices, and obtain multiple target videos, wherein the interface of the application on each target device responds to the corresponding application operation thread.

[0078] Optionally, in the response time determination device provided in the embodiment of the present application, the recording unit includes: a triggering subunit, used to trigger a video recording instruction according to a preset script; and a second recording subunit, used to record the interface based on the video recording instruction to obtain a target video.

[0079] Optionally, in the response time determination device provided in the embodiment of the present application, the extraction unit includes: a calling subunit, used to call the audio and video processing tool according to the image extraction instruction; and an extraction subunit, used to extract video frames in the target video based on the audio and video processing tool.

[0080] The device for determining the response time includes a processor and a memory. The above-mentioned connection unit 301, recording unit 302, extraction unit 303, determination unit 304, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0081] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the response time is determined by adjusting the kernel data.

[0082] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0083] In an exemplary embodiment of the present disclosure, a computer storage medium capable of implementing the above method is also provided. A program product capable of implementing the above method of this specification is stored thereon. In some possible embodiments, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present disclosure, such as the following steps: connecting a target device through a device interface; in response to a video recording instruction, recording a video of the interface of an application running on the target device to obtain a target video, wherein the interface of the application responds to an application operation thread; extracting video frames from the target video; determining the timestamp corresponding to the video frame, and determining the response time of the application to the application operation thread based on the timestamp.

[0084] In an optional implementation manner: a static area in a video frame is selected as a target video frame; an image grayscale value of the target video frame is obtained, and a timestamp is determined according to the image grayscale value.

[0085] In an optional implementation: multiple target video frames corresponding to multiple video frames are determined, wherein the video frames correspond to the target video frames one-to-one; a time sequence of the multiple target video frames is determined; image grayscale values ​​of two adjacent target video frames are compared in time sequence to obtain multiple grayscale difference values, and a timestamp is determined based on the multiple grayscale difference values.

[0086] In an optional implementation: the image grayscale values ​​of two adjacent target video frames are compared in order from the head frame to the tail frame to obtain a plurality of positive-sequence grayscale differences; if a target positive-sequence grayscale difference greater than a preset threshold value is detected for the first time among the plurality of positive-sequence grayscale differences, two target video frames for generating the target positive-sequence grayscale difference are obtained; the target video frame with the later order among the two target video frames is selected as the starting video frame, wherein the starting video frame undergoes a preset degree of image change compared with the previous adjacent target video frame; and the starting frame timestamp of the starting video frame is determined.

[0087] In an optional embodiment: an end frame timestamp, comparing the image grayscale values ​​of two adjacent target video frames in chronological order to obtain multiple grayscale differences, and determining the timestamp based on the multiple grayscale differences, including: comparing the image grayscale values ​​of two adjacent target video frames in order from the tail frame to the head frame to obtain multiple reverse grayscale differences; if a target reverse grayscale difference greater than a preset threshold is detected among the multiple reverse grayscale differences for the first time, then obtaining two target video frames that generate the target reverse grayscale difference; selecting the later target video frame in the two target video frames as the end video frame, wherein the end video frame undergoes a preset degree of image change compared to the previous adjacent target video frame; and determining the end frame timestamp based on the end video frame.

[0088] In an optional implementation manner: a timestamp difference between an end frame timestamp and a start frame timestamp is calculated; and a response time of the application program to the application operation thread is determined according to the timestamp difference.

[0089] In an optional implementation: the device number of each target device is obtained respectively; multiple video recording instructions are triggered according to the multiple device numbers respectively, and the interfaces of the applications running on the multiple target devices are recorded to obtain multiple target videos, wherein the interface of the application on each target device responds to the corresponding application operation thread.

[0090] In an optional implementation: trigger a video recording instruction according to a preset script; and record the interface based on the video recording instruction to obtain a target video.

[0091] In an optional implementation: calling an audio and video processing tool according to a picture extraction instruction; extracting video frames in a target video based on the audio and video processing tool.

[0092] In an optional embodiment, the embodiment of the present disclosure may also include a program product for implementing the above method, which may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0093] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0094] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0095] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0096] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0097] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0098] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0099] Refer to the following Figure 4 The electronic device 400 according to this embodiment of the present disclosure is described. Figure 4 The electronic device 400 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0100] like Figure 4 As shown, the electronic device 400 is in the form of a general computing device. The components of the electronic device 400 may include, but are not limited to: the at least one processing unit 410, the at least one storage unit 420, a bus 430 connecting different system components (including the storage unit 420 and the processing unit 410), and a display unit 440.

[0101] The storage unit 420 stores program codes, which can be executed by the processing unit 410, so that the processing unit 410 executes the steps described in the "Exemplary Method" section of the specification according to various exemplary embodiments of the present disclosure. For example, the processing unit 410 can execute the following steps: connect to the target device through the device interface; in response to the video recording instruction, record the interface of the application running on the target device to obtain the target video, wherein the interface of the application responds to the application operation thread; extract the video frame in the target video; determine the timestamp corresponding to the video frame, and determine the response time of the application to the application operation thread based on the timestamp.

[0102] In an optional implementation manner: a static area in a video frame is selected as a target video frame; an image grayscale value of the target video frame is obtained, and a timestamp is determined according to the image grayscale value.

[0103] In an optional implementation: multiple target video frames corresponding to multiple video frames are determined, wherein the video frames correspond to the target video frames one-to-one; a time sequence of the multiple target video frames is determined; image grayscale values ​​of two adjacent target video frames are compared in time sequence to obtain multiple grayscale difference values, and a timestamp is determined based on the multiple grayscale difference values.

[0104] In an optional implementation: the image grayscale values ​​of two adjacent target video frames are compared in order from the head frame to the tail frame to obtain a plurality of positive-sequence grayscale differences; if a target positive-sequence grayscale difference greater than a preset threshold value is detected for the first time among the plurality of positive-sequence grayscale differences, two target video frames for generating the target positive-sequence grayscale difference are obtained; the target video frame with the later order among the two target video frames is selected as the starting video frame, wherein the starting video frame undergoes a preset degree of image change compared with the previous adjacent target video frame; and the starting frame timestamp of the starting video frame is determined.

[0105] In an optional embodiment: an end frame timestamp, comparing the image grayscale values ​​of two adjacent target video frames in chronological order to obtain multiple grayscale differences, and determining the timestamp based on the multiple grayscale differences, including: comparing the image grayscale values ​​of two adjacent target video frames in order from the tail frame to the head frame to obtain multiple reverse grayscale differences; if a target reverse grayscale difference greater than a preset threshold is detected among the multiple reverse grayscale differences for the first time, then obtaining two target video frames that generate the target reverse grayscale difference; selecting the later target video frame in the two target video frames as the end video frame, wherein the end video frame undergoes a preset degree of image change compared to the previous adjacent target video frame; and determining the end frame timestamp based on the end video frame.

[0106] In an optional implementation manner: a timestamp difference between an end frame timestamp and a start frame timestamp is calculated; and a response time of the application program to the application operation thread is determined according to the timestamp difference.

[0107] In an optional implementation: the device number of each target device is obtained respectively; multiple video recording instructions are triggered according to the multiple device numbers respectively, and the interfaces of the applications running on the multiple target devices are recorded to obtain multiple target videos, wherein the interface of the application on each target device responds to the corresponding application operation thread.

[0108] In an optional implementation: trigger a video recording instruction according to a preset script; and record the interface based on the video recording instruction to obtain a target video.

[0109] In an optional implementation: calling an audio and video processing tool according to a picture extraction instruction; extracting video frames in a target video based on the audio and video processing tool.

[0110] The storage unit 420 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 4201 and / or a cache storage unit 4202 , and may further include a read-only storage unit (ROM) 4203 .

[0111] The storage unit 420 may also include a program / utility 4204 having a set (at least one) of program modules 4205, such program modules 4205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0112] Bus 430 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0113] The electronic device 400 may also communicate with one or more external devices 500 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 400, and / or communicate with any device that enables the electronic device 400 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 450. Furthermore, the electronic device 400 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 460. As shown, the network adapter 460 communicates with other modules of the electronic device 400 via a bus 430. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0114] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0115] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0116] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A method for determining a response time, characterized in that: include: Connect to the target device through the device interface; In response to the video recording instruction, a video recording is performed on the interface of the application program running on the target device to obtain a target video, wherein the interface of the application program responds to the application operation thread; Extracting video frames from the target video; A timestamp corresponding to the video frame is determined, and a response time of the application program to the application operation thread is determined based on the timestamp.

2. The method according to claim 1, characterized in that: Determining a timestamp corresponding to the video frame includes: Selecting a static area in the video frame as a target video frame; An image grayscale value of the target video frame is acquired, and the timestamp is determined according to the image grayscale value.

3. The method according to claim 2, characterized in that: There are a plurality of the video frames, obtaining the image grayscale value of the target video frame, and determining the timestamp according to the image grayscale value, including: Determine a plurality of target video frames corresponding to the plurality of video frames, wherein the video frames correspond to the target video frames one-to-one; Determining a temporal order of the plurality of target video frames; The image grayscale values ​​of two adjacent target video frames are compared in the time sequence to obtain a plurality of grayscale difference values, and the timestamp is determined according to the plurality of grayscale difference values.

4. The method according to claim 3, characterized in that: The time sequence is the sequence from the head frame to the tail frame, and the timestamp includes: a start frame timestamp, comparing the image grayscale values ​​of two adjacent target video frames according to the time sequence to obtain multiple grayscale differences, and determining the timestamp according to the multiple grayscale differences, including: Comparing the image grayscale values ​​of two adjacent target video frames in the order from the head frame to the tail frame to obtain a plurality of positive-sequence grayscale difference values; If a target positive-sequence grayscale difference greater than a preset threshold value is detected for the first time among the plurality of positive-sequence grayscale differences, two target video frames generating the target positive-sequence grayscale difference are acquired; Selecting a target video frame that is later in sequence among the two target video frames as a starting video frame, wherein the starting video frame undergoes a preset degree of image change compared with a previous adjacent target video frame; The start frame timestamp of the start video frame is determined.

5. The method according to claim 4, characterized in that: The time sequence is the sequence from the tail frame to the head frame, and the timestamp includes: an end frame timestamp, comparing the image grayscale values ​​of two adjacent target video frames according to the time sequence to obtain multiple grayscale differences, and determining the timestamp according to the multiple grayscale differences, including: Comparing the image grayscale values ​​of two adjacent target video frames in the order from the tail frame to the head frame to obtain a plurality of reverse-order grayscale difference values; If it is first monitored that there is a target reverse order grayscale difference value greater than the preset threshold value among the multiple reverse order grayscale differences, two target video frames generating the target reverse order grayscale difference value are acquired; Selecting a target video frame that is later in sequence among the two target video frames as an ending video frame, wherein the ending video frame undergoes an image change of the preset degree compared with a previous adjacent target video frame; The end frame timestamp is determined according to the end video frame.

6. The method according to claim 5, characterized in that: Determining a response time of the application program to the application operation thread based on the timestamp includes: Calculate the timestamp difference between the end frame timestamp and the start frame timestamp; The response time of the application program to the application operation thread is determined according to the timestamp difference.

7. The method according to claim 1, characterized in that: There are a plurality of target devices, and in response to a video recording instruction, a video recording is performed on an interface of an application running on the target device to obtain a target video, including: Obtain the device number of each target device respectively; Multiple video recording instructions are triggered respectively according to multiple device numbers, and the interfaces of applications running on the multiple target devices are recorded to obtain multiple target videos, wherein the interface of the application on each target device responds to the corresponding application operation thread.

8. The method according to claim 1, characterized in that: In response to the video recording instruction, recording the interface of the application running on the target device to obtain the target video includes: Trigger the video recording instruction according to a preset script; The interface is recorded based on the video recording instruction to obtain the target video.

9. The method according to claim 1, characterized in that: Extracting a video frame from the target video includes: Call the audio and video processing tool according to the image extraction instruction; Video frames in the target video are extracted based on the audio and video processing tool.

10. A device for determining a response time, characterized in that: include: A connection unit, used for connecting to a target device via a device interface; A recording unit, configured to record a video of an interface of an application program running on the target device in response to a video recording instruction to obtain a target video, wherein the interface of the application program responds to an application operation thread; An extraction unit, used for extracting video frames from the target video; The determining unit is used to determine a timestamp corresponding to the video frame, and determine a response time of the application program to the application operation thread based on the timestamp.

11. A computer-readable storage medium, characterized in that: The storage medium includes a stored program, wherein the program executes the response time determination method according to any one of claims 1 to 9.

12. An electronic device, characterized in that: include: One or more processors, a memory, a display device, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for determining the response time as described in any one of claims 1 to 9.

Images