Method, device and equipment for detecting end-to-end time delay

Abstract

The embodiment of the invention discloses an end-to-end time delay detection method, device and equipment. According to the method, a to-be-analyzed video containing a click instruction image and a response instruction image is generated through recording, the click instruction image is generated on a device end based on a click instruction, and the response instruction image is generated by a cloud end in response to the click instruction and is sent to the device end to be displayed; identifying a click indication image in the to-be-analyzed video, and generating a click time sequence; identifying a response indication image in the to-be-analyzed video, and generating a response time sequence; and matching the click time sequence with the response time sequence, and determining the time delay, thereby realizing accurate and stable statistics of the end-to-end time delay.

Description

[0001] This application is a divisional application of the invention patent application submitted to the China Patent Office on January 19, 2023, with application number 202310104727.3 and invention name “A method, device and equipment for detecting end-to-end time delay”. The full text of the application is incorporated into this application by reference. Technical Field

[0002] This specification relates to the field of Internet technology, and in particular to a method, apparatus, and device for detecting end-to-end time delay. Background Art

[0003] With the development of mobile internet, the use of cloud computers and cloud phones is becoming more and more widespread. In this scenario, end-to-end latency is an important performance indicator.

[0004] When a user executes a command on a local device, the response time is typically a few milliseconds, which is almost negligible. However, when the cloud is involved, the entire process is generally as follows: the user enters the command on the local device → the cloud responds and generates feedback → the cloud returns the feedback to the local device → the local device renders and displays the feedback. The total time involved in this entire chain is the end-to-end time latency.

[0005] Based on this, an accurate end-to-end time delay detection solution is needed. Summary of the Invention

[0006] The embodiments of this specification provide a method, apparatus, device, and storage medium for detecting an end-to-end time delay, to solve the following technical problem: an accurate end-to-end time delay detection solution is required.

[0007] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0008] In a first aspect, an embodiment of the present specification provides an end-to-end time delay detection method, comprising: recording and generating a video to be parsed that includes a click indication image and a response indication image, wherein the click indication image is generated on a device side based on a click instruction, and the response indication image is generated by the cloud side in response to the click instruction and sent to the device side for display; identifying the click indication image in the video to be parsed and generating a click time sequence; identifying the response indication image in the video to be parsed and generating a response time sequence; matching the click time sequence and the response time sequence to determine the time delay.

[0009] In the second aspect, an embodiment of the present specification provides an end-to-end time delay detection device, including: recording and generating a video to be parsed that includes a click indication image and a response indication image, wherein the click indication image is generated on the device side based on a click instruction, and the response indication image is generated by the cloud in response to the click instruction and sent to the device side for display; a click indication image recognition module, which recognizes the click indication image in the video to be parsed and generates a click time sequence; a response indication image recognition module, which recognizes the response indication image in the video to be parsed and generates a response time sequence; a determination module, which matches the click time sequence and the response time sequence to determine the time delay.

[0010] In a third aspect, one or more embodiments of this specification provide an electronic device, including:

[0011] at least one processor; and,

[0012] a memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect.

[0014] In a fourth aspect, an embodiment of this specification provides a non-volatile computer storage medium storing computer-executable instructions. When a computer reads the computer-executable instructions in the storage medium, the instructions enable one or more processors to execute the method described in the first aspect.

[0015] At least one of the above-mentioned technical solutions adopted in one or more embodiments of this specification can achieve the following beneficial effects: recording and generating a video to be parsed that includes a click indication image and a response indication image, wherein the click indication image is generated on the device side based on a click instruction, and the response indication image is generated by the cloud in response to the click instruction and sent to the device side for display; identifying the click indication image in the video to be parsed and generating a click time sequence; identifying the response indication image in the video to be parsed and generating a response time sequence; matching the click time sequence and the response time sequence to determine the time delay, thereby achieving accurate and stable statistics of the end-to-end time delay and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1a A schematic diagram of a flow chart of a method for detecting end-to-end time delay provided in an embodiment of this specification;

[0018] Figure 1b A schematic diagram of the architecture of a system provided in an embodiment of this specification;

[0019] Figure 2 A schematic diagram of displaying a click indication image on a device provided in an embodiment of this specification;

[0020] Figure 3 A schematic diagram of three images obtained by screen recording provided in an embodiment of this specification;

[0021] Figure 4 Schematic diagram of normal and abnormal situations provided in the embodiments of this specification;

[0022] Figure 5 A flowchart of a specific embodiment provided in the embodiments of this specification;

[0023] Figure 6 A schematic diagram of the structure of an end-to-end time delay detection device provided in an embodiment of this specification;

[0024] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0025] The embodiments of this specification provide a method, apparatus, device, and storage medium for detecting end-to-end time delay.

[0026] In order to help those skilled in the art better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0027] In the first aspect, Figure 1a As shown, Figure 1aA flow chart of a method for detecting end-to-end time delay provided in an embodiment of this specification includes the following steps:

[0028] S101: Record and generate a video to be parsed that includes a click indication image and a response indication image.

[0029] The click indication image is generated by the local device (hereinafter referred to as the device end) on the device end based on the click instruction. The click indication image may include a first identifier to indicate that the image is a click indication image. For example, the first identifier may be a click icon (cross pointer, hand icon or mouse pointer, etc.); the identifier may also be a specified color, for example, the background color of the device end remains unchanged, and when a click occurs, the display color of the device end is adjusted to another color.

[0030] Click instructions can be generated manually by the inspector or automatically by a script. Click instructions are typically given multiple times to improve the accuracy of the inspection results. The corresponding click instruction images will also be multiple images.

[0031] When the device and the cloud are connected, the click instruction will be uploaded to the cloud, and the cloud will respond based on the instruction and generate a corresponding response indication image. The response indication image can be returned to the device in various forms (for example, video stream or encoded file) and rendered and displayed by the device. Figure 1b As shown, Figure 1b This is a schematic diagram of the system architecture provided by the embodiments of this specification. Through this architecture, the cloud phone simulates the environment of the local mobile phone, so that the generated video stream can be displayed locally accordingly.

[0032] The response indication image also includes a second identification similar to the first identification. That is, the second identification can also be an indicator icon, a number, a color, etc. The second identification should be different from the first identification.

[0033] By keeping the device's display working properly, a script is used to generate simulated click commands, and at the same time, the response indication image returned by the cloud is received and displayed. At this time, the click indication image and the response indication image will appear alternately on the device's display.

[0034] At this point, you can use a device to record the phenomenon to obtain a video to be analyzed that includes a click indicator image and a response indicator image. The click indicator image and the response indicator image in the recorded video to be analyzed will appear in the form of "frame images". The background image that does not contain any information can be removed from the video to be analyzed.

[0035] For example, if a video to be parsed is 1000 seconds long and 1 second contains 24 frames, then the video to be parsed actually contains 24,000 frames of images. If the script issues 1000 click commands, then it may contain 1000 frames of discrete click indicator images and 1000 frames of discrete response indicator images. For example, the Nth click indicator image is the 800th frame of the video to be parsed, and the N+1th click indicator image is the 825th frame.

[0036] S103: Identify click indication images in the video to be analyzed and generate a click time sequence.

[0037] All frame images contained in the image to be parsed may be identified one by one, and by identifying the first identifier contained in the video to be parsed, a click indication image may be determined and a click time sequence may be generated.

[0038] The click time contained in the generated click time sequence can be expressed in "frames" or in real time. When expressed in frames, the click time sequence can be a numerical sequence such as (1, 25, 51, 77, 100, ...) indicating the frame number of the click indication image in the image to be parsed. When expressed in time points, the click time sequence is a real time sequence accurate to milliseconds such as (17:34:24:674, 17:34:26:587, ...), and the real time is the time point when the real image is displayed on the device.

[0039] S105: Identify a response indication image in the video to be parsed and generate a response time sequence.

[0040] Similar to the aforementioned method of generating a click time series, all frame images contained in the image to be parsed can also be identified one by one. By identifying the second identifier contained in the video to be parsed, the response indication image can be determined and a click time series can be generated. The generated click time series should have the same representation of click time as the click time series.

[0041] S107: Match the click time sequence and the response time sequence to determine a time delay.

[0042] Assuming that the number of click time series and the number of response time series are both 1000, that is, 1000 clicks occur and 1000 responses are generated, the two series can be directly subtracted to obtain a delay series containing 1000 time lengths.

[0043] For example, assuming that both the click time series and the response time series are represented by frames, the delay sequence obtained after subtraction may be a delay sequence such as (30, 33, 50, 27, ...) containing 1000 frame differences. For each frame, relevant statistics are performed on these 1000 frame differences, that is, the statistical value can be used to characterize the time delay.

[0044] The recording generates a video to be parsed that includes a click indication image and a response indication image, wherein the click indication image is generated on the device side based on a click instruction, and the response indication image is generated by the cloud in response to the click instruction and sent to the device side for display; the click indication image in the video to be parsed is identified to generate a click time sequence; the response indication image in the video to be parsed is identified to generate a response time sequence; the click time sequence and the response time sequence are matched to determine the time delay, thereby achieving accurate and stable statistics of the end-to-end time delay and improving the user experience.

[0045] In one embodiment, the first click identifier can be a click icon. That is, based on the click instruction, a click instruction can be received on the device side, and a click indication image containing a click icon is generated on the device side. For example, when the device side is an Android phone, a function "pointer position" is provided in the developer options of the phone. When this function is turned on, each click will display a click icon at the click position (the default is a cross, and it can also be modified to other forms of click icons). At the same time, the real time of the click when the click icon appears on the device side can also be provided, thereby generating a click indication image containing a cross. Figure 2 As shown, Figure 2 This is a schematic diagram of displaying a click indication image on a device provided in an embodiment of this specification. In this schematic diagram, the display interface on the device provides both a click icon and the actual time of the click.

[0046] In this case, when subsequently identifying click indication images in the video to be parsed, the video to be parsed can be identified frame by frame, and the frame image containing a click icon (e.g., a cross) can be determined as the click indication image. Furthermore, the time contained in the image can be obtained and determined as the actual click image.

[0047] In one embodiment, the cloud can count the number of click instructions received and determine the sequence number corresponding to the received click instructions. For example, a "touch counter" can be integrated into the cloud. For each click time sent by the client, the counter is incremented by 1, and a response indication image containing the sequence number is generated.

[0048] Furthermore, in the identification of the response indication image in the video to be parsed, it is sufficient to identify the serial number in the response indication image. By identifying the digital serial number contained in the image (for example, using an OCR algorithm), not only can the response indication image be easily identified, but the identified serial number can also be used as the corresponding response serial number, facilitating subsequent matching between the two sequences.

[0049] like Figure 3 As shown, Figure 3 A schematic diagram of three images obtained by screen recording provided in an embodiment of this specification. The first image is actually the background image displayed on the device after the fifth click, which includes the numerical sequence number 5 but does not include the click icon (i.e., does not include a cross); the second image is the image generated by the device after the fifth click (i.e., the sixth click), which includes the numerical sequence number 5 and a cross-shaped click icon; the third image is the response indicator image generated by the cloud and returned to the device for display, which includes the numerical sequence number 6.

[0050] In addition, the three images are actually extracted from the video to be parsed and are not necessarily adjacent. Also, since the video to be parsed is obtained based on the screen recording function, the image display time (i.e., recording time) can also be included in the image. For example, the display times T1, T2, and T3 are displayed in the upper right corner of the image.

[0051] In one embodiment, when generating a response indication image, the cloud may, in addition to displaying a serial number, also change the color of the screen to generate a response indication image that includes the serial number and is a different color from the previous response indication image. For example, if the Nth response indication image is green, the color of the N+1th response indication image will be blue, the color of the N+2th response indication image will be red, and the color of the N+3th response indication image will be green, and so on.

[0052] Correspondingly, when identifying the response indication image in the video to be parsed, in addition to identifying the digital serial number contained in the image, the serial number in the image frame that has undergone color change can also be identified by determining the image frame that has undergone color change in the video to be parsed.

[0053] As mentioned above, when the image is displayed on the device side, the click indication image and the response indication image are actually displayed alternately. However, since the click indication image is only one frame, the response indication image will be displayed as the background image all the time. For example, Figure 3In the schematic diagram shown, the first image is actually the response indication image for the fifth click. There may be many frames between it and the second image. The click indication image will not be displayed until the sixth click instruction is received. At the same time, the second image and the third image usually also contain many frames, but at this time the first image is still displayed on the screen.

[0054] In other words, if you identify the video frame by frame, you'll mostly end up with a response indicator image. However, by marking the image color and comparing the frames that have undergone color changes, you can accurately identify the valid response indicator image, avoiding the need for frame-by-frame digital identification and improving efficiency.

[0055] After obtaining the click time series and response time series, the two can be compared to determine the time delay. Under normal circumstances, if the user clicks N times, the corresponding response will be N times. At the same time, for the ath click (1≤a≤N) in the corresponding sequence, the ath response should be between the ath click and the a+1th click. Under normal circumstances, the ath click and the ath response constitute a valid response pair, and a valid response pair represents the end-to-end delay time generated by the click.

[0056] For multiple valid response pairs, statistics can be collected to determine the end-to-end delay. For example, statistics can be collected on the maximum delay, minimum delay, average delay, etc., as well as statistics on the proportion of delays falling within a certain delay range, such as the proportion of delays between [70ms, 100ms].

[0057] In one embodiment, if the click time sequence and the response time sequence are represented by frame numbers, the time delay between the valid response pairs can be determined based on the refresh rate of the device during video recording and the frame number difference.

[0058] For example, a high-performance high-speed camera or a device-side screen recording function can be used to generate a video to be parsed that includes a click indication image and a response indication image. During recording, the refresh rate of the corresponding device can be determined. For example, a device with a 60HZ refresh rate can only achieve a time resolution of 16.67ms, a 90HZ mobile phone can achieve 11.11ms, and a 120Hz phone can achieve 8.3ms. That is, assuming that the device refresh rate during recording is f, the resolution between frames is 1 / f. If the frame difference between a valid response pair is M, then the time delay corresponding to the valid response pair is M / f. In practical applications, devices that support high refresh rates can be used to effectively improve accuracy.

[0059] Other situations except normal situations should be considered as abnormal situations. In abnormal situations, there are invalid response pairs. Figure 4 As shown, Figure 4 This is a schematic diagram of normal and abnormal situations provided in the embodiments of this specification, wherein the abnormal situations include disorder, downlink frame loss, and uplink frame loss.

[0060] Since the click instructions are generated locally on the device, the number and order of the recognized click indication images can be guaranteed to be exactly the same as the number and order of the click instructions. In other words, in most cases, only the response time series needs to be compared.

[0061] For example, in Figure 4 In the disorder shown, the ath response actually occurs after the a+1th click (normally it should occur between the ath click and the a+1th click). This situation may usually be caused by network fluctuations and should be eliminated.

[0062] For example, in Figure 4 In the illustrated downlink frame loss, the a+1th response is actually not generated or is lost during network transmission, but the other responses are still normal. In this case, the a+1th click and response can be removed from the two sequences for subsequent matching without affecting the results.

[0063] For example, in Figure 4 The illustrated uplink frame loss is actually due to network transmission issues, resulting in click command loss (but it's unknown which click command was lost during the uplink). The resulting cloud response count doesn't match the client click count. That is, the number of click events in the click time series is less than the number of click commands. At this point, because it's impossible to identify which click on the client the cloud response matches, the experimental material is invalid and can be re-recorded for analysis.

[0064] The following is a specific example to illustrate the solution of this application. Figure 5 As shown, Figure 5 This is a flowchart of a specific embodiment provided in the embodiments of this specification. Before testing, first prepare the environment, which includes both the device side and the cloud side. On the local device, turn on the click pointer display in the mobile phone function, turn on the touch counter in the cloud, apply for the corresponding cloud phone, enter the test, and turn on the high fps screen recording function.

[0065] Then, start video recording on the local device, use a script to simulate a predetermined number of clicks (for example, 1000 times), end the screen recording after the clicks are completed, generate a video to be parsed, and perform video parsing. The video parsing includes using image parsing to identify click indicator images (by identifying a crosshair pointer) and using ORC to identify response indicator images (by identifying serial numbers), and then by matching clicks and responses, calculate the time taken.

[0066] In the second aspect, based on the same idea, the embodiments of this specification also provide devices and equipment corresponding to the above method, such as Figure 6 and Figure 7 shown.

[0067] Figure 6 This is a schematic diagram of the structure of an end-to-end time delay detection device provided in an embodiment of this specification, the device comprising:

[0068] A recording module 601 records and generates a video to be parsed that includes a click indication image and a response indication image, wherein the click indication image is generated on the device side based on the click instruction, and the response indication image is generated by the cloud side in response to the click instruction and sent to the device side for display;

[0069] A click indication image recognition module 603 is configured to recognize click indication images in the video to be analyzed and generate a click time sequence;

[0070] A response indication image recognition module 605 is configured to recognize a response indication image in the video to be parsed and generate a response time sequence;

[0071] The determination module 607 matches the click time sequence and the response time sequence to determine a time delay.

[0072] Optionally, the apparatus further includes a click indication image generation module 609, which receives a click instruction and generates a click indication image including a click icon on the device side; correspondingly, the click indication image recognition module 603 recognizes the click icon in the video to be parsed.

[0073] Optionally, the cloud counts the received click instructions to determine the serial number corresponding to the received click instructions; in response to the click instruction, generates a response indication image containing the serial number; accordingly, the response indication image recognition module 605 recognizes the serial number in the response indication image.

[0074] Optionally, the cloud generates a response indication image containing the serial number and having a color different from that of the previous response indication image; accordingly, the response indication image recognition module 605 determines the image frame with color change in the video to be analyzed, and identifies the serial number in the image frame with color change.

[0075] Optionally, the determining module 607 determines valid response pairs of the response time sequence to the click time sequence, and counts the valid response pairs to determine a time delay.

[0076] Optionally, the determination module 607 determines a frame number difference between the valid response pairs, and determines a time delay of the valid response pairs according to a refresh rate of the device end during video recording and the frame number difference.

[0077] Optionally, the recording module 601 re-records another video to be parsed if the number of click times in the click time sequence is less than the number of click instructions.

[0078] Optionally, the recording module 601 generates a video to be parsed including a click indication image and a response indication image by recording through a screen recording function of the device.

[0079] In the third aspect, Figure 7 As shown, Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification, the device comprising:

[0080] at least one processor; and,

[0081] a memory communicatively connected to the at least one processor; wherein,

[0082] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect.

[0083] In the fourth aspect, based on the same idea, an embodiment of this specification also provides a non-volatile computer storage medium corresponding to the above method, which stores computer-executable instructions. When a computer reads the computer-executable instructions in the storage medium, the instructions enable one or more processors to execute the method described in the first aspect.

[0084] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0085] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0086] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0087] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0088] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0090] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0092] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0093] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0094] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0095] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0096] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0097] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0098] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A method for detecting end-to-end time delay, applied to cloud computing or cloud mobile phone scenarios, comprising: Recording and generating a video to be parsed that includes a click indication image and a response indication image, wherein the click indication image is generated on the device side based on a click instruction, the response indication image is generated by the cloud side in response to the click instruction and sent to the device side for display, and the response indication image includes a sequence number corresponding to the click instruction, and there are multiple response indication images; Identifying a click indication image in the video to be parsed and generating a click time sequence; Identifying a response indication image in the video to be parsed and generating a response time sequence; The click time sequence and the response time sequence are matched to determine a time delay.

2. The method according to claim 1, wherein The click indication image is generated on the device side based on the click instruction, including: receiving a click instruction and generating a click indication image including a click icon on the device side; Correspondingly, identifying the click indication image in the video to be parsed includes: identifying the click icon in the video to be parsed.

3. The method according to claim 1, wherein The response indication image is generated by the cloud in response to the click instruction, and includes: The cloud counts the received click instructions and determines the sequence number corresponding to the received click instructions; In response to the click instruction, generating a response indication image including the serial number; Correspondingly, the identifying the response indication image in the video to be parsed includes: identifying a serial number in the response indication image.

4. The method according to claim 3, wherein: Generating a response indication image containing the sequence number, including: generating a response indication image including the serial number and having a color different from that of the previous response indication image; Correspondingly, the identifying the response indication image in the video to be analyzed includes: determining an image frame in which a color change occurs in the video to be analyzed, and identifying a serial number in the image frame in which a color change occurs.

5. The method according to claim 1, wherein Matching the click time sequence and the response time sequence to determine a time delay includes: Determine valid response pairs of the response time sequence to the click time sequence, and count the valid response pairs to determine a time delay.

6. The method according to claim 5, wherein: Determining valid response pairs of the response time sequence to the click time sequence, and counting the valid response pairs to determine a time delay, including: The frame number difference between the valid response pairs is determined, and the time delay of the valid response pairs is determined according to the refresh rate of the device end during video recording and the frame number difference.

7. The method according to claim 1, before matching the click time sequence and the response time sequence to determine the time delay, the method further comprises: If the number of click times in the click time sequence is less than the number of click instructions, another video to be parsed is re-recorded.

8. The method of claim 1, wherein: The video to be parsed is recorded and generated, including click indication images and response indication images, including: The video to be parsed, which includes a click indication image and a response indication image, is recorded and generated by the screen recording function of the device.

9. An end-to-end time delay detection device, applied to cloud computers or cloud phones, comprising: a recording module for recording and generating a video to be parsed that includes a click indication image and a response indication image, wherein the click indication image is generated on the device side based on a click instruction, the response indication image is generated by the cloud side in response to the click instruction and sent to the device side for display, and the response indication image includes a sequence number corresponding to the click instruction, and there are multiple response indication images; A click indication image recognition module, which recognizes the click indication image in the video to be analyzed and generates a click time sequence; A response indication image recognition module is configured to recognize a response indication image in the video to be analyzed and generate a response time sequence; A determination module matches the click time sequence and the response time sequence to determine a time delay.

10. An electronic device comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images