Interface response time delay test method, system, medium and equipment

Abstract

The embodiment of the invention provides an interface response time delay test method and system, a medium and equipment, and the method comprises the steps: collecting continuous image frames of a response process after a target interface is triggered, and recognizing a target feature point in each continuous image frame, and constructing a comprehensive evaluation feature fusing the quantity feature, the spatial distribution feature and the hue feature of the target feature point, judging a response image frame when the target interface enters a steady state for the first time based on the change trend of the comprehensive evaluation feature in the time dimension, and determining a response time delay between the response time corresponding to the response image frame and the trigger time. According to the method, the state of the target interface in the response process is represented through the multi-dimensional features, so that the response image frame is accurately judged, and the time delay test precision is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computers, and particularly relates to an interface response time delay testing method, system, medium and device. BACKGROUND

[0002] A human-computer interaction interface is a key carrier for user interaction with a device, and interface response time delay is a key indicator for measuring device interaction performance and user experience. Long interface response time delay can cause user operation feedback lag. In particular, in high-frequency interaction scenarios such as an electrical program guide (EPG), interface response time delay directly affects user program selection and viewing experience. Therefore, accurate and efficient testing of interface response time delay is a key link in the device research and development, production and operation process.

[0003] At present, the response time is generally determined by image similarity evaluation, and then the interface response time delay is calculated. That is, based on the image similarity algorithm, the continuous image frames of the human-computer interaction interface after the operation trigger are compared frame by frame, and when the similarity of adjacent image frames reaches a preset threshold, it is determined that the interface enters a stable state, and the interface response time delay is determined based on the response completion time. However, this image similarity evaluation method has poor recognition sensitivity to small changes, and is prone to misjudgment of the response completion time in the unstable transition stage, resulting in poor time delay test accuracy. SUMMARY

[0004] The embodiments of the present application provide an interface response time delay testing method, system, medium and device, which can improve the time delay test accuracy.

[0005] In a first aspect, the embodiments of the present application provide an interface response time delay testing method, which comprises: triggering a time delay test task in response to a time delay test instruction of a target interface; recording a triggering time of the time delay test task; collecting continuous image frames of the target interface in a response process after the triggering time; identifying target feature points in each of the continuous image frames; for each of the continuous image frames, determining a spatial distribution feature according to positions of the target feature points in the continuous image frame; for each of the continuous image frames, determining a comprehensive evaluation feature of the continuous image frame according to a number of the target feature points, the spatial distribution feature of the target feature points and a color tone feature of the continuous image frame; determining a starting frame entering a steady state in the continuous image frames as a response image frame according to a change of the comprehensive evaluation feature of each of the continuous image frames in a time dimension. The response latency of the target interface is determined based on the response time corresponding to the response image frame and the trigger time.

[0006] In one feasible implementation, identifying target feature points in each of the consecutive image frames specifically includes: Construct the scale space for each of the consecutive image frames; For each of the consecutive image frames, candidate feature points in the consecutive image frames are identified by scale space extremum detection based on the scale space of the consecutive image frames. For each of the consecutive image frames, the contrast of each candidate feature point is determined by fitting a three-dimensional quadratic function based on the position and scale parameters of each candidate feature point in the consecutive image frame. For each of the consecutive image frames, target feature points are selected from the candidate feature points based on the contrast of each candidate feature point and a preset contrast threshold.

[0007] In one feasible implementation, for each of the consecutive image frames, spatial distribution features are determined based on the positions of the target feature points in the consecutive image frames, specifically including: For each of the consecutive image frames, the mean position and standard deviation position are determined based on the position of each target feature point in the consecutive image frame; For each of the consecutive image frames, the discreteness of the target feature points is determined based on the position mean and the position standard deviation, and is used as the spatial distribution feature.

[0008] In one feasible implementation, based on the changes in the comprehensive evaluation features of each of the consecutive image frames over time, a starting frame that enters a steady state is determined as the response image frame, specifically including: According to the time sequence of the response process, the comprehensive evaluation features of each consecutive image frame are sorted to obtain the image frame feature sequence; Determine the rate of change of each comprehensive evaluation feature relative to the previous comprehensive evaluation feature in the image frame feature sequence; Continuous image frames whose rate of change is within a preset slope range are taken as steady-state image frames; The starting frame in the time dimension of each of the steady-state image frames is taken as the response image frame.

[0009] In one feasible implementation, for each of the consecutive image frames, a comprehensive evaluation feature is determined based on the quantity characteristics of each target feature point in the consecutive image frame, the spatial distribution characteristics of the target feature points, and the tonal characteristics of the consecutive image frame. Specifically, this includes: The quantitative features, the spatial distribution features, and the hue features are normalized. For each of the consecutive image frames, the normalized quantitative features, spatial distribution features, and hue features are weighted and fused according to a preset target weight coefficient group to obtain the comprehensive evaluation features of the consecutive image frames.

[0010] In one feasible implementation, before weighted fusing the normalized quantitative features, spatial distribution features, and hue features according to a preset target weight coefficient group for each consecutive image frame to obtain the comprehensive evaluation features of the consecutive image frame, the method further includes: Construct the initial set of weight coefficients; Acquire multiple time-delay training sets, each of which includes multiple consecutive image frame samples and label response image frames; According to the initial weight coefficient group, the quantitative features, spatial distribution features and tone features of each of the consecutive image frame samples are weighted and fused to obtain the comprehensive evaluation features of each of the consecutive image frame samples; Based on the changes of each comprehensive evaluation feature over time, the predicted response image frames for each of the time-delay training sets are selected. With the goal of minimizing the temporal difference between the predicted response image frame and the labeled response image frame, the initial weight coefficient set is iteratively optimized based on the multiple time-delay training sets until a preset termination condition is reached, thus obtaining the target weight coefficient set.

[0011] In one feasible implementation, after determining the response latency of the target interface based on the response time of the response image frame and the trigger time, the method further includes: In each of the consecutive image frames, the first time of the actual response of the latency test task is determined; Determine the response deviation between the first time and the response time; If the response deviation is greater than a preset deviation threshold, the target weight coefficient group is adjusted. Based on the adjusted target weight coefficient set, the next latency test task is executed.

[0012] Secondly, embodiments of this application provide an interface response latency testing device, the device comprising: The response module is used to trigger latency test tasks in response to latency test commands from the target interface; The trigger time module is used to record the trigger time of the latency test task; The continuous image frame module is used to acquire continuous image frames of the target interface during the response process after the trigger time. The feature point recognition module is used to identify target feature points in each of the consecutive image frames; The spatial distribution module is used to determine spatial distribution features for each of the consecutive image frames based on the positions of the target feature points in the consecutive image frames. The comprehensive evaluation module is used to determine the comprehensive evaluation features of each of the consecutive image frames based on the number of target feature points in each of the consecutive image frames, the spatial distribution features of the target feature points, and the tonal features of the consecutive image frames. The response image frame module is used to determine, based on the changes in the comprehensive evaluation features of each of the consecutive image frames in the time dimension, the starting frame that enters a steady state in the consecutive image frames, and use it as the response image frame; The response latency module is used to determine the response latency of the target interface based on the response time corresponding to the response image frame and the trigger time.

[0013] Thirdly, embodiments of this application provide an interface response latency testing device, the device comprising: a processor, and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement any of the above-mentioned interface response latency testing methods.

[0014] Fourthly, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement any of the above-mentioned interface response latency testing methods.

[0015] Fifthly, the embodiments of this application provide a computer program product in which the instructions are executed by the processor of an electronic device, causing the electronic device to execute any of the above-mentioned interface response latency testing methods.

[0016] This application discloses a method, system, medium, and device for testing interface response latency. It can acquire continuous image frames of the response process after a target interface is triggered, and identify target feature points in each continuous image frame. This allows for the construction of a comprehensive evaluation feature that integrates the quantity, spatial distribution, and hue features of the target feature points. Based on the changing trend of the comprehensive evaluation feature over time, it determines the response image frame at which the target interface first enters a steady state, and thus determines the response latency between the response time and the trigger time corresponding to the response image frame. In other words, this application characterizes the state of the target interface during the response process using multi-dimensional features to accurately determine the response image frame, thereby improving the accuracy of latency testing.

[0017] Furthermore, this application can also introduce a dynamically adjusted target weight coefficient group to adaptively weight and fuse quantitative features, spatial distribution features, and hue features, thereby improving the reliability and versatility of response delay testing. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an interface response latency testing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the EPG interface response latency test provided in an embodiment of this application; Figure 3 This is a schematic diagram of the process for dynamically adjusting the target weight coefficient group provided in the embodiments of this application; Figure 4 This is a schematic diagram of the interactive screen of the automated testing device provided in the embodiments of this application; Figure 5 This is a schematic diagram of the device interface of the automated testing apparatus provided in the embodiments of this application; Figure 6 This is a schematic diagram of the device connections of the automated testing apparatus provided in the embodiments of this application; Figure 7 This is a schematic diagram illustrating the cloud-edge collaboration method for completing interface response latency testing, as provided in an embodiment of this application. Figure 8 This is a schematic diagram of interface response latency testing under the platform task testing scenario provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of an interface response delay testing device provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of an interface response delay testing device provided in an embodiment of this application. Detailed Implementation

[0020] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0021] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0022] Human-computer interfaces (HCIs) are the key medium for user interaction with devices, and interface response latency is a crucial indicator for measuring device interaction performance and user experience. Excessive interface response latency leads to delayed user feedback. Especially in high-frequency interaction scenarios such as Electronic Program Guides (EPGs), interface response latency directly impacts users' program selection and viewing experience. Therefore, accurate and efficient testing of interface response latency is a critical aspect of device research and development, production, and maintenance.

[0023] Currently, response time is generally determined by image similarity evaluation, and then the interface response latency is calculated. This involves comparing consecutive image frames of the human-computer interaction interface captured after an operation is triggered, frame by frame. When the similarity between adjacent image frames reaches a preset threshold, the interface is considered to have entered a stable state, and this is used as the response completion time to determine the interface response latency. However, this image similarity evaluation method has poor sensitivity to subtle changes and is prone to misjudging unstable transition phases as response completion times, resulting in poor latency testing accuracy.

[0024] To address the problems in the prior art, embodiments of this application provide a method, system, medium, and device for testing interface response latency.

[0025] In practical applications, the execution subject of the interface response latency testing method in this application embodiment can be a terminal device, such as a desktop computer or laptop computer, or a remote device similar to a server. Of course, this application embodiment can also adopt an execution subject in the form of software, such as a client or software program installed on a terminal device. The specific type of execution subject corresponding to the technical solution provided in this application embodiment is not strictly limited here, and can be flexibly selected according to the actual application scenario and actual needs.

[0026] The following describes specific embodiments of an interface response latency testing method, system, medium, and device provided in this application. First, an interface response latency testing method is introduced.

[0027] Figure 1 This diagram illustrates a flowchart of an interface response latency testing method according to an embodiment of this application. Figure 1 As shown, the method includes steps S100 to S105: S100: In response to the latency test command on the target interface, trigger the latency test task.

[0028] S101: Record the trigger time of the latency test task.

[0029] In one or more embodiments of this application, in order to ensure the accuracy of latency testing in subsequent steps, in steps S100 to S101, this application needs to complete the initiation triggering of the latency test task and mark the triggering time of the latency test task to provide a time reference for response latency.

[0030] Specifically, this application receives a latency test command for a target interface, which can be manually triggered or automatically generated by the device based on interactive operations. Upon receiving the latency test command, a latency test task is initiated, and the trigger time of the latency test task is recorded synchronously as the starting point for latency calculation of the target interface.

[0031] It should be noted that this application does not restrict the triggering method of the latency test task, and it can be set according to actual needs.

[0032] S102: After the trigger time, acquire continuous image frames of the target interface during the response process.

[0033] In one or more embodiments of this application, in order to fully acquire the screen information of the target interface from the triggering of the latency test task to entering a steady state, so as to provide data support for feature extraction and state determination in subsequent steps, this application needs to continuously acquire continuous image frames of the target interface after the triggering time.

[0034] It should be noted that this application does not limit the frequency or method of acquiring continuous image frames, such as acquiring continuous image frames through screenshot operations at certain frame intervals. Taking EPG image acquisition as an example, during the time delay test from start to finish, the EPG image acquisition module performs uninterrupted screenshot operations on the EPG image at 50Hz frame intervals to capture the continuous changes of the EPG page throughout the entire time delay test.

[0035] S103: Identify target feature points in each of the consecutive image frames.

[0036] In one or more embodiments of this application, in order to accurately extract the quantitative and spatial distribution features of consecutive image frames in subsequent steps, and to provide a data foundation for the construction of comprehensive evaluation features, this step requires the identification of target feature points in each consecutive image frame.

[0037] It should be noted that this application does not limit the specific method of identifying target feature points, and can be set according to actual needs, such as corner detection algorithms; or using algorithms such as SIFT and SURF to achieve scale-invariant feature point recognition; or for specific scenes such as solid color transition frames, using edge detection algorithms to extract image edge feature points as target feature points. In one or more embodiments of this application, this application can construct a scale space for each consecutive image frame; for each consecutive image frame, based on the scale space of the consecutive image frame, candidate feature points in the consecutive image frame are identified through scale space extremum detection; for each consecutive image frame, based on the position and scale parameters of each candidate feature point in the consecutive image frame, the contrast of each candidate feature point is determined through three-dimensional quadratic function fitting; for each consecutive image frame, based on the contrast of each candidate feature point and a preset contrast threshold, target feature points are selected from each candidate feature point. This application does not limit the specific method of constructing the scale space, and can be set according to actual needs, such as Gaussian blur.

[0038] In this embodiment, the present application adopts a multi-scale representation method. By performing Gaussian blur on continuous image frames at different scales, the scale space of continuous image frames is constructed. In the scale space, local extreme points are detected by comparing the gray values ​​of each pixel with its neighboring pixels and the gray values ​​of neighboring pixels at different scales. The detected local extreme points are used as target feature points.

[0039] S104: For each of the continuous image frames, determine the spatial distribution features based on the position of each of the target feature points in the continuous image frame.

[0040] In one or more embodiments of this application, in order to construct a comprehensive evaluation feature that can accurately characterize the interface structure state in subsequent steps, this step requires quantifying the spatial arrangement pattern of target feature points based on the position of each target feature point in the continuous image frames to obtain spatial distribution features.

[0041] It should be noted that this application does not limit the determination method and specific content of spatial distribution features. The methods can be set according to actual needs, such as traversing all target feature points to obtain their distribution, calculating the mean and standard deviation of the coordinates of the feature points in the x and y directions in two-dimensional space, and using the dispersion assessed by the standard deviation and mean as spatial distribution features; combining three-dimensional quadratic function fitting techniques to determine the coordinates and scale information of feature points, removing low-contrast and unstable feature points, and then calculating parameters such as the cluster center distance and distribution density of the remaining effective feature points as spatial distribution features; or constructing an adjacency matrix of feature points to analyze the relative positional relationships between feature points, and using the matrix features as spatial distribution features. In one or more embodiments of this application, for each consecutive image frame, this application can determine the position mean and position standard deviation based on the position of each target feature point in the consecutive image frame; for each consecutive image frame, the dispersion of the target feature points can be determined based on the position mean and position standard deviation as spatial distribution features.

[0042] S105: For each of the continuous image frames, determine the comprehensive evaluation features of the continuous image frame based on the number of target feature points in each of the continuous image frames, the spatial distribution features of the target feature points, and the tonal features of the continuous image frame.

[0043] It should be noted that this application does not limit the specific type of hue features. They can be set according to actual needs, such as analyzing the color values ​​of all pixels in consecutive image frames and calculating the overall color mean as a hue feature; or extracting the color histogram of consecutive image frames and statistically analyzing the pixel distribution ratio of different color channels as a hue feature; or calculating the color moment features of consecutive image frames, using the first moment (mean), second moment (variance), and third moment (skewness) as hue features; or for grayscale consecutive image frames, directly using the average grayscale value of the consecutive image frames as the hue feature. This application does not limit the specific method for determining the comprehensive evaluation features. They can be set according to actual needs, such as using a weighted summation method, assigning weights to the quantity feature, spatial distribution feature, and hue feature of the target feature points and summing them to obtain the comprehensive evaluation feature; or using feature concatenation, sequentially concatenating the quantified values ​​of the three types of features into a one-dimensional feature vector, which is then used as the comprehensive evaluation feature.

[0044] S106: Based on the changes in the comprehensive evaluation features of each of the consecutive image frames in the time dimension, determine the starting frame that enters a steady state in the consecutive image frames as the response image frame.

[0045] In one or more embodiments of this application, in order to accurately calculate the response latency of the target interface in subsequent steps, this step requires analyzing the temporal variation patterns of the comprehensive evaluation features to determine the response image frame at which the target interface first enters a stable state.

[0046] It should be noted that this application does not limit the specific method of determining the response image frame. It can be set according to actual needs, such as calculating the variance of the comprehensive evaluation features of multiple consecutive frames. When the variance is lower than a preset threshold and remains stable for multiple consecutive frames, the starting frame of the stable phase is taken as the response image frame. Alternatively, a benchmark threshold for the comprehensive evaluation features can be set. When the deviation between the comprehensive evaluation features of a certain image frame and the features of subsequent multiple frames is less than the benchmark threshold, the starting frame with a deviation less than the benchmark threshold is taken as the response image frame. In one or more embodiments of this application, the comprehensive evaluation features of each consecutive image frame can be sorted according to the temporal sequence of the response process to obtain an image frame feature sequence; the rate of change of each comprehensive evaluation feature relative to the previous comprehensive evaluation feature in the image frame feature sequence can be determined; consecutive image frames with a rate of change within a preset slope range can be taken as steady-state image frames; and the starting frame in the time dimension of each steady-state image frame can be taken as the response image frame.

[0047] For example, the slope of adjacent comprehensive evaluation features in the time dimension is used as the rate of change, with the slope range set to -0.2 to 0.2. By comparing the slopes of different continuous portions, the portion in a set of data where a gentle trend first appears is identified. This gentle portion is the steady-state portion and is considered the static portion of the image. In the static portion, the rate of change tends to stabilize, without a significant increasing or decreasing trend. Continuous image frames with a rate of change close to 0 or within the slope range are considered steady-state image frames, thus determining the response image frame that first enters the steady state.

[0048] S107: Determine the response delay of the target interface based on the response time corresponding to the response image frame and the trigger time.

[0049] It should be noted that this application does not limit the specific method of determining the response latency. It can be set according to actual needs, such as directly calculating the time difference between the response time and the trigger time corresponding to the response image frame as the response latency of the target interface; or for test scenarios with multiple sets of continuous image frames, calculating the latency variance of each set of data and combining it with the mean to obtain a latency result that has both accuracy and stability.

[0050] In the aforementioned interface response latency testing method, after acquiring a set of consecutive image frames sorted by timestamps, this application needs to analyze and process the features of the consecutive image frames. Specifically, image processing techniques are used to acquire the quantity, spatial distribution, and tonal characteristics of target feature points in each consecutive image frame. Then, according to the appropriate target weight coefficient group, the proportion of each consecutive image frame in these three feature dimensions is allocated to obtain accurate comprehensive evaluation features. After normalizing each comprehensive evaluation feature, based on the changes in the time dimension, the continuous range in which the comprehensive evaluation features enter a stable state is statistically determined, and the starting frame of this continuous range is identified as the response image frame. The response time corresponding to the response image frame is the response end point, marking the end of the response. Based on this response time and the trigger time, the response latency of the target interface can be determined, completing the interface response latency test.

[0051] Figure 2 This is a schematic diagram illustrating the EPG interface response latency test provided in an embodiment of this application. Figure 2 As shown, the test begins by entering the latency test start page and recording the test start time (trigger time) as the starting benchmark for latency calculation. Then, EPG continuous screenshotting is initiated, continuously capturing consecutive image frames of the target interface during the response process. Image feature analysis is then performed on the captured consecutive image frames to determine whether the current image is static or if a preset target image appears. If this condition is not met, the image feature analysis continues. Conversely, if the condition is met, the corresponding response image frame is marked, and the response time of the response image frame is recorded as the test end time. Finally, the response latency of the EPG interface is calculated based on the difference between the recorded trigger time and response time.

[0052] It should be noted that this application does not limit the specific content of the target image and can be set according to actual needs. For example, when switching between EPG program sets, a completely black screen is used as a transition, which can be used as a target image that needs to be specifically identified during the EPG interface response latency test. The characteristic of continuous image frames in the completely black state is that "feature points are empty and the color is black". Therefore, this application can identify the target image corresponding to the completely black state based on this characteristic. That is, in one or more embodiments of this application, this application can identify the target image based on the similarity between the quantity characteristics, spatial distribution characteristics, and tone characteristics of continuous image frames and the characteristics corresponding to the target image, thereby avoiding test errors caused by misjudgment of specific target images such as completely black screens.

[0053] In step S106, in order to ensure the accuracy of feature fusion, in one or more embodiments of this application, this application can normalize the quantitative features, spatial distribution features and tone features of each continuous image frame, and then perform weighted fusion of the normalized quantitative features, spatial distribution features and tone features according to a preset target weight coefficient group to obtain the comprehensive evaluation features of the continuous image frame.

[0054] It should be noted that this application does not limit the specific values ​​of the target weight coefficient group, and can be set according to actual needs. For example, the proportion of each feature dimension can be adaptively adjusted according to the interaction triggers, screen transitions, and response completion characteristics of the target interface. In one or more embodiments of this application, this application can train an initial weight coefficient group based on training data within a specific application scenario to obtain a target weight coefficient group adapted to that specific application scenario, as follows: First, this application requires image acquisition and annotation, namely, obtaining multiple time-delay training sets, each of which includes multiple consecutive image frame samples and labeled response image frames. Simultaneously, the quantitative characteristics, spatial distribution characteristics, and tonal characteristics of each consecutive image frame sample are acquired.

[0055] Taking the EPG interface response latency test scenario as an example, under the same latency metric, different types of programs such as live broadcasts, TV dramas, and movies are selected from the EPG. Using infrared transmitters and other equipment, the user's operation steps on the EPG interface are simulated, and continuous image frames of the EPG test process are captured within a preset time interval. After acquiring the continuous image frames, the image frame marking the end point of the latency test for each group of continuous image frames is designated as the tag response image frame for subsequent deviation calculation.

[0056] Secondly, an initial set of weight coefficients is constructed. Then, according to this initial set of weight coefficients, the quantitative features, spatial distribution features, and tonal features of each consecutive image frame sample are weighted and fused to obtain the comprehensive evaluation features of each consecutive image frame sample. Based on the changes of each comprehensive evaluation feature over time, the predicted response image frames for each time-delay training set are selected. Finally, with the goal of minimizing the temporal difference between the predicted response image frames and the labeled response image frames, the initial set of weight coefficients is iteratively optimized based on multiple time-delay training sets until a preset termination condition is met, resulting in the target set of weight coefficients.

[0057] To continuously improve the fusion accuracy of the comprehensive evaluation features, in one or more embodiments of this application, the application may further determine the first time of the actual response of the latency test task in each consecutive image frame; determine the response deviation between the first time and the response time; adjust the target weight coefficient group if the response deviation is greater than a preset deviation threshold; and execute the next latency test task based on the adjusted target weight coefficient group. This application is not limited in its specific method of determining the first time; it can be set according to actual needs. For example, this application may send each consecutive image frame to the user, who can then filter out the image frame with the actual response from the consecutive image frames and mark the time corresponding to that image frame as the first time of the actual response of the latency test task.

[0058] Figure 3 This is a schematic diagram illustrating the dynamic adjustment process of the target weight coefficient group provided in an embodiment of this application. Figure 3 As shown, the target weight coefficient set can be obtained by training an initial weight coefficient set composed of multiple initial coefficients under specific application scenarios. In application, this application can also dynamically adjust the target weight coefficient set according to the actual response deviation. This application does not limit the specific training method during the training phase; it can be set according to actual needs, such as the gradient descent algorithm. During the application phase, the target weight coefficient set will also be dynamically adjusted based on the results of subsequent tests. Users can directly provide feedback on the results. That is, users can view screenshots of the testing process and, based on these screenshots, filter out the continuous image frames corresponding to the final endpoint position. When the image changes little or the judgment is inaccurate, users can manually select the first time corresponding to the accurate endpoint position in the platform, and then readjust the target weight coefficient set based on the deviation between the first time and the response time to improve adaptability.

[0059] Based on the aforementioned interface response latency testing method, this application also provides a specific embodiment of an interface response latency testing system. The system includes a simulation triggering module, an image acquisition module, an image analysis module, and a latency calculation module, comprising: The simulation triggering module is used to simulate user operations to trigger a latency test task in response to a latency test command from the target interface; record the trigger time of the latency test task; and send the trigger time to the latency calculation module. The image acquisition module is used to acquire continuous image frames of the target interface during the response process after the trigger time; send each of the continuous image frames to the image analysis module; the image analysis module is used to identify target feature points in each of the continuous image frames; for each of the continuous image frames, determine spatial distribution features based on the position of each of the target feature points in the continuous image frame; for each of the continuous image frames, determine comprehensive evaluation features based on the quantity features of each of the target feature points in the continuous image frame, the spatial distribution features of the target feature points, and the hue features of the continuous image frame; determine the starting frame of the continuous image frames that enters a steady state based on the changes in the comprehensive evaluation features of each of the continuous image frames in the time dimension, and use it as the response image frame; and send the response time corresponding to the response image frame to the delay calculation module. The latency calculation module determines the response latency of the target interface based on the response time corresponding to the response image frame and the trigger time.

[0060] In one or more embodiments of this application, a non-intrusive, integrated automated testing device can be configured to execute the aforementioned interface response latency testing method. This automated testing device is equipped with an interactive screen, an infrared simulation operation module, an EPG image acquisition module, an image analysis module, and a latency calculation module. Specifically, the infrared simulation operation module simulates the infrared radiation emitted by the EPG via a remote control, converting it into infrared signals according to the operation script corresponding to the latency test task, for remote control operation of the EPG. The EPG image acquisition module performs screenshot operations during the EPG latency test to acquire continuous image frames. The image analysis module calculates the comprehensive evaluation characteristics of each continuous image frame. The latency calculation module determines the response latency of the EPG.

[0061] Figure 4 This is a schematic diagram of the interactive screen of the automated testing device provided in the embodiments of this application.

[0062] Figure 5 This is a schematic diagram of the device interface of the automated testing apparatus provided in the embodiments of this application.

[0063] This application does not limit the specific configuration of the device interface, which can be set according to actual needs. For example, the wired network configuration can be "1000 / 100Mbps; RJ45 interface ≥1"; the video output configuration can be "HDMI 2.0 interface *1, supporting 4K@60fps output; HDMI IN interface *1, supporting 1080P@60fps input, realizing image data acquisition function"; the USB interface configuration can be "USB 3.0 HOST ≥2, using stacked A-type sockets; USB 2.0 HOST ≥1"; the infrared transmitter interface configuration can be "infrared transmitter interface (socket) *1 (external pull-out transmitter head)".

[0064] Figure 6 This is a schematic diagram of the device connections for the automated testing apparatus provided in an embodiment of this application. Figure 6 As shown, the HDMI port of this automated testing device connects to a set-top box (the carrier of the EPG) via an HDMI cable, and can also connect to a router via DHCP to communicate with the automated testing platform for cloud-based testing. That is, in one or more embodiments of this application, the automated testing device does not rely on the external network during side-side testing and can complete testing even when the device is offline, adapting to scenarios with network security restrictions or unavailable external networks. Testers can operate this integrated device independently, connecting it to the set-top box via an HDMI cable to issue latency test tasks. The test results of the latency test tasks can be manually synchronized to the cloud platform or automatically uploaded when the external network connection is restored, ensuring the timeliness and integrity of the data. In cloud-based testing, testers can issue latency test tasks to different test points through the automated testing platform to achieve remote command and monitoring. Test results can be automatically aggregated to the automated testing platform, allowing testers to obtain and analyze the response latency data of different test points.

[0065] In the latency test based on the automated testing device, the automated testing device uses the confirmation button of the latency test task triggered by the infrared simulation module as the trigger time, and the screenshot time of the response image frame statistically analyzed by the image analysis module as the response time. The time difference between the trigger time and the response time is the response latency of the latency test task. In determining the response time, the automated testing device acquires continuous image frames of the response process through the image acquisition module, and then extracts the quantity features, spatial distribution features, and tonal features of target feature points in each continuous image frame. Based on a preset target weight coefficient set, it comprehensively calculates the comprehensive evaluation features of each continuous image frame. For situations where there are minor changes or near-static states in the image during the response process, this application can dynamically adjust the proportional allocation parameters of the quantity features, spatial distribution features, and tonal features (i.e., the target weight coefficient set) through machine learning to strengthen the weight of features with changes in the continuous image frames and reduce the weight of other parts, ensuring accurate identification of the response image frame even with subtle changes in the image.

[0066] Figure 7 This is a schematic diagram illustrating the cloud-edge collaboration method for testing interface response latency, as provided in an embodiment of this application.

[0067] In this embodiment, the automated testing device adopts an independent, non-intrusive deployment scheme, enabling dynamic switching between cloud and edge dual-channel testing modes after deployment. For offline testing scenarios, users only need to select the latency indicator to be tested on the interactive screen and issue the corresponding latency test task, allowing the automated testing device to independently complete the entire automated testing process. For platform task testing scenarios, the automated testing device supports remote control via the cloud. Whether it is single-point testing of a single device or batch testing of a multi-device cluster, remote issuance and execution of latency test tasks can be achieved, adapting to diverse testing needs.

[0068] Figure 8 This is a schematic diagram illustrating the interface response latency test under a platform task testing scenario provided in this application embodiment. Figure 8 As shown, to improve the execution efficiency and resource utilization of latency test tasks in platform task testing scenarios, in one or more embodiments of this application, a priority scheduling algorithm based on resource utilization can be adopted to realize the autonomous decision-making and intelligent distribution of platform test tasks. Specifically, the automated testing platform receives and analyzes resource parameters such as CPU usage, memory occupancy, and network bandwidth reported by each automated testing device in real time. Then, by incorporating a priority scheduling algorithm, it provides an appropriate latency test task distribution strategy based on the remaining resource status of the automated testing devices. For example, devices with ample remaining resources will be prioritized for test tasks to ensure efficient execution, while devices with limited remaining resources will have their task execution sequence rationally arranged to ensure that the test tasks can be completed smoothly without affecting the normal operation of their current tasks.

[0069] Based on the above-described interface response delay testing method, this application also provides a specific embodiment of an interface response delay testing device.

[0070] like Figure 9 As shown, Figure 9 This is a schematic diagram of an interface response delay testing device provided in an embodiment of this application. The device 900 includes a response module 901, a trigger time module 902, a continuous image frame module 903, a feature point recognition module 904, a spatial distribution module 905, a comprehensive evaluation module 906, a response image frame module 907, and a response delay module 908.

[0071] The response module 901 is used to trigger a latency test task in response to a latency test command from the target interface. The trigger time module 902 is used to record the trigger time of the latency test task; The continuous image frame module 903 is used to acquire continuous image frames of the target interface during the response process after the trigger time. The feature point recognition module 904 is used to identify target feature points in each of the consecutive image frames; The spatial distribution module 905 is used to determine spatial distribution features for each of the continuous image frames based on the positions of the target feature points in the continuous image frames. The comprehensive evaluation module 906 is used to determine the comprehensive evaluation features of each of the continuous image frames based on the number of target feature points in each of the continuous image frames, the spatial distribution features of the target feature points, and the tonal features of the continuous image frames. The response image frame module 907 is used to determine, based on the changes in the comprehensive evaluation features of each of the consecutive image frames in the time dimension, the starting frame that enters a steady state in the consecutive image frames, and use it as the response image frame; The response delay module 908 is used to determine the response delay of the target interface based on the response time corresponding to the response image frame and the trigger time.

[0072] In one feasible implementation, the feature point recognition module is specifically used to construct the scale space of each of the consecutive image frames; for each of the consecutive image frames, based on the scale space of the consecutive image frames, identify candidate feature points in the consecutive image frames through scale space extremum detection; for each of the consecutive image frames, based on the position and scale parameters of each of the candidate feature points in the consecutive image frames, determine the contrast of each of the candidate feature points through three-dimensional quadratic function fitting; for each of the consecutive image frames, based on the contrast of each of the candidate feature points and a preset contrast threshold, select target feature points from each of the candidate feature points.

[0073] In one feasible implementation, the spatial distribution module is specifically used to determine the position mean and position standard deviation for each of the consecutive image frames based on the position of each of the target feature points in the consecutive image frames; and to determine the dispersion of the target feature points for each of the consecutive image frames based on the position mean and the position standard deviation, as the spatial distribution feature.

[0074] In one feasible implementation, the response image frame module is specifically used to sort the comprehensive evaluation features of each consecutive image frame according to the time sequence of the response process to obtain an image frame feature sequence; determine the rate of change of each comprehensive evaluation feature with respect to the previous comprehensive evaluation feature in the image frame feature sequence; take consecutive image frames whose rate of change is within a preset slope range as steady-state image frames; and take the starting frame in the time dimension of each steady-state image frame as the response image frame.

[0075] In one feasible implementation, the comprehensive evaluation module is specifically used to normalize the quantitative features, spatial distribution features, and hue features; for each consecutive image frame, the normalized quantitative features, spatial distribution features, and hue features are weighted and fused according to a preset target weight coefficient group to obtain the comprehensive evaluation features of the consecutive image frame.

[0076] In one feasible implementation, the comprehensive evaluation module can also be used to construct an initial weight coefficient set; acquire multiple time-delay training sets, each of which includes multiple consecutive image frame samples and label response image frames; according to the initial weight coefficient set, perform weighted fusion on the quantity features, spatial distribution features, and tone features of each consecutive image frame sample to obtain the comprehensive evaluation features of each consecutive image frame sample; based on the changes of each comprehensive evaluation feature in the time dimension, select the predicted response image frames of each time-delay training set; with the goal of minimizing the temporal difference between the predicted response image frames and the label response image frames, iteratively optimize the initial weight coefficient set based on the multiple time-delay training sets until a preset termination condition is reached to obtain the target weight coefficient set.

[0077] In one feasible implementation, the comprehensive evaluation module can also be used to determine the first time of the actual response of the latency test task in each of the consecutive image frames; determine the response deviation between the first time and the response time; adjust the target weight coefficient group if the response deviation is greater than a preset deviation threshold; and execute the next latency test task based on the adjusted target weight coefficient group.

[0078] Figure 10 A schematic diagram of the hardware structure of an interface response latency testing device provided in an embodiment of this application is shown.

[0079] An interface response latency testing device may include a processor 1001 and a memory 1002 storing computer program instructions.

[0080] Specifically, the processor 1001 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0081] Memory 1002 may include mass storage for data or instructions. For example, and not limitingly, memory 1002 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one embodiment, memory 1002 may include removable or non-removable (or fixed) media, or memory 1002 may be non-volatile solid-state memory. Memory 1002 may be internal or external to the integrated gateway disaster recovery device.

[0082] In one instance, memory 1002 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0083] Memory 1002 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the interface response latency test method according to one aspect of this application.

[0084] The processor 1001 reads and executes computer program instructions stored in the memory 1002 to achieve... Figure 1 The interface response latency testing method in the illustrated embodiment.

[0085] In one example, an interface response latency testing device may further include a communication interface 503 and a bus 504. Wherein, as Figure 10 As shown, the processor 1001, memory 1002, and communication interface 1003 are connected through bus 1004 and complete communication with each other.

[0086] The communication interface 1003 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0087] Bus 1004 includes hardware, software, or both, that couples components of an interface response latency test device together. For example, and not as a limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 504 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0088] Furthermore, in conjunction with the interface response latency testing method in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the interface response latency testing methods in the above embodiments.

[0089] This application also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the interface response latency testing methods described in the above embodiments.

[0090] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0091] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0092] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0093] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0094] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for testing interface response latency, characterized in that, include: In response to the latency test command on the target interface, a latency test task is triggered; Record the trigger time of the latency test task; After the trigger time, consecutive image frames of the target interface are acquired during the response process; Identify target feature points in each of the consecutive image frames; For each of the consecutive image frames, spatial distribution features are determined based on the positions of the target feature points in the consecutive image frames; For each of the consecutive image frames, the comprehensive evaluation features of the consecutive image frames are determined based on the number of target feature points in each of the consecutive image frames, the spatial distribution features of the target feature points, and the tonal features of the consecutive image frames. Based on the changes in the comprehensive evaluation features of each of the consecutive image frames in the time dimension, the starting frame that enters a steady state in the consecutive image frames is determined as the response image frame; The response latency of the target interface is determined based on the response time corresponding to the response image frame and the trigger time.

2. The method according to claim 1, characterized in that, Identifying target feature points in each of the consecutive image frames specifically includes: Construct the scale space for each of the consecutive image frames; For each of the consecutive image frames, candidate feature points in the consecutive image frames are identified by scale space extremum detection based on the scale space of the consecutive image frames. For each of the consecutive image frames, the contrast of each candidate feature point is determined by fitting a three-dimensional quadratic function based on the position and scale parameters of each candidate feature point in the consecutive image frame. For each of the consecutive image frames, target feature points are selected from the candidate feature points based on the contrast of each candidate feature point and a preset contrast threshold.

3. The method according to claim 1, characterized in that, For each of the consecutive image frames, spatial distribution features are determined based on the positions of the target feature points in the consecutive image frames, specifically including: For each of the consecutive image frames, the mean position and standard deviation position are determined based on the position of each target feature point in the consecutive image frame; For each of the consecutive image frames, the discreteness of the target feature points is determined based on the position mean and the position standard deviation, and is used as the spatial distribution feature.

4. The method according to claim 1, characterized in that, Based on the changes in the comprehensive evaluation features of each of the consecutive image frames over time, the starting frame that enters a steady state in the consecutive image frames is determined as the response image frame, specifically including: According to the time sequence of the response process, the comprehensive evaluation features of each consecutive image frame are sorted to obtain the image frame feature sequence; Determine the rate of change of each comprehensive evaluation feature relative to the previous comprehensive evaluation feature in the image frame feature sequence; Continuous image frames whose rate of change is within a preset slope range are taken as steady-state image frames; The starting frame in the time dimension of each of the steady-state image frames is taken as the response image frame.

5. The method according to claim 1, characterized in that, For each of the consecutive image frames, based on the quantity characteristics of each target feature point in the consecutive image frame, the spatial distribution characteristics of the target feature points, and the tonal characteristics of the consecutive image frame, a comprehensive evaluation feature for the consecutive image frame is determined, specifically including: The quantitative features, the spatial distribution features, and the hue features are normalized. For each of the consecutive image frames, the normalized quantitative features, spatial distribution features, and hue features are weighted and fused according to a preset target weight coefficient group to obtain the comprehensive evaluation features of the consecutive image frames.

6. The method according to claim 5, characterized in that, Before performing weighted fusion of the normalized quantitative features, spatial distribution features, and hue features according to a preset target weight coefficient group for each consecutive image frame to obtain the comprehensive evaluation features of the consecutive image frame, the method further includes: Construct the initial set of weight coefficients; Acquire multiple time-delay training sets, each of which includes multiple consecutive image frame samples and label response image frames; According to the initial weight coefficient group, the quantitative features, spatial distribution features and tone features of each of the consecutive image frame samples are weighted and fused to obtain the comprehensive evaluation features of each of the consecutive image frame samples; Based on the changes of each comprehensive evaluation feature over time, the predicted response image frames for each of the time-delay training sets are selected. With the goal of minimizing the temporal difference between the predicted response image frame and the labeled response image frame, the initial weight coefficient set is iteratively optimized based on the multiple time-delay training sets until a preset termination condition is reached, thus obtaining the target weight coefficient set.

7. The method according to claim 5, characterized in that, After determining the response latency of the target interface based on the response time of the response image frame and the trigger time, the method further includes: In each of the consecutive image frames, the first time of the actual response of the latency test task is determined; Determine the response deviation between the first time and the response time; If the response deviation is greater than a preset deviation threshold, the target weight coefficient group is adjusted. Based on the adjusted target weight coefficient set, the next latency test task is executed.

8. A system for testing interface response latency, characterized in that, The system includes a simulation triggering module, an image acquisition module, an image analysis module, and a time delay calculation module, including: The simulation triggering module is used to simulate user operations to trigger a latency test task in response to a latency test command from the target interface; record the trigger time of the latency test task; and send the trigger time to the latency calculation module. The image acquisition module is used to acquire continuous image frames of the target interface during the response process after the trigger time; send each of the continuous image frames to the image analysis module; the image analysis module is used to identify target feature points in each of the continuous image frames; for each of the continuous image frames, determine spatial distribution features based on the position of each of the target feature points in the continuous image frame; for each of the continuous image frames, determine comprehensive evaluation features based on the quantity features of each of the target feature points in the continuous image frame, the spatial distribution features of the target feature points, and the hue features of the continuous image frame; determine the starting frame of the continuous image frames that enters a steady state based on the changes in the comprehensive evaluation features of each of the continuous image frames in the time dimension, and use it as the response image frame; and send the response time corresponding to the response image frame to the delay calculation module. The latency calculation module determines the response latency of the target interface based on the response time corresponding to the response image frame and the trigger time.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the interface response latency testing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the interface response latency testing method as described in any one of claims 1-7.

Images