End-to-end delay detection method and apparatus

Abstract

This application provides an end-to-end delay detection method and apparatus. The method includes: acquiring externally recorded video; if the current state is waiting for a trigger frame and the difference between the brightness statistics of the display area in the current frame video image and the ambient reference brightness value is greater than a preset brightness jump threshold, then the current frame video image is determined to be a trigger frame; collecting N frames of video images after the trigger frame, averaging the brightness of each pixel position in the response area of ​​the N frames of video images to generate an average reference frame, skipping K frames after the trigger frame, and starting from the K+1th frame, calculating the image brightness difference between the response area in the current frame and the average reference frame frame by frame; if the image brightness difference exceeds the response judgment threshold, then the current frame is determined to be a screen response frame; calculating the end-to-end response delay based on the frame number of the trigger frame and the frame number of the screen response frame, which can provide a fully automatic, high-precision, anti-interference, and simple end-to-end delay detection method.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to an end-to-end delay detection method and apparatus. Background Technology

[0002] In scenarios such as game performance testing and mobile phone touch response testing, it is necessary to analyze the screen's response time to user operations in externally recorded videos, i.e., end-to-end latency. This metric directly reflects the device's overall response performance from receiving user input to the screen displaying changes, and is a key parameter for measuring user experience.

[0003] Currently, the main methods for detecting end-to-end latency are as follows: Method 1: Manual Frame-by-Frame Annotation. This method uses a high-speed camera to film the manual operation of the equipment, acquiring multiple frames at a preset frame rate. Testers then visually inspect each frame to determine the image frame corresponding to the click moment and the image frame corresponding to the screen response moment. Finally, the response latency is calculated by converting the frame differences. The disadvantages of this method are low automation, low efficiency, and poor consistency in annotation results among different testers.

[0004] Method 2: Audio-assisted detection. This method records the click sound and screen image, using sudden changes in the audio waveform as trigger points. However, due to buffer delays in the audio acquisition system (typically 50ms to 200ms, and this delay is not fixed), the detection accuracy is insufficient.

[0005] Method 3: Frame Difference Method. This method directly calculates the pixel difference between two adjacent frames in an externally recorded video. When the difference exceeds a threshold, it is considered a screen response. However, externally recorded videos contain inherent interference such as reflections, moiré patterns, and video compression noise. The ordinary frame difference method is prone to misinterpreting these environmental disturbances as changes in screen content, resulting in a high false detection rate.

[0006] Method 4: Image comparison method based on structural similarity (SSIM) algorithm. The SSIM algorithm is sensitive to structural changes in images, but it is also sensitive to interference such as noise and reflections. It is prone to misjudgment under complex shooting conditions, resulting in large measurement errors.

[0007] In addition, the prior art CN113141433A discloses a method for testing screen sensitivity, which locates the display lamp area through template matching, determines the LED lighting moment as the click frame by the average brightness of the HSV (Hue, Saturation, Value) color space, selects a stable frame from the video frame as the reference image, and uses the SSIM algorithm and HSV color model to determine the dual-condition response frame. Although this method achieves partial automation, it still has the following shortcomings: (1) It compares the average brightness of the display lamp area with the brightness threshold to determine the trigger frame, which has weak noise resistance and is prone to misjudgment; (2) It uses a single frame image as the reference, and the reference frame itself carries random noise, which leads to a decrease in the signal-to-noise ratio of the subsequent SSIM comparison; (3) It does not consider the interference of motion blur at the moment of clicking on the response determination.

[0008] In summary, existing technologies suffer from problems such as reliance on manual annotation, low accuracy, and insufficient noise resistance. There is an urgent need for a fully automated, high-precision, and interference-resistant end-to-end delay detection method. Summary of the Invention

[0009] To address the technical problems existing in the prior art, embodiments of this application provide an end-to-end delay detection method and apparatus.

[0010] In a first aspect, embodiments of this application provide an end-to-end delay detection method, including: Acquire external video, wherein the external video includes a series of multiple frames of video images obtained by capturing the screen to be tested through an image acquisition device. The video images simultaneously include an indicator light area and a response area of ​​the screen to be tested. A control device is connected to the indicator light and is used to control the device to which the screen to be tested belongs. The indicator light lights up synchronously when the control device is controlled. For the current frame video image in the externally recorded video, if it is determined that the current state is waiting for a trigger frame and the difference between the brightness statistics value of the display lamp area in the current frame video image and the ambient reference brightness value is greater than a preset brightness jump threshold, then the current frame video image is determined to be a trigger frame. The ambient reference brightness value is determined based on the brightness statistics value of the display lamp area in at least one previous video image. The brightness statistics value is calculated based on the pixels in the display lamp area whose brightness value is greater than a preset high brightness threshold. After determining the trigger frame, N video images following the trigger frame are collected. The brightness of each pixel position in the response area of ​​the N video images is averaged to generate an average reference frame. K frames following the trigger frame are skipped. Starting from the (K+1)th frame, the image brightness difference between the response area in the current frame and the average reference frame is calculated frame by frame. If it is determined that the image brightness difference exceeds a preset response determination threshold, the current frame is determined to be a screen response frame. Here, N is an integer greater than 1, K is an integer greater than 0, and the screen response frame is located after the N video images. The frame difference is calculated based on the frame number of the trigger frame and the frame number of the screen response frame, and the end-to-end response delay is obtained based on the frame rate of the externally recorded video.

[0011] Secondly, embodiments of this application also provide an end-to-end delay detection device, comprising: An acquisition unit is used to acquire externally recorded video, wherein the externally recorded video includes a series of video images obtained by capturing the screen to be tested through an image acquisition device. The video images simultaneously include an indicator light area and a response area of ​​the screen to be tested. A control device is connected to the indicator light and is used to control the device to which the screen to be tested belongs. The indicator light lights up synchronously when the control device is controlled. The trigger frame determination unit is used to determine the current frame video image as a trigger frame if it is determined that the current frame is in a state of waiting for a trigger frame and the difference between the brightness statistics value of the display lamp area in the current frame video image and the ambient reference brightness value is greater than a preset brightness jump threshold. The ambient reference brightness value is determined based on the brightness statistics value of the display lamp area in at least one previous frame video image. The brightness statistics value is calculated based on the pixels in the display lamp area whose brightness value is greater than a preset high brightness threshold. The response frame determination unit is used to collect N video images after the trigger frame is determined, average the brightness of the response area in the N video images pixel by pixel to generate an average reference frame, skip the K frames after the trigger frame, and calculate the image brightness difference between the response area in the current frame and the average reference frame starting from the (K+1)th frame. If it is determined that the image brightness difference exceeds a preset response determination threshold, the current frame is determined to be a screen response frame, where N is an integer greater than 1, K is an integer greater than 0, and the screen response frame is located after the N video images. The delay calculation unit is used to calculate the frame difference based on the frame number of the trigger frame and the frame number of the screen response frame, and to convert the end-to-end response delay based on the frame rate of the externally recorded video.

[0012] The end-to-end delay detection method and apparatus provided in this application determine the trigger frame by the difference between the brightness statistics value and the ambient reference brightness value, replacing the inaccurate audio detection and inefficient manual annotation in the background technology. After the trigger frame is determined, the response area images of N frames after the trigger are collected and the brightness is averaged pixel by pixel to generate an average reference frame. This solves the problem of high false detection rate caused by interference such as reflection and moiré patterns in the ordinary frame difference method and the defect of SSIM algorithm being sensitive to noise. At the same time, compared with the existing patent CN113141433A: First, this solution dynamically determines the ambient reference brightness value based on the brightness statistics value of the display lamp area in at least one previous video image, and compares the brightness statistics value of the display lamp area in the current video image with the ambient reference brightness value to determine the trigger frame. On the one hand, since the brightness statistics value can directly exclude the reflection and noise of the intermediate brightness, the noise resistance is far superior. This solution addresses several key issues. First, it avoids using a single frame image as a reference. It collects response regions from N frames after triggering and averages the brightness of each pixel to generate an average reference frame. This multi-frame averaging smooths single-frame noise, significantly improving the signal-to-noise ratio for subsequent brightness difference calculations. Second, it directly compares the single-dimensional brightness difference with a threshold to determine the response frame. Compared to the existing patent's indirect SSIM plus HSV dual-dimensional judgment method, this results in a shorter judgment chain, simpler implementation, and avoids the composite errors inherent in the brightness, contrast, and structure dimensions of SSIM calculations caused by single-frame noise. Third, it considers the interference of motion blur at the moment of click on response judgment, actively skipping K frames after triggering to avoid the effects of device jitter and motion blur. In summary, this solution solves the problems of existing technologies relying on manual annotation or semi-automation and insufficient noise resistance, providing a fully automatic, high-precision, highly interference-resistant, and simple-to-implement end-to-end delay detection method. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating an embodiment of an end-to-end delay detection method provided in this application. Figure 2 This is a schematic diagram of an embodiment of an end-to-end delay detection device provided in this application. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0015] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0016] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0017] Reference Figure 1 The diagram shown is a flowchart illustrating an end-to-end delay detection method provided in an embodiment of this application. The method includes: S10. Acquire external video, wherein the external video includes a series of video images obtained by capturing the screen to be tested through an image acquisition device. The video images simultaneously include an indicator light area and a response area of ​​the screen to be tested. A control device is connected to the indicator light and is used to control the device to which the screen to be tested belongs. The indicator light lights up synchronously when the control device is controlled. In this embodiment, the indicator light can be placed on the screen to be tested (preferably without obstructing the response area) or placed next to the screen to be tested. When the image acquisition device captures a picture, the field of view must simultaneously cover the response areas of both the indicator light and the screen to be tested. Thus, the indicator light area and the response area can be divided from the captured video image: the indicator light area is the image area at a preset position in the video image, which is predetermined based on the placement of the indicator light. For example, if the video image is divided into a 3x5 grid, and the indicator light is located in the 1st row and 5th column grid, then the indicator light area can be the image area where the 1st row and 5th column grid is located; the response area is a preset detection area on the display screen of the screen to be tested, used to monitor whether the image of the screen to be tested changes when the control device is operated. For example, the response area can be the button area, menu area, or central area of ​​the screen to be tested.

[0018] The control device is connected to an indicator light, which illuminates each time the control device interacts with the device whose screen is being monitored. The control device can include a mouse, gamepad, or keyboard, while the device whose screen is being monitored can include a mobile phone, computer, smart wearable devices (such as watches or fitness trackers), in-vehicle systems, or a television. To reduce computational load, the video image from the externally recorded video can be scaled before subsequent calculations.

[0019] S11. For the current frame video image in the externally recorded video, if it is determined that the current state is waiting for a trigger frame and the difference between the brightness statistics value of the display lamp area in the current frame video image and the ambient reference brightness value is greater than a preset brightness jump threshold, then the current frame video image is determined to be a trigger frame. The ambient reference brightness value is determined based on the brightness statistics value of the display lamp area in at least one previous frame video image. The brightness statistics value is calculated based on the pixels in the display lamp area whose brightness value is greater than a preset high brightness threshold. In this embodiment, trigger frames and screen response frames always alternate in the order of trigger frames first, followed by screen response frames. Being in a state of waiting for a trigger frame means that no trigger frame has been detected since the detection began, or the latest detected frame during the detection process is a screen response frame. The brightness statistics can be the number of pixels in the display area whose brightness value (the brightness value of a pixel can be the grayscale value of a pixel after converting the image to grayscale) is greater than a preset high-brightness threshold (the high-brightness threshold can be set as needed, for example, to 220), the sum of the brightness values ​​of pixels in the display area whose brightness value is greater than the preset high-brightness threshold, the average brightness value of pixels in the display area whose brightness value is greater than the preset high-brightness threshold, etc. The environmental reference brightness value can be the median, minimum, or average of the brightness statistics values ​​of the display area in at least one video image preceding the current frame. At least one video image can be a set of multiple video images preceding and closest to the current frame; the number of these multiple video images can be set as needed, for example, to 10. If the number of video images preceding the current frame is less than the number of at least one video frame, then all video images preceding the current frame constitute at least one video frame.

[0020] It should be noted that the existing patent CN113141433A calculates the average HSV brightness of the display area when detecting the trigger frame and compares the average brightness with a preset brightness threshold. However, the average brightness will combine dark and bright pixels in the display area. When there is intermediate brightness interference such as reflection and compressed noise, the mean value will be inflated, leading to misjudgment. In this embodiment, the ambient reference brightness value is dynamically determined by the brightness statistics of the display area in at least one frame before the current frame. The difference between the brightness statistics of the display area in the current frame and this dynamic reference is used to determine the trigger frame. The brightness statistics value can directly exclude reflection and noise in intermediate brightness, so the noise resistance is far better than the mean value. In addition, the existing patent uses a fixed preset threshold to determine whether it is lit. When the ambient light becomes brighter, the display is not lit, but the average HSV brightness may have exceeded the fixed threshold, resulting in misjudgment. In this embodiment, the ambient reference brightness value dynamically determined in at least one previous frame is used as the comparison benchmark. When the ambient light drifts slowly, the benchmark automatically follows. Only relative jumps are detected instead of absolute brightness, which fundamentally eliminates misjudgment caused by ambient light drift.

[0021] S12. After determining the trigger frame, collect N video images after the trigger frame, average the brightness of each pixel position of the response area in the N video images to generate an average reference frame, skip the K frames after the trigger frame, and start from the (K+1)th frame to calculate the image brightness difference between the response area in the current frame and the average reference frame frame by frame. If it is determined that the image brightness difference exceeds a preset response determination threshold, then the current frame is determined to be a screen response frame, where N is an integer greater than 1, K is an integer greater than 0, and the screen response frame is located after the N video images. In this embodiment, the N video frames can be any N consecutive video frames following the trigger frame. N and K can be set as needed, for example, to 10 and 3 respectively. When the number of video frames between the trigger frame and the current frame is less than N, the N video frames after the trigger frame can be at least a portion of the video frames between the trigger frame and the current frame. The response determination threshold can be set as needed, for example, to 3.8.

[0022] It should be noted that the existing patent CN113141433A uses a single-frame image as a benchmark and combines the SSIM algorithm and HSV color model to determine the response frame under two conditions. This embodiment, however, makes three improvements: First, it collects the response regions of N frames after triggering and averages the brightness of each pixel to generate an average benchmark frame. Compared to a single-frame image, the noise variance is significantly reduced after averaging N frames, thus significantly improving the signal-to-noise ratio during subsequent frame-by-frame comparisons. This makes image changes easier to identify and less likely to be obscured by noise in the presence of a clean benchmark. Second, it considers the physical jitter and motion blur of the device during operation and actively skips K frames after triggering to determine the response frame, preventing interference from physical disturbances. Third, it directly compares the single-dimensional brightness difference instead of using the SSIM plus HSV dual-condition indirect determination, resulting in a shorter link and avoiding the composite errors caused by single-frame noise in the inherent brightness, contrast, and structure dimensions of SSIM calculations. These three improvements work together to significantly improve the accuracy and robustness of response frame detection compared to existing patents.

[0023] S13. Calculate the frame difference based on the frame number of the trigger frame and the frame number of the screen response frame, and calculate the end-to-end response delay based on the frame rate of the externally recorded video.

[0024] Steps S10-S12 determine at least one pair of trigger frames and screen response frames. For each pair, the frame difference (the difference between the frame number of the screen response frame and the frame number of the trigger frame) is calculated based on the frame number of the trigger frame and the frame number of the screen response frame. The end-to-end response delay is then calculated based on the frame difference and the frame rate of the externally recorded video. The end-to-end response delay equals the frame difference × 1000 / frame rate (in milliseconds). This yields at least one end-to-end response delay. The final end-to-end response delay can be determined based on this at least one end-to-end response delay, for example, by averaging or medianing the at least one end-to-end response delay.

[0025] The end-to-end delay detection method provided in this application determines the trigger frame by comparing the difference between the brightness statistics and the ambient reference brightness, replacing the inaccurate audio detection and inefficient manual annotation in the prior art. After the trigger frame is determined, the response area images of N frames after the trigger are collected, and the brightness is averaged pixel by pixel to generate an average reference frame. This solves the problem of high false detection rate caused by interference such as reflection and moiré patterns in ordinary frame difference methods and the defect of SSIM algorithm being sensitive to noise. At the same time, compared with the existing patent CN113141433A: First, this solution dynamically determines the ambient reference brightness value based on the brightness statistics of the display lamp area in at least one previous video image, and compares the brightness statistics of the display lamp area in the current video image with the ambient reference brightness value to determine the trigger frame. On the one hand, since the brightness statistics can directly exclude the reflection and noise of intermediate brightness, the noise resistance is far superior to that of the previous method. The existing patent uses the average value; on the other hand, since the ambient light reference brightness value can automatically follow the slow drift of ambient light, it can fundamentally eliminate misjudgments caused by ambient light drift; secondly, this solution does not use a single frame image as a reference, but collects the response area of ​​N frames after triggering and averages the brightness of each pixel to generate an average reference frame. It uses multi-frame averaging to smooth single-frame noise, which significantly improves the signal-to-noise ratio of subsequent brightness difference calculation. Moreover, this solution directly compares the single-dimensional brightness difference with the threshold to determine the response frame. Compared with the existing patent's SSIM plus HSV dual-dimensional indirect judgment method, the judgment link is shorter and the implementation is simpler. It can also avoid the composite error caused by single-frame noise in the three dimensions of brightness, contrast, and structure inherent in SSIM calculation; thirdly, this solution considers the interference of motion blur at the moment of click on the response judgment. It actively skips K frames after triggering to avoid the influence of device jitter and motion blur. In summary, this solution can solve the problems of existing technologies that rely on manual annotation or semi-automation and have insufficient noise resistance, and provides a fully automatic, high-precision, anti-interference and simple end-to-end delay detection method.

[0026] Based on the aforementioned method embodiments, the step of calculating the image brightness difference between the response region in the current frame and the average reference frame frame by frame may include: The brightness difference between the response region in the current frame and the average reference frame is calculated frame by frame, and the brightness difference is processed by a preset difference aggregation function to obtain the image brightness difference degree.

[0027] Based on the aforementioned method embodiments, the brightness value difference can be the absolute value of the frame difference between the response region in the current frame and the average reference frame. The absolute value of the frame difference can be the absolute value of the difference between the pixel brightness values ​​at each pixel position in the response region and the average reference frame. The difference aggregation function can be an averaging function. The image brightness difference can be the average, median, root mean square value, or weighted average of the absolute values ​​of the pixel brightness values ​​at each pixel position obtained by subtracting the response region from the average reference frame in the current frame.

[0028] Reference Figure 2 The diagram shown is a structural schematic of an end-to-end delay detection device provided in an embodiment of this application. The device includes: The acquisition unit 20 is used to acquire externally recorded video, wherein the externally recorded video includes a series of video images obtained by capturing the screen to be tested through an image acquisition device. The video images simultaneously include an indicator light area and a response area of ​​the screen to be tested. The control device is connected to the indicator light and is used to control the device to which the screen to be tested belongs. The indicator light lights up synchronously when the control device is controlled. The trigger frame determination unit 21 is used to determine the current frame video image as a trigger frame if it is determined that the current frame is in a state of waiting for a trigger frame and the difference between the brightness statistics value of the display lamp area in the current frame video image and the ambient reference brightness value is greater than a preset brightness jump threshold. The ambient reference brightness value is determined based on the brightness statistics value of the display lamp area in at least one previous frame video image. The brightness statistics value is calculated based on the pixels in the display lamp area whose brightness value is greater than a preset high brightness threshold. The response frame determination unit 22 is used to collect N video images after the trigger frame is determined, average the brightness of the response area in the N video images pixel by pixel to generate an average reference frame, skip the K frames after the trigger frame, and calculate the image brightness difference between the response area in the current frame and the average reference frame starting from the (K+1)th frame. If it is determined that the image brightness difference exceeds a preset response determination threshold, the current frame is determined to be a screen response frame, where N is an integer greater than 1, K is an integer greater than 0, and the screen response frame is located after the N video images. The delay calculation unit 23 is used to calculate the frame difference based on the frame number of the trigger frame and the frame number of the screen response frame, and to calculate the end-to-end response delay based on the frame rate of the externally recorded video.

[0029] The end-to-end delay detection device provided in this application determines the trigger frame by comparing the difference between the brightness statistics value and the ambient reference brightness value, replacing the inaccurate audio detection and inefficient manual annotation in the background technology. After the trigger frame is determined, the response area images of N frames after the trigger are collected, and the brightness is averaged pixel by pixel to generate an average reference frame. This solves the problem of high false detection rate caused by interference such as reflection and moiré patterns in the ordinary frame difference method and the defect of SSIM algorithm being sensitive to noise. At the same time, compared with the existing patent CN113141433A: First, this solution dynamically determines the ambient reference brightness value based on the brightness statistics value of the display lamp area in at least one previous video image, and compares the brightness statistics value of the display lamp area in the current video image with the ambient reference brightness value to determine the trigger frame. On the one hand, since the brightness statistics value can directly exclude the reflection and noise of the intermediate brightness, the noise resistance is far superior to that of the previous method. The existing patent uses the average value; on the other hand, since the ambient light reference brightness value can automatically follow the slow drift of ambient light, it can fundamentally eliminate misjudgments caused by ambient light drift; secondly, this solution does not use a single frame image as a reference, but collects the response area of ​​N frames after triggering and averages the brightness of each pixel to generate an average reference frame. It uses multi-frame averaging to smooth single-frame noise, which significantly improves the signal-to-noise ratio of subsequent brightness difference calculation. Moreover, this solution directly compares the single-dimensional brightness difference with the threshold to determine the response frame. Compared with the existing patent's SSIM plus HSV dual-dimensional indirect judgment method, the judgment link is shorter and the implementation is simpler. It can also avoid the composite error caused by single-frame noise in the three dimensions of brightness, contrast, and structure inherent in SSIM calculation; thirdly, this solution considers the interference of motion blur at the moment of click on the response judgment. It actively skips K frames after triggering to avoid the influence of device jitter and motion blur. In summary, this solution can solve the problems of existing technologies that rely on manual annotation or semi-automation and have insufficient noise resistance, and provides a fully automatic, high-precision, anti-interference and simple end-to-end delay detection method.

[0030] The end-to-end delay detection device provided in this application embodiment is implemented in the same way as the end-to-end delay detection method provided in this application embodiment, and the effect it achieves is also the same as the end-to-end delay detection method provided in this application embodiment, so it will not be described again here.

[0031] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An end-to-end delay detection method, characterized in that, include: Acquire external video, wherein the external video includes a series of multiple frames of video images obtained by capturing the screen to be tested through an image acquisition device. The video images simultaneously include an indicator light area and a response area of ​​the screen to be tested. A control device is connected to the indicator light and is used to control the device to which the screen to be tested belongs. The indicator light lights up synchronously when the control device is controlled. For the current frame video image in the externally recorded video, if it is determined that the current state is waiting for a trigger frame and the difference between the brightness statistics value of the display lamp area in the current frame video image and the ambient reference brightness value is greater than a preset brightness jump threshold, then the current frame video image is determined to be a trigger frame. The ambient reference brightness value is determined based on the brightness statistics value of the display lamp area in at least one previous video image. The brightness statistics value is calculated based on the pixels in the display lamp area whose brightness value is greater than a preset high brightness threshold. After determining the trigger frame, N video images following the trigger frame are collected. The brightness of each pixel position in the response area of ​​the N video images is averaged to generate an average reference frame. K frames following the trigger frame are skipped. Starting from the (K+1)th frame, the image brightness difference between the response area in the current frame and the average reference frame is calculated frame by frame. If it is determined that the image brightness difference exceeds a preset response determination threshold, the current frame is determined to be a screen response frame. Here, N is an integer greater than 1, K is an integer greater than 0, and the screen response frame is located after the N video images. The frame difference is calculated based on the frame number of the trigger frame and the frame number of the screen response frame, and the end-to-end response delay is obtained based on the frame rate of the externally recorded video.

2. The method as described in claim 1, characterized in that, The brightness statistics value is the number of pixels in the display lamp area whose brightness value is greater than a preset high brightness threshold, and the ambient reference brightness value is the minimum value of the brightness statistics value of the display lamp area in at least one video image before the current frame video image.

3. The method as described in claim 2, characterized in that, The at least one video frame is a set of video frames that are preceding the current video frame and are closest to the current video frame.

4. The method according to any one of claims 1 to 3, characterized in that, The step of calculating the image brightness difference between the response region in the current frame and the average reference frame frame frame by frame includes: The brightness difference between the response region in the current frame and the average reference frame is calculated frame by frame, and the brightness difference is processed by a preset difference aggregation function to obtain the image brightness difference degree.

5. The method as described in claim 4, characterized in that, The brightness difference is the absolute value of the frame difference between the response region in the current frame and the average reference frame. The absolute value of the frame difference is the absolute value of the difference between the pixel brightness values ​​at each pixel position in the response region and the average reference frame. The difference aggregation function is an averaging function. The image brightness difference is the average value of the absolute values ​​of the pixel brightness values ​​at each pixel position obtained by subtracting the response region from the average reference frame in the current frame.

6. The method as described in claim 1, characterized in that, N is 10, K is 3.

7. An end-to-end delay detection device, characterized in that, include: An acquisition unit is used to acquire externally recorded video, wherein the externally recorded video includes a series of video images obtained by capturing the screen to be tested through an image acquisition device. The video images simultaneously include an indicator light area and a response area of ​​the screen to be tested. A control device is connected to the indicator light and is used to control the device to which the screen to be tested belongs. The indicator light lights up synchronously when the control device is controlled. The trigger frame determination unit is used to determine the current frame video image as a trigger frame if it is determined that the current frame is in a state of waiting for a trigger frame and the difference between the brightness statistics value of the display lamp area in the current frame video image and the ambient reference brightness value is greater than a preset brightness jump threshold. The ambient reference brightness value is determined based on the brightness statistics value of the display lamp area in at least one previous frame video image. The brightness statistics value is calculated based on the pixels in the display lamp area whose brightness value is greater than a preset high brightness threshold. The response frame determination unit is used to collect N video images after the trigger frame is determined, average the brightness of the response area in the N video images pixel by pixel to generate an average reference frame, skip the K frames after the trigger frame, and calculate the image brightness difference between the response area in the current frame and the average reference frame starting from the (K+1)th frame. If it is determined that the image brightness difference exceeds a preset response determination threshold, the current frame is determined to be a screen response frame, where N is an integer greater than 1, K is an integer greater than 0, and the screen response frame is located after the N video images. The delay calculation unit is used to calculate the frame difference based on the frame number of the trigger frame and the frame number of the screen response frame, and to convert the end-to-end response delay based on the frame rate of the externally recorded video.

Images