Video signal input time delay determination method and device, electronic equipment and storage medium

Abstract

The invention provides a video signal input time delay determination method and device, electronic equipment and a storage medium. The method comprises the following steps: inserting a target pixel point at a specified position of a video frame to obtain a target video frame; and recording a sending timestamp of the target video frame, and sending the sending timestamp and the target video frame to a receiving end, so that the receiving end records a receiving timestamp when detecting a target pixel point in the target video frame, and determines a video signal input time delay according to the sending timestamp and the receiving timestamp. According to the invention, the measurement precision of the video signal input time delay can be improved on the basis of not increasing the complexity and the measurement cost of the system.

Description

Technical Field

[0001] The present application relates to the technical field of latency detection, and particularly to a method, apparatus, electronic device, and storage medium for determining the input latency of a video signal. Background Art

[0002] In a video transmission system, accurately measuring the input latency of a video signal from the sending end to the receiving end is crucial for ensuring a high-quality user experience. Whether it is a real-time video conference, an online game, or a high-definition live broadcast, any delay will affect the user's interactive experience and viewing effect.

[0003] To optimize the performance in these applications, accurately measuring and reducing latency is the key. Traditional video transmission systems usually rely on the timestamps of network layer protocols (such as TCP (Transmission Control Protocol) / IP (Internet Protocol), etc.) or frame synchronization-based methods to estimate latency. However, with the improvement of video resolution and frame rate, as well as application scenarios with increasingly high requirements for low latency (such as virtual reality, remote surgery, etc.), traditional methods gradually show their limitations. Some high-precision latency measurement methods may require dedicated hardware support, increasing the complexity and cost of the system. Therefore, it is necessary to design a simple and effective method for measuring the input latency of a video signal. Summary of the Invention

[0004] In view of the above problems, embodiments of the present application are proposed to provide a method, apparatus, electronic device, and storage medium for determining the input latency of a video signal that can overcome or at least partially solve the above problems.

[0005] In a first aspect, an embodiment of the present application provides a method for determining the input latency of a video signal, which is applied to a sending end, and the method includes:

[0006] Insert a target pixel point at a specified position in the video frame to obtain a target video frame;

[0007] Record the sending timestamp of the target video frame, and send the sending timestamp and the target video frame to the receiving end, so that the receiving end records the receiving timestamp when detecting the target pixel point in the target video frame, and determines the input latency of the video signal according to the sending timestamp and the receiving timestamp.

[0008] Optionally, the inserting a target pixel point at a specified position in the video frame to obtain a target video frame includes:

[0009] Insert the target pixel point at a specified position in the video frame to obtain the target video frame; or

[0010] Insert the target pixel points at multiple specified positions of the video frame to obtain the target video frame.

[0011] Optionally, recording the transmission timestamp of the target video frame and sending the transmission timestamp and the target video frame to a receiving end includes:

[0012] Encode the target video frame to obtain a marked video stream;

[0013] Record the transmission timestamp of the marked video stream, add the transmission timestamp to the metadata field of the marked video stream, and send the marked video stream to the receiving end, so that the receiving end decodes the marked video stream to obtain the target video frame, detects the target pixel points in the target video frame, and records the reception timestamp, and determines the video signal input delay according to the transmission timestamp and the reception timestamp.

[0014] In a second aspect, an embodiment of the present application provides a method for determining video signal input delay, which is applied to a receiving end and includes:

[0015] Obtain a target video frame sent by a sending end and the transmission timestamp corresponding to the target video frame;

[0016] When detecting the target pixel points in the target video frame, record the reception timestamp corresponding to the current time;

[0017] Determine the video signal input delay according to the transmission timestamp and the reception timestamp.

[0018] Optionally, the obtaining the target video frame sent by the sending end and the transmission timestamp corresponding to the target video frame includes:

[0019] Parse the marked video stream sent by the sending end to obtain the transmission timestamp in the metadata field of the marked video stream;

[0020] Decode the marked video stream to obtain the target video frame.

[0021] Optionally, the target pixel points are photosensitive pixel points,

[0022] When detecting the target pixel points in the target video frame, recording the reception timestamp corresponding to the current time includes:

[0023] Detect the target pixel points in the target video frame based on a pre-configured photosensitive element;

[0024] When detecting the target pixel points, record the reception timestamp.

[0025] In a third aspect, an embodiment of the present application provides a video signal input delay determination device, which is applied to a sending end and includes:

[0026] A target frame acquisition module, configured to insert a target pixel point at a specified position of a video frame to obtain a target video frame;

[0027] A delay determination module, configured to record the sending timestamp of the target video frame, send the sending timestamp and the target video frame to a receiving end, so that the receiving end records a receiving timestamp when detecting the target pixel point in the target video frame, and determine the video signal input delay according to the sending timestamp and the receiving timestamp.

[0028] Optionally, the target frame acquisition module includes:

[0029] A first acquisition unit, configured to insert the target pixel point at a specified position of the video frame to obtain the target video frame;

[0030] A second acquisition unit, configured to insert the target pixel point at multiple specified positions of the video frame to obtain the target video frame.

[0031] Optionally, the delay determination module includes:

[0032] A video stream acquisition unit, configured to encode the target video frame to obtain a marked video stream;

[0033] A delay determination unit, configured to record the sending timestamp of the marked video stream, add the sending timestamp to a metadata field of the marked video stream, and send the marked video stream to the receiving end, so that the receiving end decodes the marked video stream to obtain the target video frame, detects the target pixel point in the target video frame, and records a receiving timestamp, and determines the video signal input delay according to the sending timestamp and the receiving timestamp.

[0034] Optionally, the target pixel point is a photosensitive pixel point, and the target pixel point is at least one of a pixel point of a specified color and a pixel point of a specified brightness.

[0035] In a fourth aspect, an embodiment of the present application provides a video signal input delay determination device, which is applied to a receiving end and includes:

[0036] A video frame acquisition module, configured to receive a target video frame sent by a sending end and the sending timestamp corresponding to the target video frame;

[0037] A timestamp recording module, configured to record a receiving timestamp corresponding to the current time when detecting the target pixel point in the target video frame;

[0038] An input delay determination module, configured to determine the input delay of the video signal according to the sending timestamp and the receiving timestamp.

[0039] Optionally, the video frame acquisition module includes:

[0040] A first recording unit, configured to parse the marked video stream sent by the sending end to obtain the sending timestamp in the metadata field of the marked video stream;

[0041] A target frame acquisition unit, configured to decode the marked video stream to obtain the target video frame.

[0042] Optionally, the target pixel is a photosensitive pixel.

[0043] The timestamp recording module includes:

[0044] A target pixel detection unit, configured to detect the target pixel in the target video frame based on a pre-configured photosensitive element;

[0045] A second recording unit, configured to record the receiving timestamp when the target pixel is detected.

[0046] In a fifth aspect, an embodiment of the present application provides an electronic device, including:

[0047] One or more processors; and

[0048] One or more machine-readable media storing instructions, which when executed by the one or more processors, cause the device to execute the video signal input delay determination method described in any one of the above.

[0049] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, and the computer program stored therein causes a processor to execute the video signal input delay determination method described in any one of the above.

[0050] The embodiments of the present application include the following advantages:

[0051] The solution provided by the embodiments of this application obtains a target video frame by inserting target pixel points at specified positions in a video frame. Record the transmission timestamp of the target video frame, and send the transmission timestamp and the target video frame to the receiving end, so that the receiving end records the reception timestamp when detecting the target pixel points in the target video frame, and determines the input delay of the video signal according to the transmission timestamp and the reception timestamp. The embodiments of this application insert detectable target pixel points in the source video frame, and the receiving end can measure the input delay of the video signal by detecting the target pixel points and recording the timestamps. This method can improve the measurement accuracy of the input delay of the video signal compared with estimating the delay based on the timestamp of the network layer protocol or the method based on frame synchronization. At the same time, this measurement method does not require dedicated hardware support. Therefore, the measurement of the input delay of the video signal can be realized without increasing the complexity and measurement cost of the system. Description of the Drawings

[0052] Figure 1 It is a flowchart of the steps of a method for determining the input delay of a video signal provided by the embodiments of this application;

[0053] Figure 2 It is a flowchart of the steps of a method for obtaining a target video frame provided by the embodiments of this application;

[0054] Figure 3 It is a flowchart of the steps of a method for sending a video frame provided by the embodiments of this application;

[0055] Figure 4 It is a flowchart of the steps of another method for determining the input delay of a video signal provided by the embodiments of this application;

[0056] Figure 5 It is a flowchart of the steps of another method for obtaining a target video frame provided by the embodiments of this application;

[0057] Figure 6 It is a flowchart of the steps of a method for recording timestamps provided by the embodiments of this application;

[0058] Figure 7 It is a schematic diagram of a system architecture provided by the embodiments of this application;

[0059] Figure 8 It is a schematic diagram of the structure of a device for determining the input delay of a video signal provided by the embodiments of this application;

[0060] Figure 9 It is a schematic diagram of the structure of another device for determining the input delay of a video signal provided by the embodiments of this application. Detailed Embodiments

[0061] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] Referring to Figure 1 , a flowchart of steps of a method for determining the input delay of a video signal provided by an embodiment of the present application is shown. This method can be applied to the sending end. As Figure 1 shown, the method for determining the input delay of a video signal may include: step 101 and step 102.

[0063] Step 101: Insert a target pixel at a specified position in the video frame to obtain a target video frame.

[0064] The embodiment of the present application can be applied to the sending end, that is, the execution subject is the sending end.

[0065] In this embodiment, the target pixel is a pixel that can be detected by the receiving end and is different from other pixels in the video frame. In this example, the target pixel can be a photosensitive pixel, such as at least one of pixels of a specified color, pixels of a specified brightness, etc.

[0066] When measuring the input delay of a video signal, for the video frame to be sent, a target pixel can be inserted at a specified position in the video frame to obtain a target video frame. In this example, the target pixel can be inserted at one specified position or multiple specified positions in the video frame to obtain the target video frame. The implementation process will be described in detail below in conjunction with Figure 2 as follows.

[0067] Referring to Figure 2 , a flowchart of steps of a method for obtaining a target video frame provided by an embodiment of the present application is shown. As Figure 2 shown, the method for obtaining a target video frame may include: step 201 and step 202.

[0068] Step 201: Insert the target pixel at one specified position in the video frame to obtain the target video frame.

[0069] In this embodiment, when inserting the target pixel, the target pixel can be inserted at one specified position in the video frame to obtain the target video frame. For example, a red pixel or a highlighted pixel can be inserted at a certain position in the lower right corner, upper left corner, upper right corner, lower left corner, etc. of the video frame (such as the position with coordinates width - 10, height - 10 in the lower right corner) to obtain the target video frame.

[0070] It can be understood that the above examples are only examples listed for better understanding of the technical solutions of the embodiments of the present application and do not serve as the sole limitation of this embodiment.

[0071] In the embodiment of the present application, by inserting a target pixel point at a specific position in the video frame, a clear and unique reference point can be provided for the receiving end. The receiving end can accurately identify the position information of this pixel point, which is convenient for accurately measuring the input delay of the video signal subsequently. At the same time, only one target pixel point is inserted. When the receiving end detects and processes, it only needs to focus on the information of this specific point, without having to process the complex data brought by multiple points, greatly reducing the amount of calculation and the processing difficulty. This helps to improve the measurement efficiency, especially in video processing scenarios with high real-time requirements, and can quickly obtain the measurement result of the input delay of the video signal.

[0072] Step 202: Insert the target pixel points at multiple specified positions in the video frame to obtain the target video frame.

[0073] When inserting the target pixel points, the target pixel points can be inserted at multiple specified positions in the video frame to obtain the target video frame. For example, the target pixel points can be purple pixel points or highlighted pixel points, etc., inserted at two or more positions such as the lower right corner, upper left corner, upper right corner, lower left corner, etc. of the video frame (for example, the position of the lower right corner has coordinates: width - 10, height - 10, and the position of the lower left corner has coordinates width 10, height - 10), so as to obtain the target video frame.

[0074] It can be understood that the above examples are only examples listed for better understanding the technical solution of the embodiment of the present application, and do not serve as the sole limitation of this embodiment.

[0075] In the embodiment of the present application, by inserting the target pixel points at multiple specified positions, the receiving end can obtain more information about the propagation and processing of the video signal. The measurement results of multiple points can confirm and complement each other, reducing the influence of possible errors or abnormal situations of a single point on the measurement result, so as to more comprehensively and accurately reflect the delay situation of the video signal during transmission and improve the accuracy and reliability of the measurement.

[0076] In specific implementation, for a certain video, when measuring the input delay of the video signal, this video may contain many video frames. Different types of target pixel points can be inserted for each video frame of this video, for example, pixel points of different colors, different brightness, etc. It is also possible to insert the same type of target pixel points for all video frames of this video, etc. This embodiment does not limit this.

[0077] After inserting the target pixel points at the specified positions in the video frame to obtain the target video frame, step 102 is executed.

[0078] Step 102: Record the transmission timestamp of the target video frame, and send the transmission timestamp and the target video frame to the receiving end, so that the receiving end records the reception timestamp when detecting the target pixel points in the target video frame, and determines the video signal input delay according to the transmission timestamp and the reception timestamp.

[0079] A timestamp refers to a mark that records the specific time when an event occurs in a computer system.

[0080] Delay refers to the time required for data to travel from the sending end to the receiving end, usually measured in milliseconds (ms).

[0081] Video signal input delay refers to the time delay between the signal sent from a video signal source (such as a computer, game console, camera, etc.) and the image finally displayed on a display device (such as a monitor, TV, projector, etc.).

[0082] After inserting target pixel points at the specified position in the video frame to obtain the target video frame, the transmission timestamp of the target video frame can be recorded. At the same time, the transmission timestamp and the target video frame can be sent to the receiving end.

[0083] After receiving the target video frame and the transmission timestamp, the receiving end can detect the target pixel points in the target video frame and record the reception timestamp when detecting the target pixel points. Then, the video signal input delay can be determined according to the transmission timestamp and the reception timestamp, that is, the time difference between the transmission timestamp and the reception timestamp is used as the video signal input delay.

[0084] In the embodiment of the present application, by inserting detectable target pixel points in the source video frame, the receiving end can measure the video signal input delay by detecting the target pixel points and recording timestamps. This method can improve the measurement accuracy of the video signal input delay compared with the method of estimating the delay based on the timestamp of the network layer protocol or frame synchronization. At the same time, by adding photosensitive pixel points to the source video frame, the receiving end can detect special pixel points through built-in photosensitive elements (such as cameras or other devices that monitor specific optical characteristics), without the support of dedicated hardware. Therefore, the measurement of the video signal input delay can be realized without increasing the complexity of the system and the measurement cost.

[0085] In this embodiment, when sending the target video frame and the transmission timestamp, the target video frame can be encoded into a video stream, and at the same time, the transmission timestamp is added to the metadata field of the video stream, and the video stream is sent. The implementation process can be combined with Figure 3 to be described in detail as follows.

[0086] Refer to Figure 3, showing the step flowchart of a video frame sending method provided by an embodiment of the present application. As Figure 3 shown, the video frame sending method may include: step 301 and step 302.

[0087] Step 301: Encode the target video frame to obtain a marked video stream.

[0088] In this embodiment, after obtaining the target video frame, the target video frame can be encoded to obtain a marked video stream. Specifically, the target video frame added with special pixel points can be encoded and converted into a marked video stream suitable for transmission in environments such as networks. The encoding process will compress and convert the video frame according to a certain video encoding standard (such as H.264, H.265, etc.) to efficiently transmit and store video data.

[0089] After encoding the target video frame to obtain a marked video stream, step 302 is executed.

[0090] Step 302: Record the sending timestamp of the marked video stream, add the sending timestamp to the metadata field of the marked video stream, and send the marked video stream to the receiving end, so that the receiving end decodes the marked video stream to obtain the target video frame, detects the target pixel points in the target video frame, and records the receiving timestamp, and determines the video signal input delay according to the sending timestamp and the receiving timestamp.

[0091] Before sending the marked video stream, record the current time as the sending timestamp. This timestamp precisely marks the moment when the video stream leaves the sending end, providing a starting time reference for subsequent calculation of the video signal input delay.

[0092] Then, the recorded sending timestamp can be added to the metadata field of the marked video stream. Metadata is some additional information about the video stream. In this way, after the receiving end receives the video stream, it can conveniently obtain the sending timestamp information without an additional channel to transmit this time information, ensuring the close association between the time information and the video data. After that, the marked video stream with the sending timestamp metadata can be sent to the receiving end through the network or other transmission media.

[0093] In the embodiments of the present application, by accurately recording the transmission timestamp at the sending end, accurately recording the reception timestamp at the receiving end, and combining the detection of target pixel points, the time experienced by the video signal from transmission to reception can be accurately measured, thereby accurately obtaining the input delay of the video signal. This method can effectively exclude the interference of other irrelevant factors and provide relatively accurate delay measurement results, which is very important for application scenarios that require strict control of video transmission delay (such as video conferencing, real-time monitoring, etc.). Adding the transmission timestamp to the metadata field that marks the video stream enables the time information to be transmitted together with the video data, ensuring the accuracy and integrity of the time information and facilitating synchronization and processing at the receiving end. The receiving end can conveniently obtain the transmission timestamp while decoding the video stream without the need for additional complex synchronization mechanisms or separate time transmission channels, reducing the complexity and implementation cost of the system.

[0094] Understandably, the sending end and the receiving end can perform clock synchronization to improve the measurement accuracy of the input delay of the video signal. The clock synchronization methods can include:

[0095] 1. Based on the Network Time Protocol (NTP): NTP is a protocol used to synchronize computer clocks in a network. It sends timestamp messages over the network to keep the clocks of different devices synchronized with an authoritative time source. In a local area network or wide area network environment, there is usually one or more NTP servers as time reference points. Both the sending end and the receiving end synchronize their times with the NTP server, thereby indirectly achieving clock synchronization between each other.

[0096] 2. Based on the Precision Time Protocol (PTP): High-precision clock synchronization is achieved by exchanging accurate timestamp information between the master and slave clocks and compensating for factors such as network latency. Different from NTP, PTP usually adopts hardware implementation and can achieve synchronization accuracy at the sub-microsecond level.

[0097] 3. Based on hardware synchronization signals: Synchronization signals are sent through specialized hardware devices. For example, a high-precision clock source is used to generate clock pulse signals, which are transmitted to the sending end and the receiving end through physical media such as cables or optical fibers. The clock circuits at the sending end and the receiving end adjust the local clocks according to the received synchronization signals to achieve clock synchronization.

[0098] 4. Synchronization based on software algorithms: Software algorithms are used to perform clock synchronization between the sending end and the receiving end. For example, the sending end attaches local clock information when sending data, and the receiving end adjusts according to the received data and the clock information therein in combination with its own clock.

[0099] Understandably, the above four clock synchronization methods are only examples listed for better understanding the technical solutions of the embodiments of the present application and do not serve as the sole limitation of this embodiment.

[0100] The video signal input delay determination method provided by the embodiment of the present application obtains a target video frame by inserting a target pixel point at a specified position of a video frame. Record the transmission timestamp of the target video frame, and send the transmission timestamp and the target video frame to the receiving end, so that the receiving end records the reception timestamp when detecting the target pixel point in the target video frame, and determines the video signal input delay according to the transmission timestamp and the reception timestamp. By inserting detectable target pixel points in the source video frame in the embodiment of the present application, the receiving end can measure the video signal input delay by detecting the target pixel points and recording timestamps. Compared with the method of estimating delay based on the timestamp of the network layer protocol or frame synchronization, this method can improve the measurement accuracy of the video signal input delay. At the same time, this measurement method does not require dedicated hardware support. Therefore, the measurement of the video signal input delay can be realized without increasing the complexity and measurement cost of the system.

[0101] Refer to Figure 4 , which shows a flowchart of steps of another video signal input delay determination method provided by the embodiment of the present application. This method can be applied to the receiving end. As Figure 4 shown, the video signal input delay determination method may include: step 401, step 402, and step 403.

[0102] Step 401: Obtain the target video frame sent by the sending end and the transmission timestamp corresponding to the target video frame.

[0103] The embodiment of the present application can be applied to the receiving end, that is, the execution subject is the receiving end.

[0104] The target video frame refers to the video frame with target pixel points added sent by the sending end to the receiving end.

[0105] The transmission timestamp refers to the timestamp corresponding to the time when the sending end sends the target video frame to the receiving end.

[0106] When measuring the video signal input delay, the sending end can send the target video frame and the transmission timestamp corresponding to the target video frame to the receiving end. In this embodiment, the sending end can send a video stream to the receiving end, and the video stream contains the target video frame and the corresponding transmission timestamp. The process for the receiving end to obtain the target video frame and the transmission timestamp will be described in detail in combination with Figure 5 as follows.

[0107] Refer to Figure 5 , which shows a flowchart of steps of another target video frame acquisition method provided by the embodiment of the present application. As Figure 5 shown, the target video frame acquisition method may include: step 501 and step 502.

[0108] Step 501: Parse the marked video stream sent by the sending end to obtain the sending timestamp in the metadata field of the marked video stream.

[0109] In this embodiment, the sending end can send a marked video stream to the receiving end. The marked video stream can be obtained after the sending end encodes the video frame with the target pixel points added and simultaneously adds the sending timestamp in the metadata field of the video stream.

[0110] After receiving the marked video stream sent by the sending end, the marked video stream can be parsed to obtain the sending timestamp in the metadata field of the marked video stream.

[0111] After parsing the marked video stream to obtain the sending timestamp, step 502 is executed.

[0112] Step 502: Decode the marked video stream to obtain the target video frame.

[0113] After parsing the marked video stream to obtain the sending timestamp, the marked video stream can be decoded to obtain the target video frame. Specifically, the OpenCV library or the like can be used to decode the marked video stream to obtain the target video frame.

[0114] In the embodiment of the present application, by accurately parsing the sending timestamp from the marked video stream, it provides a basis for the subsequent measurement of the input delay of the video signal. At the same time, decoding the video stream can efficiently obtain the target video frame, providing a basis for the subsequent detection of the target pixel points.

[0115] After obtaining the target video frame sent by the sending end and the corresponding sending timestamp of the target video frame, step 402 is executed.

[0116] Step 402: When the target pixel point in the target video frame is detected, record the receiving timestamp corresponding to the current time.

[0117] After obtaining the target video frame, the target pixel point in the target video frame can be detected, and the receiving timestamp corresponding to the current time when the target pixel point in the target video frame is detected is recorded. In this example, the target pixel point inserted in the target video frame can be a photosensitive pixel point, and the receiving end can detect the target pixel point through a photosensitive element and record the receiving timestamp at the same time. For this implementation process, it can be combined with Figure 6 to be described in detail as follows.

[0118] Refer to Figure 6 , which shows the step flowchart of a timestamp recording method provided by the embodiment of the present application. As Figure 6 shown, the timestamp recording method can include: step 601 and step 602.

[0119] Step 601: Detect the target pixel point in the target video frame based on a pre-configured photosensitive element.

[0120] In this embodiment, the target pixel point can be a photosensitive pixel point. After receiving the target video frame, a pre-configured photosensitive element (such as a camera or other device for monitoring specific optical characteristics, etc.) can be used to detect the target pixel point in the target video frame.

[0121] Step 602: When the target pixel point is detected, record the reception timestamp.

[0122] When the target pixel point in the target video frame is detected, the timestamp corresponding to the current time, that is, the reception timestamp, can be recorded.

[0123] In the embodiment of the present application, the photosensitive element can be used to detect the inserted photosensitive pixel point in the target video frame, without the support of dedicated hardware, which can reduce the cost investment in measuring the video signal input delay. At the same time, when the target pixel point is detected, the current reception timestamp is recorded in a timely manner, providing data support for the subsequent analysis of the video signal input delay.

[0124] After recording the reception timestamp corresponding to the current time, step 403 is executed.

[0125] Step 403: Determine the video signal input delay according to the transmission timestamp and the reception timestamp.

[0126] After recording the reception timestamp corresponding to the current time, the video signal input delay can be determined according to the transmission timestamp and the reception timestamp. Specifically, the time difference between the transmission timestamp and the reception timestamp can be used as the video signal input delay.

[0127] The video signal input delay determination method provided by the embodiment of the present application obtains the target video frame sent by the sending end and the transmission timestamp corresponding to the target video frame. When the target pixel point in the target video frame is detected, the reception timestamp corresponding to the current time is recorded. The video signal input delay is determined according to the transmission timestamp and the reception timestamp. In the embodiment of the present application, by inserting detectable target pixel points in the source video frame, the video signal input delay can be measured at the receiving end by detecting the target pixel points and recording the timestamps. Compared with the method of estimating the delay based on the timestamp of the network layer protocol or frame synchronization, this method can improve the measurement accuracy of the video signal input delay. At the same time, this measurement method does not require the support of dedicated hardware. Therefore, the measurement of the video signal input delay can be realized without increasing the complexity of the system and the measurement cost.

[0128] For the measurement method of the video signal input delay provided in this embodiment, it can be combined withFigure 7 A detailed description is as follows.

[0129] Figure 7 It is a schematic diagram of a system architecture provided by an embodiment of the present application. As Figure 7 shown, the system may include: a delay detection server, a video sender, and a video receiver.

[0130] Among them, the delay detection server can synchronize the clocks of the video sender and the video receiver through NTP (Network Time Protocol). Specifically, the delay detection server can send the time t1 to the video sender and send the reception time t2 to the video receiver, and the video sender and the video receiver can perform clock synchronization according to the sending time t1 and the reception time t2 respectively.

[0131] During the video transmission process, the video sender can first insert pixel points into the video frame. Specifically, the sender can first select special pixel points and insert special pixel points at one or more specific positions in each video, such as pixel points with special brightness or color.

[0132] Then, the video sender performs an encoding operation. Specifically, these special pixel points can be embedded into the video stream during the encoding stage by modifying the video encoder. After that, for each video frame containing special pixel points, record its sending timestamp and send it to the video receiver together with the video frame as metadata.

[0133] At the video receiver, the received video stream can be first parsed to obtain the sending timestamp. After that, the video stream can be decoded to obtain the video frame with special pixel points inserted. Furthermore, a pre-configured camera or other devices suitable for detecting specific optical characteristics can be used to detect the special pixel points in the video frame, and when the special pixel points are detected, record the current timestamp, that is, the reception timestamp. Finally, the time difference between the recorded sending timestamp and the reception timestamp can be calculated, which is the input delay of the video signal.

[0134] By directly detecting the special pixel points in the video frame in the embodiment of the present application, the errors that may exist in the traditional method are avoided. At the same time, no complex hardware support is required, and only a simple software module needs to be added to the existing video transmission system. This video signal input delay measurement method is applicable to various types of video transmission environments and can work effectively whether it is a local area network or a wide area network.

[0135] It should be noted that, for method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the embodiments of the present application are not limited by the described action sequences, because according to the embodiments of the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present application.

[0136] Referring to Figure 8 , a schematic structural diagram of a video signal input delay determination device provided by an embodiment of the present application is shown. This device can be applied to a sending end. As Figure 8 shown, the video signal input delay determination device 800 may include the following modules:

[0137] A target frame acquisition module 810, configured to insert a target pixel point at a specified position of a video frame to obtain a target video frame;

[0138] A delay determination module 820, configured to record the transmission timestamp of the target video frame, send the transmission timestamp and the target video frame to a receiving end, so that the receiving end records a reception timestamp when detecting the target pixel point in the target video frame, and determines the video signal input delay according to the transmission timestamp and the reception timestamp.

[0139] Optionally, the target frame acquisition module includes:

[0140] A first acquisition unit, configured to insert the target pixel point at a specified position of the video frame to obtain the target video frame;

[0141] A second acquisition unit, configured to insert the target pixel point at multiple specified positions of the video frame to obtain the target video frame.

[0142] Optionally, the delay determination module includes:

[0143] A video stream acquisition unit, configured to encode the target video frame to obtain a marked video stream;

[0144] A delay determination unit, configured to record the transmission timestamp of the marked video stream, add the transmission timestamp to the metadata field of the marked video stream, and send the marked video stream to the receiving end, so that the receiving end decodes the marked video stream to obtain the target video frame, detects the target pixel point in the target video frame, and records a reception timestamp, and determines the video signal input delay according to the transmission timestamp and the reception timestamp.

[0145] The video signal input delay determination device provided by the embodiment of the present application obtains a target video frame by inserting a target pixel point at a specified position in a video frame. Record the transmission timestamp of the target video frame, and send the transmission timestamp and the target video frame to the receiving end, so that the receiving end records the reception timestamp when detecting the target pixel point in the target video frame, and determines the video signal input delay according to the transmission timestamp and the reception timestamp. By inserting detectable target pixel points in the source video frame in the embodiment of the present application, the receiving end can measure the video signal input delay by detecting the target pixel points and recording timestamps. Compared with the method of estimating delay based on the timestamp of the network layer protocol or frame synchronization, this method can improve the measurement accuracy of the video signal input delay. At the same time, this measurement method does not require dedicated hardware support. Therefore, the measurement of the video signal input delay can be realized without increasing the complexity of the system and the measurement cost.

[0146] Referring to Figure 9 , a schematic structural diagram of another video signal input delay determination device provided by the embodiment of the present application is shown. This device can be applied to the receiving end. As Figure 9 shown, the video signal input delay determination device 900 may include the following modules:

[0147] A video frame acquisition module 910, configured to receive the target video frame sent by the sending end and the transmission timestamp corresponding to the target video frame;

[0148] A timestamp recording module 920, configured to record the reception timestamp corresponding to the current time when detecting the target pixel point in the target video frame;

[0149] An input delay determination module 930, configured to determine the video signal input delay according to the transmission timestamp and the reception timestamp.

[0150] Optionally, the video frame acquisition module includes:

[0151] A first recording unit, configured to parse the marked video stream sent by the sending end to obtain the transmission timestamp in the metadata field of the marked video stream;

[0152] A target frame acquisition unit, configured to decode the marked video stream to obtain the target video frame.

[0153] Optionally, the target pixel point is a photosensitive pixel point,

[0154] The timestamp recording module includes:

[0155] A target pixel detection unit, configured to detect the target pixel point in the target video frame based on a pre-configured photosensitive element;

[0156] A second recording unit, configured to record the reception timestamp when the target pixel point is detected.

[0157] The video signal input delay determination device provided by the embodiments of the present application obtains a target video frame sent by a sending end and a sending timestamp corresponding to the target video frame. When a target pixel point in the target video frame is detected, the reception timestamp corresponding to the current time is recorded. According to the sending timestamp and the reception timestamp, the video signal input delay is determined. In the embodiments of the present application, by inserting detectable target pixel points into the source video frame, the video signal input delay can be measured at the receiving end by detecting the target pixel points and recording the timestamps. Compared with the method of estimating the delay based on the timestamp of the network layer protocol or the frame synchronization method, this method can improve the measurement accuracy of the video signal input delay. At the same time, this measurement method does not require dedicated hardware support. Therefore, the measurement of the video signal input delay can be realized without increasing the complexity of the system and the measurement cost.

[0158] For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the related parts, please refer to the partial description of the method embodiments.

[0159] In the embodiments of the present application, an electronic device is further provided. The electronic device may include one or more processors, and one or more machine-readable media storing instructions thereon, such as application programs. When the instructions are executed by the one or more processors, the processors are caused to execute the video signal input delay determination method according to any one of the above.

[0160] In the embodiments of the present application, a non-transitory computer-readable storage medium is further provided, having a computer program stored thereon, and the program can be executed by a processor of an electronic device to implement the video signal input delay determination method according to any one of the above. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0161] The embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other.

[0162] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, apparatuses, or computer program products. Therefore, the embodiments of the present application can take the form of all-hardware embodiments, all-software embodiments, or embodiments combining software and hardware aspects. Moreover, the embodiments of the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) that contain computer-usable program code.

[0163] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks.

[0164] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one or more of the flows Figure 1 or blocks.

[0165] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks.

[0166] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0167] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the said element.

[0168] The above has introduced in detail a method, apparatus, electronic device and storage medium for determining the input time delay of a video signal provided by this application. Specific examples are used in this article to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A method for determining a video signal input delay, applied to a transmitting end, characterized in that: include: Inserting a target pixel point at a specified position of a video frame to obtain a target video frame; Record the sending timestamp of the target video frame, send the sending timestamp and the target video frame to the receiving end, so that the receiving end records the receiving timestamp when detecting the target pixel point in the target video frame, and determines the video signal input delay according to the sending timestamp and the receiving timestamp.

2. The method according to claim 1, characterized in that Inserting a target pixel point at a designated position of a video frame to obtain a target video frame includes: Inserting the target pixel point at a specified position of the video frame to obtain the target video frame; or The target pixel points are inserted into a plurality of designated positions of the video frame to obtain the target video frame.

3. The method according to claim 1, characterized in that The recording of the sending timestamp of the target video frame and sending the sending timestamp and the target video frame to the receiving end includes: Encoding the target video frame to obtain a marked video stream; Record the sending timestamp of the marked video stream, add the sending timestamp to the metadata field of the marked video stream, and send the marked video stream to the receiving end, so that the receiving end decodes the marked video stream to obtain the target video frame, detects the target pixel point in the target video frame, and records the receiving timestamp, and determines the video signal input delay according to the sending timestamp and the receiving timestamp.

4. A method for determining a video signal input delay, applied to a receiving end, characterized in that: include: Obtain a target video frame sent by a sender and a sending timestamp corresponding to the target video frame; When a target pixel point in the target video frame is detected, a receiving timestamp corresponding to the current time is recorded; A video signal input delay is determined according to the sending timestamp and the receiving timestamp.

5. The method according to claim 4, characterized in that The obtaining of a target video frame sent by a sending end and a sending timestamp corresponding to the target video frame includes: Parsing the marked video stream sent by the sending end to obtain the sending timestamp in the metadata field of the marked video stream; The marked video stream is decoded to obtain the target video frame.

6. The method according to claim 4, characterized in that The target pixel is a photosensitive pixel. When the target pixel point in the target video frame is detected, recording the receiving timestamp corresponding to the current time includes: Detecting the target pixel point in the target video frame based on a pre-configured photosensitive element; When the target pixel is detected, the receiving timestamp is recorded.

7. A video signal input delay determination device, applied to a transmitting end, characterized in that: include: A target frame acquisition module is used to insert a target pixel point at a specified position of a video frame to obtain a target video frame; A delay determination module is used to record the sending timestamp of the target video frame, send the sending timestamp and the target video frame to the receiving end, so that the receiving end records the receiving timestamp when the target pixel point in the target video frame is detected, and determines the video signal input delay according to the sending timestamp and the receiving timestamp.

8. A video signal input delay determination device, applied to a receiving end, characterized in that: include: A video frame acquisition module, used for the target video frame sent by the sending end and the sending timestamp corresponding to the target video frame; A timestamp recording module, used to record a receiving timestamp corresponding to the current time when a target pixel point in the target video frame is detected; The input delay determination module is used to determine the video signal input delay according to the sending timestamp and the receiving timestamp.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, cause the apparatus to perform the video signal input delay determination method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the video signal input delay determination method according to any one of claims 1 to 6.

Images