A real-time video delay calculation method and device

Abstract

The present disclosure relates to a real-time video delay calculation method, device, electronic device, and storage medium. The method comprises: simultaneously displaying a timestamp reference image and a timestamp to-be-tested image based on a real-time video stream terminal signal monitoring module, and calculating the real-time video delay using a real-time video stream delay signal processing module. The present disclosure replaces the manual photography method with a more intelligent and automated measurement solution, ensuring both the universality of engineering applications and the ease of operation of the delay test system. It also eliminates the high error associated with manual intervention and improves the intuitiveness of delay data testing results.

Description

Technical Field

[0001] The present disclosure relates to the field of video measurement, and in particular to a real-time video delay calculation method, device, electronic device, and computer-readable storage medium. Background Art

[0002] With the rapid evolution and development of multimedia technology, smart security and live streaming are increasingly pervasive across industries, and users are increasingly demanding higher quality video transmission. Among the factors affecting video transmission quality, latency is one of the most significant issues affecting user experience. From camera capture to client playback, video stream data undergoes transmission, encoding, decoding, and transcoding, all of which contribute to latency. Therefore, accurately measuring real-time video latency from the capture end to the playback end is essential.

[0003] Among existing technical solutions, many documents cite the timestamp marking method. The so-called timestamp marking method refers to embedding timestamp information in the video stream before the video signal enters the transmission network, and then obtaining the timestamp information at the output moment when the video stream signal is output; the output timestamp is subtracted from the input timestamp to calculate the video delay data. This method first requires that the input and output ends of the video signal have the same time reference, and that the input end supports the embedding of millisecond-level timestamp information. However, in the field of security monitoring, most cameras do not have the function of embedding millisecond-level timestamp information, and the consistency of the time reference between the input and output ends cannot be guaranteed. Therefore, this solution is rarely used in the actual production environment of the security monitoring field.

[0004] In addition to the timestamp method, the stopwatch photography method is currently widely used. The so-called stopwatch photography method involves running a digital stopwatch on a PC and displaying it on a monitor. Using the system under test, the system takes a photo of the PC monitor and displays it on another monitor under test. The PC monitor and the monitor under test are placed side by side and photographed together using a test camera. The difference between the stopwatches on the two monitors is then calculated based on the photographic results to generate video delay data. This method requires at least two monitors, a video camera, a camera, and a human operator. Its major drawback is that it requires manual intervention to continuously photograph the two monitors and manually calculate the difference between the input and output images at different times, making it difficult to implement in scenarios where real-time delay data display is required. Another drawback is that the different shutter speeds of the cameras used and the varying quality of the video output resulting from the real-time video stream network transmission device can easily cause ghosting in the photographic results, affecting the accuracy of the delay measurement. It is known that the delay data of a real-time video stream does not remain constant but rather fluctuates within a certain range due to network fluctuations. Manual photography and calculation cannot quickly calculate real-time delay data.

[0005] To accurately calculate video transmission delay, invention patent CN 113194307 A proposes an oscilloscope video voltage signal comparison method. This method transmits the video signal generated by a video signal generator to the video device or system under test. The output signal is then simultaneously transmitted to an oscilloscope along with the signal generator's output signal. The voltage waveform differences between the two light field signals or video signals are compared to calculate the delay. While this method can produce relatively accurate delay data, it requires the system under test to have an open input interface, making it unsuitable for most production scenarios.

[0006] Therefore, one or more methods are needed to solve the above problems.

[0007] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0008] The purpose of the present disclosure is to provide a real-time video delay calculation method, device, electronic device and computer-readable storage medium, thereby overcoming one or more problems caused by the limitations and defects of related technologies to at least a certain extent.

[0009] According to one aspect of the present disclosure, a real-time video delay calculation method is provided, comprising:

[0010] Generate a time signal based on a digital clock generation module, and send the clock signal to a real-time video stream terminal signal monitoring module;

[0011] The real-time video stream terminal signal monitoring module receives the time signal sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the time signal;

[0012] The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module;

[0013] The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module;

[0014] The real-time video stream terminal signal monitoring module receives the video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp to be tested picture based on the video signal and displays it on the same display device as the timestamp reference picture;

[0015] The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage;

[0016] The real-time video stream delay signal processing module retrieves the to-be-processed picture from the delayed video data storage module, identifies the reference timestamp and the to-be-tested timestamp in the to-be-processed picture based on the character recognition submodule, and calculates the real-time video delay based on the reference timestamp and the to-be-tested timestamp based on the delay calculation submodule.

[0017] In an exemplary embodiment of the present disclosure, the method further includes:

[0018] The first time generation submodule based on the digital clock generation module generates a first time signal, the second time generation submodule based on the digital clock generation module generates a second time signal, and sends the first time signal and the second time signal to the real-time video stream terminal signal monitoring module;

[0019] The real-time video stream terminal signal monitoring module receives the first time signal and the second time signal sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the first time signal and the second time signal, wherein the first time signal and the second time signal in the timestamp reference picture are displayed at a first preset position and a second preset position on the display device respectively;

[0020] The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module;

[0021] The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module;

[0022] The real-time video stream terminal signal monitoring module receives a video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp test picture based on the video signal and displays it on the same display device as the timestamp reference picture, wherein the timestamp test picture includes a first time signal to be tested and a second time signal to be tested, and the first time signal to be tested and the second time signal to be tested are displayed at a third preset position and a fourth preset position on the display device;

[0023] The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage;

[0024] The real-time video stream delay signal processing module retrieves the picture to be processed from the delayed video data storage module, and based on the character recognition submodule, respectively identifies the reference first timestamp, the first timestamp to be measured, the reference second timestamp, and the second timestamp to be measured in the picture to be processed, and based on the delay calculation submodule, calculates the real-time video delay according to the reference first timestamp, the first timestamp to be measured, the reference second timestamp, and the second timestamp to be measured.

[0025] In an exemplary embodiment of the present disclosure, the method further includes:

[0026] The stopwatch time generation submodule based on the digital clock generation module generates a stopwatch time signal, the millisecond time generation submodule based on the digital clock generation module generates a millisecond time signal, and sends the stopwatch time signal and the millisecond time signal to the real-time video stream terminal signal monitoring module;

[0027] The real-time video stream terminal signal monitoring module receives the stopwatch time signal and the millisecond time signal sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the stopwatch time signal and the millisecond time signal, wherein the stopwatch time signal and the millisecond time signal in the timestamp reference picture are displayed at a first preset position and a second preset position on the display device respectively;

[0028] The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module;

[0029] The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module;

[0030] The real-time video stream terminal signal monitoring module receives a video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp test picture based on the video signal and displays it on the same display device as the timestamp reference picture, wherein the timestamp test picture includes a stopwatch time signal to be tested and a millisecond time signal to be tested, and the stopwatch time signal to be tested and the millisecond time signal to be tested are displayed at a third preset position and a fourth preset position on the display device;

[0031] The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage;

[0032] The real-time video stream delay signal processing module retrieves the picture to be processed from the delayed video data storage module, and based on the character recognition submodule, respectively identifies the reference stopwatch timestamp, the stopwatch timestamp to be measured, the reference millisecond timestamp, and the millisecond timestamp to be measured in the picture to be processed, and based on the delay calculation submodule, calculates the real-time video delay according to the reference stopwatch timestamp, the stopwatch timestamp to be measured, the reference millisecond timestamp, and the millisecond timestamp to be measured.

[0033] In an exemplary embodiment of the present disclosure, the method further includes:

[0034] The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module based on a preset time interval to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module;

[0035] In an exemplary embodiment of the present disclosure, the method further includes:

[0036] The real-time video stream delay signal processing module retrieves a preset number of to-be-processed frames from the delay video data storage module, identifies reference timestamps and to-be-tested timestamps in the preset number of to-be-processed frames based on the character recognition submodule, and calculates the real-time video delay based on the reference timestamps and to-be-tested timestamps based on the delay calculation submodule;

[0037] An average value of the real-time video delays corresponding to the preset number of frames to be processed is calculated as a video delay calculation result.

[0038] In an exemplary embodiment of the present disclosure, the method further includes:

[0039] N time generation submodules based on the digital clock generation module respectively generate N time signals, and send the N time signals to the real-time video stream terminal signal monitoring module, where N is a positive integer greater than or equal to 1;

[0040] The real-time video stream terminal signal monitoring module receives the N time signals sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the N time signals, wherein the N time signals in the timestamp reference picture are respectively displayed at a 2N-1th preset position on the display device;

[0041] The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module;

[0042] The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module;

[0043] The real-time video stream terminal signal monitoring module receives a video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp test picture based on the video signal and displays it on the same display device as the timestamp reference picture, wherein the timestamp test picture includes the Nth time signal to be tested, and the Nth time signal to be tested is displayed at the 2Nth preset position on the display device;

[0044] The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage;

[0045] The real-time video stream delay signal processing module retrieves the to-be-processed picture from the delay video data storage module, identifies the reference Nth timestamp and the to-be-tested Nth timestamp in the to-be-processed picture based on the character recognition submodule, and calculates the real-time video delay based on the reference Nth timestamp and the to-be-tested Nth timestamp based on the delay calculation submodule;

[0046] Calculate an average value of the real-time video delay calculated based on the reference Nth timestamp and the Nth timestamp to be measured as the video delay calculation result.

[0047] In an exemplary embodiment of the present disclosure, the method further includes:

[0048] The real-time video stream delayed signal processing module retrieves the to-be-processed picture from the delayed video data storage module, and identifies the reference Nth timestamp and the to-be-tested Nth timestamp in the to-be-processed picture based on the character recognition submodule;

[0049] When the character recognition submodule identifies that there is a ghosting phenomenon in the reference Nth timestamp and the Nth timestamp to be tested, discard the reference Nth timestamp / the Nth timestamp to be tested and its corresponding Nth timestamp to be tested / the reference Nth timestamp;

[0050] The delay calculation submodule calculates the real-time video delay according to the reference Nth timestamp and the Nth timestamp to be measured;

[0051] Calculate a mode value of the real-time video delay calculated based on the reference Nth timestamp and the Nth timestamp to be measured as the video delay calculation result.

[0052] In one aspect of the present disclosure, a real-time video delay calculation device is provided, comprising:

[0053] A digital clock generation module, the digital clock generation module is integrated in the real-time video stream terminal signal monitoring module, the digital clock generation module is used to generate a time signal and send the clock signal to the real-time video stream terminal signal monitoring module;

[0054] A real-time video stream terminal signal monitoring module, the real-time video stream terminal signal monitoring module is used to receive the time signal sent by the digital clock generation module, and generate and display a timestamp reference picture based on the time signal; the real-time video stream terminal signal monitoring module is also used to receive the video signal collected by the real-time video stream signal acquisition module and sent by the real-time video stream transmission network module after processing the video stream based on the network transmission, and generate a timestamp to be tested picture based on the video signal and display it on the same display device as the timestamp reference picture;

[0055] A real-time video stream signal acquisition module, which is used to acquire the timestamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the timestamp reference picture and send it to the real-time video stream transmission network module;

[0056] A real-time video stream transmission network module, the real-time video stream transmission network module is used to receive the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and send the video signal to the real-time video stream terminal signal monitoring module after performing video stream processing based on network transmission;

[0057] A real-time video stream delay signal processing module, wherein the real-time video stream delay signal processing module is used to take screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generate a picture to be processed and send it to the delayed video data storage module; the real-time video stream delay signal processing module is also used to retrieve the picture to be processed from the delayed video data storage module, and respectively identify the reference timestamp and the timestamp to-be-tested in the picture to be processed based on the character recognition submodule; the real-time video stream delay signal processing module is also used to calculate the real-time video delay based on the reference timestamp and the timestamp to-be-tested based on the delay calculation submodule;

[0058] The delayed video data storage module is used to receive and store the to-be-processed images sent by the real-time video stream delayed signal processing module.

[0059] In one aspect of the present disclosure, there is provided an electronic device, comprising:

[0060] processor; and

[0061] A memory having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by the processor, implement the method according to any one of the above items.

[0062] In one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method according to any one of the above items is implemented.

[0063] In an exemplary embodiment of the present disclosure, a method for calculating real-time video delay is provided. The method includes: simultaneously displaying a timestamp reference image and a timestamp test image based on a real-time video stream terminal signal monitoring module, and calculating the real-time video delay using a real-time video stream delay signal processing module. This method replaces the manual photography method with a more intelligent and automated calculation solution, ensuring both the universality of engineering applications and the ease of use of the delay test system. It also eliminates the high error associated with manual intervention and improves the intuitiveness of delay data testing results.

[0064] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The above and other features and advantages of the present disclosure will become more apparent by describing in detail example embodiments thereof with reference to the accompanying drawings.

[0066] Figure 1 A flowchart of a method for calculating real-time video delay according to an exemplary embodiment of the present disclosure is shown;

[0067] Figures 2A-2B A schematic diagram of an application scenario of a real-time video delay calculation method according to an exemplary embodiment of the present disclosure is shown;

[0068] Figure 3 A schematic diagram of an application scenario of a real-time video delay calculation method according to an exemplary embodiment of the present disclosure is shown;

[0069] Figure 4 A schematic block diagram of a real-time video delay calculation device according to an exemplary embodiment of the present disclosure is shown;

[0070] Figure 5A block diagram schematically illustrates an electronic device according to an exemplary embodiment of the present disclosure; and

[0071] Figure 6 A schematic diagram schematically illustrates a computer-readable storage medium according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0072] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. Like reference numerals in the drawings represent like or similar parts, and thus repetitive description thereof will be omitted.

[0073] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, materials, devices, steps, etc. can be adopted. In other cases, well-known structures, methods, devices, implementations, materials or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0074] The blocks shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. Specifically, these functional entities may be implemented in software, or in one or more software-hardened modules, or in different networks and / or processor devices and / or microcontroller devices.

[0075] In this example embodiment, a real-time video delay calculation method is first provided; Figure 1 As shown in , the real-time video delay calculation method may include the following steps:

[0076] Step S110, generating a time signal based on the digital clock generation module, and sending the clock signal to the real-time video stream terminal signal monitoring module;

[0077] Step S120: The real-time video stream terminal signal monitoring module receives the time signal sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the time signal;

[0078] Step S130: The real-time video stream signal acquisition module acquires the timestamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the timestamp reference picture and sends it to the real-time video stream transmission network module;

[0079] Step S140: The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, processes the video signal based on the network transmission video stream, and then sends it to the real-time video stream terminal signal monitoring module;

[0080] Step S150: The real-time video stream terminal signal monitoring module receives the video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp to be tested picture based on the video signal and displays it on the same display device as the timestamp reference picture;

[0081] Step S160: The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp test picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage;

[0082] In step S170, the real-time video stream delay signal processing module retrieves the picture to be processed from the delayed video data storage module, identifies the reference timestamp and the timestamp to be measured in the picture to be processed based on the character recognition submodule, and calculates the real-time video delay based on the reference timestamp and the timestamp to be measured based on the delay calculation submodule.

[0083] In an exemplary embodiment of the present disclosure, a method for calculating real-time video delay is provided. The method includes: simultaneously displaying a timestamp reference image and a timestamp test image based on a real-time video stream terminal signal monitoring module, and calculating the real-time video delay using a real-time video stream delay signal processing module. This method replaces the manual photography method with a more intelligent and automated calculation solution, ensuring both the universality of engineering applications and the ease of use of the delay test system. It also eliminates the high error associated with manual intervention and improves the intuitiveness of delay data testing results.

[0084] Next, a method for calculating real-time video delay in this exemplary embodiment will be further described.

[0085] Example 1:

[0086] In step S110, a time signal may be generated based on the digital clock generation module, and the clock signal may be sent to the real-time video stream terminal signal monitoring module.

[0087] In step S120, the real-time video stream terminal signal monitoring module may receive the time signal sent by the digital clock generating module, and generate and display a timestamp reference picture based on the time signal.

[0088] In step S130, the real-time video stream signal acquisition module may acquire the timestamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the timestamp reference picture and send it to the real-time video stream transmission network module.

[0089] In step S140, the real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, processes the video signal based on the network transmission video stream, and sends it to the real-time video stream terminal signal monitoring module.

[0090] In step S150, the real-time video stream terminal signal monitoring module can receive the video signal collected by the real-time video stream signal acquisition module and sent by the real-time video stream transmission network module after processing the video stream based on the network transmission, and generate a timestamp to be tested picture based on the video signal and display it on the same display device as the timestamp reference picture.

[0091] In step S160, the real-time video stream delayed signal processing module can capture the timestamp reference picture and the timestamp test picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generate a picture to be processed and send it to the delayed video data storage module for storage.

[0092] In step S170, the real-time video stream delay signal processing module can retrieve the to-be-processed picture from the delayed video data storage module, identify the reference timestamp and the to-be-tested timestamp in the to-be-processed picture based on the character recognition submodule, and calculate the real-time video delay based on the reference timestamp and the to-be-tested timestamp based on the delay calculation submodule.

[0093] In this exemplary embodiment, the method further includes:

[0094] The first time generation submodule based on the digital clock generation module generates a first time signal, the second time generation submodule based on the digital clock generation module generates a second time signal, and sends the first time signal and the second time signal to the real-time video stream terminal signal monitoring module;

[0095] The real-time video stream terminal signal monitoring module receives the first time signal and the second time signal sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the first time signal and the second time signal, wherein the first time signal and the second time signal in the timestamp reference picture are displayed at a first preset position and a second preset position on the display device respectively;

[0096] The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module;

[0097] The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module;

[0098] The real-time video stream terminal signal monitoring module receives a video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp test picture based on the video signal and displays it on the same display device as the timestamp reference picture, wherein the timestamp test picture includes a first time signal to be tested and a second time signal to be tested, and the first time signal to be tested and the second time signal to be tested are displayed at a third preset position and a fourth preset position on the display device;

[0099] The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage;

[0100] The real-time video stream delay signal processing module retrieves the picture to be processed from the delayed video data storage module, and based on the character recognition submodule, respectively identifies the reference first timestamp, the first timestamp to be measured, the reference second timestamp, and the second timestamp to be measured in the picture to be processed, and based on the delay calculation submodule, calculates the real-time video delay according to the reference first timestamp, the first timestamp to be measured, the reference second timestamp, and the second timestamp to be measured.

[0101] In this exemplary embodiment, the method further includes:

[0102] The stopwatch time generation submodule based on the digital clock generation module generates a stopwatch time signal, the millisecond time generation submodule based on the digital clock generation module generates a millisecond time signal, and sends the stopwatch time signal and the millisecond time signal to the real-time video stream terminal signal monitoring module;

[0103] The real-time video stream terminal signal monitoring module receives the stopwatch time signal and the millisecond time signal sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the stopwatch time signal and the millisecond time signal, wherein the stopwatch time signal and the millisecond time signal in the timestamp reference picture are displayed at a first preset position and a second preset position on the display device respectively;

[0104] The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module;

[0105] The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module;

[0106] The real-time video stream terminal signal monitoring module receives a video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp test picture based on the video signal and displays it on the same display device as the timestamp reference picture, wherein the timestamp test picture includes a stopwatch time signal to be tested and a millisecond time signal to be tested, and the stopwatch time signal to be tested and the millisecond time signal to be tested are displayed at a third preset position and a fourth preset position on the display device;

[0107] The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage;

[0108] The real-time video stream delay signal processing module retrieves the picture to be processed from the delayed video data storage module, and based on the character recognition submodule, respectively identifies the reference stopwatch timestamp, the stopwatch timestamp to be measured, the reference millisecond timestamp, and the millisecond timestamp to be measured in the picture to be processed, and based on the delay calculation submodule, calculates the real-time video delay according to the reference stopwatch timestamp, the stopwatch timestamp to be measured, the reference millisecond timestamp, and the millisecond timestamp to be measured.

[0109] In this exemplary embodiment, the method further includes:

[0110] The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module based on a preset time interval to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module;

[0111] In this exemplary embodiment, the method further includes:

[0112] The real-time video stream delay signal processing module retrieves a preset number of to-be-processed frames from the delay video data storage module, identifies reference timestamps and to-be-tested timestamps in the preset number of to-be-processed frames based on the character recognition submodule, and calculates the real-time video delay based on the reference timestamps and to-be-tested timestamps based on the delay calculation submodule;

[0113] An average value of the real-time video delays corresponding to the preset number of frames to be processed is calculated as a video delay calculation result.

[0114] In this exemplary embodiment, the method further includes:

[0115] N time generation submodules based on the digital clock generation module respectively generate N time signals, and send the N time signals to the real-time video stream terminal signal monitoring module, where N is a positive integer greater than or equal to 1;

[0116] The real-time video stream terminal signal monitoring module receives the N time signals sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the N time signals, wherein the N time signals in the timestamp reference picture are respectively displayed at a 2N-1th preset position on the display device;

[0117] The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module;

[0118] The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module;

[0119] The real-time video stream terminal signal monitoring module receives a video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp test picture based on the video signal and displays it on the same display device as the timestamp reference picture, wherein the timestamp test picture includes the Nth time signal to be tested, and the Nth time signal to be tested is displayed at the 2Nth preset position on the display device;

[0120] The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage;

[0121] The real-time video stream delay signal processing module retrieves the to-be-processed picture from the delay video data storage module, identifies the reference Nth timestamp and the to-be-tested Nth timestamp in the to-be-processed picture based on the character recognition submodule, and calculates the real-time video delay based on the reference Nth timestamp and the to-be-tested Nth timestamp based on the delay calculation submodule;

[0122] Calculate an average value of the real-time video delay calculated based on the reference Nth timestamp and the Nth timestamp to be measured as the video delay calculation result.

[0123] In this exemplary embodiment, the method further includes:

[0124] The real-time video stream delayed signal processing module retrieves the to-be-processed picture from the delayed video data storage module, and identifies the reference Nth timestamp and the to-be-tested Nth timestamp in the to-be-processed picture based on the character recognition submodule;

[0125] When the character recognition submodule identifies that there is a ghosting phenomenon in the reference Nth timestamp and the Nth timestamp to be tested, discard the reference Nth timestamp / the Nth timestamp to be tested and its corresponding Nth timestamp to be tested / the reference Nth timestamp;

[0126] The delay calculation submodule calculates the real-time video delay according to the reference Nth timestamp and the Nth timestamp to be measured;

[0127] Calculate a mode value of the real-time video delay calculated based on the reference Nth timestamp and the Nth timestamp to be measured as the video delay calculation result.

[0128] Example 2:

[0129] The real-time video delay measurement method provided by an embodiment of the present invention begins with the camera's sampling phase, captures the reference image through the digital clock signal generating device, and continues until the output image to be measured is displayed on the real-time video monitoring device. This encompasses all stages of video stream transmission, including acquisition, processing, encoding, packaging, transmission, transcoding, distribution, decoding, and playback. The reference image can be considered a captured image of the target scene; the image to be measured can be considered the image of the target scene displayed on the real-time video monitoring device after being encoded and decoded by the camera and the real-time video transmission network. Ideally, both images would have the same digital clock value, or equivalent timestamp, at the same moment. However, since the video stream signal is transmitted from the input to the output through the complete network transmission process described above, network jitter may occur during this process, and the encoding and decoding of the video stream also takes a certain amount of time, resulting in inevitable delay. Depending on the performance of the real-time video transmission device or the impact of network transmission performance during transmission, the resulting delay can range from tens of milliseconds to tens of seconds. To ensure real-time feedback for users viewing real-time video streams, delay measurement is essential. The measurement method adopted in the embodiment of the present invention monitors and measures the real-time video stream acquisition process from the first stage to the real-time video stream playback process in the final stage, which can ensure that the tested delay data includes the complete process of real-time video stream transmission, thereby ensuring the accuracy of the measurement results.

[0130] Currently, applications of the stopwatch photography method focus on capturing a single digital stopwatch playback time snapshot within a single display area, transmitting it, and then displaying it in another display area. This other display area may be on the same monitor as the digital stopwatch generator or on a different monitor. Due to varying camera shutter parameter settings and monitor display effects from different real-time video stream network transmission devices, the millisecond-level digits of the previous moment may appear just as the next moment appears, resulting in ghosting of the time snapshot. In this case, manual recognition or optical character recognition is performed between the reference image and the image to be measured, and the difference between the recognition results is calculated to obtain real-time delay data. However, in this delay measurement scenario, measuring the delay performance of real-time video streams can result in certain errors.

[0131] To improve the accuracy of real-time video stream delay measurement, avoid or at least reduce the impact of ghosting on real-time delay measurement results, and replace manual operations with intelligently controlling the sampling rate of real-time video stream delay data, embodiments of the present invention provide an intelligent, automated method for real-time video delay measurement in a monitoring system. First, a real-time video delay performance measurement system must be established. A computer monitor can be used as the real-time video stream terminal signal monitoring device. For ease of description and understanding, the real-time video stream terminal signal monitoring device will be referred to as the PC monitor. A digital clock generator is run within the PC monitor, which displays the current millisecond time. The captured image is referred to as the first reference image and can be used as a reference to a digital clock operating with millisecond accuracy on the Internet. It is worth noting that the current time in any time zone does not affect the delay measurement results of embodiments of the present invention. Therefore, the current time in any time zone can be selected as the first reference time. A digital stopwatch generator is also run, which displays the current millisecond time elapsed by the digital stopwatch. The captured image is referred to as the second reference image and can be used as a reference to a digital stopwatch operating with millisecond accuracy on the Internet. The significance of choosing a digital clock and a digital stopwatch is that the timestamps displayed by the two are different, which makes it easier to compare the processed delay data. As an alternative embodiment, two digital stopwatch generating devices can also be used to generate timestamps, and the same effect can be achieved by running the two digital stopwatch generating devices separately. A network camera can be used as the real-time video stream acquisition device. For the sake of ease of description and understanding, the real-time video stream acquisition device will be referred to as a camera below. Aim the camera at the first reference picture and the second reference picture of the PC monitor. The first reference picture and the second reference picture can be taken as follows Figure 2A The arrangement shown can also be Figure 2B In the arrangement shown, the obtained first test frame and the second test frame can be placed on the other side of the PC according to the arrangement of the first reference frame and the second reference frame.

[0132] The digital clock generator and the digital stopwatch generator operate sequentially, continuously generating dynamic timestamps. A camera captures the first and second reference frames, which are then transmitted via a real-time video streaming network. After transmission is complete, the first and second frames to be measured are displayed on opposite sides of the same PC monitor. Four timestamp values ​​are displayed on the PC monitor: the current timestamp generated by the digital clock generator, the digital clock timestamp output after transmission, the stopwatch timestamp generated by the digital stopwatch generator, and the stopwatch timestamp output after transmission. For convenient and accurate subsequent processing, the display of the timestamp values ​​must meet the following two conditions: Condition 1: The four timestamp values ​​must have a high contrast with their background on the PC monitor, meaning they must stand out against the background color. Similar colors can cause unnecessary recognition and calculation errors in subsequent processing. Condition 2: The PC monitor should avoid displaying any irrelevant characters other than the four timestamp values. This ensures that no interfering characters other than the timestamps are recognized during the subsequent recognition process, as this can cause unnecessary recognition and calculation errors in the delay measurement results.

[0133] After the four timestamp values ​​are successfully displayed on the PC monitor, the real-time video stream delay signal processing device and the delay data storage device are run. The real-time video stream delay signal processing device includes two parts. Part one is a module for automatically taking screenshots of the PC display screen, wherein the interval time of the automatic screenshot is adjustable. The user can input the interval value parameter according to their own needs to adjust the interval time of the automatic screenshot. Every time the set interval time is reached, the automatic screenshot module will take a screenshot sample of the monitoring screen of the PC monitor and feed the screenshot result into the delay data storage device as soon as possible. Part two of the real-time video stream delay signal processing device is an optical character recognition module. For the convenience of description and understanding, this module will be referred to as the OCR module below. The OCR module performs optical character recognition and character segmentation operations on the screenshot result taken out from the delay data storage device. Among them, the delayed data storage device is set as a first-in-first-out queue, that is, the screenshot sampling result that first enters the storage device will be processed with absolute priority compared to the screenshot sampling result that enters the storage device later. This ensures that the sampling order and processing order of the reference picture and the picture to be tested are consistent, and does not cause the problem of identifying the delay result of the latter moment as the delay result of the previous moment.

[0134] In the embodiment of the present invention, each time the OCR module performs optical character recognition and character segmentation, four different timestamp values ​​will be output. These four different timestamp values ​​will be put into a two-dimensional array and output. Figure 2A and Figure 2BThe different arrangements of the reference picture and the picture to be tested require pairing of four time stamp values. Figure 2A The output two-dimensional array is [[first reference picture, second reference picture], [first test picture, second test picture]]; if the arrangement is arranged according to Figure 2B If the arrangement shown is performed, the output two-dimensional array is [[first reference picture, first test picture], [second reference picture, second test picture]]. The present invention does not limit the arrangement of the reference pictures and the test pictures. It is only important to understand that different arrangements will produce different results in the final output two-dimensional array of timestamps. Different value pairings should be selected according to different arrangements for subsequent calculations.

[0135] After the recognition and segmentation are completed in the previous step and the timestamp two-dimensional array is output, Figure 2A The timestamp two-dimensional array A[[first reference picture, second reference picture], [first test picture, second test picture]] obtained by the arrangement shown in the figure, when performing the difference operation, A[0,0] and A[1,0] should be selected as a group for calculation; A[0,1] and A[1,1] should be selected as a group for calculation, that is, [first reference picture, first test picture] are extracted as one group, and [second reference picture, second test picture] are extracted as another group; for Figure 2B For the two-dimensional array of timestamps B[[first reference picture, first picture to be tested], [second reference picture, second picture to be tested]] obtained by the arrangement shown, when performing the difference operation, B[0,0] and B[0,1] should be selected as a group for calculation, and B[1,0] and B[1,1] should be selected as a group for calculation. Similarly, [first reference picture, first picture to be tested] are extracted as one group, and [second reference picture, second picture to be tested] are extracted as another group.

[0136] In other alternative arrangements included in the embodiments of the present invention, there may be an arrangement similar to [[first test picture, first reference picture], [second test picture, second reference picture]]. Due to the existence of delay, the timestamp of the test picture must be delayed after the timestamp of the reference picture. Therefore, in this case, directly subtracting the latter from the former will result in a negative value. In order to avoid the occurrence of negative delay results and cause ambiguity to users, whether it is the form provided by the embodiments of the present invention, Figure 2A or Figure 2B , or the alternative arrangement mentioned in the embodiment of the present invention, the absolute value of the difference between the reference picture and the picture to be tested is taken, and the obtained value is the delay result data.

[0137] In the measurement system proposed in the embodiments of the present invention, even though a more optimal sampling method is proposed to replace manual photography, ghosting will no longer occur in the timestamps generated by the array clock generation device. However, due to the varying performance of real-time video streaming network devices in different production environments, ghosting may still occur to varying degrees depending on the quality of video transmission. Generally speaking, while the probability of ghosting has been reduced, it is still unavoidable even if other technical solutions, such as color block changes, are employed instead of a digital clock timestamp generation device. It should be noted that ghosting occurs because the timestamp value of the previous time has not yet disappeared when the timestamp of the subsequent time is displayed. The fact that the timestamp of the subsequent time is displayed at the moment of sampling indicates that the current sampling time is either the subsequent time or closer to the subsequent time. Therefore, regardless of whether the OCR device identifies the previous or subsequent time, the result with a smaller difference from the true reference timestamp is closer to the current accurate delay time. Therefore, the smaller value between [first reference time, first time to be measured] and [second reference time, second time to be measured] is taken as the delay measurement result data in this test. The error value between this result and the actual delay data at the current moment is smaller.

[0138] Based on the description of the above embodiments of the present invention, the first concept that needs to be clarified is that the occurrence of ghosting is random. For a scenario in which multiple sets of reference images and images to be tested appear on a PC monitor at the same time, after the real-time video stream delay signal processing device performs a screenshot sampling operation, if ghosting occurs, not all images to be tested will have ghosting at the same time. Instead, depending on the performance of the real-time video stream transmission network device to be tested currently in use, a timestamp where the ghosting occurs will randomly appear among all timestamps of the images to be tested.

[0139] Based on this, in order to further reduce the error value between the delay measurement result and the actual delay time, the present invention provides a more optimal embodiment for reference. When a set of [reference picture, picture to be measured] screenshot results have a ghosting phenomenon, resulting in a large error between the delay measurement result obtained by calculating the difference and the actual delay data, there is no corresponding error correction scheme to eliminate or at least reduce the error. It can be determined that whether it is a digital clock timestamp generated by a digital clock generating device or a digital stopwatch timestamp generated by a digital stopwatch generating device, even if all timestamp generating devices are not running at the same time, during the operation process, the time scale recorded by each timestamp generating device is absolutely the same. In other words, within one second, the time recorded by all timestamp generating devices will pass through one second. Therefore, in the same screenshot sampling result, the actual delay time of all [reference picture, picture to be measured] groups should be completely equal. Therefore, in the first more optimal embodiment of the present invention, it can be set as follows Figure 3The multiple sets of [reference picture, test picture] are arranged as shown. Alternatively, other arrangements of the multiple sets of [reference picture, test picture] can also be designed. The embodiment of the present invention does not limit the arrangement. It should only be noted that for timestamp pairs with different arrangements, the dimensions of the multidimensional arrays obtained after passing through the real-time video stream delay signal processing device and the delay data storage device are different. When calculating the difference, the corresponding [reference picture, test picture] timestamp group should be extracted for calculation.

[0140] After setting up the real-time video stream delay calculation system, n digital stopwatch generation devices (for example, 20) are sequentially run on the PC monitor. Without changing the overall structure of the real-time video stream delay calculation system, the 20 digital stopwatch timestamps are transmitted to the real-time video stream transmission network device. Finally, the corresponding 20 time-stamped images to be measured are displayed on another portion of the PC monitor. Similarly, 20 sets of [reference images, measured images] are fed into the real-time video stream delay signal processing device. After automatic screenshot capture, optical character recognition, and difference calculation modules, 20 real-time video stream delay measurement results are obtained. According to the aforementioned theory, the actual delay time corresponding to all delay results should be equal. Therefore, taking the mode of the 20 values ​​can ensure the accuracy of the result while discarding bad values ​​that generate erroneous results due to ghosting. This allows the delay measurement result with the smallest error from the actual delay value to be determined.

[0141] In this preferred embodiment of the present invention, the number of digital stopwatch generation devices designed has no impact on the performance of the real-time video stream transmission network device and the delayed data storage device under test, and has essentially no impact on the performance of the real-time video stream terminal signal monitoring device and the real-time video stream delay signal processing device. Therefore, the number of reference images can be dynamically adjusted based on the performance of the real-time video stream transmission network device under test in a production environment to maximize the accuracy of real-time video stream delay measurement.

[0142] Based on the technical concepts proposed in this invention, in actual production environments, users or viewers of real-time video do not care about millisecond-level changes in delay time. Generally speaking, the delay time should be dynamically updated at regular intervals, such as once per second. This ensures that users do not experience visual fatigue when viewing the real-time delay of the video stream, and also ensures that the delay data is displayed to users or viewers immediately.

[0143] Based on this, the present invention proposes a second, more preferred embodiment for reference. While the structure of the real-time video stream delay measurement system proposed in the present invention remains unchanged, it is known that the larger the number of sampled samples, the closer their average value is to the average value of the actual delay data. Based on the technical feature of the real-time video stream delay signal processing device proposed in the present invention, which has an adjustable automatic screenshot frequency, assuming that the delay time is ultimately displayed to the user once every second, in one possible embodiment, the automatic screenshot sampling frequency can be set to perform a screenshot operation every 10 milliseconds. After a successful screenshot, the screenshot result is immediately entered into the delay data storage device. In this way, 100 screenshot results can be obtained within one second. These 100 results are then simultaneously retrieved from the delay data storage device and entered into the OCR module of the real-time video stream delay signal processing device for optical character recognition. The difference calculation module calculates the delay time of each screenshot sample, ultimately obtaining 100 delay result data within one second. The average value of these 100 delay result data values ​​is as close to the actual delay average value within that second as possible.

[0144] By maximizing the average sample size within a unit of time, we can approximate the actual latency data within that unit of time. This can further reduce the impact of ghosting on the accuracy of latency measurement results. The resulting real-time video stream latency results presented to users or live video viewers are both consistent with human viewing habits and provide more accurate latency data.

[0145] In view of the real production scenario based on the second preferred embodiment described above, when the real-time video stream delay data is updated once per unit time for users or video stream viewers, in order to obtain a more accurate delay result, the present invention proposes an alternative embodiment of the second preferred embodiment. Taking the example of updating the delay data once per second, on the basis of the first embodiment proposed by the present invention, the sources of the first reference picture and the second reference picture can be set to the same digital clock generating device, so as to ensure that the timestamp values ​​of the first reference picture and the second reference picture are completely consistent. Similarly, the purpose of using two reference pictures generated by the same digital clock generating devices is only to obtain the same reference picture timestamp every time the screenshot is sampled. Therefore, this embodiment does not impose any restrictions on the time zone of the digital clock generating device, and only needs to ensure that the time zones of the two digital clock generating devices are consistent.

[0146] Assuming the same scenario of capturing 100 snapshots of the combined playback of the reference and test images within one second, 100 sets of [reference image, test image] timestamps within that second can be obtained. The acquisition method has been detailed previously and will not be repeated here. When the timestamps of the first reference image and the second reference image are absolutely equal, theoretically, the first test image and the second test image in the same screenshot should also have the same timestamp value. That is, the delay values ​​of [first reference image, first test image] and [second reference image, second test image] should be equal. If the delay values ​​are unequal, this indicates that one or both of the results experienced ghosting during optical character recognition, resulting in an erroneous timestamp value. Unequal timestamp values ​​can be discarded in the real-time video stream delay signal processing device. Each of the remaining 100 accurate timestamp values ​​should be the true delay value for the corresponding sampling moment. Taking the average of these values ​​yields the true average delay value for that second.

[0147] It should be noted that although the steps of the method disclosed herein are depicted in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in that particular order, or that all steps must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one, and / or one step may be decomposed into multiple steps.

[0148] In addition, in this exemplary embodiment, a real-time video delay calculation device is also provided. Figure 4 As shown, the real-time video delay calculation device 400 may include: a digital clock generation module 410, a real-time video stream terminal signal monitoring module 420, a real-time video stream signal acquisition module 430, a real-time video stream transmission network module 440, a real-time video stream delay signal processing module 450, and a delayed video data storage module 460.

[0149] A digital clock generation module 410, which is integrated into the real-time video stream terminal signal monitoring module and is used to generate a time signal and send the clock signal to the real-time video stream terminal signal monitoring module;

[0150] A real-time video stream terminal signal monitoring module 420 is configured to receive a time signal sent by the digital clock generation module, and generate and display a timestamp reference picture based on the time signal; the real-time video stream terminal signal monitoring module is further configured to receive a video signal collected by the real-time video stream signal acquisition module and sent by the real-time video stream transmission network module after processing the video stream based on network transmission, and generate and display a timestamp to-be-tested picture based on the video signal on the same display device as the timestamp reference picture;

[0151] A real-time video stream signal acquisition module 430 is configured to acquire a timestamp reference picture displayed in the real-time video stream terminal signal monitoring module, generate a video signal containing the timestamp reference picture, and send the video signal to the real-time video stream transmission network module;

[0152] A real-time video stream transmission network module 440 is configured to receive the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, perform network-based video stream processing on the video signal, and then send the video signal to the real-time video stream terminal signal monitoring module;

[0153] A real-time video stream delay signal processing module 450 is configured to capture the timestamp reference image and the timestamp to-be-tested image in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generate a to-be-processed image, and send it to the delayed video data storage module; the real-time video stream delay signal processing module is further configured to retrieve the to-be-processed image from the delayed video data storage module, and identify the reference timestamp and the to-be-tested timestamp in the to-be-processed image based on the character recognition submodule; the real-time video stream delay signal processing module is further configured to calculate the real-time video delay based on the reference timestamp and the to-be-tested timestamp based on the delay calculation submodule;

[0154] The delayed video data storage module 460 is used to receive and store the to-be-processed images sent by the real-time video stream delayed signal processing module.

[0155] The specific details of each of the above-mentioned real-time video delay calculation device modules have been described in detail in the corresponding real-time video delay calculation method, and will not be repeated here.

[0156] It should be noted that although the above detailed description mentions several modules or units of a real-time video delay calculation device 400, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in a single module or unit. Conversely, the features and functions of a single module or unit described above can be further divided and embodied by multiple modules or units.

[0157] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0158] Those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods, or program products. Accordingly, various aspects of the present invention may be implemented as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or a combination of hardware and software embodiments, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0159] Refer to the following Figure 5 An electronic device 500 according to such an embodiment of the present invention will be described. Figure 5 The electronic device 500 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0160] like Figure 5 As shown, electronic device 500 is implemented as a general-purpose computing device. Components of electronic device 500 may include, but are not limited to, the aforementioned at least one processing unit 510, the aforementioned at least one storage unit 520, a bus 530 connecting various system components (including storage unit 520 and processing unit 510), and a display unit 540.

[0161] The storage unit stores program codes, which can be executed by the processing unit 510, so that the processing unit 510 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 510 can perform the following steps: Figure 1 Steps S110 to S170 shown in FIG.

[0162] The storage unit 520 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 5201 and / or a cache memory unit 5202 , and may further include a read-only memory unit (ROM) 5203 .

[0163] The storage unit 520 may also include a program / utility 5204 having a set (at least one) of program modules 5203, such program modules 5205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0164] Bus 550 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0165] The electronic device 500 can also communicate with one or more external devices 570 (e.g., a keyboard, a pointing device, a Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 500, and / or any device that enables the electronic device 500 to communicate with one or more other computing devices (e.g., a router, a modem, etc.). Such communication can occur via an input / output (I / O) interface 550. Furthermore, the electronic device 500 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 560. As shown, the network adapter 560 communicates with other modules of the electronic device 500 via the bus 550. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 500, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0166] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0167] In exemplary embodiments of the present disclosure, a computer-readable storage medium is also provided, storing a program product capable of implementing the aforementioned methods of this specification. In some possible embodiments, various aspects of the present invention may also be implemented in the form of a program product comprising program code. When the program product is executed on a terminal device, the program code is configured to cause the terminal device to perform the steps according to various exemplary embodiments of the present invention described in the "Exemplary Methods" section of this specification.

[0168] refer to Figure 6 , a program product 600 for implementing the above-described method according to an embodiment of the present invention is described. The program product 600 may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0169] The program product may be implemented in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0170] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0171] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0172] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0173] Furthermore, the figures above are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the figures above do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0174] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow from the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

[0175] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A real-time video delay calculation method, characterized in that: The method comprises: Generate a time signal based on a digital clock generation module, and send the time signal to a real-time video stream terminal signal monitoring module; The real-time video stream terminal signal monitoring module receives the time signal sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the time signal; The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module; The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module; The real-time video stream terminal signal monitoring module receives the video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp to be tested picture based on the video signal and displays it on the same display device as the timestamp reference picture; The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage; The real-time video stream delay signal processing module retrieves the to-be-processed picture from the delayed video data storage module, identifies the reference timestamp and the to-be-tested timestamp in the to-be-processed picture based on the character recognition submodule, and calculates the real-time video delay based on the reference timestamp and the to-be-tested timestamp based on the delay calculation submodule.

2. The method according to claim 1, wherein The method further comprises: The first time generation submodule based on the digital clock generation module generates a first time signal, the second time generation submodule based on the digital clock generation module generates a second time signal, and sends the first time signal and the second time signal to the real-time video stream terminal signal monitoring module; The real-time video stream terminal signal monitoring module receives the first time signal and the second time signal sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the first time signal and the second time signal, wherein the first time signal and the second time signal in the timestamp reference picture are displayed at a first preset position and a second preset position on the display device respectively; The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module; The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module; The real-time video stream terminal signal monitoring module receives a video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp test picture based on the video signal and displays it on the same display device as the timestamp reference picture, wherein the timestamp test picture includes a first time signal to be tested and a second time signal to be tested, and the first time signal to be tested and the second time signal to be tested are displayed at a third preset position and a fourth preset position on the display device; The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage; The real-time video stream delay signal processing module retrieves the picture to be processed from the delayed video data storage module, and based on the character recognition submodule, respectively identifies the reference first timestamp, the first timestamp to be measured, the reference second timestamp, and the second timestamp to be measured in the picture to be processed, and based on the delay calculation submodule, calculates the real-time video delay according to the reference first timestamp, the first timestamp to be measured, the reference second timestamp, and the second timestamp to be measured.

3. The method according to claim 2, wherein The method further comprises: The stopwatch time generation submodule based on the digital clock generation module generates a stopwatch time signal, the millisecond time generation submodule based on the digital clock generation module generates a millisecond time signal, and sends the stopwatch time signal and the millisecond time signal to the real-time video stream terminal signal monitoring module; The real-time video stream terminal signal monitoring module receives the stopwatch time signal and the millisecond time signal sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the stopwatch time signal and the millisecond time signal, wherein the stopwatch time signal and the millisecond time signal in the timestamp reference picture are displayed at a first preset position and a second preset position on the display device respectively; The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module; The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module; The real-time video stream terminal signal monitoring module receives a video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp test picture based on the video signal and displays it on the same display device as the timestamp reference picture, wherein the timestamp test picture includes a stopwatch time signal to be tested and a millisecond time signal to be tested, and the stopwatch time signal to be tested and the millisecond time signal to be tested are displayed at a third preset position and a fourth preset position on the display device; The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage; The real-time video stream delay signal processing module retrieves the picture to be processed from the delayed video data storage module, and based on the character recognition submodule, respectively identifies the reference stopwatch timestamp, the stopwatch timestamp to be measured, the reference millisecond timestamp, and the millisecond timestamp to be measured in the picture to be processed, and based on the delay calculation submodule, calculates the real-time video delay according to the reference stopwatch timestamp, the stopwatch timestamp to be measured, the reference millisecond timestamp, and the millisecond timestamp to be measured.

4. The method according to claim 1, wherein The method further comprises: The real-time video stream signal acquisition module acquires the timestamp reference picture displayed in the real-time video stream terminal signal monitoring module based on a preset time interval to generate a video signal containing the timestamp reference picture and sends it to the real-time video stream transmission network module.

5. The method according to claim 4, wherein The method further comprises: The real-time video stream delay signal processing module retrieves a preset number of to-be-processed frames from the delay video data storage module, identifies reference timestamps and to-be-tested timestamps in the preset number of to-be-processed frames based on the character recognition submodule, and calculates the real-time video delay based on the reference timestamps and to-be-tested timestamps based on the delay calculation submodule; An average value of the real-time video delays corresponding to the preset number of frames to be processed is calculated as a video delay calculation result.

6. The method according to claim 1, wherein The method further comprises: N time generation submodules based on the digital clock generation module respectively generate N time signals, and send the N time signals to the real-time video stream terminal signal monitoring module, where N is a positive integer greater than or equal to 1; The real-time video stream terminal signal monitoring module receives the N time signals sent by the digital clock generation module, and generates and displays a timestamp reference picture based on the N time signals, wherein the N time signals in the timestamp reference picture are respectively displayed at a 2N-1th preset position on the display device; The real-time video stream signal acquisition module acquires the time stamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the time stamp reference picture and sends it to the real-time video stream transmission network module; The real-time video stream transmission network module receives the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and processes the video signal based on the video stream transmitted over the network and then sends it to the real-time video stream terminal signal monitoring module; The real-time video stream terminal signal monitoring module receives a video signal collected by the real-time video stream signal acquisition module and sent after being processed by the real-time video stream transmission network module based on the video stream transmitted over the network, and generates a timestamp test picture based on the video signal and displays it on the same display device as the timestamp reference picture, wherein the timestamp test picture includes the Nth time signal to be tested, and the Nth time signal to be tested is displayed at the 2Nth preset position on the display device; The real-time video stream delayed signal processing module takes screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generates a picture to be processed and sends it to the delayed video data storage module for storage; The real-time video stream delay signal processing module retrieves the to-be-processed picture from the delay video data storage module, identifies the reference Nth timestamp and the to-be-tested Nth timestamp in the to-be-processed picture based on the character recognition submodule, and calculates the real-time video delay based on the reference Nth timestamp and the to-be-tested Nth timestamp based on the delay calculation submodule; Calculate an average value of the real-time video delay calculated based on the reference Nth timestamp and the Nth timestamp to be measured as the video delay calculation result.

7. The method according to claim 6, wherein The method further comprises: The real-time video stream delayed signal processing module retrieves the to-be-processed picture from the delayed video data storage module, and identifies the reference Nth timestamp and the to-be-tested Nth timestamp in the to-be-processed picture based on the character recognition submodule; When the character recognition submodule identifies that there is a ghosting phenomenon in the reference Nth timestamp and the Nth timestamp to be tested, discard the reference Nth timestamp / the Nth timestamp to be tested and its corresponding Nth timestamp to be tested / the reference Nth timestamp; The delay calculation submodule calculates the real-time video delay according to the reference Nth timestamp and the Nth timestamp to be measured; Calculate a mode value of the real-time video delay calculated based on the reference Nth timestamp and the Nth timestamp to be measured as the video delay calculation result.

8. A real-time video delay calculation device, characterized in that: The device comprises: A digital clock generation module, the digital clock generation module is integrated in the real-time video stream terminal signal monitoring module, the digital clock generation module is used to generate a time signal and send the time signal to the real-time video stream terminal signal monitoring module; A real-time video stream terminal signal monitoring module, the real-time video stream terminal signal monitoring module is used to receive the time signal sent by the digital clock generation module, and generate and display a timestamp reference picture based on the time signal; the real-time video stream terminal signal monitoring module is also used to receive the video signal collected by the real-time video stream signal acquisition module and sent by the real-time video stream transmission network module after processing the video stream based on the network transmission, and generate a timestamp to be tested picture based on the video signal and display it on the same display device as the timestamp reference picture; A real-time video stream signal acquisition module, which is used to acquire the timestamp reference picture displayed in the real-time video stream terminal signal monitoring module to generate a video signal containing the timestamp reference picture and send it to the real-time video stream transmission network module; A real-time video stream transmission network module, the real-time video stream transmission network module is used to receive the video signal containing the timestamp reference picture sent by the real-time video stream signal acquisition module, and send the video signal to the real-time video stream terminal signal monitoring module after performing video stream processing based on network transmission; A real-time video stream delay signal processing module, wherein the real-time video stream delay signal processing module is used to take screenshots of the timestamp reference picture and the timestamp to-be-tested picture in the display device of the real-time video stream terminal signal monitoring module based on the screenshot submodule, generate a picture to be processed and send it to the delayed video data storage module; the real-time video stream delay signal processing module is also used to retrieve the picture to be processed from the delayed video data storage module, and respectively identify the reference timestamp and the timestamp to-be-tested in the picture to be processed based on the character recognition submodule; the real-time video stream delay signal processing module is also used to calculate the real-time video delay based on the reference timestamp and the timestamp to-be-tested based on the delay calculation submodule; The delayed video data storage module is used to receive and store the to-be-processed images sent by the real-time video stream delayed signal processing module.

9. An electronic device, characterized in that: include processor; and A memory having computer-readable instructions stored thereon, wherein the computer-readable instructions are executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by a processor.

Images