Dynamic video transmission system based on real-time performance optimization detection

Abstract

The present invention relates to a video transmission system based on real-time performance optimization detection, which includes: a first transmission link, in which a distributed system encoder encodes an image into a network stream signal, which is decoded by a distributed system decoder to extract the image frame by frame, and each frame of the image is subjected to OCR recognition to identify the timestamp on the image, and the corresponding timestamp when decoding the image frame is recorded, and the difference between each timestamp related to the same image is calculated; a second transmission link, in which a desktop screen capture client encodes an image containing a timestamp into a network stream signal, which is decoded by a centralized system processor, and the image is extracted frame by frame during the decoding process, and each frame is subjected to OCR recognition to identify the timestamp on the image, and the timestamp when decoding the image frame is recorded, and the difference between each timestamp related to the same image is calculated; a comprehensive management and control platform, which switches the transmission link. The present invention can maximize the capabilities of the video transmission system.

Description

Technical Field

[0001] The invention relates to a video transmission system, in particular to a video transmission system based on real-time performance optimization detection. Background Art

[0002] In the prior art, the video transmission process of the video transmission system involves steps such as encoding, network transmission, decoding, and splicing processing. The connection method between the various components in the video transmission system is fixed, and the achievable transmission methods or capabilities are also very fixed. Summary of the invention

[0003] In view of the above existing technical problems, the present application proposes a video transmission system based on real-time performance optimization detection, which includes:

[0004] The first transmission link includes a distributed system encoder, a display control network switch, a distributed system decoder and a display device; the distributed system encoder is connected to a video signal source through a video line to obtain an image displayed thereon and encode the image to form a first network stream signal; the distributed system decoder is used to decode the first network stream signal, extract the image frame by frame during the decoding process, perform OCR recognition on each frame of the image to identify the timestamp on the image, and record the timestamp corresponding to the decoding when decoding the image frame, and calculate the difference between each timestamp related to the same image in the first transmission chain;

[0005] A second transmission link, comprising a desktop screen capture client, a data network switch, a centralized system processor and the display device, wherein the desktop screen capture client is installed on a video signal source and presents a timestamp on the video signal source, and encodes the image with the timestamp into a second network stream signal; the centralized system processor is used to decode the second network stream signal, extract the image frame by frame during the decoding process, perform OCR recognition on each frame to identify the timestamp on the image, record the timestamp when decoding the image frame, and calculate the difference between each timestamp related to the same image in the second transmission chain;

[0006] An integrated management and control platform, which is used to simultaneously access the display and control network switch and the data network switch through the core network switch, obtain the timestamp information of each of the first transmission link and the second transmission link and the difference between the timestamps in real time, and compare them, and switch the transmission link to the first transmission link or the second transmission link that is most suitable for video transmission at the moment according to the comparison result;

[0007] The display device has a first video interface and a second video interface, which are respectively used to connect to a distributed system decoder and a centralized system processor, and the display device can freely switch between the first video interface and the second video interface.

[0008] According to some embodiments of the present invention, encoding the image to form a first network stream signal also includes providing the video image frame with a second timestamp; the distributed system decoder is also used to identify the second timestamp on the image during decoding, and to record the corresponding timestamp during decoding when decoding the image frame, and to calculate the difference between each timestamp related to the same image in the first transmission chain.

[0009] According to some embodiments of the present invention, the desktop screen capture client timestamps the SEI frame of the video image in the video stream of the video signal source through the H.264 video coding standard; the distributed system encoder, the distributed system decoder and the centralized system processor extract the timestamp of the SEI frame and calculate the difference between the timestamp and the timestamp when the SEI frame is extracted.

[0010] According to some embodiments of the present invention, timestamping a SEI frame of a video image in a video stream of a video signal source includes:

[0011] Use encoding tools to analyze the video stream and determine the location of SEI frames;

[0012] Write a script that reads a video file and inserts a timestamp when it finds a SEI frame.

[0013] Determine the format of the timestamp;

[0014] In the script, when a SEI frame is detected, the current timestamp is inserted into the video frame in the specified format;

[0015] Repackage the modified video frames into a new video file.

[0016] According to some embodiments of the present invention, the video transmission system further includes:

[0017] The third transmission link includes the distributed system encoder and display control network switch in the first transmission link, the centralized system processor and display device in the second transmission link, and the core switch; the distributed system encoder is used to encode the image of the video signal source to form a third network stream signal; the centralized system processor is used to decode the third network stream signal, and extract the image frame by frame and perform OCR recognition on each frame during the decoding process to identify the time stamp on the image, and record the corresponding time stamp when decoding the image frame, and calculate the difference between each time stamp related to the same image in the third transmission chain; and

[0018] A fourth transmission link, comprising a desktop screen capture client and a data network switch in the second transmission link, a distributed system decoder and a display device in the first transmission link, and a core switch; the desktop screen capture client is installed on a video signal source and presents a timestamp on the video signal source, and encodes an image of the video signal source with the timestamp into a fourth network stream signal; the distributed system decoder is used to decode the fourth network stream signal, extract images frame by frame during the decoding process, perform OCR recognition on each frame to identify the timestamp on the image, and record the timestamp when decoding the image frame, and calculate the difference between each timestamp related to the same image in the fourth transmission chain;

[0019] The integrated management and control platform is also used to obtain each timestamp in the third transmission link and the fourth transmission link and the difference between the timestamps, and compare them, and switch the transmission link to the third transmission link or the fourth transmission link that is most suitable for video transmission at the moment according to the comparison result.

[0020] According to some embodiments of the present invention, the distributed system encoder is also used to encode the video image containing the timestamp obtained in the third transmission link, so that the video image frame encoded here is further provided with a third timestamp; the centralized system processor is also used to identify the third timestamp on the image during decoding, and to record the corresponding timestamp during decoding when decoding the image frame, and to calculate the difference between each timestamp related to the same image in the third transmission chain.

[0021] According to some embodiments of the present invention, the integrated management and control platform is further used to calculate the difference between the maximum delay and the minimum delay on each transmission link, and to switch one of the transmission links with a relatively larger value of the difference to another transmission link.

[0022] According to some embodiments of the present invention, the integrated management and control platform is also used to pre-count the differences between the timestamps when each transmission link transmits images of video signal sources of different sizes, establish a correspondence between different transmission links, images of video signal sources of different sizes and the differences between the timestamps, and select a suitable transmission link according to the size of the image of the video signal source, the correspondence and the comparison result.

[0023] According to some embodiments of the present invention, the integrated management and control platform is also used to pre-count the differences between the timestamps when images of video signal sources of different sizes are transmitted in different time periods of each transmission link, establish a correspondence between the images of video signal sources of different sizes and the timestamp differences in different transmission links, different time periods, and different sizes, and select a suitable transmission link according to the transmission time period, the size of the image of the video signal source, the correspondence and the comparison result.

[0024] According to some embodiments of the present invention, the integrated management and control platform is also used to determine whether the centralized system processor is operating at full capacity and whether the video signal needs to be processed before output based on the monitored delay associated with the centralized system processor, wherein the processing includes controlling the video signal source to reduce its image resolution before output, or reducing its output frame rate.

[0025] The present invention can overcome the limitations of the prior art and provide a more flexible, efficient, reliable, easily expandable and low-cost video transmission solution. Specifically:

[0026] High fault tolerance: When a node fails or a link is interrupted, the system can automatically switch to a backup link or reconfigure the link to ensure the continuity and stability of transmission.

[0027] Easy to expand: Adaptive systems can more easily integrate new devices and services, supporting the expansion of system scale.

[0028] Reduced maintenance costs: Since the system can automatically adjust the configuration, the need for human intervention is reduced, thereby reducing maintenance costs.

[0029] Improve adaptability: The system can automatically select the most appropriate transmission strategy based on different types of video content or transmission requirements.

[0030] Improve user experience: By optimizing the transmission link, the system can provide higher quality video transmission services and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A video transmission system based on real-time performance optimization detection according to some embodiments of the present invention is shown;

[0032] Figure 2a Show Figure 1 A partial schematic diagram of the system shown;

[0033] Figure 2b Show Figure 1 A partial schematic diagram of the system shown;

[0034] Figure 3 A flow chart of a dynamic video transmission method based on real-time performance optimization detection according to some embodiments of the present invention is shown;

[0035] Figure 4 A module diagram showing a comprehensive management device according to some embodiments of the present invention;

[0036] Figure 5 A schematic structural diagram of a comprehensive management device according to some embodiments of the present invention is shown. DETAILED DESCRIPTION

[0037] In this application, all references to implementation through a network are essentially intended to cover wired or wireless network connections implemented through necessary firmware or software such as switches and routers, and also cover wired or wireless network connections implemented through some server terminals or other computers as intermediaries. In order to simplify the description and highlight the invention, this application sometimes omits the description of routers / switches when describing network connections.

[0038] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to enable the scope of the present disclosure to be fully conveyed to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined and referenced with each other.

[0039] Figure 1 A topology diagram of a video transmission system based on real-time performance optimization detection according to some embodiments of the present invention is shown. Figure 1 The reference numerals in the accompanying drawings correspond to the following: 1 network cable; 2 network cable; 3 video cable; 4 video cable; 5 network cable; 6 network cable; 7 network cable; 8 optical fiber cable: 9 optical fiber cable; 11 network cable; 12 video cable; 10 video signal source; 101 distributed system encoder; 102 distributed system decoder; 103 display control network switch; 202 centralized system processor; 203 data network switch; 401 display device.

[0040] like Figure 1 As shown in , the system includes a first transmission link, a second transmission link and an integrated management and control platform, wherein the first transmission link and the second transmission link are identified by L1 and L2.

[0041] The first transmission link includes a distributed system encoder, a display control network switch, a distributed system decoder and a display device; the distributed system encoder is connected to the video signal source through a video line to obtain the image displayed thereon and encode it to form a first network stream signal. Therefore, the image obtained by the distributed system encoder is the image finally presented on the video signal source, that is, the image containing timestamp information mentioned in the next paragraph; the distributed system decoder is used to decode the first network stream signal, extract the image frame by frame during the decoding process, and perform OCR recognition on each frame of the image to identify the timestamp on the image, and record the corresponding timestamp when decoding the image frame, and calculate the difference between each timestamp related to the same image in the first transmission chain;

[0042] The second transmission link includes a desktop screen capture client, a data network switch, a centralized system processor and the display device. The desktop screen capture client is installed on the video signal source and presents a timestamp on the video signal source, and encodes the image with the timestamp into a second network stream signal; the centralized system processor is used to decode the second network stream signal, extract the image frame by frame during the decoding process, and perform OCR recognition on each frame to identify the timestamp on the image and record the timestamp when decoding the image frame, and calculate the difference between each timestamp related to the same image in the second transmission chain. The centralized system processor and the data network switch are connected by two network cables, one of which is used to transmit the video signal and the other is used to transmit management instructions.

[0043] An integrated management and control platform, which is used to simultaneously access the display and control network switch and the data network switch through the core network switch, obtain the timestamp information of each of the first transmission link and the second transmission link and the difference between the timestamps, and compare them, and switch the transmission link to the first transmission link or the second transmission link that is most suitable for video transmission at the moment according to the comparison result;

[0044] The display device has a first video interface and a second video interface, which are respectively used to connect to a distributed system decoder and a centralized system processor, and the display device can freely switch between the first video interface and the second video interface.

[0045] Figure 2a A local topology diagram of a video transmission system based on real-time performance optimization detection according to some embodiments of the present invention is shown, and some annotations are made therein for the convenience of explanation. The various components involved in the figure are: video signal source 10, distributed system encoder 101, display control network switch 103, core switch 402, integrated management platform 301, time system platform 108, distributed system decoder 102 and display device 401.

[0046] The desktop screen capture client installed on the video signal source is connected to the time system platform 108, and displays the time stamp signal SY1 of the time system platform at a certain position (such as the lower right corner) of the video signal source 10. This time stamp signal will also be used as the unique number of the video frame image of the video signal source. Since the display devices involved in video splicing and other occasions are usually large, putting a time stamp in the lower right corner of the image will not affect the actual use effect of the entire large screen.

[0047] The distributed system encoder 101 is connected to the time synchronization platform 108 to unify its own time with the external standard time, and its internal time is set as FB1, and the actual standard time (the time on the time synchronization platform 108) is set as SFB1. The distributed system encoder 101 is used to obtain the video signal of the video signal source 10 and encode it, so that the encoded video image frame here is further provided with the timestamp signal SFB1, and the video image frame is finally encoded to form a first network stream signal.

[0048] The distributed system decoder 102 is connected to the time standard platform 108 to unify its own time with the external standard time, and its internal time is set as FJ1, and the actual standard time is SFJ1. In addition to being used to obtain the first network stream signal through the network and decode it, and send the decoded video signal to the first video interface of the display device 401, the distributed system decoder 102 is also used to extract the aforementioned timestamp signal in the image obtained during the decoding process, and calculate the delay FJS01=SFJ1-SY1 of the image from the video signal source to the location, and the delay FJB01=SFJ1-SFB1 of the image from the distributed system encoder to the location with reference to the external standard time SFJ1 at this time.

[0049] According to some embodiments of the present invention, during the video decoding process, images are extracted frame by frame, and OCR recognition is performed on each frame to identify the timestamp on the image and to record the decoding time using a high-resolution timing function when decoding the image frame, such as clock_gettime or QueryPerformanceCounter.

[0050] According to some embodiments of the present invention, it is also possible to encode timestamp information in the video frames of the video signal source through the H.264 video coding standard, decode the video frames at the node where the delay needs to be determined to obtain the initial timestamp, record the timestamp during decoding, and determine the delay of the node by taking the difference between the two.

[0051] According to some embodiments of the present invention, encoding the timestamp information in the video frame of the video signal source through the H.264 video coding standard includes: timestamping the SEI frame of the video image in the video stream of the video signal source, wherein the SEI frame (sequence parameter set) is a key frame in the H.264 video coding standard, which contains global parameters of the video sequence, such as frame rate, resolution and other information. The following are steps for timestamping the SEI frame:

[0052] Analyze the video stream: Use an encoding tool (such as FFmpeg) to analyze the video stream and determine the location of the SEI frame. This step can be done by viewing the metadata of the video file.

[0053] Write a script: Write a script that reads a video file and inserts a timestamp when it finds a SEI frame.

[0054] Determine the format of the timestamp: The timestamp includes seconds, milliseconds, and microseconds, and the format can be seconds:milliseconds:microseconds.

[0055] Insert timestamp: In the script, when a SEI frame is detected, the current timestamp is inserted into the video frame in the specified format.

[0056] Encapsulate Video: Re-encapsulate the modified video frames into a new video file.

[0057] The distributed system decoder and the centralized system processor can calculate the delay by extracting the timestamp of the SEI frame and taking the difference between the timestamp and the timestamp when the SEI frame is extracted. Assuming that each GOP is basically sent in 1 second, the timestamp frequency of the present application can also be about 1 second, and the end-to-end delay collection or calculation cycle of the whole network can also be once per second.

[0058] Figure 2b The local topology diagram of the video transmission system based on real-time performance optimization detection according to some embodiments of the present invention is shown, and some annotations are made therein for the convenience of explanation. The various components involved in the figure are: video signal source 10, data network switch 203, core switch 402, integrated management platform 301, time system platform 108, centralized system processor 202 and display device 401.

[0059] The desktop screen capture client installed on the video signal source 10 is connected to the time synchronization platform 108. As mentioned above, the desktop screen capture client presents the time stamp signal SY1 of the time synchronization platform at a certain position (such as the lower right corner) of the video signal source 10. When the aforementioned distributed encoder obtains a video image from the video signal source 10, and when the desktop screen capture client obtains a video image from the video signal source 10, it can be determined by the timestamp on the video image whether the object obtained by both and the subsequent processing object are the same object, thereby ensuring that various measurements on the two transmission links are carried out for the same object.

[0060] The desktop screen capture client encodes the image with the timestamp into a second network stream signal.

[0061] The centralized system processor 202 is connected to the time synchronization platform 108 to unify its own time with the external standard time, assuming that its internal time is JC1 and the actual standard time is SJC1. The centralized system processor 202 obtains the second network stream signal through the network and decodes it. After decoding, the timestamp SY1 can be obtained, and the delay JCS01=SJC1-SY1 for the image to reach there from the desktop screen capture client is calculated with reference to the external standard time SJC1 at this time. The decoded video signal is sent by the centralized system processor 202 to the second video interface of the display device 401. During the decoding process, the centralized system processor extracts the image frame by frame and performs OCR recognition on each frame to identify the timestamp on the image and, when decoding the image frame, records the corresponding timestamp when decoding.

[0062] Since the video image is captured with a timestamp signal at its lower right corner, due to the uniqueness of time, this timestamp can also be used as a unique identifier of the video image. Therefore, the aforementioned centralized system processor can decode and identify the received image numbered SY1.

[0063] The integrated management and control platform is simultaneously connected to the display and control network switch and the data network switch through the core network switch, and further obtains the timestamp information SY1, SFB1, SFJ1 and SJC1 of the distributed system encoder, distributed system decoder, desktop screen capture client and centralized system processor, as well as the delay between each node, including the delay FJS01=SFJ1-SY1 of the image from the video signal source 10 to the distributed system decoder, the delay FJB01=SFJ1-SFB1 of the image from the distributed system encoder to the distributed system decoder, and the delay JCS01=SJC1-SY1 of the image from the desktop screen capture client to the centralized system processor.

[0064] The integrated management and control platform can further compare the timestamp information in the first transmission link and the second transmission link and the difference between the timestamps, and switch the transmission link to the first transmission link or the second transmission link that is most suitable for video transmission at the moment.

[0065] By allowing the above process to continue for a period of time, multiple measurement results of the first transmission link and the second transmission link for video frame images can be obtained in turn. If an image frame with a certain number in one of the transmission links happens to be lost, the image frame is directly skipped, and the capabilities of the two transmission links under other image frames with the same number are examined and compared.

[0066] As mentioned above, since the components of the first transmission link and the second transmission link can specifically process the same frame of video signal source image when implementing the above actions, it can be ensured that various measurements and calculation values ​​on the two transmission links are generated around the same object, thereby better measuring the transmission capabilities of the two transmission links for the same transmission object (including delay, frame loss, etc.).

[0067] According to other embodiments of the present invention, the integrated management and control platform can monitor the performance parameters of each component in the first and second transmission links in real time, calculate the scores of the first and second transmission links based on the performance parameters, and select the link with the higher score as the current transmission link.

[0068] The present invention can overcome the limitations of the prior art and provide a more flexible, efficient, reliable, easily expandable and low-cost video transmission solution.

[0069] According to some embodiments of the present invention, the integrated management and control platform can also enable the distributed video encoder in the first transmission link and the centralized system processor in the second transmission link to perform encoding and decoding combination, and finally output the video signal to the display device, such as Figure 1 As shown in L3 (set as the third transmission link).

[0070] Referring to the above process, the distributed video encoder can further encode the image SY2 (assuming that the lower right corner of the video signal source image is stamped with a timestamp SY2, which is also the identifier of the image frame) so that it carries the external time SFB2 at this time to form a network stream signal. The centralized system processor is used to decode the network stream signal, extract the image frame by frame during the decoding process, and perform OCR recognition on each frame to identify the timestamp on the image, and record the timestamp SJC2 corresponding to the decoding of SY2 when decoding the image frame, and calculate the difference between the timestamps related to the same image in the transmission link.

[0071] The integrated management and control platform can obtain the delay FBS02=SFB2-SY2 of the image from the video signal source to the distributed system encoder, and the delay JCFB02=SJC2-SFB2 of the image from the distributed system encoder to the centralized system processor, as well as the corresponding timestamp information SY2, SFB2, SJC2, wherein SY2 is the timestamp signal of the timing platform displayed at a certain position (such as the lower right corner) of the video signal source 10, and this timestamp signal will subsequently be used as the unique number of the video frame image of the video signal source; SFB2 is: when the distributed system encoder 101 obtains the video signal of the video signal source 10 and encodes it, the encoded video signal is further added with the timestamp SFB2, and this timestamp is also regarded as the time when the distributed encoder obtains the video image SY2; SJC2 is the external standard time SJC2 when the centralized system processor 202 obtains the second network stream signal through the network and decodes it to obtain the video image SY2.

[0072] According to some embodiments of the present invention, the integrated management and control platform can make the desktop screen capture client in the second transmission link and the distributed system decoder in the first transmission link perform encoding and decoding combination, and finally output the video signal to the display device, such as Figure 1 As shown in L4 (set as the fourth transmission link).

[0073] Referring to the above process, after the desktop screen capture client obtains the desktop image SY3 (assuming that the video signal source image is marked with a timestamp SY3 in the lower right corner, and this SY3 is also the identifier of the image frame), it is transmitted to the distributed system decoder as a network stream signal. The distributed system decoder is used to decode the network stream signal, and extract the image frame by frame during the decoding process, and perform OCR recognition on each frame to identify the timestamp on the image, and record the timestamp SFJ3 of the decoded SY3 image when decoding the image frame, and calculate the difference between each timestamp related to the same image in the fourth transmission chain.

[0074] During this process, the integrated management and control platform can obtain the delay FJS03=SFJ3-SY3 of the image from the video signal source to the distributed system decoder, as well as the corresponding timestamp information SY3, SFJ3, where SY3 is the timestamp signal of the timing platform displayed at a certain position (such as the lower right corner) of the video signal source 10, and this timestamp signal will subsequently be used as the unique number of the video frame image of the video signal source; SFJ3 is: the external standard time SFJ3 when the distributed system decoder decodes the second network stream signal to obtain the image frame SY2.

[0075] In the above embodiment, the integrated management and control platform can obtain the various timestamp information and the difference between the timestamps in the first transmission link and the second transmission link in real time when the first transmission link and the second transmission link are running simultaneously, and compare them, and switch the transmission link to the first transmission link or the second transmission link that is most suitable for video transmission at the moment according to the comparison result; and the integrated management and control platform can also obtain the various timestamp information and the difference between the timestamps in the third transmission link and the fourth transmission link in real time when the third transmission link and the fourth transmission link are running simultaneously, and compare them, and switch the transmission link to the third transmission link or the fourth transmission link that is most suitable for video transmission at the moment according to the comparison result. In addition, according to some embodiments of the present invention, the integrated management and control platform can also pre-count the difference between the timestamps when each transmission link transmits images of video signal sources of different sizes, establish a corresponding relationship between different transmission links, images of video signal sources of different sizes and the difference between the timestamps, and select a suitable transmission link according to the corresponding relationship and the comparison result. The data required for statistics can be generated by the user's controlled selection of images of video signal sources of different sizes and different transmission links.

[0076] According to some embodiments of the present invention, the integrated management and control platform can also pre-count the difference between the timestamps when the images of video signal sources of different sizes are transmitted in different time periods of each transmission link, establish the corresponding relationship between the difference between the images of different transmission links, different time periods, and video signal sources of different sizes and the timestamps, and select a suitable transmission link according to the transmission time period, the size of the image of the video signal source, the corresponding relationship and the comparison result. The data required for statistics can be generated by the user controlling the selection of images of video signal sources of different sizes and different transmission links in different time periods.

[0077] In addition, in addition to the timestamp and the delay performance associated therewith, the user can also similarly establish corresponding correspondences for other performances of the transmission link in advance, and the other performances include: picture quality, number of freezes, etc. For example, within 2 days, the number of freezes that occurred in the first transmission link from 8:00 to 12:00 was 4 and 6 times, respectively, and the number of freezes that occurred in the second transmission link from 8:00 to 12:00 was 8 and 10 times, respectively. Therefore, the correspondence may include that the average number of freezes that occurred from 8:00 to 12:00 corresponding to the first transmission link was 5 times, and the average number of freezes that occurred from 8:00 to 12:00 corresponding to the second transmission link was 9 times.

[0078] In addition, according to some embodiments of the present invention, the integrated management and control platform can also supplement or calibrate the correspondence between the images of different transmission links and video signal sources of different sizes and the difference between the timestamps that have been established, or supplement or calibrate the correspondence between the images of different transmission links, different time periods, and video signal sources of different sizes and the difference between the timestamps that have been established, based on the timestamp information obtained in real time and the difference between the timestamps.

[0079] According to some embodiments of the present invention, the integrated management and control platform can control the video transmission frame rate of the video signal source so that the frequency of change of the timestamp presented on the video signal source is much greater than the transmission frame rate, thereby ensuring that each image sent is timestamped.

[0080] Through the above embodiments of the present invention, the limitations of the prior art can be overcome and a more flexible, efficient and reliable video transmission solution can be provided, which is specifically manifested as follows:

[0081] High fault tolerance: When a node fails or a link is interrupted, the system can automatically switch to a backup link or reconfigure the link to ensure the continuity and stability of transmission.

[0082] Easy to expand: Adaptive systems can more easily integrate new devices and services, supporting the expansion of system scale.

[0083] Reduced maintenance costs: Since the system can automatically adjust the configuration, the need for human intervention is reduced, thereby reducing maintenance costs.

[0084] Improve adaptability: The system can automatically select the most appropriate transmission strategy based on different types of video content or transmission requirements.

[0085] Improve user experience: By optimizing the transmission link, the system can provide higher quality video transmission services and improve user experience.

[0086] Figure 3 A flow chart of a video transmission method based on the video transmission system based on real-time performance optimization detection according to some embodiments of the present invention is shown. As shown in the figure, the method includes:

[0087] S1. The integrated management and control platform switches the video transmission system to the first transmission link, the second transmission link, the third transmission link, and the fourth transmission link in sequence, and calculates the delays between the video signal source image in each transmission link and the centralized video processor or distributed video decoder in the link under each transmission link;

[0088] S2. According to the delay, select a suitable transmission link for video transmission.

[0089] According to some embodiments of the present invention, when it is monitored that any logical node in the first transmission link is abnormal, the first transmission link of the video transmission system is switched to the second transmission link.

[0090] According to some embodiments of the present invention, the integrated management and control platform can control the video transmission frame rate of the video signal source so that the frequency of change of the timestamp presented on the video signal source is much greater than the transmission frame rate, thereby ensuring that each image sent is timestamped.

[0091] According to some embodiments of the present invention, the integrated management and control platform is also used to pre-count the difference between the maximum delay and the minimum delay on each transmission link, and switch one of the transmission links with a relatively large difference to another transmission link. Network jitter is the difference between the maximum delay and the minimum delay of the network. Through the present invention, the network jitter from the video source to the sink can be effectively described.

[0092] According to some embodiments of the present invention, the integrated management and control platform is also used to determine whether the centralized system processor is already operating at full capacity and whether the video signal needs to be processed before output based on acquired delays related to the centralized system processor, for example, the delay of acquiring the signal source image frame from the centralized system processor. The integrated management and control platform can control the centralized system processor to reduce the image resolution of the video signal source before output, thereby reducing resource consumption of the centralized system processor.

[0093] According to some embodiments of the present invention, the integrated management and control platform is further used to determine whether it is necessary to adjust the output frame rate of the video signal source based on the acquired delay related to the centralized system processor, for example, the delay of acquiring the signal source image frame from the centralized system processor. For example, when the image of a certain video signal source places too much load on the centralized system processor, the integrated management and control platform controls the video signal source to reduce its output frame rate, thereby reducing the load on the centralized system processor.

[0094] According to some embodiments of the present invention, the integrated management and control platform can also pre-count the differences between the timestamps when each transmission link transmits images of video signal sources of different sizes, establish a correspondence between different transmission links, images of video signal sources of different sizes and the differences between the timestamps, and select a suitable transmission link based on the correspondence and the comparison result.

[0095] According to some embodiments of the present invention, the integrated management and control platform can also pre-count the differences between the timestamps when images of video signal sources of different sizes are transmitted in different time periods of each transmission link, establish a correspondence between the images of different transmission links, different time periods, and different sizes of video signal sources and the differences between the timestamps, and select a suitable transmission link based on the transmission time period, the size of the image of the video signal source, the correspondence, and the comparison result.

[0096] In addition, in addition to the timestamp and its associated delay performance, the user can also similarly establish corresponding correspondences in advance for other performances of the transmission link, and the other performances include: picture quality, number of freezes, etc.

[0097] In addition, according to some embodiments of the present invention, the integrated management and control platform can also supplement or calibrate the correspondence between the images of different transmission links and video signal sources of different sizes and the difference between the timestamps that have been established, or supplement or calibrate the correspondence between the images of different transmission links, different time periods, and video signal sources of different sizes and the difference between the timestamps that have been established, based on the timestamp information obtained in real time and the difference between the timestamps.

[0098] In addition to the timestamp and its associated delay performance, the user can also similarly establish corresponding correspondences in advance for other performances of the transmission link, including: picture quality, number of freezes, etc.

[0099] According to some embodiments of the present invention, the integrated management and control platform can also supplement or calibrate the correspondence between the images of different transmission links and video signal sources of different sizes and the difference between the timestamps that have been established, or supplement or calibrate the correspondence between the images of different transmission links, different time periods, and video signal sources of different sizes and the difference between the timestamps that have been established, based on the timestamp information and the difference between the timestamps that are acquired in real time.

[0100] According to some embodiments of the present invention, the integrated management and control platform can pre-count the difference between the maximum delay and the minimum delay on each transmission link, and switch one of the transmission links with a relatively large difference to another transmission link. Network jitter is the difference between the maximum delay and the minimum delay of the network. Through the present invention, the network jitter of the video source can be effectively described.

[0101] Figure 4 A module diagram of an integrated management device 4000 according to some embodiments of the present invention is shown.

[0102] As shown in the figure, the device includes: a receiving module 4001, a calculating module 4002, a judging module 4003 and a switching module 4004.

[0103] The receiving module 4001 is used to receive data sent by various components, and the data may include, for example, the delay obtained on each node, the timestamp, the timestamp on the image frame, etc.

[0104] The calculation module 4002 is used to calculate the data to obtain the delay and / or packet loss of each transmission link.

[0105] The judgment module 4003 is used to determine whether it is necessary to switch the transmission link according to the calculation result.

[0106] The switching module 4004 is used to switch the transmission link of the video transmission system according to the judgment result. For example, when any logical node in the first transmission link is monitored to be abnormal (infinite delay), the first transmission link of the video transmission system is switched to the second transmission link.

[0107] The integrated management device can be used as a part of the management control platform to implement the video transmission method or a part thereof. Accordingly, according to some embodiments of the present invention, the integrated management device can also include more modules to implement more detailed processes in the above-mentioned video transmission method.

[0108] Each of the modules mentioned above may be a program module or may be a hardware component that implements the corresponding function.

[0109] Figure 5 1 is a schematic structural diagram of an integrated management device 500 according to an embodiment of the present invention. As shown in the figure, the integrated management device 500 includes a processor 51, a memory 52 and a bus 53.

[0110] In some examples, the integrated management device 500 may also include multiple input devices 501, multiple input ports 502, multiple output ports 503, and multiple output devices 504. For simplicity, only one input device, one input port, one output port, and one output device are shown in the figure. Among them, the input port 502, the processor 51, the memory 52, and the output port 503 are interconnected through the bus 53, and the input device 501 and the output device 504 are connected to the bus 53 through the input port 502 and the output port 503 respectively, and then connected to other components of the integrated management device 500. It should be noted that the input port and the output port here can also be represented by an I / O interface.

[0111] Each input device 501 transmits each input information to the processor 51 through each input port 502; the processor 51 processes or responds to the input information based on the computer executable instructions stored in the memory 52 to generate output information, and temporarily or permanently stores the output information in the memory 52, and then transmits the output information to each output device 504 through each output port 504; the output device 504 outputs the output information to another terminal as needed, and the input signal can be, for example, each time stamp. The process of the processor 51 processing or responding to each input information based on the computer executable instructions stored in the memory 52 to generate output information can be understood in conjunction with the aforementioned video transmission method.

[0112] According to actual needs, the above-mentioned memory may be various types of memory, such as flash memory, optical disk, magnetic disk or a combination of two or more of these; it may also include removable or non-removable media; it may be inside or outside the device; it may be a non-volatile solid-state memory, read-only memory, erasable PROM (EPROM) or a combination of two or more of these.

[0113] The present application also provides a computer-readable storage medium, which may include instructions. When the computer-readable storage medium is executed on a computer, the computer may execute the above-mentioned video transmission method or part of the steps thereof.

[0114] The present invention can overcome the limitations of the prior art and provide a more flexible, efficient, reliable, easily expandable and low-cost video transmission solution.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video transmission system based on real-time performance optimization detection, comprising: The first transmission link includes a distributed system encoder, a display control network switch, a distributed system decoder and a display device; The distributed system encoder is connected to the video signal source through a video line to obtain the image displayed thereon and encode the image to form a first network stream signal; The distributed system decoder is used to decode the first network stream signal, extract images frame by frame during the decoding process, perform OCR recognition on each frame of the image to identify the timestamp on the image, and record the timestamp corresponding to the decoding when decoding the image frame, and calculate the difference between each timestamp related to the same image in the first transmission chain; A second transmission link includes a desktop screen capture client, a data network switch, a centralized system processor and the display device, wherein the desktop screen capture client is installed on a video signal source and presents a timestamp on the video signal source, and encodes an image with the timestamp into a second network stream signal; The centralized system processor is used to decode the second network stream signal, extract the image frame by frame during the decoding process, perform OCR recognition on each frame to identify the time stamp on the image, record the time stamp when decoding the image frame, and calculate the difference between each time stamp related to the same image in the second transmission chain; An integrated management and control platform, which is used to simultaneously access the display and control network switch and the data network switch through the core network switch, obtain the timestamp information of each of the first transmission link and the second transmission link and the difference between the timestamps in real time, and compare them, and switch the transmission link to the first transmission link or the second transmission link that is most suitable for video transmission at the moment according to the comparison result; The display device has a first video interface and a second video interface, which are respectively used to connect to a distributed system decoder and a centralized system processor, and the display device can freely switch between the first video interface and the second video interface.

2. The video transmission system according to claim 1, wherein: The encoding of the image to form the first network stream signal also includes providing the video image frame with a second timestamp; the distributed system decoder is also used to identify the second timestamp on the image during decoding, and to record the corresponding timestamp during decoding when decoding the image frame, and to calculate the difference between the timestamps related to the same image in the first transmission chain.

3. The video transmission system according to claim 1, wherein: The desktop screen capture client timestamps the SEI frame of the video image in the video stream of the video signal source through the H.264 video encoding standard; the distributed system encoder, distributed system decoder and centralized system processor extract the timestamp of the SEI frame and calculate the difference between the timestamp and the timestamp when the SEI frame is extracted.

4. The video transmission system according to claim 1, wherein: Timestamping the SEI frame of a video image in a video stream of a video signal source includes: Use encoding tools to analyze the video stream and determine the location of SEI frames; Write a script that reads a video file and inserts a timestamp when it finds a SEI frame. Determine the format of the timestamp; In the script, when a SEI frame is detected, the current timestamp is inserted into the video frame in the specified format; Repackage the modified video frames into a new video file.

5. The video transmission system according to claim 1 or 2, further comprising: The third transmission link includes a distributed system encoder and a display control network switch in the first transmission link, a centralized system processor and a display device in the second transmission link, and a core switch; the distributed system encoder is used to encode the image of the video signal source to form a third network stream signal; the centralized system processor is used to decode the third network stream signal, and extract the image frame by frame during the decoding process and perform OCR recognition on each frame to identify the timestamp on the image, and record the corresponding timestamp when decoding the image frame, and calculate the difference between each timestamp related to the same image in the third transmission chain; and A fourth transmission link, comprising a desktop screen capture client and a data network switch in the second transmission link, a distributed system decoder and a display device in the first transmission link, and a core switch; the desktop screen capture client is installed on a video signal source and presents a timestamp on the video signal source, and encodes an image of the video signal source with the timestamp into a fourth network stream signal; The distributed system decoder is used to decode the fourth network stream signal, extract the image frame by frame during the decoding process, perform OCR recognition on each frame to identify the timestamp on the image, and record the timestamp when decoding the image frame, and calculate the difference between each timestamp related to the same image in the fourth transmission chain; The integrated management and control platform is also used to obtain each timestamp in the third transmission link and the fourth transmission link and the difference between the timestamps, and compare them, and switch the transmission link to the third transmission link or the fourth transmission link that is most suitable for video transmission at the moment according to the comparison result.

6. The video transmission system according to claim 5, wherein: The distributed system encoder is also used to encode the video image containing the timestamp obtained in the third transmission link, so that the video image frame encoded here is further provided with the third timestamp; the centralized system processor is also used to identify the third timestamp on the image during decoding, and record the corresponding timestamp during decoding when decoding the image frame, and calculate the difference between each timestamp related to the same image in the third transmission chain.

7. The video transmission system according to claim 5, wherein: The integrated management and control platform is also used to calculate the difference between the maximum delay and the minimum delay on each transmission link, and to switch one of the transmission links with a relatively larger difference value to another transmission link.

8. The video transmission system according to claim 5, wherein: The integrated management and control platform is also used to pre-count the differences between the timestamps when each transmission link transmits images of video signal sources of different sizes, establish a correspondence between different transmission links, images of video signal sources of different sizes and the differences between the timestamps, and select a suitable transmission link based on the size of the image of the video signal source, the correspondence and the comparison result.

9. The video transmission system according to claim 5, wherein: The integrated management and control platform is also used to pre-count the differences between the timestamps when images of video signal sources of different sizes are transmitted in different time periods of each transmission link, establish a correspondence between the images of video signal sources of different sizes and the differences between the timestamps in different transmission links, different time periods, and different transmission links, and select a suitable transmission link based on the transmission time period, the size of the image of the video signal source, the correspondence and the comparison result.

10. The video transmission system according to claim 5, wherein: The integrated management and control platform is also used to determine whether the centralized system processor is operating at full capacity and whether the video signal source needs to be processed before output based on the monitored delay associated with the centralized system processor. The processing includes controlling the video signal source to reduce its image resolution before output, or reducing its output frame rate.

Images