Video transmission method, system, device, electronic device and storage medium

Abstract

The embodiments of the present application provide a video transmission method, system, device, electronic device and storage medium, which can be applied to LED display products. The method is applied to a target input device, which is any input device in a video transmission system. The video transmission system also includes other input devices other than the target input device. The method includes: receiving a first video frame from its own receiving port, and setting a corresponding frame number for the received first video frame; obtaining a frame number update time; when the frame number update time is reached, determining the first frame number in accordance with the method of setting the frame number for the received video frame; when the unified sending time is reached, sending the video frame corresponding to the first frame number. In this way, it is possible to avoid the situation where the video images displayed by multiple video display terminals are out of sync.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a video transmission method, system, device, electronic device and storage medium. Background Art

[0002] With the development of the communication technology field, large-scale and highly complex multi-input videos can be displayed through multiple video display terminals.

[0003] In the related art, multiple video input devices can respectively obtain video images from a source end, and each video image corresponds to a different display area of ​​the same display screen. Furthermore, each video input device can send the received video image to a video display terminal, and multiple video display terminals can respectively display the video images they have received, so that multiple video display terminals present a complete display screen.

[0004] However, when multiple input videos are displayed, since different video input devices have time differences when receiving video images and send them at a unified sending time, the video images displayed by multiple video display terminals may be out of sync. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a video transmission method, system, device, electronic device and storage medium to avoid the situation where the video images displayed by multiple video display terminals are out of sync. The specific technical solution is as follows:

[0006] In a first aspect of an embodiment of the present application, a video transmission method is first provided. The method is applied to a target input device, the target input device is any input device in a video transmission system, and the video transmission system further includes other input devices except the target input device. The method includes:

[0007] Receive a first video frame from its own receiving port, and set a corresponding frame sequence number for the received first video frame;

[0008] Obtaining a frame number update time; wherein the frame number update time is obtained by adding a preset time length to the receiving end time corresponding to the designated input device, and the frame number update time is located after the receiving end time when each input device receives the first video frame from its respective receiving port; the receiving end time corresponding to the designated input device indicates: the receiving end time when the designated input device receives the first video frame from its own receiving port; the first video frames received by each input device have the same corresponding timestamp in the original video image at the source end;

[0009] When the frame sequence number update time is reached, a first frame sequence number is determined in accordance with a method of setting a frame sequence number for a received video frame; wherein the first frame sequence number is used to indicate a video frame to be sent next;

[0010] When a unified sending time is reached, the video frame corresponding to the first frame sequence number is sent; wherein the unified sending time represents the time when all input devices in the video transmission system uniformly send video frames.

[0011] In some embodiments, the designated input device is a main input device, and obtaining the frame sequence number update time includes:

[0012] If it is the main input device, then obtain the end time of receiving the first video frame from its receiving port; based on the obtained end time of receiving, increase the preset duration to obtain the frame number update time, and send the frame number update time to other input devices; and / or,

[0013] If it is not the main input device, it receives the frame sequence number update time sent by the main input device.

[0014] In some embodiments, the method further comprises:

[0015] When receiving the first video frame from its own receiving port, sending the receiving start time of receiving the first video frame from its own receiving port to other input devices;

[0016] Determine the earliest receiving start time of the receiving start times of the first video frame received by each input device from its respective receiving port as the first time, and determine the latest receiving start time of the receiving start times of the first video frame received by each input device from its respective receiving port as the second time;

[0017] Determine a time corresponding to an equinox of a time interval with the first time and the second time as endpoints as a third time;

[0018] The input device corresponding to the receiving start time with the shortest duration between the third time moments is determined as the main input device.

[0019] In some embodiments, the step of increasing the preset duration based on the acquired receiving end time to obtain the frame sequence number update time includes:

[0020] Add the preset duration to the acquired receiving end time to obtain a fourth time;

[0021] If the fourth moment is outside the preset time interval, the fourth moment is determined to be the frame sequence number update moment; wherein the center of the preset time interval is the unified sending moment;

[0022] If the fourth moment is within the preset time interval, a fifth moment that is not earlier than the end moment of the preset time interval is determined as the frame sequence number update moment.

[0023] In some embodiments, the method further comprises:

[0024] If it is not the main input device and has not determined the first frame number based on the frame number update time sent by the main input device before receiving the next video frame from its own receiving port, then the first frame number is determined when the next video frame is received from its own receiving port.

[0025] A second aspect of the embodiments of the present application provides a video transmission system, the video transmission system comprising a plurality of input devices, and an output device corresponding to each input device;

[0026] Each input device is used to execute any of the above-mentioned video transmission methods;

[0027] Each output device is used to receive and display the video frames sent by the corresponding input device.

[0028] According to a third aspect of the embodiments of the present application, a video transmission device is provided. The device is applied to a target input device, wherein the target input device is any input device in a video transmission system, and the video transmission system further includes other input devices except the target input device. The device includes:

[0029] The video frame receiving module is used to receive the first video frame from its own receiving port and set a corresponding frame sequence number for the received first video frame;

[0030] A first acquisition module is used to acquire a frame number update time; wherein the frame number update time is obtained by adding a preset time length to the receiving end time corresponding to the designated input device, and the frame number update time is located after the receiving end time when each input device receives the first video frame from its respective receiving port; the receiving end time corresponding to the designated input device indicates: the receiving end time when the designated input device receives the first video frame from its own receiving port; the first video frames received by each input device have the same corresponding timestamp in the original video image at the source end;

[0031] A first determining module, configured to determine a first frame sequence number in a manner of setting a frame sequence number for a received video frame when the frame sequence number update time is reached; wherein the first frame sequence number is used to indicate a video frame to be sent next;

[0032] The video frame sending module is used to send the video frame corresponding to the first frame sequence number when a unified sending time is reached; wherein the unified sending time represents the time when all input devices in the video transmission system uniformly send video frames.

[0033] In some embodiments, the designated input device is a main input device, and the first acquisition module includes:

[0034] A first sending submodule is used for obtaining the end time of receiving the first video frame from its receiving port if it is the main input device; based on the obtained end time of receiving, adding the preset duration to obtain the frame sequence number update time, and sending the frame sequence number update time to other input devices;

[0035] and / or,

[0036] The first receiving submodule is configured to receive the frame sequence number update time sent by the main input device if the first receiving submodule is not the main input device.

[0037] In some embodiments, the apparatus further comprises:

[0038] A first sending module is used to send a receiving start time of receiving the first video frame from its own receiving port to other input devices when the first video frame is received from its own receiving port;

[0039] A second determining module is used to determine the earliest receiving start time of receiving the first video frame from the respective receiving ports of the input devices as the first time, and to determine the latest receiving start time of receiving the first video frame from the respective receiving ports of the input devices as the second time;

[0040] A third determining module, configured to determine a time corresponding to an equinox of a time interval having the first time and the second time as endpoints as a third time;

[0041] The master device determination module is used to determine the input device corresponding to the receiving start time with the shortest duration between the third time moments as the master input device.

[0042] In some embodiments, the first sending submodule is specifically used to:

[0043] Add the preset duration to the acquired receiving end time to obtain a fourth time;

[0044] If the fourth moment is outside the preset time interval, the fourth moment is determined to be the frame sequence number update moment; wherein the center of the preset time interval is the unified sending moment;

[0045] If the fourth moment is within the preset time interval, a fifth moment that is not earlier than the end moment of the preset time interval is determined as the frame sequence number update moment.

[0046] In some embodiments, the apparatus further comprises:

[0047] The fourth determination module is used to determine the first frame number when the next video frame is received from its own receiving port if it is not the main input device and has not determined the first frame number based on the frame number update time sent by the main input device before receiving the next video frame from its own receiving port.

[0048] According to a fourth aspect of the embodiments of the present application, an electronic device is provided, including:

[0049] Memory, used to store computer programs;

[0050] The processor is used to implement any of the above-mentioned video transmission methods when executing the program stored in the memory.

[0051] According to a fifth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned video transmission methods is implemented.

[0052] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-mentioned video transmission methods.

[0053] Beneficial effects of the embodiments of the present application:

[0054] A video transmission method provided by an embodiment of the present application is applied to a target input device, the target input device is any input device in a video transmission system, and the video transmission system also includes other input devices except the target input device. The method includes: receiving a first video frame from its own receiving port, and setting a corresponding frame sequence number for the received first video frame; obtaining a frame sequence number update time; wherein the frame sequence number update time is obtained based on adding a preset duration on the basis of the receiving end time corresponding to the designated input device, and the frame sequence number update time is located after the receiving end time when each input device receives the first video frame from its respective receiving port; the receiving end time corresponding to the designated input device indicates: the receiving end time when the designated input device receives the first video frame from its own receiving port; the corresponding timestamps of the first video frames received by each input device in the original video image at the source end are the same; when the frame sequence number update time is reached, the first frame sequence number is determined in a manner of setting a frame sequence number for the received video frame; wherein the first frame sequence number is used to indicate the video frame to be sent next; when the unified sending time is reached, the video frame corresponding to the first frame sequence number is sent; wherein the unified sending time indicates the time when all input devices in the video transmission system uniformly send video frames.

[0055] Based on the above processing, since the frame number update time is obtained by adding a preset time length to the receiving end time corresponding to the designated input device, and is located after the receiving end time of each input device receiving the first video frame from its respective receiving port, each input device can determine the first frame number at the same time, and the reception of the first video frame has been completed at this time. Accordingly, each input device sends the video frame corresponding to the first frame number at the same sending time, which enables each input device to send the video frame with the same frame number at the same time, thereby avoiding the situation where the video images displayed by multiple video display terminals are out of sync.

[0056] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0058] Figure 1 A schematic diagram of an input device receiving and sending a video frame provided in an embodiment of the present application;

[0059] Figure 2A schematic diagram of a flow chart of a video transmission method provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of another video transmission method provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of an input device receiving a video frame provided in an embodiment of the present application;

[0062] Figure 5 A schematic diagram of a process for determining a first frame sequence number provided in an embodiment of the present application;

[0063] Figure 6 A schematic diagram of the structure of a video transmission system provided in an embodiment of the present application;

[0064] Figure 7 A schematic diagram of the structure of an input device interaction provided in an embodiment of the present application;

[0065] Figure 8 A structural diagram of a video transmission device provided in an embodiment of the present application;

[0066] Fig. 9 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0068] In the related art, a video transmission system includes multiple input devices, and each input device corresponds to a video sent by a source end. For any input device, when the input device receives a video frame, the input device can determine the frame number used to indicate the next video frame to be sent. Since there is a time difference between the moments when each input device starts to receive the video frame, there is also a time difference between the moments when each input device receives the video frame. Correspondingly, there is also a time difference between the moments when each input device determines the frame number. Furthermore, when a unified sending time is reached, if each input device sends the video frame corresponding to the determined frame number, each input device will send a video frame with an inconsistent frame number, causing the video images displayed by multiple video display terminals to be out of sync.

[0069] like Figure 1 As shown, Figure 1A schematic diagram of an input device receiving and sending video frames provided in an embodiment of the present application. Figure 1 In , each input device can obtain video frames from the source and send them to their corresponding video display terminals. Each input device works on the same timeline. Figure 1 In the example, input device #0 starts receiving video frames at time a0 and sets its frame number to 1. At this time, the frame number of the video frame to be sent next time determined by input device #0 is 0. At time b0, input device #0 completes receiving the video frame with frame number 1. At this time, the frame number of the video frame to be sent next time determined by input device #0 is 1. Furthermore, at the unified sending time c, input device #0 sends the video frame with frame number 1. Subsequently, input device #0 starts receiving the next video frame at time a'0 and sets its frame number to 2.

[0070] Similarly, input device #1 starts receiving video frames at time a1 and sets its frame number to 1. At this time, the frame number of the video frame to be sent next time determined by input device #1 is 0. Therefore, at the unified sending time c, input device #1 sends the video frame with frame number 0. Subsequently, at time b1, input device #1 completes receiving the video frame with frame number 1. At this time, input device #1 determines the frame number of the video frame to be sent next time to be 1. Input device #1 starts receiving the next video frame at time a'1 and sets its frame number to 2.

[0071] Input device #n starts receiving video frames at time an and sets its frame number to 1. At this time, the frame number of the video frame to be sent next time determined by input device #n is 0. At time bn, input device #n completes receiving the video frame with frame number 1. At this time, the frame number of the video frame to be sent next time determined by input device #n is 1. Furthermore, at the unified sending time c, input device #n sends the video frame with frame number 1. Subsequently, input device #n starts receiving the next video frame at time a'n and sets its frame number to 2.

[0072] It can be seen that at the unified sending time c, each input device will send video frames with inconsistent frame numbers, which will cause the video images displayed by the video display terminals corresponding to each input device to be out of sync.

[0073] In order to solve the above problems, the embodiment of the present application provides a video transmission method, which is applied to a target input device, the target input device is any input device in a video transmission system, and the video transmission system also includes other input devices except the target input device, and the input devices can communicate data through Ethernet, serial port, USB (Universal Serial Bus), or hardware connection. Figure 2 , Figure 2A schematic diagram of a video transmission method provided in an embodiment of the present application, the method may include the following steps:

[0074] S201: Receive a first video frame from its own receiving port, and set a corresponding frame sequence number for the received first video frame.

[0075] S202: Obtain the frame number update time.

[0076] Among them, the frame number update time is obtained by adding a preset time length to the receiving end time corresponding to the designated input device, and the frame number update time is after the receiving end time when each input device receives the first video frame from its respective receiving port; the receiving end time corresponding to the designated input device indicates: the receiving end time when the designated input device receives the first video frame from its own receiving port; the first video frames received by each input device have the same corresponding timestamp in the original video image at the source end.

[0077] S203: When the frame number update time is reached, the first frame number is determined in accordance with the manner of setting the frame number for the received video frame.

[0078] The first frame sequence number is used to indicate the video frame to be sent next time.

[0079] S204: When the unified sending time is reached, the video frame corresponding to the first frame sequence number is sent.

[0080] The unified sending time refers to the time when all input devices in the video transmission system uniformly send video frames.

[0081] Based on the above processing, since the frame number update time is obtained by adding a preset time length to the receiving end time corresponding to the designated input device, and is located after the receiving end time of each input device receiving the first video frame from its respective receiving port, each input device can determine the first frame number at the same time, and the reception of the first video frame has been completed at this time. Accordingly, each input device sends the video frame corresponding to the first frame number at the same sending time, which enables each input device to send the video frame with the same frame number at the same time, thereby avoiding the situation where the video images displayed by multiple video display terminals are out of sync.

[0082] With respect to step S201, the source end may obtain multiple channels of video based on the original video image, and the number of the obtained videos is the same as the number of input devices, that is, one channel of video corresponds to one input device.

[0083] For example, the multiple videos obtained by the source end are the same. That is, the source end can send video frames in the original video image to each input device respectively. In this way, each first video frame received by each input device is the same video frame in the original video image, that is, each first video frame received by each input device has the same corresponding timestamp in the original video image.

[0084] For another example, the multiple videos obtained by the source end are different. For example, for each video frame in the original video image, the source end can divide the video frame into multiple image areas according to the number of input devices, and thus multiple videos can be obtained. One video channel contains a video screen of an image area in the original video image. In this way, each first video frame received by each input device is a different image area of ​​a video frame in the original video image, that is, each first video frame received by each input device has the same corresponding timestamp in the original video image.

[0085] Each input device can sequentially receive each video frame in a video sent by the source end from its own receiving port. The videos sent by the source end can be the same or different. For example, if the videos are different, the videos can correspond to different areas in the same display screen, so that the device that obtains the video frame from each input device (i.e., the output device in this application) can present the complete display screen.

[0086] For example, the source end may be a video server providing multiple channels of video. In the present application, for each video frame, the time when the video frame starts to be received may be referred to as the receiving start time, and correspondingly, the time when the video frame is completed to be received may be referred to as the receiving end time.

[0087] The manner in which each input device sets the frame sequence number for the received video frames may be the same or different.

[0088] For example, each input device may set corresponding frame numbers for received video frames in a cyclic manner based on the values ​​in a preset frame number set. For example, the frame number set may be [0, 1, 2]. When receiving the first video frame from the source, the frame number of the video frame may be set to 0, when receiving the second video frame from the source, the frame number of the video frame may be set to 1, when receiving the third video frame from the source, the frame number of the video frame may be set to 2, when receiving the fourth video frame from the source, the frame number of the video frame may be set to 0, and so on.

[0089] Alternatively, each input device may also set a corresponding frame number for the received video frame based on a natural number increment method. For example, when receiving the first video frame from the source, the frame number of the video frame may be set to 0, when receiving the second video frame from the source, the frame number of the video frame may be set to 1, when receiving the third video frame from the source, the frame number of the video frame may be set to 2, and when receiving the fourth video frame from the source, the frame number of the video frame may be set to 3.

[0090] With respect to step S202, the frame number update time indicates: the time when the input device determines the frame number of the video frame to be sent next.

[0091] The designated input device can be any input device in the video transmission system.

[0092] For example, the designated input device may be pre-set by a technician according to current business requirements when the video transmission system is initialized. That is, during the process of the video transmission system transmitting the video, the designated input device may not change. Alternatively, the designated input device may be an input device determined from the input devices according to the first preset election method when each input device receives each video frame from the source. That is, during the process of the video transmission system transmitting the video, the designated input device may change.

[0093] In one implementation, the frame number update time can be obtained by adding a preset duration to the end time of receiving the first video frame from the receiving port of the designated input device. Accordingly, the designated input device can send the obtained frame number update time to other input devices. In this way, the designated input device is the main input device in the embodiment of the present application. The main input device will be described in detail in subsequent embodiments. For example, the preset duration can be 5 microseconds.

[0094] In another implementation, the frame number update time can also be obtained by adding a preset time length to the end time of receiving the first video frame by any input device other than the designated input device. That is, the designated input device can send the end time of receiving the first video frame to the any input device, and after receiving the end time of receiving, the any input device can add the preset time length to the end time of receiving to obtain the frame number update time. Accordingly, the any input device can send the frame number update time to other input devices.

[0095] In another implementation, the designated input device may also send the end time of receiving the first video frame by the designated input device to other input devices. Accordingly, each other input device may add a preset time length to the end time of receiving the first video frame by the designated input device to obtain the frame sequence number update time, and the designated input device may also add a preset time length to the end time of receiving the first video frame to obtain the frame sequence number update time.

[0096] For step S203, the way in which each input device determines the first frame number is consistent with the way in which the input device sets the frame number for the received video frame. For example, if the input device sets the corresponding frame number for the received video frame in a cyclic manner based on the frame number set [0, 1, 2] when receiving the video frame, the input device can determine that the first frame number is 0 when the frame number update time is reached for the first time, and can determine that the first frame number is 1 when the frame number update time is reached for the second time, and can determine that the first frame number is 2 when the frame number update time is reached for the third time, and can determine that the first frame number is 0 when the frame number update time is reached for the fourth time, and so on.

[0097] For step S204, the unified sending time is used to instruct each input device to send a video frame at that time. The unified sending time corresponding to each input device in the video transmission system is the same. In other words, each input device sends a video frame at the unified sending time on the same time axis.

[0098] In one embodiment, the designated input device is a primary input device. Figure 3 ,exist Figure 2 On the basis of Figure 3 A schematic flow chart of another video transmission method provided in an embodiment of the present application, step S202 includes:

[0099] S2021: If it is the main input device, obtain the receiving end time of the first video frame received from its own receiving port; based on the obtained receiving end time, increase the preset duration to obtain the frame number update time, and send the frame number update time to other input devices.

[0100] and / or,

[0101] S2022: If it is not the main input device, receive the frame sequence number update time sent by the main input device.

[0102] Among them, the main input device can be any input device in the video transmission system, and the main input device is used to determine the frame number update time. The main input device can obtain the end time of receiving the first video frame from its own receiving port, and obtain the frame number update time based on the obtained end time of receiving by adding a preset time length, and send the frame number update time to other input devices. In this way, the main input device is the designated input device in this application. In this application, input devices other than the main input device can be called slave input devices.

[0103] In one implementation, the main input device can be pre-set by a technician according to current business requirements when the video transmission system is initialized. That is, the main input device can remain unchanged during the video transmission system's video transmission process.

[0104] In another implementation, the main input device may be an input device determined from the input devices according to a second preset selection method when each input device receives each video frame from the source. That is, the main input device may be changed during the video transmission process of the video transmission system.

[0105] Based on the above processing, the main input device uniformly determines the frame number update time, which can ensure the accuracy of the frame number update time, and does not need each input device to calculate the frame number update time, thereby reducing the data calculation amount of the entire system.

[0106] In one embodiment, the method further comprises:

[0107] Step 1: When the first video frame is received from its own receiving port, the receiving start time of receiving the first video frame from its own receiving port is sent to other input devices.

[0108] Step 2: Determine the earliest receiving start time among the receiving start times when each input device receives the first video frame from its respective receiving port as the first time, and determine the latest receiving start time among the receiving start times when each input device receives the first video frame from its respective receiving port as the second time.

[0109] Step 3: Determine the time corresponding to the equinox of the time interval with the first time and the second time as endpoints, as the third time.

[0110] Step 4: Determine the input device corresponding to the receiving start time with the shortest duration between the third time points as the main input device.

[0111] For example, for any input device, when the input device receives the first video frame from its own receiving port, it can send the receiving start time of receiving the first video frame to other input devices. That is to say, each input device can receive the receiving start time of receiving the first video frame sent by other input devices, and thus the receiving start time of receiving the first video frame by each input device in the system can be obtained. That is, each input device can obtain the receiving start time of receiving the first video frame by all input devices in the video transmission system. Furthermore, each input device can determine the main input device according to the second preset election method based on the receiving start time of all input devices. Since each input device determines the main input device in the same way, the main input device determined by each input device is also the same.

[0112] Based on the above processing, each input device can send each other the start time of receiving the first video frame to elect a master input device, so that the slave input device can receive the frame sequence number update time sent by the elected master input device. Compared with the method of pre-setting the master input device, the reliability of each input device obtaining the frame sequence number update time can be improved, and further, the video images displayed by multiple video display terminals can be further prevented from being out of sync.

[0113] For each input device, after the input device obtains the receiving start time of all input devices, the earliest receiving start time (i.e., the first time) among the receiving start times of each input device and the latest receiving start time (i.e., the second time) among the receiving start times of each input device can be determined.

[0114] The third moment is the moment corresponding to the equinox of the time interval with the first moment and the second moment as endpoints. That is, the third moment is after the first moment and before the second moment, and the time difference between the first moment and the third moment is the same as the time difference between the third moment and the second moment. Determine the duration between the receiving start moment of each input device and the third moment, that is, calculate the time difference between the receiving start moment of each input device and the third moment. Then, the input device corresponding to the receiving start moment with the smallest time difference can be determined as the main input device.

[0115] In addition, the main input device determined by each input device may be verified. For example, after determining the main input device, each input device may send the device information of the main input device determined by itself to other input devices. For example, the identity of the main input device. If the device information of the main input device received by any input device is inconsistent with the device information of the main input device determined by itself, the input device may send an error reminder to other input devices, and accordingly, each input device may re-determine the main input device.

[0116] Based on the above processing, since the input devices corresponding to other larger time differences may be abnormal input devices, the input device corresponding to the receiving start time with the smallest time difference is determined as the main input device. This can avoid the situation where the determined frame number update time is abnormal due to the abnormal input device, and thus can improve the fault tolerance of the video transmission system when displaying multiple input videos.

[0117] like Figure 4 As shown, Figure 4 A schematic diagram of an input device receiving a video frame provided in an embodiment of the present application. Figure 4 In the example, each input device works under the same time axis. The receiving start time of input device #0 is a0, the receiving start time of input device #1 is a1, the receiving start time of input device #i is ai, the receiving start time of input device #j is aj, and the receiving start time of input device #n is an. Among them, the receiving start time of input device #n is the earliest, that is, the first time is an; the receiving start time of input device #j is the latest, that is, the second time is aj. Determine the time corresponding to the bisection point of the time interval with the first time and the second time as endpoints, that is, the third time is d. Then, the input device corresponding to the receiving start time with the smallest duration between the third time (that is, input device #1) can be determined as the main input device.

[0118] In addition, in actual scenarios, in order to ensure the validity of the determined frame number update time, the time difference between the source end sending each video frame can be reduced so that the time difference between the receiving start time corresponding to each input device is no more than 1 / 3 of the frame interval of the input device receiving the video frame. Thus, the validity of the correction of the frame number update time can be guaranteed. For example, for Figure 4 , △t1, △t2, △tn are all less than 1 / 3 of the frame interval of the video frame received by the input device. Among them, the frame interval of the video frame received by the input device represents the time difference between the start time of receiving two adjacent video frames, and the time difference is the reciprocal of the frame rate of the video sent by the source. For example, the frame rate of the video sent by the source is 60fps, that is, the source sends 60 video frames to each input device per second. Correspondingly, the frame interval of the video frame received by each input device is 1S / 60, that is, 16.6ms (milliseconds).

[0119] In the present application, the process of adding a preset duration to the reception end time of the first video frame by the main input device can be called a calibration. For example, when the reception of the first video frame is completed, that is, when the reception end time of the first video frame is reached, the main input device can add a preset duration to the reception end time to obtain the frame number update time.

[0120] The size of the preset duration can be determined by technical personnel based on current business needs, so that the fourth moment obtained by a calibration is located after the end moment of reception of the first video frame by each input device and before the start moment of reception of the next frame by the main input device.

[0121] For example, the frame interval of the input device receiving the video frame can be increased based on the receiving start time of the first video frame to obtain the receiving start time of the input device receiving the next frame. The source end sends the same frame rate of each video.

[0122] In one embodiment, the above step S2021 includes:

[0123] Step 1: Add a preset duration to the acquired receiving end time to obtain a fourth time.

[0124] Step 2: If the fourth moment is outside the preset time interval, the fourth moment is determined as the frame number update moment.

[0125] The center of the preset time interval is the unified sending time.

[0126] Step 3: If the fourth moment is within the preset time interval, a fifth moment that is not earlier than the end moment of the preset time interval is determined as the frame number update moment.

[0127] The center of the preset time interval is the unified sending time. For example, the start time of the preset time interval can be obtained by extending the jitter duration forward from the unified sending time, and the end time of the preset time interval can be obtained by extending the jitter duration backward from the unified sending time. Correspondingly, in the present application, the preset time interval can also be called a jitter window. For example, the jitter duration can be 1 microsecond.

[0128] In actual scenarios, although a unified sending time is set for each input device, when each input device sends a video frame according to the unified sending time, there will be a certain error between the actual sending times (which can be called the sending start time), that is, the actual sending start time of each input device may not be consistent. Therefore, a preset time interval can be determined based on the unified sending time, so that the actual sending start time of each input device is within the preset time interval.

[0129] If the fourth moment obtained after a calibration is within the preset time interval, there may be input devices whose actual sending start time is before the fourth moment, and there may be input devices whose actual sending start time is after the fourth moment. This will cause some input devices to send video frames after determining the first frame number, while some input devices will determine the first frame number after sending the video frame. This will cause each input device to send video frames with inconsistent frame numbers, causing the video images displayed by multiple video display terminals to be out of sync.

[0130] Therefore, in order to avoid the influence caused by the time difference between the actual sending start times of each input device, calibration can be performed again, that is, the frame number update time is calibrated to outside the preset time interval. This process can be called secondary calibration.

[0131] For example, the end time of the preset time interval can be determined as the frame number update time; or, any time after the end time of the preset time interval and before the start time of the main input device receiving the next frame can be determined as the frame number update time.

[0132] Based on the above processing, the frame number update time is calibrated to outside the preset time interval, so that each input device determines the first frame number after sending the video frame. This can unify the timing for each input device to determine the first frame number, and further, can avoid each input device sending video frames with inconsistent frame numbers, thereby avoiding the situation where the video images displayed by multiple video display terminals are out of sync.

[0133] like Figure 5 As shown, Figure 5 A schematic diagram of a process for determining a first frame sequence number provided in an embodiment of the present application.

[0134] S501: Waiting for receiving the start time.

[0135] That is, each input device receives the first video frame, and sends the start time of receiving the first video frame to other input devices, thereby determining the main input device.

[0136] S502: The main input device obtains the receiving end time VIN_T_end of the first video frame.

[0137] S503: The master input device delays the receiving end time VIN_T_end by Δt.

[0138] The main input device performs a calibration, that is, the main input device adds a preset time length △t based on the end time of receiving the first video frame to obtain the delay time (that is, the fourth time in the present application).

[0139] S504: Determine whether the delay time is within the jitter window.

[0140] Determine whether the fourth moment is within the jitter window (ie, the preset time interval in this application). If the fourth moment is within the preset time interval, execute step S506; if the fourth moment is outside the preset time interval, execute step S505.

[0141] S505: Determine the delay time of the main input device that is delayed by Δt as the frame sequence number update time.

[0142] S506: Determine the time SEND_T_back corresponding to the trailing edge of the jitter window as the frame number update time.

[0143] The main input device performs secondary calibration, that is, obtains the time SEND_T_back corresponding to the trailing edge of the jitter window. The time corresponding to the trailing edge of the jitter window is also the end time of the preset time interval in this application.

[0144] S507: Determine the frame sequence number.

[0145] That is, the main input device sends the frame number update time to other input devices, and then, when the frame number update time is reached, each input device can determine the first frame number at this time.

[0146] S508: Calibration ends.

[0147] Subsequently, the above steps S501-S507 are repeated, and each input device re-determines the main input device and the first frame number when receiving a new video frame.

[0148] In one embodiment, the method further comprises:

[0149] If it is not the main input device and has not determined the first frame number based on the frame number update time sent by the main input device before receiving the next video frame from its own receiving port, the first frame number is determined when the next video frame is received from its own receiving port.

[0150] For each input device, after starting to receive the first video frame, if the input device determines that it is not the main input device, it can receive the frame sequence number update time sent by the main input device.

[0151] However, when the master input device performs a calibration, the frame number update time may be determined as a time after the receiving start time of the next frame received by the input device. In addition, when the master input device performs a calibration and a secondary calibration, the frame number update time may also be determined as a time after the receiving start time of the next frame received by the input device.

[0152] The above two situations will cause the input device to not determine the first frame number based on the frame number update time sent by the main input device when it starts to receive the next frame. Therefore, in order to make itself and other input devices send video frames with consistent frame numbers at the same time, the input device can actively determine the first frame number.

[0153] In addition, due to reasons such as communication failure with the main input device, the input device may not receive the frame number update time sent by the main input device, which means that the input device fails to determine the first frame number before receiving the next video frame. Therefore, in order to allow itself and other input devices to send video frames with the same frame number at the same time, the input device can actively determine the first frame number.

[0154] Based on the above processing, the first frame number can be actively determined from the input device, so that each input device sends a video frame with the same frame number at the same time, thereby avoiding the situation where the video images displayed by multiple video display terminals are out of sync.

[0155] Based on the same inventive concept, an embodiment of the present application further provides a video transmission system, the video transmission system comprising a plurality of input devices, and an output device corresponding to each input device;

[0156] Each input device is used to execute the steps of any video transmission method in the above embodiments;

[0157] Each output device is used to receive and display the video frames sent by the corresponding input device.

[0158] like Figure 6 As shown, Figure 6 A schematic diagram of the structure of a video transmission system provided in an embodiment of the present application. The video transmission system includes a source end, multiple input devices, a video data exchange network, multiple output devices, and multiple video display terminals. When each input device receives a video frame from its respective receiving port, it can send the receiving start time of the received video frame to other input devices. Then, based on the receiving start time of each input device, the main input device is determined, for example, input device #0. Figure 7 As shown, Figure 7 A schematic diagram of an input device interaction structure provided in an embodiment of the present application. Figure 7 In the example, the main input device (i.e., input device #0) can communicate data with other input devices (input device #1-input device #n).

[0159] When the unified sending time corresponding to each input device is reached, each input device can obtain the first frame number and send the video frame corresponding to the first frame number to the corresponding output device through the video data exchange network. Each output device is connected to each video display terminal. After receiving the corresponding video frame, each output device enables the corresponding video display terminal to display the video image at the unified display time. Each input device and each output device work on the same timeline, and the unified sending time of each input device is consistent with the output display time of each output device.

[0160] Based on the above processing, after receiving the video frame sent by the input device, the output device can store the obtained video frame in the video memory without processing, and multiple video display terminals can directly display the video frame received by the output device, and finally present a complete display picture.

[0161] Based on the same inventive concept, the embodiment of the present application also provides a video transmission device, which is applied to a target input device, wherein the target input device is any input device in a video transmission system, and the video transmission system also includes other input devices except the target input device. Figure 8 , Figure 8 A structural diagram of a video transmission device provided in an embodiment of the present application, the device comprising:

[0162] The video frame receiving module 801 is used to receive the first video frame from its own receiving port and set a corresponding frame sequence number for the received first video frame;

[0163] The first acquisition module 802 is used to acquire the frame number update time; wherein the frame number update time is obtained by adding a preset time length to the receiving end time corresponding to the designated input device, and the frame number update time is located after the receiving end time when each input device receives the first video frame from its respective receiving port; the receiving end time corresponding to the designated input device indicates: the receiving end time when the designated input device receives the first video frame from its own receiving port; the first video frames received by each input device have the same corresponding timestamp in the original video image at the source end;

[0164] A first updating module 803 is configured to determine a first frame sequence number by setting a frame sequence number for a received video frame when the frame sequence number update time is reached; wherein the first frame sequence number is used to indicate a video frame to be sent next;

[0165] The video frame sending module 804 is used to send the video frame corresponding to the first frame sequence number when a unified sending time is reached; wherein the unified sending time represents the time when all input devices in the video transmission system uniformly send video frames.

[0166] In some embodiments, the first acquisition module 802 includes:

[0167] A first sending submodule is used for obtaining the end time of receiving the first video frame from its receiving port if it is the main input device; based on the obtained end time of receiving, adding the preset duration to obtain the frame sequence number update time, and sending the frame sequence number update time to other input devices;

[0168] and / or,

[0169] The first receiving submodule is configured to receive the frame sequence number update time sent by the main input device if the first receiving submodule is not the main input device.

[0170] In some embodiments, the apparatus further comprises:

[0171] A first sending module is used to send a receiving start time of receiving the first video frame from its own receiving port to other input devices when the first video frame is received from its own receiving port;

[0172] A second determining module is used to determine the earliest receiving start time of receiving the first video frame from the respective receiving ports of the input devices as the first time, and to determine the latest receiving start time of receiving the first video frame from the respective receiving ports of the input devices as the second time;

[0173] A third determining module, configured to determine a time corresponding to an equinox of a time interval having the first time and the second time as endpoints as a third time;

[0174] The master device determination module is used to determine the input device corresponding to the receiving start time with the shortest duration between the third time moments as the master input device.

[0175] In some embodiments, the first sending submodule is specifically used to:

[0176] Add the preset duration to the acquired receiving end time to obtain a fourth time;

[0177] If the fourth moment is outside the preset time interval, the fourth moment is determined to be the frame sequence number update moment; wherein the center of the preset time interval is the unified sending moment;

[0178] If the fourth moment is within the preset time interval, a fifth moment that is not earlier than the end moment of the preset time interval is determined as the frame sequence number update moment.

[0179] In some embodiments, the apparatus further comprises:

[0180] The fourth determination module is used to determine the first frame number when the next video frame is received from its own receiving port if it is not the main input device and has not determined the first frame number based on the frame number update time sent by the main input device before receiving the next video frame from its own receiving port.

[0181] The present application also provides an electronic device, such as Fig. 9 As shown, including:

[0182] Memory 901, used for storing computer programs;

[0183] The processor 902 is used to implement the steps of any video transmission method in the above embodiments when executing the program stored in the memory 901.

[0184] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 902, the communication interface, and the memory 901 communicate with each other via the communication bus.

[0185] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0186] The communication interface is used for communication between the above electronic device and other devices.

[0187] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0188] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0189] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned video transmission methods are implemented.

[0190] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the video transmission methods in the above embodiments.

[0191] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), etc.

[0192] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0193] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system, device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0194] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A video transmission method, characterized in that: The method is applied to a target input device, the target input device is any input device in a video transmission system, and the video transmission system also includes other input devices except the target input device. The method includes: Receive a first video frame from its own receiving port, and set a corresponding frame sequence number for the received first video frame; Obtaining a frame number update time; wherein the frame number update time is obtained by adding a preset time length to the receiving end time corresponding to the designated input device, and the frame number update time is located after the receiving end time when each input device receives the first video frame from its respective receiving port, and before the receiving start time when the main input device receives the next frame; the receiving end time corresponding to the designated input device indicates: the receiving end time when the designated input device receives the first video frame from its own receiving port; the first video frames received by each input device have the same corresponding timestamp in the original video image at the source end; the main input device is an input device used to determine the frame number update time; When the frame sequence number update time is reached, a first frame sequence number is determined in accordance with a method of setting a frame sequence number for a received video frame; wherein the first frame sequence number is used to indicate a video frame to be sent next; When a unified sending time is reached, the video frame corresponding to the first frame sequence number is sent; wherein the unified sending time represents the time when all input devices in the video transmission system uniformly send video frames.

2. The method according to claim 1, characterized in that The designated input device is a main input device, and the obtaining of the frame sequence number update time comprises: If it is the main input device, then obtain the end time of receiving the first video frame from its receiving port; based on the obtained end time of receiving, increase the preset duration to obtain the frame number update time, and send the frame number update time to other input devices; and / or, If it is not the main input device, it receives the frame sequence number update time sent by the main input device.

3. The method according to claim 2, characterized in that The method further comprises: When receiving the first video frame from its own receiving port, sending the receiving start time of receiving the first video frame from its own receiving port to other input devices; Determine the earliest receiving start time of the receiving start times of the first video frame received by each input device from its respective receiving port as the first time, and determine the latest receiving start time of the receiving start times of the first video frame received by each input device from its respective receiving port as the second time; Determine a time corresponding to an equinox of a time interval with the first time and the second time as endpoints as a third time; The input device corresponding to the receiving start time with the shortest duration between the third time moments is determined as the main input device.

4. The method according to claim 2, characterized in that: The step of increasing the preset duration based on the acquired receiving end time to obtain the frame sequence number update time includes: Add the preset duration to the acquired receiving end time to obtain a fourth time; If the fourth moment is outside the preset time interval, the fourth moment is determined to be the frame sequence number update moment; wherein the center of the preset time interval is the unified sending moment; If the fourth moment is within the preset time interval, a fifth moment that is not earlier than the end moment of the preset time interval is determined as the frame sequence number update moment.

5. The method according to claim 2, characterized in that: The method further comprises: If it is not the main input device and has not determined the first frame number based on the frame number update time sent by the main input device before receiving the next video frame from its own receiving port, then the first frame number is determined when the next video frame is received from its own receiving port.

6. A video transmission system, characterized in that: The video transmission system includes a plurality of input devices, and an output device corresponding to each input device; Each input device is used to execute the method described in any one of claims 1 to 5 above; Each output device is used to receive and display the video frames sent by the corresponding input device.

7. A video transmission device, characterized in that: The device is applied to a target input device, the target input device is any input device in a video transmission system, the video transmission system also includes other input devices except the target input device, and the device includes: The video frame receiving module is used to receive the first video frame from its own receiving port and set a corresponding frame sequence number for the received first video frame; The first acquisition module is used to acquire the frame number update time; wherein the frame number update time is obtained by adding a preset time length to the receiving end time corresponding to the designated input device, and the frame number update time is located after the receiving end time of each input device receiving the first video frame from its respective receiving port, and before the receiving start time of the main input device receiving the next frame; the receiving end time corresponding to the designated input device indicates: the receiving end time of the first video frame received by the designated input device from its own receiving port; the first video frames received by each input device have the same corresponding timestamp in the original video image at the source end; the main input device is an input device used to determine the frame number update time; A first determining module, configured to determine a first frame sequence number in a manner of setting a frame sequence number for a received video frame when the frame sequence number update time is reached; wherein the first frame sequence number is used to indicate a video frame to be sent next; The video frame sending module is used to send the video frame corresponding to the first frame sequence number when a unified sending time is reached; wherein the unified sending time represents the time when all input devices in the video transmission system uniformly send video frames.

8. The device according to claim 7, characterized in that The designated input device is a main input device, and the first acquisition module includes: A first sending submodule is used for obtaining the end time of receiving the first video frame from its receiving port if it is the main input device; based on the obtained end time of receiving, adding the preset duration to obtain the frame sequence number update time, and sending the frame sequence number update time to other input devices; and / or, The first receiving submodule is configured to receive the frame sequence number update time sent by the main input device if the first receiving submodule is not the main input device.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-5 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images