Video transmission method, device, equipment and storage medium

Abstract

The present invention discloses a video transmission method, apparatus, device and storage medium. The video transmission method includes the following steps: obtaining a list of available candidate terminals; binding the available candidate terminals with layout video streams to form a binding list, wherein the layout video stream is the video stream being played in the video conference; upon receiving a key frame acquisition request from a conference terminal, selecting an auxiliary conference terminal according to the key frame acquisition request and the binding list; sending a re-encoding instruction to the auxiliary conference terminal, wherein the re-encoding instruction is used to instruct the auxiliary conference terminal to re-encode the target image frame to form a target key frame, and sending the target key frame to the conference terminal that issued the key frame acquisition request. The present invention enables each video code stream to basically only transmit P frames, and only transmits I frames when periodic I frames arrive, thereby reducing bandwidth transmission requirements and transmission fluctuations.

Description

Technical Field

[0001] The present invention relates to the technical field of video network transmission, and more specifically, to a video transmission method, apparatus, device and storage medium. Background Art

[0002] With the development of video conferencing technology, multi-screen technology has emerged. In a video conference, each conference terminal can capture images and audio. After connecting to the server to join the conference, each conference terminal encodes the captured images and audio and sends the encoded data to the server. The server then transmits the images and audio sent by each conference terminal to other terminals according to the conference requirements, thus enabling audio and video calls for multiple participants. This is the "send, receive, multiple" feature of a conference terminal.

[0003] Currently, when a new terminal joins a conference or a conference terminal hasn't received data from a particular video stream for an extended period, it requests a keyframe (I-frame) from the source conference terminal. The source conference terminal responds to this request by immediately generating and sending a keyframe. Because the source conference terminal only generates one video stream, it generates a keyframe at the location of the original interframe (P-frame or B-frame). This newly generated keyframe is sent not only to the terminal that requested it but also to other conference terminals that don't require it. These conference terminals, which should normally receive interframes, suddenly become heavily bandwidth-intensive, disrupting existing network policies and affecting network stability. Summary of the Invention

[0004] In order to overcome at least one of the above-mentioned defects of the prior art, the present invention provides a video transmission method, device, equipment and storage medium. The technical solution adopted by the present invention is as follows.

[0005] In a first aspect, the present invention provides a video transmission method, comprising the steps of:

[0006] Obtain a list of available candidate terminals, where the candidate terminals are terminals that only need to pull video streams from the media server in the video conference;

[0007] Binding available candidate terminals with layout video streams to form a binding list, wherein the layout video streams are video streams being played in the video conference;

[0008] When a key frame acquisition request is received from a conference terminal, an auxiliary conference terminal is selected according to the key frame acquisition request and the binding list, wherein the key frame acquisition request includes: information of a target image frame;

[0009] A re-encoding instruction is sent to the auxiliary conference terminal, wherein the re-encoding instruction is used to instruct the auxiliary conference terminal to re-encode the target image frame to form a target key frame, and send the target key frame to the conference terminal that issued the key frame acquisition request.

[0010] In one embodiment, the process of binding available candidate terminals and layout video streams to form a binding list includes the following steps:

[0011] Bind a layout video stream to at least one available candidate terminal.

[0012] In one embodiment, the process of binding available candidate terminals and layout video streams to form a binding list includes the following steps:

[0013] Bind the multi-channel layout video streams as a whole to at least one available candidate terminal.

[0014] In one embodiment, when the number of available candidate terminals is less than the number of video streams being played in the video conference, the process of binding the available candidate terminals with the layout video streams to form a binding list includes the steps of:

[0015] Bind all layout video streams as a whole to all available candidate terminals.

[0016] In one embodiment, when the number of available candidate terminals is not less than the number of video streams being played in the video conference, the process of binding the available candidate terminals with the layout video streams to form a binding list includes the steps of:

[0017] Bind each layout video stream to at least one available candidate terminal.

[0018] In one embodiment, the process of binding available candidate terminals and layout video streams to form a binding list further includes the steps of:

[0019] When adding candidate terminals for binding, the added candidate terminals are bound to the layout video stream with the least number of bound terminals.

[0020] In one embodiment, the process of binding available candidate terminals and layout video streams to form a binding list further includes the steps of:

[0021] When removing bound candidate terminals, if there are layout video streams without bound candidate terminals after the removal, the candidate terminals will be unbound from the layout video stream with the largest number of bound terminals, and the unbound candidate terminals will be bound to the layout video stream without bound candidate terminals.

[0022] In one embodiment, the re-encoding instruction is also used to instruct the auxiliary conference terminal to re-encode the image frames starting from the target image frame in the GOP where the target image frame is located to form a target video code stream, and send the target video code stream to the conference terminal that issued the key frame acquisition request.

[0023] In one embodiment, the re-encoding instruction is also used to instruct the auxiliary conference terminal to re-encode the image frames starting from the target image frame in the GOP where the target image frame is located to form a target video code stream, and send the target video code stream to the conference terminal that issued the key frame acquisition request.

[0024] In one embodiment, the method further includes the following steps: regularly checking available candidate terminals according to the established inspection items, and removing candidate terminals whose inspection items do not meet the preset standards from the binding list.

[0025] In one embodiment, the formulated inspection items include: encoding and decoding capabilities and / or network performance.

[0026] In one embodiment, the steps are further included:

[0027] Inspect auxiliary conference terminals according to the established inspection items;

[0028] When an inspection item of the auxiliary conference terminal does not meet the preset standard, the auxiliary conference terminal is removed from the binding list and a new auxiliary conference terminal is selected.

[0029] In one embodiment, the formulated inspection items include: encoding and decoding capabilities and / or network performance.

[0030] In a second aspect, the present invention provides a video transmission device, characterized by comprising:

[0031] An acquisition module is used to obtain a list of available candidate terminals, wherein the candidate terminals are terminals in the video conference that only need to pull video streams from the media server;

[0032] A binding module, configured to bind available candidate terminals to a layout video stream to form a binding list, wherein the layout video stream is a video stream being played in a video conference;

[0033] A selection module is configured to select an auxiliary conference terminal according to the key frame acquisition request and the binding list when receiving a key frame acquisition request from a conference terminal, wherein the key frame acquisition request includes: information of a target image frame;

[0034] The sending module is used to send a re-encoding instruction to the auxiliary conference terminal, wherein the re-encoding instruction is used to instruct the auxiliary conference terminal to re-encode the target image frame to form a target key frame, and send the target key frame to the conference terminal that issued the key frame acquisition request.

[0035] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned methods when executing the program.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program implements the method of any of the above-mentioned embodiments when executed by a processor.

[0037] In the present invention, when a new conference terminal joins the conference, or other conference terminals need to obtain the current video frame in I frame format, the conference terminal that only needs to pull the video stream from the media server is arranged to undertake the task of I frame re-encoding, so that it forms the required I frame and sends it to the conference terminal that issues the I frame request, thereby avoiding the source conference terminal from generating and sending the required I frame to all conference terminals, so that each video code stream basically only transmits P frames, and only transmits I frames when periodic I frames arrive, reducing bandwidth transmission requirements and reducing transmission fluctuations.

[0038] Furthermore, compared to using dedicated video stream processing equipment to re-encode key frames, this method reduces the number of devices by fully utilizing "idle" conference terminals. Furthermore, when a new terminal joins the conference and multiple video channels need to be re-encoded, this method effectively schedules each terminal, reducing workload and bandwidth usage compared to using only one video stream processing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flowchart of embodiment 1 of the present invention.

[0040] Figure 2 This is a schematic diagram of video transmission according to the first embodiment of the present invention.

[0041] Figure 3 This is a schematic diagram of playback of a conference terminal according to the first embodiment of the present invention.

[0042] Figure 4 It is a schematic diagram of the overall structure of embodiment 2 of the present invention. DETAILED DESCRIPTION

[0043] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0044] It should be noted that the terms "first, second, ..." used in the embodiments of the present invention are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that the terms "first, second, ..." may interchangeably represent a specific order or precedence, where permitted. It should be understood that the objects distinguished by "first, second, ..." may be interchangeable, where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0045] Example 1

[0046] See Figure 1 , Figure 1 1 is a flow chart of a video transmission method provided in the first embodiment of the present invention, the method includes step S110, step S120, step S130 and step S140. It should be noted that step S110, step S120, step S130 and step S140 are only reference numerals for the purpose of clearly explaining the embodiment and the appendix. Figure 1 The corresponding relationship does not limit the order of the steps in this embodiment.

[0047] Step S110 : obtaining a list of available candidate terminals, wherein the candidate terminals are terminals that only need to pull video streams from the media server in the video conference.

[0048] like Figure 2 As shown, Figure 2 The example in Figure 1 illustrates a video conference involving at least six conference terminals T1-T6. In a video conference, servers typically include a signaling server responsible for sending signaling and a media server responsible for forwarding video. The method of the present invention is applied to the signaling server. Of course, it is also possible for a single server to perform both signaling and media forwarding. In this case, the method is also applicable to such servers.

[0049] like Figure 3 As shown, Figure 3 The layout of the video conference displayed by each conference terminal is shown in the figure. In a video conference, usually only the images collected by some important conference terminals need to be played. Figure 3In the layout screen, there are only the screens of conference terminals T1-T4. Since the videos of T1-T4 need to be played in the layout screen, conference terminals T1-T4 need to push the video streams they have collected and encoded to the media server. At the same time, they also need to pull the video streams of other terminals in the layout from the media server. However, for conference terminals other than conference terminals T1-T4 (such as T5 and T6), only the video streams of conference terminals T1-T4 need to be pulled from the media server. In this step S110, what is obtained is a list of conference terminals of the type T5 and T6, and this list is a list of available candidate terminals. In addition, Figure 3 One video stream in this type of layout screen is the layout video stream.

[0050] It should be noted that the list of available candidate terminals may include all candidate terminals that only need to pull video streams from the media server, or some conference terminals may be excluded from the list of candidate terminals according to actual conditions.

[0051] Step S120 : Bind the available candidate terminals with the layout video streams to form a binding list, wherein the layout video streams are the video streams being played in the video conference.

[0052] Step S130 : upon receiving a key frame acquisition request from a conference terminal, selecting an auxiliary conference terminal according to the key frame acquisition request and the binding list, wherein the key frame acquisition request includes information of a target image frame.

[0053] When a new conference terminal joins the conference, or when one of the conference terminals has not received data from a certain video stream for a long time, or in other situations, these conference terminals will issue a key frame acquisition request, requesting to obtain the current video frame in I-frame format. This is because these conference terminals are all joined midway during a GOP (Group of Pictures), and without the I-frame in the current GOP, the image cannot be restored, so the request is to obtain the I-frame. Of course, the I-frame obtained by this request is not necessarily the I-frame in the current GOP, it can be the current frame. In most cases, the current frame is a P-frame, so the decoded video frame needs to be re-encoded by a conference terminal that can decode the P-frame and restore it to the current frame, so as to obtain the I-frame.

[0054] Candidate terminals are actually "receive-only" terminals, so they occupy less bandwidth, meaning they have a larger bandwidth available for borrowing. Available candidate terminals are selected from the candidate terminals and are suitable as auxiliary conference terminals. If there are no available candidate terminals, it means there are no "spare" conference terminals that can be borrowed. In this case, the only option is to use existing methods to have the source conference terminal generate key frames at the location where the key frames are requested. If there are available candidate terminals, it means there are "spare" conference terminals that can be borrowed. At this time, based on a preset strategy, one of these candidate terminals is selected as an auxiliary conference terminal. This auxiliary conference terminal is used to "produce" and upload the key frames required by the conference terminal that issued the key frame request.

[0055] This method uses "idle" conference terminals to re-encode key frames, so it involves the problem of adopting different scheduling methods for "idle" conference terminals according to different situations. Step S120 binds the available candidate terminals with the layout video streams to form a binding list, which facilitates the scheduling of available candidate terminals according to the binding list in the future.

[0056] Step S130 is the process of selecting available candidate terminals after receiving a key frame acquisition request for a layout video stream from a conference terminal. The key frame acquisition request contains information about the target image frame. This information indicates which layout video stream the key frame acquisition request is intended for re-encoding. This information can be combined with the binding list to determine which available candidate terminals should be used as auxiliary conference terminals.

[0057] It should be noted here that if Figure 3 In the example shown, T1-T4 actually have four video streams, and the key frame acquisition request here is only issued for one of the streams.

[0058] In some cases, more than one candidate terminal may be selected from the binding list. In this case, a variety of other methods can be used to determine a single auxiliary conferencing terminal from among these terminals. For example, candidate terminals can rotate as auxiliary conferencing terminals according to a predetermined order, or tasks can be allocated based on the available bandwidth of each candidate terminal. Furthermore, priority can be assigned to terminals during binding and the auxiliary conferencing terminal can be determined based on priority. In short, those skilled in the art can reasonably select an appropriate auxiliary conferencing terminal based on the actual situation.

[0059] Step S140: Send a re-encoding instruction to the auxiliary conference terminal, wherein the re-encoding instruction is used to instruct the auxiliary conference terminal to re-encode the target image frame to form a target key frame, and send the target key frame to the conference terminal that issued the key frame acquisition request.

[0060] After selecting an auxiliary conference terminal, a command is sent to it, instructing it to generate the key frame requested. Although the auxiliary conference terminal receives P frames from the media server, it receives the entire GOP and is therefore capable of converting the current P frames into regular image frames. This re-encodes the frames to form the key frames requested by the key frame request. Once the target key frames are generated, they are sent to the requesting conference terminal. The conference terminal then decodes and restores the video stream based on the target key frames and the P frames in the video stream forwarded by the media server.

[0061] It should be noted that in this method, relevant information can be directly added to the re-encoding instruction to inform the auxiliary conference terminal which frame needs to be re-encoded, or an instruction can be sent to the media server, causing the media server to add a mark to the corresponding frame, thereby letting the auxiliary conference terminal know which frame needs to be re-encoded. Those skilled in the art can select an appropriate method to inform the auxiliary conference terminal of the video frame that needs to be re-encoded based on actual circumstances.

[0062] It should be noted that the auxiliary conference terminal may send the key frame to the conference terminal that issued the key frame request directly or indirectly, such as returning the key frame to the signaling server, which then forwards it to the target terminal.

[0063] In this method, when a new conference terminal joins the conference, or other conference terminals need to obtain the current video frame in I frame format, the conference terminals that only need to pull the video stream from the media server are arranged to take on the task of re-encoding the I frame, so that they form the required I frame and send it to the conference terminal that issued the I frame request, thereby avoiding the source conference terminal from generating and sending the required I frame to all conference terminals, so that each video code stream basically only transmits P frames, and only transmits I frames when periodic I frames arrive, reducing bandwidth transmission requirements and reducing transmission fluctuations.

[0064] Furthermore, compared to using dedicated video stream processing equipment to re-encode key frames, this method reduces the number of devices by fully utilizing "idle" conference terminals. Furthermore, when a new terminal, or even multiple new terminals, join a conference, a large number of key frames need to be re-encoded. Compared to using only a single video stream processing device, this method can fully schedule each terminal to simultaneously re-encode, reducing the workload and bandwidth usage of individual terminals.

[0065] In one embodiment, the process of step S120 includes: step S121.

[0066] Step S121: Bind one layout video stream to at least one available candidate terminal.

[0067] like Figure 2 、 3 As shown, frames T1-T4 correspond to the video streams of conference terminals T1-T4, respectively. In this implementation, at least one available candidate terminal is bound to one of the video streams. For example, T6 is bound to T1's video stream. When a terminal issues a key frame request for T1's video stream, T6 is selected as the auxiliary conference terminal and is instructed to re-encode the required I-frame.

[0068] Dedicating available candidate terminals to a specific video stream can effectively reduce the preparation time of encoder resources.

[0069] It should be noted here that when a video stream is bound to multiple available candidate terminals, there may be more than one candidate terminal selected through the aforementioned binding list. At this time, a suitable auxiliary conference terminal can be reasonably selected according to the actual situation (such as the method mentioned above).

[0070] In one embodiment, the process of step S120 includes: step S122.

[0071] Step S122: Bind the multi-channel layout video streams as a whole to at least one available candidate terminal.

[0072] like Figure 2 、 3 As shown, screens T1-T4 correspond to the video streams of conference terminals T1-T4, respectively. In this embodiment, the multi-channel layout video streams are bound as a whole to at least one available candidate terminal, such as in Scenario 1: T5 is bound to the video streams of T1 and T2 (only one candidate terminal is bound), or Scenario 2: T5 and T6 are bound to the video streams of T1, T2, and T3 (multiple candidate terminals are bound), or Scenario 3: T5 and T6 are bound to the video streams of T1-T4 (all candidate terminals are bound).

[0073] In the case where T5 is bound to T1 and T2 video streams, when a terminal sends a key frame request for T1 and T2 video streams, T5 is selected as the auxiliary conference terminal.

[0074] In the case where T5 and T6 are bound to T1, T2, and T3 video streams, when a terminal issues a key frame request for T1, T2, or T3, T5 or T6 is selected as the auxiliary conference terminal.

[0075] In the case where T5, T6 and T1-T4 video streams are bound together, when a terminal sends a key frame request, T5 or T6 is selected as the auxiliary conference terminal.

[0076] In situations 2 and 3, there may be more than one candidate terminal selected through the binding list. A suitable auxiliary conference terminal can be reasonably selected according to the actual situation.

[0077] If there are a lot of key frame acquisition requests for a certain layout video stream, this implementation can reduce the encoding burden of the encoder.

[0078] In one embodiment, when the number of available candidate terminals is less than the number of video streams being played in the video conference, the process of step S120 includes: step S123.

[0079] Step S123: Bind all layout video streams as a whole to all available candidate terminals.

[0080] When the number of available candidate terminals is less than the number of video streams being played in the video conference, it means that it is not possible to equip each video stream with a "full-time" conference terminal. In this case, a better approach is to let the available candidate terminals take turns to take on the task of re-editing I frames to reduce the encoding burden of the encoder.

[0081] In one embodiment, when the number of available candidate terminals is not less than the number of video streams being played in the video conference, the process of step S120 includes: step S124.

[0082] Step S124: Bind each layout video stream to at least one available candidate terminal.

[0083] In this embodiment, at least one "dedicated" conference terminal is equipped for each layout video stream. When a key frame of the video is requested, the "dedicated" conference terminal is responsible for re-editing the I frame to reduce the preparation time of the encoder resources.

[0084] Preferably, this embodiment further includes: step 125 and / or step S126.

[0085] Step 125 : When adding candidate terminals for binding, the added candidate terminals are bound to the layout video stream with the least number of bound terminals.

[0086] Step 126: When removing bound candidate terminals, if there are layout video streams without bound candidate terminals after the removal, unbind the candidate terminals from the layout video stream with the largest number of bound terminals, and bind the unbound candidate terminals to the layout video stream without bound candidate terminals.

[0087] During a video conference, when a new conference terminal joins, this new conference terminal has the opportunity to become an available candidate terminal. Alternatively, during the video conference, the layout of video streams will be adjusted and some layout video streams will be removed. In this case, the terminals corresponding to these video streams will have the opportunity to become available candidate terminals again. Alternatively, terminals that were previously excluded due to poor network conditions will become available candidate terminals again as the network conditions improve. When these terminals become new available candidate terminals, they must be bound to the new available candidate terminals.

[0088] At this time, to balance the number of terminals bound to each layout video stream, you can bind it to the layout video stream with the least number of bound terminals. If there are multiple least bound terminals, you can bind it to one of the video streams based on certain rules, such as a predetermined order or video stream priority.

[0089] In a video conference, the conference terminals in the binding list may be removed from the binding list due to various reasons, such as being disconnected, exiting the conference, being incompetent, or becoming the terminal for layout video streams, etc. In this case, the candidate terminals for binding must be removed. After the binding is removed, there may be a layout video stream that does not have a "full-time" conference terminal responsible for re-editing the I frame. At this time, the previously bound candidate terminals need to be adjusted. In step S126, the adjustment method is to unbind a terminal from the layout video stream with the most bound terminals, and transfer the unbound terminal to the name of the video stream that does not have a "full-time" conference terminal responsible for re-editing the I frame. If there are multiple most bound terminals, one can be selected for unbinding according to certain rules, such as a predetermined order or the priority of the video stream.

[0090] This method utilizes "idle" conference terminals to re-encode key frames, but the premise of utilization is that these conference terminals meet the basic requirements for re-encoding key frames. The following implementation method is developed around ensuring that the conference terminals meet the requirements for re-encoding key frames.

[0091] In one embodiment, the video transmission method further includes step S150.

[0092] Step S140 : regularly checking available candidate terminals according to the established inspection items, and removing candidate terminals whose inspection items do not meet the preset standards from the binding list.

[0093] The available candidate terminals may not be able to generate or send key frames due to their own reasons. For example, the encoding and decoding capabilities of the candidate terminals are poor, and it takes a long time to complete the re-editing of the key frames, or the candidate terminals are set with special permissions and cannot perform re-editing of the key frames, or the network conditions of the candidate terminals are very poor, and the re-edited key frames cannot be sent out smoothly within the specified time. Therefore, in this embodiment, inspection items are formulated in advance, and the available candidate terminals are inspected regularly, and terminals that fail the inspection items are removed from the binding list.

[0094] Preferably, the formulated inspection items include: encoding and decoding capabilities and / or network performance.

[0095] If a candidate terminal cannot re-encode a P-frame into an I-frame within the preset timeframe, the candidate terminal's encoding capability is considered substandard. The time between the media server sending a video packet and the candidate terminal receiving the acknowledgment packet is calculated. If this timeframe falls outside the preset timeframe, the candidate terminal's network capability is considered substandard.

[0096] In one embodiment, the video transmission method further includes the steps of:

[0097] Inspect auxiliary conference terminals according to the established inspection items;

[0098] When an inspection item of the auxiliary conference terminal does not meet the preset standard, the auxiliary conference terminal is removed from the binding list and a new auxiliary conference terminal is selected.

[0099] Checking the auxiliary conference terminal is helpful to ensure that the re-encoded key frame can be sent to the conference terminal that issued the key frame request on time. Even if the regular check of the previous implementation method is not performed, the task can be completed on time.

[0100] Similar to the previous embodiment, preferably, the formulated inspection items include: encoding and decoding capabilities and / or network performance.

[0101] In one embodiment, the re-encoding instruction is also used to instruct the auxiliary conference terminal to re-encode the image frames starting from the target image frame in the GOP where the target image frame is located to form a target video code stream, and send the target video code stream to the conference terminal that issued the key frame acquisition request.

[0102] If the conference terminal that issues a key frame acquisition request forms the bitstream for the current GOP using the target key frame and the P-frames forwarded by the media server, there's a risk of artifacts in the video due to some missing data in the re-formed target key frame compared to the pre-encoded image. In this implementation, the image frames starting with the target image frame in the current GOP are re-encoded to form the target video bitstream. This prevents artifacts and ensures the quality of the video output by the conference terminal that requested the key frame acquisition.

[0103] It should be noted here that the auxiliary conference terminal is in an occupied state when re-encoding I frames and other frames in GOP. When a key frame acquisition request is received at this time, the selection process in step S130 must also consider this occupied situation and reasonably select a suitable auxiliary conference terminal.

[0104] Example 2

[0105] Corresponding to the method of embodiment 1, Figure 4 As shown, the present invention further provides a video transmission device 4, comprising: an acquisition module 401, a binding module 402, a selection module 403 and a sending module 404.

[0106] An acquisition module 401 is configured to acquire a list of available candidate terminals, wherein the candidate terminals are terminals in a video conference that only need to pull video streams from a media server;

[0107] A binding module 402 is configured to bind available candidate terminals to a layout video stream to form a binding list, wherein the layout video stream is a video stream being played in a video conference;

[0108] The selection module 403 is configured to select an auxiliary conference terminal according to the key frame acquisition request and the binding list when receiving a key frame acquisition request from a conference terminal, wherein the key frame acquisition request includes information of a target image frame;

[0109] The sending module 404 is configured to send a re-encoding instruction to the auxiliary conference terminal, wherein the re-encoding instruction is configured to instruct the auxiliary conference terminal to re-encode the target image frame to form a target key frame, and send the target key frame to the conference terminal that issued the key frame acquisition request.

[0110] In one embodiment, the binding module 402 performs a process of binding available candidate terminals and layout video streams to form a binding list, including the following steps:

[0111] Bind a layout video stream to at least one available candidate terminal.

[0112] In one embodiment, the binding module 402 performs a process of binding available candidate terminals and layout video streams to form a binding list, including the following steps:

[0113] Bind the multi-channel layout video streams as a whole to at least one available candidate terminal.

[0114] In one embodiment, when the number of available candidate terminals is less than the number of video streams being played in the video conference, the binding module 402 performs a process of binding the available candidate terminals with the layout video streams to form a binding list, including the following steps:

[0115] Bind all layout video streams as a whole to all available candidate terminals.

[0116] In one embodiment, when the number of available candidate terminals is not less than the number of video streams being played in the video conference, the binding module 402 performs a process of binding the available candidate terminals with the layout video streams to form a binding list, including the following steps:

[0117] Bind each layout video stream to at least one available candidate terminal.

[0118] In one embodiment, the process of binding available candidate terminals and layout video streams to form a binding list further includes the steps of:

[0119] When adding candidate terminals for binding, the added candidate terminals are bound to the layout video stream with the least number of bound terminals.

[0120] In one embodiment, the process of binding available candidate terminals and layout video streams to form a binding list further includes the steps of:

[0121] When removing bound candidate terminals, if there are layout video streams without bound candidate terminals after the removal, the candidate terminals will be unbound from the layout video stream with the largest number of bound terminals, and the unbound candidate terminals will be bound to the layout video stream without bound candidate terminals.

[0122] In one embodiment, further comprising a removal module;

[0123] The removal module is used to periodically check available candidate terminals according to the established inspection items, and remove candidate terminals whose inspection items do not meet the preset standards from the binding list.

[0124] Preferably, the formulated inspection items include: encoding and decoding capabilities and / or network performance.

[0125] In one embodiment, further comprising a removal module;

[0126] The removal module is used to check the auxiliary conference terminal according to the established inspection items. When an inspection item of the auxiliary conference terminal does not meet the preset standard, the auxiliary conference terminal is removed from the binding list and a new auxiliary conference terminal is selected.

[0127] Preferably, the formulated inspection items include: encoding and decoding capabilities and / or network performance.

[0128] In one embodiment, the re-encoding instruction is also used to instruct the auxiliary conference terminal to re-encode the image frames starting from the target image frame in the GOP where the target image frame is located to form a target video code stream, and send the target video code stream to the conference terminal that issued the key frame acquisition request.

[0129] In this device, when a new conference terminal joins the conference, or other conference terminals need to obtain the current video frame in I frame format, the conference terminals that only need to pull the video stream from the media server are arranged to take on the task of re-encoding the I frame, so that they form the required I frame and send it to the conference terminal that issued the I frame request, thereby avoiding the source conference terminal from generating and sending the required I frame to all conference terminals, so that each video code stream basically only transmits P frames, and only transmits I frames when periodic I frames arrive, reducing bandwidth transmission requirements and reducing transmission fluctuations.

[0130] Furthermore, compared to using dedicated video stream processing equipment to re-encode key frames, this method reduces the number of devices by fully utilizing "idle" conference terminals. Furthermore, when a new terminal joins the conference and multiple video channels need to be re-encoded, this method effectively schedules each terminal, reducing workload and bandwidth usage compared to using only one video stream processing device.

[0131] Example 3

[0132] An embodiment of the present invention further provides a storage medium on which computer instructions are stored. When the instructions are executed by a processor, the video transmission method of any of the above embodiments is implemented.

[0133] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, random access memory (RAM), read-only memory (ROM), magnetic disk or optical disk, etc. Various media that can store program codes.

[0134] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, RAM, ROM, magnetic disks or optical disks.

[0135] Corresponding to the above-mentioned computer storage medium, in one embodiment, a computer device is also provided, which includes a memory, an encoder, and a computer program stored in the memory and runnable on the encoder, wherein when the encoder executes the program, any one of the video transmission methods in the above-mentioned embodiments is implemented.

[0136] When a new conference terminal joins the conference, or other conference terminals need to obtain the current video frame in I-frame format, the above-mentioned computer device arranges the conference terminal that only needs to pull the video stream from the media server to undertake the task of re-encoding the I frame, so that it forms the required I frame and sends it to the conference terminal that issues the I-frame request, thereby avoiding the source conference terminal from generating and sending the required I frame to all conference terminals, so that each video code stream basically only transmits P frames, and only transmits I frames when periodic I frames arrive, reducing bandwidth transmission requirements and reducing transmission fluctuations.

[0137] Furthermore, compared to using dedicated video stream processing equipment to re-encode key frames, this method reduces the number of devices by fully utilizing "idle" conference terminals. Furthermore, when a new terminal joins the conference and multiple video channels need to be re-encoded, this method effectively schedules each terminal, reducing workload and bandwidth usage compared to using only one video stream processing device.

[0138] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A video transmission method, characterized in that: Including steps: Obtain a list of available candidate terminals, where the candidate terminals are terminals that only need to pull video streams from the media server in the video conference; Binding available candidate terminals with layout video streams to form a binding list, wherein the layout video streams are video streams being played in the video conference; When a key frame acquisition request is received from a conference terminal, an auxiliary conference terminal is selected according to the key frame acquisition request and the binding list, wherein the key frame acquisition request includes: information of a target image frame; Sending a re-encoding instruction to the auxiliary conference terminal, wherein the re-encoding instruction is used to instruct to re-encode the image frames starting from the target image frame in the GOP where the target image frame is located to form a target video stream, and sending the target video stream to the conference terminal that issued the key frame acquisition request; Check the auxiliary conference terminal according to the established inspection items. If any inspection item of the auxiliary conference terminal does not meet the preset standard, remove the auxiliary conference terminal from the binding list and reselect the auxiliary conference terminal. The inspection items include: coding and decoding capabilities; The process of binding available candidate terminals and layout video streams to form a binding list includes the following steps: When the number of available candidate terminals is less than the number of video streams being played in the video conference, all layout video streams are bound as a whole to all available candidate terminals; When the number of available candidate terminals is not less than the number of video streams being played in the video conference, each layout video stream is bound to at least one available candidate terminal.

2. The video transmission method according to claim 1, wherein: The process of binding available candidate terminals and layout video streams to form a binding list further includes the steps of: When the number of available candidate terminals is not less than the number of video streams being played in the video conference, when adding a binding candidate terminal, the added candidate terminal is bound to the layout video stream with the least number of bound terminals.

3. The video transmission method according to claim 1, wherein: The process of binding available candidate terminals and layout video streams to form a binding list further includes the steps of: After removing the bound candidate terminals, if the number of available candidate terminals after removal is still not less than If there are a certain number of video streams being played in a video conference and there are no candidate terminals bound to them, the candidate terminals are unbound from the layout video stream with the largest number of bound terminals, and the unbound candidate terminals are bound to the layout video stream without any bound candidate terminals, so that each layout video stream is bound to at least one available candidate terminal.

4. The video transmission method according to any one of claims 1 to 3, characterized in that: The inspection items also include: network performance.

5. A video transmission device, characterized in that: include: An acquisition module is used to obtain a list of available candidate terminals, wherein the candidate terminals are terminals in the video conference that only need to pull video streams from the media server; a binding module for binding available candidate terminals with layout video streams to form a binding list, wherein the layout video stream is a video stream being played in a video conference; The process of binding available candidate terminals and layout video streams to form a binding list includes the following steps: When the number of available candidate terminals is less than the number of video streams being played in the video conference, all layout video streams are bound as a whole to all available candidate terminals; When the number of available candidate terminals is not less than the number of video streams being played in the video conference, each layout video stream is bound to at least one available candidate terminal; The selection module is configured to select an auxiliary conference terminal according to the key frame acquisition request and the binding list when receiving a key frame acquisition request from the conference terminal, wherein the key frame acquisition request Including: information of target image frame; a sending module, configured to send a re-encoding instruction to the auxiliary conference terminal, wherein the re-encoding instruction is used to instruct the auxiliary conference terminal to re-encode the target image frame to form a target key frame, and sending the target key frame to the conference terminal that issues the key frame acquisition request; Remove the module, which is used to check the auxiliary conference terminal according to the specified inspection items. When an inspection item does not meet the preset standard, the auxiliary conference terminal is removed from the binding list and a new auxiliary conference terminal is selected, wherein the inspection item includes: codec capability.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and operable on the processor. A computer program running on a computer, characterized in that When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images