Video processing method and device, computer equipment, storage medium and program product

By distributing and encoding video frames on the sending end, it is directly transmitted to the target decoder, which solves the problem of low decoding efficiency caused by aggregation delay during video transmission, and improves video playback efficiency.

CN119071535APending Publication Date: 2024-12-03TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310648348.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-01
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

During video transmission, the aggregation delay of the sub-data packets leads to low decoding efficiency on the receiver side, and low video efficiency after playback and decoding.

Method used

By obtaining the encoding information of real-time video at the sending end, frame allocation processing is performed, the target encoder is determined, and the target encoder is used to encode and transmit the video frames, which are directly transmitted to the target decoder, avoiding the transmission after aggregation through the server.

Benefits of technology

Improve the playback efficiency of videos after decoding, reduce delay and packet loss, and improve the overall performance of video transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071535A_ABST
    Figure CN119071535A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a video processing method and device, computer equipment, a storage medium and a program product, and the method comprises the steps: obtaining a to-be-encoded real-time video, and obtaining the encoding information of encoders in an encoder cluster for the real-time video; according to the encoding information, frame distribution processing is carried out on a current video frame of the real-time video, and at least one target encoder corresponding to the current video frame in the encoders is determined; adopting a target encoder to encode the current video frame to obtain at least one encoded video frame corresponding to the current video frame; and transmitting the encoded video frame to a target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each target encoder, decoding the encoded video frame through the target decoder to obtain a decoded video, and transmitting the decoded video through a receiving end corresponding to the transmitting end. And playing the decoded video. According to the invention, the playing efficiency of the decoded video can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and specifically to a video processing method, apparatus, computer equipment, storage medium and program product, wherein the storage medium is a computer-readable storage medium, and the program product is a computer program product. Background Art

[0002] When the sending end sends a video to the receiving end, the sending end divides the video data packet into multiple sub-data packets and transmits them to the server. Then, the server aggregates the sub-data packets and sends the aggregated data packet to the receiving end. The receiving end then uses a decoder to decode the aggregated data packet to obtain the decoded video. However, in the process of the sending end transmitting the sub-data packets to the receiving end, the aggregation of the sub-data packets is prone to delay, which will cause the receiving end to have low decoding efficiency when the decoder receives the video, resulting in low playback efficiency during the playback of the decoded video.

[0003] In summary, there is currently a problem that the playback efficiency of the decoded video is low. Summary of the invention

[0004] The embodiments of the present application provide a video processing method, apparatus, computer equipment, storage medium and program product, which can improve the playback efficiency of decoded videos.

[0005] A video processing method, applied to a sending end, comprising:

[0006] Acquire a real-time video to be encoded, and acquire encoding information of an encoder in an encoder cluster for the real-time video;

[0007] According to the encoding information, performing frame allocation processing on a current video frame of the real-time video, and determining at least one target encoder corresponding to the current video frame in the encoder;

[0008] Using the target encoder, encoding the current video frame to obtain at least one encoded video frame corresponding to the current video frame;

[0009] The encoded video frame is transmitted to the target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each of the target encoders, so that the encoded video frame is decoded by the target decoder to obtain the decoded video, so that the decoded video is played through the receiving end corresponding to the sending end.

[0010] Accordingly, an embodiment of the present application provides a video processing device, including:

[0011] A first acquisition unit may be used to acquire a real-time video to be encoded, and to acquire encoding information of an encoder in an encoder cluster for the real-time video;

[0012] An allocation unit may be configured to perform frame allocation processing on a current video frame of the real-time video according to the encoding information, and determine at least one target encoder corresponding to the current video frame in the encoder;

[0013] The encoding unit may be configured to use the target encoder to perform encoding processing on the current video frame to obtain at least one encoded video frame corresponding to the current video frame;

[0014] The transmission unit can be used to transmit the encoded video frame to the target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each target encoder, so that the encoded video frame can be decoded by the target decoder to obtain the decoded video, so as to play the decoded video through the receiving end corresponding to the sending end.

[0015] In some embodiments, the allocation unit can be specifically used to obtain video transmission parameters that match the real-time video with the video transmission channel based on the encoding information; perform frame allocation processing on the current video frame of the real-time video based on the video transmission parameters, and determine at least one target encoder corresponding to the current video frame in the encoder.

[0016] In some embodiments, the allocation unit may be specifically configured to obtain channel characteristic information of a video transmission channel; and obtain video transmission parameters matching the video transmission channel according to the encoding information and the channel characteristic information.

[0017] In some embodiments, the allocation unit can be specifically used to sort the transmission parameter acquisition order of the video transmission channel according to the channel characteristic information to obtain the transmission parameter acquisition order of the video transmission channel; according to the transmission parameter acquisition order, the encoding information is mapped to obtain the video transmission parameters that match the video transmission channel.

[0018] In some embodiments, the allocation unit can be specifically used to perform weighted processing on the video transmission parameters to obtain weighted video transmission parameters; perform parameter fusion processing on the weighted video transmission parameters and the video transmission parameters to obtain fused parameters; and based on the fused parameters, filter out at least one target encoder corresponding to the current video frame from the encoder.

[0019] In some embodiments, the transmission unit can be specifically used to obtain the encoder type corresponding to each target encoder; determine the target decoder corresponding to the target encoder in the decoder cluster according to the encoder type; and transmit the encoded video frame to the target decoder corresponding to the target encoder through at least one video transmission channel corresponding to the target encoder.

[0020] The present application also provides another video processing method, which is applied to a receiving end, including:

[0021] Obtaining at least one encoded video frame of a target encoder through at least one video transmission channel corresponding to each target decoder in the decoder cluster, wherein the encoded video frame is a video frame obtained by encoding a current video frame of a real-time video through at least one target encoder corresponding to the target decoder in the encoder cluster; the target encoder is an encoder determined by performing frame allocation processing on the current video frame of the real-time video based on encoding information of encoders in the encoder cluster of the real-time video;

[0022] Decoding the encoded video frame by the target decoder to obtain a decoded video frame corresponding to the real-time video;

[0023] Generate a decoded video corresponding to the real-time video according to the frame feature information of the decoded video frame;

[0024] The decoded video is played.

[0025] Accordingly, an embodiment of the present application provides another video processing device, including:

[0026] The second acquisition unit may be used to acquire at least one encoded video frame of a target encoder through at least one video transmission channel corresponding to each target decoder in the decoder cluster, wherein the encoded video frame is a video frame obtained by encoding a current video frame of a real-time video through at least one target encoder corresponding to the target decoder in the encoder cluster; the target encoder is an encoder determined by performing frame allocation processing on the current video frame of the real-time video based on encoding information of encoders in the encoder cluster of the real-time video;

[0027] A decoding unit may be used to decode the encoded video frame through a target decoder to obtain a decoded video frame corresponding to the real-time video;

[0028] A generating unit, which can be used to generate a decoded video corresponding to the real-time video according to the frame feature information of the decoded video frame;

[0029] The playback unit can be used to play the decoded video.

[0030] In some embodiments, the generation unit can be used to obtain a preset fusion time interval for the decoded video frame; based on the frame feature information of the decoded video frame, the decoded video frame is fused within the preset fusion time interval to obtain a decoded video corresponding to the real-time video.

[0031] In some embodiments, the generating unit may be used to obtain the frame arrival time of the decoded video frame; and obtain a preset fusion time interval for the decoded video frame according to the frame arrival time and the frame feature information of the decoded video frame.

[0032] In some embodiments, the generation unit can be used to perform calculations on the frame arrival time and the decoded video frame time in the frame feature information to obtain the difference between the frame arrival time and the decoded video frame time; based on the difference value, obtain the preset fusion time interval for the decoded video frame.

[0033] In some embodiments, the generation unit can be used to obtain the channel buffer time corresponding to the video transmission channel; perform time interval calculation processing according to the channel buffer time, frame arrival time and frame feature information of the decoded video frame to obtain a preset fusion time interval for the decoded video frame.

[0034] In addition, an embodiment of the present application also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any video processing method provided in the embodiment of the present application.

[0035] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for loading by a processor to execute any video processing method provided in the embodiment of the present application.

[0036] In addition, an embodiment of the present application also provides a computer program product, including a computer program, which implements any video processing method provided in the embodiment of the present application when the computer program is executed by a processor.

[0037] The present application can obtain real-time video to be encoded, and obtain encoding information of encoders in an encoder cluster for real-time video; according to the encoding information, perform frame allocation processing on the current video frame of the real-time video, and determine at least one target encoder corresponding to the current video frame in the encoder; use the target encoder to encode the current video frame to obtain at least one encoded video frame corresponding to the current video frame; transmit the encoded video frame to the target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each target encoder, so as to decode the encoded video frame through the target decoder to obtain the decoded video, and play the decoded video through the receiving end corresponding to the sending end. Since the present application can perform frame allocation processing on the current video frame of the real-time video based on the encoding information of the encoder, determine the target encoder corresponding to the current video frame in the encoder, and then use the target encoder to encode the current video frame, the encoded video frame can be directly transmitted to the target decoder corresponding to the target encoder through the video transmission channel corresponding to the target encoder, without the need to aggregate the encoded video frame through the server and then transmit it to the decoder. The present application can directly use the target decoder to decode the encoded video frame to obtain the decoded video, thereby improving the playback efficiency of the decoded video. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 is a scene schematic diagram of the video processing method provided in an embodiment of the present application;

[0040] Figure 2 1 is a flowchart of a video processing method provided in an embodiment of the present application;

[0041] Figure 3 FIG2 is a flow chart of a video processing method provided in an embodiment of the present application;

[0042] Figure 4 is a timing diagram of a video processing method provided in an embodiment of the present application;

[0043] Figure 5 is an architecture diagram of a video processing method provided in an embodiment of the present application;

[0044] Figure 6 is a schematic diagram of the structure of the video processing method provided in an embodiment of the present application;

[0045] Figure 7 2 is a schematic diagram of the structure of the video processing method provided in the embodiment of the present application;

[0046] Figure 8 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0048] The embodiment of the present application provides a video processing method, apparatus, computer equipment and computer readable storage medium. The video processing apparatus can be integrated in a computer equipment, which can be a server or a terminal.

[0049] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, car terminal, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0050] For example, see Figure 1 Taking the video processing device integrated in a computer device as an example, the computer device can obtain the real-time video to be encoded, and obtain the encoding information of the encoder in the encoder cluster for the real-time video; according to the encoding information, perform frame allocation processing on the current video frame of the real-time video, and determine at least one target encoder corresponding to the current video frame in the encoder; use the target encoder to encode the current video frame to obtain at least one encoded video frame corresponding to the current video frame; transmit the encoded video frame to the target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each target encoder, so as to decode the encoded video frame through the target decoder to obtain the decoded video, and play the decoded video through the receiving end corresponding to the sending end.

[0051] The encoding information may refer to information related to the encoder itself and encoding parameter recommendation information received by the encoder or encoder cluster. The information related to the encoder itself may include the correction parameters of the encoder and the compression rate of the encoder; the encoding parameter recommendation information may include the transmission bit rate recommendation information fed back by the video transmission channel.

[0052] The video transmission channel may refer to a transmission channel established between a target encoder and a target decoder and used to transmit encoded video frames.

[0053] The encoder cluster may include at least one encoder, and the decoder cluster may include at least one decoder. The encoders in the encoder cluster and the decoders in the decoder cluster may have a one-to-one correspondence.

[0054] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0055] This embodiment will be described from the perspective of a video processing device, which can be specifically integrated in a computer device, which can be a server or a terminal. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device, or other smart devices that can acquire data.

[0056] First of all, it should be explained that in the existing related technologies, when the sender sends a video to the receiver, the sender divides the video data packet into multiple sub-data packets through the network layer and transmits them to the server, then aggregates the sub-data packets through the server, and then sends the aggregated data packet to the receiver through the server, and then the receiver decodes the aggregated data packet to obtain the decoded video. Among them, in the existing related technologies, the process of the sender transmitting the sub-data packet to the receiver occurs in the network layer, and the network layer cannot perceive whether the sub-data packet is successfully transmitted, nor can it perceive whether the sub-data packet is a video data packet. Based on this, the existing related technologies are prone to packet loss and invalid retransmission. In addition, in the process of the sender transmitting the sub-data packet to the receiver, the aggregation of the sub-data packet is prone to delay, which will cause the receiver to have low decoding efficiency when the decoder receives the video, resulting in low playback efficiency in the process of playing the decoded video.

[0057] Based on the above, the present application provides a video processing method, which can firstly improve the playback efficiency of the decoded video. Figure 2As shown, the video processing method can be applied to a sending end, wherein the sending end can be the aforementioned computer device, and the specific flow of the video processing method is shown in steps S201 to S204:

[0058] It should be noted here that the video processing method of the present application is implemented at the application layer. The application layer can adopt the protocol stack of real-time video transmission itself, which has its own anti-packet loss capabilities and no additional aggregation transmission overhead.

[0059] S201: Acquire a real-time video to be encoded, and acquire encoding information of encoders in an encoder cluster for the real-time video.

[0060] In some application embodiments, the sending end may obtain real-time video to be encoded online.

[0061] The encoding information may include but is not limited to at least one of sending bit rate recommendation information, encoder correction parameters and encoder compression rate.

[0062] Among them, the present application can adopt a congestion control algorithm to obtain the recommended sending bit rate information.

[0063] It can be understood here that before encoding the real-time video, it is necessary to prioritize frame allocation for the real-time video to determine the target encoder for encoding the current video frame in the real-time video. Based on this, the present application needs to collect the encoding information of the encoder to facilitate the subsequent frame allocation of the real-time video.

[0064] In some application embodiments, a method for obtaining encoding information of an encoder in an encoder cluster for real-time video may be: extracting the encoding information of the encoder from configuration information of the encoder cluster for real-time video.

[0065] The configuration information may include configuration information corresponding to each encoder in the encoder cluster, and the configuration information may configure various parameters and information of the encoder during the encoding process, such as the correction parameters of the encoder and the compression rate of the encoder.

[0066] S202: Perform frame allocation processing on a current video frame of the real-time video according to the encoding information, and determine at least one target encoder corresponding to the current video frame in the encoder.

[0067] There are multiple ways to perform step S202, such as the following ways (1) and (2).

[0068] Mode (1), according to the encoding information, the current video frame of the real-time video is subjected to frame allocation processing, and the method of obtaining the target encoder corresponding to the current video frame in the encoder can be as shown in steps S2021 to S2022:

[0069] S2021. Obtain video transmission parameters that match the real-time video with the video transmission channel according to the encoding information.

[0070] The video transmission parameter may refer to a relevant parameter of the encoded video frame during the transmission process. For example, the video transmission parameter may be a frame rate of the encoded video frame.

[0071] In one example, according to the encoding information, the method for obtaining video transmission parameters that match the real-time video and the video transmission channel can be: obtaining channel characteristic information of the video transmission channel; and obtaining video transmission parameters that match the video transmission channel according to the encoding information and the channel characteristic information.

[0072] The channel characteristic information may refer to information characterizing the characteristics of the video transmission channel, for example, the channel characteristic information may refer to the round trip delay of the video transmission channel. Each video transmission channel has its own corresponding channel characteristic information.

[0073] In one example, a method for obtaining video transmission parameters that match a video transmission channel based on coding information and channel characteristic information may be as follows: according to the channel characteristic information, the video transmission channels are sorted in a transmission parameter acquisition order to obtain a transmission parameter acquisition order for the video transmission channels; according to the transmission parameter acquisition order, the coding information is mapped to obtain video transmission parameters that match the video transmission channel.

[0074] It can be understood that the present application can sort the video transmission channels from small to large according to the round-trip delay to obtain the transmission parameter acquisition order of the video transmission channels; then, based on the transmission parameter acquisition order, the video transmission parameters matching each video transmission channel can be obtained. For example, the video transmission channels include a first video transmission channel, a second video transmission channel, and a third video transmission channel. The video transmission channels are sorted from small to large according to the round-trip delay to obtain the transmission parameter acquisition order of the first video transmission channel, the second video transmission channel, and the third video transmission channel. Based on this, the present application can calculate and process the video transmission parameters corresponding to each video transmission channel in the order of the first video transmission channel, the second video transmission channel, and the third video transmission channel.

[0075] In one example, according to the order of obtaining transmission parameters, the encoding information is mapped and processed to obtain video transmission parameters that match the video transmission channel. The method can be: obtain a parameter mapping function; according to the order of obtaining transmission parameters, the encoding information is mapped and processed by the parameter mapping function to obtain video transmission parameters that match the video transmission channel.

[0076] The parameter mapping function can be shown as formula (1):

[0077] Ni=Ri / belta_i / video frame memory size formula (1)

[0078] Among them, Ni can refer to the video transmission parameters such as frame rate corresponding to the i-th encoder; belta_i can refer to the compression rate of the encoder in the encoding information; the memory size occupied by the video frame can refer to the size of the memory occupied by the current video frame, or it can refer to the average size of the memory occupied by the video frame of the real-time video over a period of time; Ri can refer to the sending bit rate recommendation information corresponding to the i-th encoder.

[0079] In one example, the coding information is mapped according to the order in which the transmission parameters are obtained, and the method for obtaining the video transmission parameters that match the video transmission channel can also be: obtaining a mapping relationship set, the mapping relationship set including the mapping relationship between preset coding information and preset video transmission parameters; extracting the target preset video transmission parameters corresponding to the coding information from the mapping relationship according to the order in which the transmission parameters are obtained; and using the target preset video transmission parameters as the video transmission parameters that match the video transmission channel.

[0080] S2022. Perform frame allocation processing on a current video frame of the real-time video according to the video transmission parameters, and determine at least one target encoder corresponding to the current video frame in the encoder.

[0081] In one example, frame allocation processing is performed on the current video frame of the real-time video according to the video transmission parameters, and a method for determining at least one target encoder corresponding to the current video frame in the encoder may be: weighting the video transmission parameters to obtain weighted video transmission parameters; performing parameter fusion processing on the weighted video transmission parameters and the video transmission parameters to obtain fused parameters; and based on the fused parameters, screening out at least one target encoder corresponding to the current video frame from the encoder.

[0082] In one example, each video transmission channel may have a corresponding video transmission parameter. In the embodiment of the present application, the video transmission parameter corresponding to each video transmission channel may be weighted to obtain a weighted video transmission parameter, wherein the weight of the video transmission parameter corresponding to each video transmission channel may be 1.

[0083] In one example, a parameter fusion function may be used to perform parameter fusion processing on the weighted video transmission parameters and the video transmission parameters to obtain fused parameters.

[0084] The parameter fusion function can be found in formula (2):

[0085] Pi=Ni / N Formula (2)

[0086] Among them, Pi may refer to the fused parameters of the i-th encoder, and the fused parameters may represent the allocation ratio of the real-time video to the i-th encoder; N may refer to the weighted video transmission parameters, where N may represent the sum of the video transmission parameters corresponding to each video transmission channel.

[0087] In one example, the weighted video transmission parameters and the video transmission parameters are subjected to parameter fusion processing, and the method for obtaining the fused parameters can be: determining whether the weighted video transmission parameters are equal to the parameter threshold; if so, performing parameter fusion processing on the weighted video transmission parameters and the video transmission parameters to obtain the fused parameters; if not, returning to the step of performing weighted processing on the video transmission parameters to obtain the weighted video transmission parameters, until determining whether the weighted video transmission parameters are equal to the parameter threshold.

[0088] The parameter threshold may be the same as the video frame rate of the real-time video.

[0089] In one example, based on the fused parameters, a target encoder corresponding to the current video frame can be selected from encoders in the following manner: for each encoder, the fused parameters and video transmission parameters such as frame rate are processed to obtain the calculated parameters; and based on the calculated parameters, a target encoder corresponding to the current video frame is selected from encoders.

[0090] Among them, a parameter operation function can be used to perform operation processing on the fused parameters and video transmission parameters such as frame rate to obtain the post-operation parameters. The parameter operation function can be referred to in formula (3):

[0091] L=Ni / Pi Formula (3)

[0092] Among them, L may refer to the post-operation parameters; Ni may refer to the video transmission parameters corresponding to the i-th encoder, such as frame rate; Pi may refer to the post-fusion parameters of the i-th encoder.

[0093] Among them, according to the post-operation parameters, the method for filtering out the target encoder corresponding to the current video frame from the encoders can be: according to the size of the post-operation parameters, determining the encoder corresponding to the minimum post-operation parameter as the target encoder corresponding to the current video frame; if the encoder that encodes the previous current video frame is the encoder corresponding to the minimum post-operation parameter, then the encoder corresponding to the maximum post-fusion parameter in the post-fusion parameters is used as the target encoder corresponding to the current video frame.

[0094] In one example, according to the fused parameters, the method for filtering out the target encoder corresponding to the current video frame from the encoders can be: according to the fused parameters, obtaining the video frame allocation value of each encoder; detecting the candidate encoder that encodes the previous video frame of the current video frame, the candidate encoder belongs to the encoder; obtaining the candidate video frame allocation value corresponding to the candidate encoder from the video allocation value; detecting whether the number of video frames encoded by the candidate encoder reaches the candidate video frame allocation value corresponding to the candidate encoder; if reached, the encoder with a serial number after the candidate encoder is used as the target encoder corresponding to the current video frame; if not reached, the candidate encoder is used as the target encoder corresponding to the current video frame.

[0095] The video frame allocation value may refer to the number of video frames allocated to the encoder. For example, the encoder includes a first encoder and a second encoder, and the fusion parameter between the first encoder and the second encoder, that is, the allocation ratio of the real-time video to the encoder is 1:2, then the video allocation value of the first encoder is 1, and the video allocation value of the second encoder is 2.

[0096] Method (2): In one example, according to the encoding information, frame allocation processing is performed on the current video frame of the real-time video, and the method for obtaining the target encoder corresponding to the current video frame in the encoder can be: when the encoding information includes the encoder's sending bit rate recommendation information and the encoder's correction parameters, the encoder's sending bit rate recommendation information in the encoding information and the encoder's correction parameters in the encoding information are fused to obtain fused data that matches the real-time video with the video transmission channel; based on the fused data, the target encoder corresponding to the current video frame is screened out from the encoder.

[0097] In the above, a data fusion function may be used to fuse the transmission bit rate recommendation information of the encoder in the coding information and the correction parameters of the encoder in the coding information to obtain fused data matching the real-time video with the video transmission channel.

[0098] The data fusion function can be found in formula (4):

[0099] Gi = Ri*alpha_i / sum(Ri*alpha_i) Formula (4)

[0100] Among them, Gi may refer to the fused data, which can represent the allocation ratio of the real-time video to the i-th encoder; Ri represents the sending bit rate recommendation information of the i-th encoder; alpha_i represents the correction parameter of the i-th encoder, wherein when the encoder is a layered encoder, alpha_i can be equal to 0.8, and when the encoder is an independent encoder, alpha_i can be equal to 1; sum(Ri*alpha_i) means summing the product of the encoder's sending bit rate recommendation information and the correction parameter.

[0101] S203: Use a target encoder to encode the current video frame to obtain at least one encoded video frame corresponding to the current video frame.

[0102] In some application embodiments, a target encoder is used to encode a current video frame, and a method for obtaining at least one encoded video frame corresponding to the current video frame may be: according to an encoder type of the target encoder, an encoding strategy corresponding to the encoder type is obtained; according to the encoding strategy, the current video frame is encoded to obtain at least one encoded video frame corresponding to the current video frame.

[0103] The current video frame may include at least one current video frame.

[0104] The encoder type may be a property of the target encoder itself. Target encoders of different encoder types have different encoding rules. For example, the encoder type may include an independent encoder of an independent encoding type and a layered encoder of a layered encoding type. The layered encoder is also called a scalable video encoding encoder, and the layered encoder may refer to an encoder capable of performing layered encoding on video frames in real-time video; the independent encoder of the independent type may refer to an ordinary encoder that does not perform layered encoding on video frames in real-time video.

[0105] The coding strategy can be characterized as a coding algorithm. The coding strategy can include a layered coding strategy of a layered encoder and an independent coding strategy of an independent encoder.

[0106] For example, the layered coding strategy can encode the current video frame into a base layer and multiple enhancement layers. The base layer can provide the most basic video quality, frame rate and resolution, while the enhancement layer improves the video quality. The independent coding strategy can divide the current video frame into multiple image blocks; then, based on the image blocks, perform intra-frame prediction on the current video frame to obtain the predicted image blocks corresponding to the image blocks; based on the predicted image blocks, determine the encoded video frame corresponding to the current video frame.

[0107] S204. Transmit the encoded video frames to the target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each target encoder, so that the encoded video frames are decoded by the target decoder to obtain the decoded video, and the decoded video is played through the receiving end corresponding to the sending end.

[0108] In some application embodiments, the method for transmitting the encoded video frame to the target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each of the target encoders can be: obtaining the encoder type corresponding to each of the target encoders; determining the target decoder corresponding to the target encoder in the decoder cluster according to the encoder type; and transmitting the encoded video frame to the target decoder corresponding to the target encoder through at least one video transmission channel corresponding to the target encoder.

[0109] In one example, a method for determining a target decoder corresponding to a target encoder in a decoder cluster according to an encoder type may be as follows: obtaining the decoder type of the decoders in the decoder cluster; extracting a target decoder type corresponding to the encoder type from the decoder type according to the encoder type; and determining a target decoder corresponding to the target decoder type from the decoders in the decoder cluster according to the target decoder type.

[0110] For example, when the encoder type is a layered encoding type, correspondingly, the target decoder type corresponding to the encoder type can be a layered decoding type; and only one layered encoder corresponding to the layered encoding type can be set, and correspondingly, only one target layered decoder corresponding to the layered decoding type can also be set.

[0111] In one example, according to the target decoder type, a target decoder corresponding to the target decoder type can be determined from the decoders of the decoder cluster as follows: according to the target decoder type, at least two candidate decoders corresponding to the target decoder type are determined from the decoders of the decoder cluster; an order of the candidate decoders is obtained; and according to the order, a target decoder corresponding to the target encoder in the decoder cluster is screened out from the candidate decoders.

[0112] For example, when the encoder type is an independent encoding type, correspondingly, the target decoder type corresponding to the encoder type may be an independent decoding type; and multiple independent encoders corresponding to the independent encoding type may be set, and correspondingly, multiple candidate independent decoders corresponding to the independent decoding type may be set. Then, the order of the candidate independent decoders may be used to determine the corresponding target decoders for the independent encoders one by one from the candidate independent decoders.

[0113] For another example, when the encoder type includes an independent encoding type and a layered encoding type, correspondingly, the target decoder type corresponding to the encoder type may include an independent decoding type and a layered decoding type; and at least one independent encoder of the independent encoding type and at least one layered encoder of the layered encoding type may be set, and correspondingly, at least one candidate independent decoder corresponding to the independent decoding type and at least one candidate layered decoder corresponding to the layered decoding type may be set. Then, the order of the candidate independent decoders may be used to determine the corresponding target decoders for the independent encoders one by one from the candidate independent encoders; and the order of the candidate layered decoders may be used to extract the corresponding target decoders for the layered decoders one by one from the candidate layered decoders.

[0114] Among them, at least one video transmission channel can be set, and each video transmission channel can realize the transmission of video frames. Specifically, the video transmission channel can be a channel established by the first local IP and the first local port of the sending end, and the second local IP and the second local port of the receiving end where the decoder cluster is located. For example, the sending end has multiple first local network cards, and the receiving end has a second local network card, then the first local network cards of the sending end can establish video transmission channels with the second local network card of the receiving end. Among them, the first local network card can provide the first local IP and the first local port, and the second local network card can provide the second local IP and the second local port.

[0115] When there are multiple target encoders, any target encoder has at least one corresponding video transmission channel.

[0116] For example, when the encoder type is a layered coding type, and only one layered encoder corresponding to the layered coding type is set, the layered encoder can set multiple video transmission channels, and the layered encoder can perform layered encoding processing on the current video frame to obtain an encoded video frame; when the encoded video frame belongs to the base layer, it can be transmitted on the video transmission channel with the best network quality; when the encoded video frame belongs to the enhancement layer, it can be transmitted to the target decoder on other video transmission channels with the second best quality in order of importance.

[0117] For another example, when the encoder type is an independent encoding type, and multiple independent encoders corresponding to the independent encoding type are set, the current video frame is assigned to a target independent encoder among the independent encoders for encoding, and then the encoded video frame is obtained; then, it is transmitted to the target decoder through the video transmission channel corresponding to the target independent encoder. In this case, each independent encoder can have a corresponding video transmission channel, and when any independent encoder is used as a target encoder, the encoded video frame generated by the target encoder is transmitted to the target decoder through the video transmission channel corresponding to the target encoder.

[0118] For another example, when the encoder type includes an independent encoding type and a layered encoding type, the current video frame generated by the real-time video each time is allocated to an independent encoder of the independent encoding type and a layered encoder of the layered encoding type for encoding processing according to the allocation ratio. The encoded video frame encoded by the independent encoder is transmitted through the video transmission channel corresponding to the independent encoder; the encoded video frame encoded by the layered encoder is transmitted through at least two video transmission channels corresponding to the layered encoder.

[0119] In one embodiment, the receiving end may receive the decoded video, and then render the decoded video frames in the decoded video to obtain rendered video frames; the receiving end performs display processing on the rendered video frames to play the decoded video.

[0120] The present application can obtain the real-time video to be encoded, and obtain the encoding information of the encoder in the encoder cluster for the real-time video; according to the encoding information, the current video frame of the real-time video is subjected to frame allocation processing to obtain the target encoder corresponding to the current video frame in the encoder; the target encoder is used to encode the current video frame to obtain the encoded video frame corresponding to the current video frame; the encoded video frame is transmitted to the target decoder corresponding to the target encoder in the decoder cluster through the video transmission channel corresponding to the target encoder, so that the encoded video frame is decoded by the target decoder to obtain the decoded video; the decoded video is played by the receiving end corresponding to the transmitting end. Since the present application can perform frame allocation processing on the current video frame of the real-time video based on the encoding information of the encoder, and obtain the target encoder corresponding to the current video frame in the encoder, and then the target encoder can be used to encode the current video frame, so that the encoded video frame can be directly transmitted to the target decoder corresponding to the target encoder through the video transmission channel corresponding to the target encoder, without the need to aggregate the encoded video frame through the server and then transmit the decoder, the present application can directly use the target decoder to decode the encoded video frame to obtain the decoded video, thereby improving the playback efficiency of the decoded video.

[0121] According to the method described in the above embodiment, the following is further described in detail with examples.

[0122] In this embodiment, the video processing device is specifically integrated in the receiving end, and the receiving end can also be the aforementioned computer device. In practical applications, the sending end can also be used as a receiving end to receive the encoded video frame, decode the encoded video frame, and play the decoded video. Of course, the receiving end can also be used as a sending end to obtain the real-time video to be encoded, and obtain the encoding information of the encoder in the encoder cluster for the real-time video, perform frame allocation processing on the current video frame of the real-time video, perform encoding processing on the current video frame, and transmit the encoded video frame to the target decoder corresponding to the target encoder in the decoder cluster through the video transmission channel corresponding to the target encoder.

[0123] like Figure 3 As shown, a video processing method, the specific process is shown in steps S301 to S304:

[0124] S301. Obtain at least one encoded video frame of a target encoder through at least one video transmission channel corresponding to each target decoder in a decoder cluster.

[0125] The encoded video frame is a video frame obtained by encoding a current video frame of a real-time video through at least one target encoder corresponding to a target decoder in an encoder cluster.

[0126] The target encoder is an encoder determined by performing frame allocation processing on a current video frame of the real-time video based on encoding information of encoders in an encoder cluster of the real-time video.

[0127] Among them, the target decoder corresponds one-to-one to the aforementioned target encoder, and the decoder type of the target decoder corresponds one-to-one to the encoder type of the target encoder. For example, when the target encoder is an independent encoder, the target decoder is also an independent decoder; when the target encoder is a layered encoder, the target decoder is also a layered decoder.

[0128] S302: Decode the encoded video frame by using a target decoder to obtain a decoded video frame corresponding to the real-time video.

[0129] There are multiple implementation methods for step 302, such as the following method A, method B and method C:

[0130] Method A: The method of decoding the encoded video frames through the target decoder to obtain the decoded video frames corresponding to the real-time video can be: merging the encoded video frames of different video transmission channels to obtain a video frame stream; and decoding each encoded video frame in the video frame stream through the target decoder to obtain the decoded video frames corresponding to the real-time video.

[0131] Among them, the situation of mode A may correspond to the situation that only one layered encoder corresponding to the aforementioned layered encoding type is set. It can be understood that the target decoder here may be a layered decoder, and only one layered decoder may be set.

[0132] Method B: The method of decoding the encoded video frames through the target decoder to obtain the decoded video frames corresponding to the real-time video can be: for each video transmission channel, the decoded video frames corresponding to the video transmission channel are input into the target decoder corresponding to the video transmission channel; the encoded video frames are decoded through the target decoder to obtain the decoded video frames corresponding to the real-time video.

[0133] Among them, the situation of method B may correspond to the situation where multiple independent encoders corresponding to the aforementioned independent encoding type are set. It can be understood that the target decoder here is an independent decoder, and multiple independent decoders can be set.

[0134] Method C: The method of decoding the encoded video frame by the target decoder to obtain the decoded video frame corresponding to the real-time video can be: when the target decoder includes an independent decoder and a layered decoder, the encoded video frame corresponding to the independent encoder can be input to the independent decoder for decoding to obtain a first decoded video frame; the encoded video frame corresponding to the layered encoder can be input to the layered decoder for decoding to obtain a second decoded video frame; and a decoded video frame can be generated according to the first decoded video frame and the second decoded video frame.

[0135] Among them, the situation of method C can correspond to the aforementioned situation, where the target decoder type corresponding to the encoder type can include an independent decoding type and a layered decoding type, and at least one of the independent encoder of the independent encoding type and the layered encoder of the layered encoding type can be set. It can be understood that the target decoder can include an independent decoder and a layered decoder.

[0136] S303: Generate a decoded video corresponding to the real-time video according to the frame feature information of the decoded video frame.

[0137] The frame feature information is information carried by the decoded video frame. The frame feature information may be information characterizing features of the decoded video frame, for example, the frame feature information may include at least one of the decoded video frame order and the decoded video frame time, wherein the decoded video frame order may refer to the order of the video frames corresponding to the decoded video frame in the real-time video, and the decoded video frame time may refer to the time of the video frame corresponding to the decoded video frame in the real-time video.

[0138] In order to make the decoded video consistent with the video frame rate of the real-time video when it is played and avoid playback delays, based on this, the present application needs to buffer the decoded video frames to obtain the decoded video.

[0139] Based on the above, in some application embodiments, a method for generating a decoded video corresponding to a real-time video according to frame feature information of the decoded video frame may be: obtaining a preset fusion time interval for the decoded video frame; and performing fusion processing on the decoded video frame within the preset fusion time interval according to the frame feature information of the decoded video frame to obtain a decoded video corresponding to the real-time video.

[0140] Among them, it can be understood that the preset fusion time interval can be the buffering time interval of the decoded video frame in the video buffer queue. When the decoded video frame arrives at the video buffer queue, it can be fused with other decoded video frames in the video buffer queue to obtain the decoded video corresponding to the real-time video.

[0141] In this application, there are multiple ways to obtain the preset fusion time interval for the decoded video frame, as follows:

[0142] Method a: In one example, the method for obtaining the preset fusion time interval for the decoded video frame can be: obtaining the frame arrival time of the decoded video frame; and obtaining the preset fusion time interval for the decoded video frame based on the frame arrival time and the frame feature information of the decoded video frame.

[0143] The frame arrival time may refer to the time when the decoded video frame arrives at the video buffer queue.

[0144] In one example, based on the frame arrival time and the frame feature information of the decoded video frame, a method for obtaining a preset fusion time interval for the decoded video frame can be: performing calculations on the decoded video frame time in the frame arrival time and the frame feature information to obtain a difference value between the frame arrival time and the decoded video frame time; based on the difference value, obtaining the preset fusion time interval for the decoded video frame.

[0145] In one example, according to the difference value, the method for obtaining the preset fusion time interval for the decoded video frame can be: obtaining a difference value mapping function, mapping the difference value to obtain a mapped difference value; according to the mapped difference value, obtaining the preset fusion time interval for the decoded video frame.

[0146] Wherein, for each video transmission channel, a post-mapping difference value can be calculated.

[0147] The difference value mapping function can be specifically referred to in formula (5):

[0148] Cj =Fx(Dij – Ti) Formula (5)

[0149] Wherein, (Dij–Ti) may refer to a difference value, wherein Dij may refer to a frame arrival time of the i-th decoded video frame in the j-th video transmission channel, and Ti may refer to a decoded video frame time of the i-th decoded video frame; Cj may refer to a mapped difference value of the j-th video transmission channel; and Fx(Dij–Ti) may refer to a value of X% distribution of (Dij–Ti), where X is a distribution percentage.

[0150] Among them, it can be understood that about Fx(Dij–Ti), for example, when (Dij–Ti) is 100 and X is 82, the value of Fx(Dij–Ti) is 82.

[0151] Based on the above, in one example, according to the mapped difference value, the method for obtaining the preset fusion time interval for the decoded video frame can be: according to the size of the mapped difference value, obtaining the maximum mapped difference value and the minimum mapped difference value; calculating the target difference value between the maximum mapped difference value and the minimum mapped difference value; according to the target difference value, determining the preset fusion time interval for the decoded video frame.

[0152] The target difference value between the maximum mapped difference value and the minimum mapped difference value can be calculated by referring to the following formula (6):

[0153] Tb= maxCj-minCj formula (6)

[0154] Among them, maxCj may refer to the maximum post-mapping difference value; minCj may refer to the minimum post-mapping difference value; and Tb may refer to the target difference value.

[0155] In one example, the target difference value can be used as a preset fusion time interval for the decoded video frame. In this case, maxCj can represent the maximum mapped difference value among the mapped difference values ​​of all video transmission channels, and minCj can refer to the minimum mapped difference value among the mapped difference values ​​of all video transmission channels.

[0156] Furthermore, in one example, since each video transmission channel has a corresponding target difference value, based on this, the embodiment of the present application can sort the target difference values ​​to obtain sorted difference values; extract the target sorted difference values ​​of preset sorting numbers; and use the target sorted difference values ​​as the preset fusion time interval for the decoded video frames.

[0157] Among them, the target post-sorting difference value of the preset sorting sequence number can be the post-sorting difference value with the largest size.

[0158] Furthermore, in one example, the target difference value of any video transmission channel may be used as a preset fusion time interval for the decoded video frame.

[0159] Furthermore, in one example, a time interval prediction algorithm may be used to predict a preset fusion time interval for the decoded video frame according to the frame arrival time and the frame feature information of the decoded video frame.

[0160] The time interval prediction algorithm may be a support vector machine algorithm (SVM), a linear regression algorithm, or other time interval prediction neural network models, and the like.

[0161] Method b: In one example, according to the frame arrival time and the frame feature information of the decoded video frame, the method for obtaining the preset fusion time interval for the decoded video frame can be: obtaining the channel buffer time corresponding to the video transmission channel; performing time interval calculation processing according to the channel buffer time, the frame arrival time and the frame feature information of the decoded video frame, and obtaining the preset fusion time interval for the decoded video frame.

[0162] The channel buffering time may refer to the buffering time of decoded video frames in the video transmission channel.

[0163] In an example, a method for obtaining a channel buffer time corresponding to a video transmission channel may be: adopting a channel buffer time obtaining strategy to collect the channel buffer time corresponding to the video transmission channel.

[0164] The channel buffer time acquisition strategy may be a jitterbuffer algorithm, such as a jitterbuffer algorithm based on a Kalman filter.

[0165] Specifically, the channel buffer time acquisition strategy is adopted, and the specific method of collecting the channel buffer time corresponding to the video transmission channel can be: for each video transmission channel, the channel buffer time acquisition strategy is adopted to obtain the target time when the encoded video frame enters the video transmission channel; the target difference between the target time and the decoded video frame time in the frame feature information is calculated; the target difference of the encoded video frame in the video transmission channel is fused to obtain the fused difference; according to the fused difference, the channel buffer time corresponding to the video transmission channel is determined.

[0166] The fused difference may be used as the channel buffer time corresponding to the video transmission channel.

[0167] Among them, reference parameters can be obtained, and the reference parameters and the fused difference are weighted to obtain weighted data; the weighted data is used as the channel buffer time corresponding to the video transmission channel.

[0168] In one example, time interval calculation processing is performed based on the channel buffer time, frame arrival time and frame feature information of the decoded video frame to obtain the preset fusion time interval for the decoded video frame. The method can be: performing time fusion processing on the decoded video frame time in the frame arrival time and the frame feature information to obtain the fused time; and obtaining the preset fusion time interval for the decoded video frame based on the fused time and the channel buffer time.

[0169] The time fusion function can be used to perform time fusion processing on the frame arrival time and the decoded video frame time in the frame feature information to obtain the fused time. The time fusion function can refer to formula (7):

[0170] Aj = F(Dij-Ti) Formula (7)

[0171] Among them, Aj may refer to the fusion time corresponding to the j-th video transmission channel; Dij may refer to the frame arrival time of the i-th decoded video frame in the j-th video transmission channel, Ti may refer to the decoded video frame time of the i-th decoded video frame; F(Dij–Ti) may refer to the minimum or average value of (Dij–Ti).

[0172] In one example, a method for obtaining a preset fusion time interval for a decoded video frame based on the post-fusion time and the channel buffer time can be: extracting the maximum channel buffer time from the channel buffer time based on the size of the channel buffer time; extracting the maximum post-fusion time and the minimum post-fusion time from the post-fusion time based on the size of the post-fusion time; and fusing the maximum channel buffer time, the maximum post-fusion time, and the minimum post-fusion time to obtain a preset fusion time interval for the decoded video frame.

[0173] Among them, an objective function can be used to fuse the maximum channel buffer time, the maximum fusion time and the minimum fusion time to obtain a preset fusion time interval for the decoded video frame. The objective function can be referred to in formula (8):

[0174] K=max(Tbi) + maxAj – min Aj Formula (8)

[0175] Among them, K may refer to a preset fusion time interval; max(Tbi) may refer to a maximum channel buffer time; maxAj may refer to a maximum post-fusion time; and min Aj may refer to a minimum post-fusion time.

[0176] In an embodiment of the present application, when the distribution difference between different fusion times Aj is greater than a first preset threshold, if the preset fusion time interval is greater than or equal to the preset time interval threshold, the receiving end sends a channel closing message to the sending end, and the channel closing message instructs the sending end to close the video transmission channel corresponding to the maximum fusion time; when the distribution difference of the mapped difference value Cj is greater than a second preset threshold, if the preset fusion time interval is greater than or equal to the preset time interval threshold, the receiving end sends a channel closing message to the sending end.

[0177] When the sender receives the channel closing information, the sender can determine whether the recommended sending bit rate information of the encoder of the video transmission channel corresponding to the maximum fusion time is greater than or equal to the target output bit rate; if so, the sender closes the video transmission channel corresponding to the maximum fusion time according to the channel closing information.

[0178] The target output bit rate may refer to the actual bit rate recommendation information of the encoder for the encoded video frame.

[0179] S304: Play the decoded video.

[0180] In one example, the receiving end renders the decoded video frames in the decoded video to obtain rendered video frames; the receiving end performs display processing on the rendered video frames to play the decoded video.

[0181] In one example, the receiving end renders the decoded video frame in the decoded video, and the method for obtaining the rendered video frame can be: the receiving end determines whether the decoded video frame time of the decoded video frame is greater than a preset time threshold; if so, the receiving end waits for the target preset time, and then renders the decoded video frame to obtain the rendered video frame; if not, the receiving end sends non-retransmission information to the sending end, and the non-retransmission information indicates that the sending end will no longer retransmit the decoded video frame whose decoded video frame time is less than or equal to the preset time threshold.

[0182] It can be understood here that the present application can reduce invalid retransmissions of decoded video frames by not retransmitting part of the decoded video frames through non-retransmission information.

[0183] Among them, the target preset time can be set to To-Tw+Ti, where To can refer to the rendering time of the previous decoded video frame, Tw can refer to the preset time threshold; Ti can refer to the decoded video frame time of the i-th decoded video frame.

[0184] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0185] The present application can obtain at least one encoded video frame of the target encoder through at least one video transmission channel corresponding to each target decoder in the decoder cluster, wherein the encoded video frame is a video frame obtained by encoding the current video frame of the real-time video through at least one target encoder corresponding to the target decoder in the encoder cluster; the target encoder is an encoder determined by performing frame allocation processing on the current video frame of the real-time video based on the encoding information of the encoder in the encoder cluster of the real-time video; the encoded video frame is decoded by the target decoder to obtain a decoded video frame corresponding to the real-time video; a decoded video corresponding to the real-time video is generated according to the frame feature information of the decoded video frame; and the decoded video is played. Since the present application does not need to aggregate the encoded video frames through the server and then transmit them to the decoder, but can directly obtain the encoded video frames of the target encoder through the video transmission channel corresponding to the target decoder in the decoder cluster, without the need for the server to aggregate the encoded video frames and then transmit them to the decoder, the present application can directly decode the encoded video frames through the target decoder to obtain the decoded video frames corresponding to the real-time video, and then directly generate the decoded video corresponding to the real-time video based on the frame feature information of the decoded video frames, thereby improving the playback efficiency of the decoded video.

[0186] According to the method described in the above embodiment, the following is further described in detail with examples.

[0187] Based on the above technical problems, the embodiment of the present application proposes a video processing method, in which the sending end can encode and transmit the video frames of the same real-time video for different video transmission channels, and then the receiving end receives the decoded video frames from the video transmission channel, and merges the decoded video frames to obtain the decoded video, thereby improving the playback efficiency of the decoded video. Specifically, the video processing method is shown in steps S401 to S408:

[0188] S401: A transmitting end obtains a real-time video to be encoded, and obtains encoding information of an encoder in an encoder cluster for the real-time video.

[0189] S402: The transmitting end performs frame allocation processing on the current video frame of the real-time video according to the encoding information to obtain at least one target encoder corresponding to the current video frame in the encoder.

[0190] S403: The transmitting end uses at least one target encoder to encode the current video frame to obtain at least one encoded video frame corresponding to the current video frame.

[0191] S404: The transmitter transmits the encoded video frame to a target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each target encoder.

[0192] Based on the above steps S401 to S404, it can be understood that the sending end encodes a real-time video through a joint encoding module into encoded video frames corresponding to multiple video transmission channels respectively, and sends the encoded video frames through the video transmission channels respectively.

[0193] S405 . The receiving end obtains at least one encoded video frame of the target encoder through the video transmission channel corresponding to the target decoder in the decoder cluster.

[0194] Among them, the encoded video frame is a video frame obtained by encoding the current video frame of the real-time video through the target encoder corresponding to the target decoder in the encoder cluster, and the target encoder is an encoder determined by performing frame allocation processing on the current video frame of the real-time video based on the encoding information of the encoder in the encoder cluster of the real-time video.

[0195] S406: The receiving end decodes the encoded video frame through a target decoder to obtain a decoded video frame corresponding to the real-time video.

[0196] S407: The receiving end generates a decoded video corresponding to the real-time video according to the frame feature information of the decoded video frame.

[0197] S408: The receiving end plays the decoded video.

[0198] Based on the above steps S405 to S408, it can be understood that the receiving end can receive decoded video frames from multiple video transmission channels respectively, and then send the decoded video frames in the decoded video to the decoder cluster, decode them into serial output decoded video frames through the target decoder corresponding to the decoder cluster, and send them to the video buffer queue for buffering to generate the decoded video.

[0199] Based on the above, the embodiments of the present application are described with practical examples.

[0200] For example, Figure 5 As shown in the figure, suppose there is a 30fps real-time video S, the sending end has two network cards A and B, and the receiving end has a network card C. The network of network card A and the network of network card B can both be connected to the network of network card C. In this way, there are two video transmission channels from the sending end to the receiving end, namely video transmission channel 1: A->C and video transmission channel 2: B->C.

[0201] The transmitter inputs the real-time video S into the encoder cluster for joint encoding. Here, the encoder cluster can also be called a joint encoder. The encoder cluster adopts two independent encoders, and the independent encoders include independent encoder 1 and independent encoder 2. Then, the transmitter splits the real-time video S into independent encoder 1 and independent encoder 2 according to different allocation ratios.

[0202] The sending end obtains channel characteristic information fed back by video transmission channel 1, such as the round-trip delay of the video transmission channel is 30ms, and the recommended sending bit rate information fed back by video transmission channel 1 is 1Mbps; the sending end obtains channel characteristic information fed back by video transmission channel 2, such as the round-trip delay of the video transmission channel is 40ms, and the recommended sending bit rate information fed back by video transmission channel 2 is 3Mbps.

[0203] Then, the sending end sorts the transmission parameter acquisition order of video transmission channel 1 and video transmission channel 2 according to the channel characteristic information. Since 30ms is less than 40ms, based on this, the transmission parameter acquisition order of the video transmission channel is video transmission channel 1 and video transmission channel 2.

[0204] Next, based on the sending bit rate recommendation information, the sender adopts the mapping relationship set in the aforementioned method (1) to preferentially obtain the video transmission parameters of video transmission channel 1, such as a frame rate of 15bps, and then obtains the video transmission parameters of video transmission channel 2, such as a frame rate of 15bps.

[0205] Next, the present application uses the aforementioned formula (2) to obtain that the fused parameter of independent encoder 1 is 1 / 2, and the fused parameter of independent encoder 2 is also 1 / 2. Based on this, it can be obtained that the distribution ratio of real-time video for independent encoder 1 and independent encoder 2 is 1 / 2:1 / 2, that is, 1:1.

[0206] Then, the sending end can distribute the current video frame of the real-time video to independent encoder 1 and independent encoder 2 in turn according to the distribution ratio of 1:1, and the first encoded video frame encoded by independent encoder 1 is sent to video transmission channel 1, and the second encoded video frame encoded by independent encoder 2 is sent to video transmission channel 2.

[0207] A decoder cluster is provided in the receiving end, which may be referred to as a joint decoder. The joint decoder includes an independent decoder 1 and an independent decoder 2. The independent decoder 1 receives a first encoded video frame, and the independent decoder 2 receives a second encoded video frame. Then, the independent decoder 1 decodes the first encoded video frame to obtain a first target decoded video frame, and the independent decoder 2 decodes the second encoded video frame to obtain a second target decoded video frame. Then, the receiving end sends the first target decoded video frame and the second target decoded video frame to a video buffer queue for buffering, so as to fuse the first target decoded video frame and the second target decoded video frame within a preset fusion time interval to obtain a decoded video corresponding to the real-time video.

[0208] The specific calculation of the preset fusion time interval can be as follows:

[0209] For example, the decoded video frame includes a first decoded video frame, a second decoded video frame, a third decoded video frame, and a fourth decoded video frame. The frame arrival time D11 of the first decoded video frame arriving at the video buffer queue through the video transmission channel 1 is 1300ms, and the decoded video frame time of the first decoded video frame is 300ms; the frame arrival time D22 of the second decoded video frame arriving at the video buffer queue through the video transmission channel 2 is 1343ms, and the decoded video frame time of the second decoded video frame is 333ms; the frame arrival time D31 of the third decoded video frame arriving at the video buffer queue through the video transmission channel 1 is 1366ms, and the decoded video frame time of the third decoded video frame is 366ms; the frame arrival time D41 of the fourth decoded video frame arriving at the video buffer queue through the video transmission channel 2 is 1415ms, and the decoded video frame time of the fourth decoded video frame is 400ms.

[0210] The receiving end adopts the aforementioned formula (5), and Fx(Dij–Ti) takes a value of 100%. It can be obtained that the difference value of the first frame of the video frame after decoding for the video transmission channel 1 is 1000ms; the difference value of the second frame of the video frame after decoding for the video transmission channel 2 is 1010ms; the difference value of the third frame of the video frame after decoding for the video transmission channel 1 is 1000ms; the difference value of the fourth frame of the video frame after decoding for the video transmission channel 2 is 1015ms.

[0211] Then, based on the above formula (6), the receiving end can calculate that the preset fusion time interval is 1015ms-1000ms=15ms.

[0212] Then, the receiving end can perform fusion processing on the decoded video frames within the preset fusion time zone of 15ms to obtain the decoded video corresponding to the real-time video.

[0213] Afterwards, the receiving end renders the first decoded video frame in the decoded video at Tc1=D11+15ms=1315ms.

[0214] Among them, the receiving end sets a preset time threshold Tw=300ms, and the rendering time of the previous decoded video frame To=1315ms. Based on this, the second frame of the decoded video can be rendered when Tc2=To-Tw+Ti=1315ms-300ms+333ms=1348ms, and so on. At 1381ms, the third frame of the decoded video can be rendered; at 1415ms, the fourth frame of the decoded video can be rendered.

[0215] The present application can obtain real-time video to be encoded, and obtain encoding information of encoders in an encoder cluster for real-time video; according to the encoding information, perform frame allocation processing on the current video frame of the real-time video, and determine at least one target encoder corresponding to the current video frame in the encoder; use the target encoder to encode the current video frame to obtain at least one encoded video frame corresponding to the current video frame; transmit the encoded video frame to the target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each target encoder, so as to decode the encoded video frame through the target decoder to obtain the decoded video, and play the decoded video through the receiving end corresponding to the sending end. Since the present application can perform frame allocation processing on the current video frame of the real-time video based on the encoding information of the encoder, determine the target encoder corresponding to the current video frame in the encoder, and then use the target encoder to encode the current video frame, the encoded video frame can be directly transmitted to the target decoder corresponding to the target encoder through the video transmission channel corresponding to the target encoder, without the need to aggregate the encoded video frame through the server and then transmit it to the decoder. The present application can directly use the target decoder to decode the encoded video frame to obtain the decoded video, thereby improving the playback efficiency of the decoded video.

[0216] In order to better implement the above method, an embodiment of the present application also provides a video processing device, which can be integrated into a computer device, such as a server or a terminal. The terminal may include a tablet computer, a laptop computer and / or a personal computer.

[0217] For example, Figure 6As shown, the video processing device may include a first acquisition unit 301, an allocation unit 302, an encoding unit 303 and a transmission unit 304, as follows:

[0218] (1) a first acquisition unit;

[0219] The first acquisition unit may be used to acquire the real-time video to be encoded, and to acquire encoding information of encoders in an encoder cluster for the real-time video.

[0220] (2) allocation unit;

[0221] The allocation unit may be configured to perform frame allocation processing on the current video frame of the real-time video according to the encoding information, and determine at least one target encoder corresponding to the current video frame in the encoder.

[0222] In some embodiments, the allocation unit can be used to obtain video transmission parameters that match the real-time video with the video transmission channel based on the encoding information; perform frame allocation processing on the current video frame of the real-time video based on the video transmission parameters, and determine at least one target encoder corresponding to the current video frame in the encoder.

[0223] In some embodiments, the allocation unit may be used to obtain channel characteristic information of a video transmission channel; and obtain video transmission parameters matching the video transmission channel according to the encoding information and the channel characteristic information.

[0224] In some embodiments, the allocation unit can be used to sort the transmission parameter acquisition order of the video transmission channel according to the channel characteristic information to obtain the transmission parameter acquisition order of the video transmission channel; according to the transmission parameter acquisition order, the encoding information is mapped to obtain the video transmission parameters that match the video transmission channel.

[0225] In some embodiments, the allocation unit can be used to perform weighted processing on the video transmission parameters to obtain weighted video transmission parameters; perform parameter fusion processing on the weighted video transmission parameters and the video transmission parameters to obtain fused parameters; and based on the fused parameters, filter out at least one target encoder corresponding to the current video frame from the encoder.

[0226] (3) Coding unit;

[0227] The encoding unit can be used to use a target encoder to encode the current video frame to obtain at least one encoded video frame corresponding to the current video frame.

[0228] (4) Transmission unit;

[0229] The transmission unit can be used to transmit the encoded video frame to the target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each target encoder, so as to decode the encoded video frame through the target decoder to obtain the decoded video, and play the decoded video through the receiving end corresponding to the sending end.

[0230] In one embodiment, a transmission unit can be used to obtain an encoder type corresponding to each target encoder; determine a target decoder corresponding to the target encoder in a decoder cluster according to the encoder type; and transmit the encoded video frame to the target decoder corresponding to the target encoder through at least one video transmission channel corresponding to the target encoder.

[0231] The first acquisition unit of the present application can be used to acquire the real-time video to be encoded, and to acquire the encoding information of the encoder in the encoder cluster for the real-time video; the allocation unit can be used to perform frame allocation processing on the current video frame of the real-time video according to the encoding information, and determine at least one target encoder corresponding to the current video frame in the encoder; the encoding unit can be used to use the target encoder to encode the current video frame to obtain at least one encoded video frame corresponding to the current video frame; the transmission unit can be used to transmit the encoded video frame to the target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each target encoder, so as to decode the encoded video frame through the target decoder to obtain the decoded video, and play the decoded video through the receiving end corresponding to the sending end. Since the present application can perform frame allocation processing on the current video frame of the real-time video based on the encoding information of the encoder, and obtain the target encoder corresponding to the current video frame in the encoder, and then use the target encoder to encode the current video frame, the encoded video frame can be directly transmitted to the target decoder corresponding to the target encoder through the video transmission channel corresponding to the target encoder, without the need to aggregate the encoded video frame through the server and then transmit it to the decoder. The present application can directly use the target decoder to decode the encoded video frame to obtain the decoded video, thereby improving the playback efficiency of the decoded video.

[0232] In order to better implement the above method, an embodiment of the present application also provides a video processing device, which can be integrated into a computer device, such as a server or a terminal. The terminal may include a tablet computer, a laptop computer and / or a personal computer.

[0233] For example, Figure 7 As shown, the video processing device may include a second acquisition unit 401, a decoding unit 402, a generation unit 403 and a playback unit 404, as follows:

[0234] (1) a second acquisition unit;

[0235] The second acquisition unit can be used to acquire at least one encoded video frame of the target encoder through at least one video transmission channel corresponding to each target decoder in the decoder cluster, wherein the encoded video frame is a video frame obtained by encoding the current video frame of the real-time video through at least one target encoder corresponding to the target decoder in the encoder cluster; the target encoder is an encoder determined by performing frame allocation processing on the current video frame of the real-time video based on the encoding information of the encoder in the encoder cluster of the real-time video.

[0236] (2) decoding unit;

[0237] The decoding unit can be used to decode the encoded video frame through a target decoder to obtain a decoded video frame corresponding to the real-time video.

[0238] (3) Generation unit;

[0239] The generating unit can be used to generate a decoded video corresponding to the real-time video according to the frame feature information of the decoded video frame.

[0240] In one example, a generation unit can be used to obtain a preset fusion time interval for a decoded video frame; based on frame feature information of the decoded video frame, the decoded video frame is fused within the preset fusion time interval to obtain a decoded video corresponding to the real-time video.

[0241] In one example, the generating unit may be used to obtain a frame arrival time of a decoded video frame; and obtain a preset fusion time interval for the decoded video frame according to the frame arrival time and frame feature information of the decoded video frame.

[0242] In one example, a generation unit can be used to perform calculations on the frame arrival time and the decoded video frame time in the frame feature information to obtain a difference value between the frame arrival time and the decoded video frame time; based on the difference value, a preset fusion time interval for the decoded video frame is obtained.

[0243] In one example, a generation unit can be used to obtain a channel buffering time corresponding to a video transmission channel; perform time interval calculation processing based on the channel buffering time, frame arrival time, and frame feature information of a decoded video frame to obtain a preset fusion time interval for the decoded video frame.

[0244] (4) playback unit;

[0245] The playback unit can be used to play the decoded video.

[0246] The second acquisition unit of the present application can be used to obtain the encoded video frame of the target encoder through at least one video transmission channel corresponding to each target decoder in the decoder cluster, wherein the encoded video frame is a video frame obtained by encoding the current video frame of the real-time video through at least one target encoder corresponding to the target decoder in the encoder cluster, and the target encoder is an encoder determined by performing frame allocation processing on the current video frame of the real-time video based on the encoding information of the encoder in the encoder cluster of the real-time video; the decoding unit can be used to decode the encoded video frame through the target decoder to obtain the decoded video frame corresponding to the real-time video; the generation unit can be used to generate a decoded video corresponding to the real-time video according to the frame feature information of the decoded video frame; the playback unit can be used to play the decoded video. Since the present application does not need to aggregate the encoded video frames through the server and then transmit them to the decoder, but can directly obtain the encoded video frames of the target encoder through the video transmission channel corresponding to the target decoder in the decoder cluster, the present application can directly decode the encoded video frames through the target decoder to obtain the decoded video frames corresponding to the real-time video, and then directly generate the decoded video corresponding to the real-time video based on the frame feature information of the decoded video frames, thereby improving the playback efficiency of the decoded video.

[0247] The present application also provides a computer device, such as Figure 8 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0248] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Figure 8 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0249] The processor 401 is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the computer device and processes data. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and computer programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0250] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0251] The computer device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0252] The computer device may further include an input unit 404, which may be used to receive input digital or character information communications and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0253] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the computer device will load the executable files corresponding to the processes of one or more computer programs into the memory 402 according to the following instructions, and the processor 401 will run the computer programs stored in the memory 402, thereby realizing various functions, as follows:

[0254] Acquire a real-time video to be encoded, and acquire encoding information of an encoder in an encoder cluster for the real-time video; perform frame allocation processing on a current video frame of the real-time video according to the encoding information, and determine at least one target encoder corresponding to the current video frame in the encoder; use the target encoder to encode the current video frame to obtain at least one encoded video frame corresponding to the current video frame; transmit the encoded video frame to a target decoder corresponding to the target encoder in a decoder cluster through at least one video transmission channel corresponding to each target encoder, so as to decode the encoded video frame through the target decoder to obtain a decoded video, and play the decoded video through a receiving end corresponding to the transmitting end.

[0255] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0256] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0257] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute any video processing method provided in the embodiment of the present application.

[0258] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0259] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0260] Since the instructions stored in the computer-readable storage medium can execute the steps in any video processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any video processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0261] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations provided in the above embodiments.

[0262] The above is a detailed introduction to a video processing method, computer equipment, device, computer-readable storage medium and computer program product provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A video processing method, It is characterized in that Applied to the sending end, including: Acquire a real-time video to be encoded, and acquire encoding information of an encoder in an encoder cluster for the real-time video; According to the encoding information, performing frame allocation processing on a current video frame of the real-time video, and determining at least one target encoder corresponding to the current video frame in the encoder; Using the target encoder, encoding the current video frame to obtain at least one encoded video frame corresponding to the current video frame; The encoded video frame is transmitted to the target decoder corresponding to the target encoder in the decoder cluster through at least one video transmission channel corresponding to each of the target encoders, so that the encoded video frame is decoded by the target decoder to obtain the decoded video, and the decoded video is played through the receiving end corresponding to the sending end.

2. The video processing method according to claim 1, It is characterized in that The step of performing frame allocation processing on the current video frame of the real-time video according to the encoding information to determine at least one target encoder corresponding to the current video frame in the encoder includes: According to the encoding information, obtaining video transmission parameters matching the real-time video with the video transmission channel; According to the video transmission parameters, frame allocation processing is performed on the current video frame of the real-time video to determine at least one target encoder corresponding to the current video frame in the encoder.

3. The video processing method according to claim 2, It is characterized in that The acquiring, according to the encoding information, video transmission parameters matching the real-time video with the video transmission channel comprises: Acquiring channel characteristic information of the video transmission channel; According to the encoding information and the channel characteristic information, a video transmission parameter matching the video transmission channel is acquired.

4. The video processing method according to claim 3, It is characterized in that The acquiring, according to the encoding information and the channel characteristic information, a video transmission parameter matching the video transmission channel comprises: According to the channel characteristic information, the video transmission channels are sorted in a transmission parameter acquisition order to obtain a transmission parameter acquisition order of the video transmission channels; According to the transmission parameter acquisition order, the encoding information is mapped to obtain video transmission parameters matching the video transmission channel.

5. The video processing method according to claim 2, It is characterized in that The step of performing frame allocation processing on the current video frame of the real-time video according to the video transmission parameter, and determining at least one target encoder corresponding to the current video frame in the encoder, comprises: Performing weighted processing on the video transmission parameters to obtain weighted video transmission parameters; Performing parameter fusion processing on the weighted video transmission parameter and the video transmission parameter to obtain fused parameters; At least one target encoder corresponding to the current video frame is selected from the encoders according to the fused parameters.

6. The video processing method according to claim 1, It is characterized in that The transmitting the encoded video frame to a target decoder corresponding to the target encoder in a decoder cluster through at least one video transmission channel corresponding to each target encoder comprises: Obtaining the encoder type corresponding to each of the target encoders; Determining, according to the encoder type, a target decoder corresponding to the target encoder in a decoder cluster; The encoded video frame is transmitted to a target decoder corresponding to the target encoder through at least one video transmission channel corresponding to the target encoder.

7. A video processing method, It is characterized in that Applied to the receiving end, including: Obtaining at least one encoded video frame of a target encoder through at least one video transmission channel corresponding to each target decoder in the decoder cluster, wherein the encoded video frame is a video frame obtained by encoding a current video frame of a real-time video through at least one target encoder corresponding to the target decoder in the encoder cluster; the target encoder is an encoder determined by performing frame allocation processing on the current video frame of the real-time video based on encoding information of encoders in the encoder cluster of the real-time video; Decoding the encoded video frame by the target decoder to obtain a decoded video frame corresponding to the real-time video; Generate a decoded video corresponding to the real-time video according to the frame feature information of the decoded video frame; The decoded video is played.

8. The video processing method according to claim 7, It is characterized in that The step of generating a decoded video corresponding to the real-time video according to the frame feature information of the decoded video frame comprises: Obtaining a preset fusion time interval for the decoded video frame; According to the frame feature information of the decoded video frame, the decoded video frame is fused within a preset fusion time interval to obtain a decoded video corresponding to the real-time video.

9. The video processing method according to claim 8, It is characterized in that The obtaining a preset fusion time interval for the decoded video frame includes: Obtaining a frame arrival time of the decoded video frame; A preset fusion time interval for the decoded video frame is obtained according to the frame arrival time and the frame feature information of the decoded video frame.

10. The video processing method according to claim 9, It is characterized in that The acquiring a preset fusion time interval for the decoded video frame according to the frame arrival time and the frame feature information of the decoded video frame includes: Performing calculation processing on the frame arrival time and the decoded video frame time in the frame feature information to obtain a difference value between the frame arrival time and the decoded video frame time; According to the difference value, a preset fusion time interval for the decoded video frame is obtained.

11. The video processing method according to claim 9, It is characterized in that The acquiring, according to the frame arrival time and the frame feature information of the decoded video frame, a preset fusion time interval for the decoded video frame comprises: Obtaining a channel buffer time corresponding to the video transmission channel; A time interval operation is performed according to the channel buffering time, the frame arrival time and the frame feature information of the decoded video frame to obtain a preset fusion time interval for the decoded video frame.

12. A video processing device, It is characterized in that include: A first acquisition unit, configured to acquire a real-time video to be encoded, and to acquire encoding information of an encoder in an encoder cluster for the real-time video; an allocation unit, configured to perform frame allocation processing on a current video frame of the real-time video according to the encoding information, and determine at least one target encoder corresponding to the current video frame in the encoder; An encoding unit, configured to use the target encoder to perform encoding processing on the current video frame to obtain at least one encoded video frame corresponding to the current video frame; A transmission unit is used to transmit the encoded video frame to a target decoder corresponding to the target encoder in a decoder cluster through at least one video transmission channel corresponding to each target encoder, so as to decode the encoded video frame through the target decoder to obtain a decoded video, so as to play the decoded video through a receiving end corresponding to the sending end.

13. A computer device, It is characterized in that It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the video processing method according to any one of claims 1 to 11.

14. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the video processing method according to any one of claims 1 to 11.

15. A computer program product, It is characterized in that The computer program product stores a computer program, and the computer program is suitable for being loaded by a processor to execute the video processing method according to any one of claims 1 to 11.