High-definition live broadcast method and device

Abstract

The present invention provides a high-definition live broadcast method and device, belonging to the field of media transmission technology. The high-definition live broadcast device includes: an encoder for encoding original media data; a controller for sending the media data encoded by the encoder to at least two CPEs, and adjusting the sending rate, adjusting the encoding bit rate of the encoder and / or controlling the CPE to perform frequency switching according to the overflow state of the transmission channel between the CPEs; the at least two CPEs are used for uplink transmission of the received media data. The technical solution of the present invention can realize high-definition live broadcast.

Description

Technical Field

[0001] The present invention relates to the technical field of media transmission, and in particular to a high-definition live broadcast method and device. Background Art

[0002] With the development of video technology and the improvement of people's appreciation level, high-resolution, high-frame-rate, and low-latency ultra-high-definition video live broadcast is gradually heating up. Wireless networks make mobile ultra-high-definition video live broadcast possible due to their ease of use and popularity. However, in occasions such as large public gatherings, the uplink bandwidth and stability of wireless networks still cannot meet the needs of ultra-high-definition live broadcast. To solve this problem, the current practice is to use multiple 4G and / or 5G cards, or customer premise equipment (CPE) to transmit data.

[0003] Related technology 1: Configure multiple network cards, i.e., multiple 4G modules, in the transmission front-end device to bind and work together to share the transmission bandwidth and even out the bit rate. The network card control module controls the transmission bandwidth of a single card to reduce the transmission of a single card; multiple 4G modules, i.e., multiple cards, back up each other. Even if a single card is disconnected, it can automatically reconnect and recover. During the reconnection process, the bandwidth is shared by other cards to avoid transmission interruption caused by a single card being disconnected. However, mobile bandwidth is limited after all. Simply diverting the bandwidth of one card to other cards during the reconnection process will cause congestion on other cards, resulting in live broadcast delays. There is also no mention of the data processing logic when the device is in the cell switching area and the way to handle lost data before network switching.

[0004] In the related technology 2, the target threshold is determined according to the network bandwidth. When the bandwidth is lower than a certain threshold, data transmission is stopped and the data is cached locally. Data is sent only when the bandwidth is restored. This has two problems. First, obtaining network bandwidth requires sending a large number of data packets, which itself occupies the bandwidth. Second, stopping transmission when the bandwidth is lower than the threshold will cause a large amount of data to be cached and cause delays.

[0005] Therefore, how to make the mobile live broadcast system adaptively switch between the two frequency bands is an urgent problem that needs to be solved, and there is no mature solution yet. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a high-definition live broadcast method and device, which can realize high-definition live broadcast.

[0007] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0008] On the one hand, a high-definition live broadcast device is provided, comprising:

[0009] An encoder, used to encode raw media data;

[0010] A controller, configured to send the media data encoded by the encoder to at least two CPEs, and adjust the sending rate, adjust the encoding bit rate of the encoder, and / or control the CPE to perform frequency switching according to the overflow state of the transmission channel between the CPEs;

[0011] The at least two CPEs are used for uplink transmission of received media data.

[0012] In some embodiments, the controller is further configured to send an encoding control instruction to the encoder;

[0013] The encoder is also used to calculate the initial bit rate of the original media data according to the resolution and frame rate of the input original media data, and adjust the encoding bit rate for encoding the original media data according to the encoding control instruction of the controller during encoding.

[0014] In some embodiments, the controller is also used to receive registration information of two CPEs, and divide the two CPEs into a main CPE and a backup CPE according to the registration time of the two CPEs, the registration time of the backup CPE being later than the registration time of the main CPE; establish transmission channels with the main CPE and the backup CPE respectively, and set the sending buffer size of each CPE.

[0015] In some embodiments, the controller includes a first local buffer and a second local buffer, the first local buffer caches the media data encoded by the encoder in a first-in-first-out queue manner, and the second local buffer stores data in a circular linked list manner. After the sending thread reads data from the first local buffer and puts it into the second local buffer and the CPE's sending buffer, the read data is deleted from the first local buffer.

[0016] In some embodiments, the controller is specifically used to initialize two circular linked lists Lp and Ls of packet sequence numbers to be sent, and record the packet sequence numbers in the sending buffers in the main CPE and the backup CPE respectively. The number of nodes in each circular linked list is FGOP, and the packet sequence number of each data packet is stored in the linked list node; the data packets read from the first local buffer are sent alternately through two transmission channels, and the packet sequence numbers are updated to the corresponding Lp or Ls; when it is detected that the overflow time of a transmission channel exceeds a preset threshold, a single-channel congestion processing step is executed; when overflow is detected in two transmission channels, a dual-channel congestion processing step is executed.

[0017] In some embodiments, when performing the single channel congestion handling step,

[0018] The controller is specifically used to reduce the sending rate of the first transmission channel where overflow occurs to (1-δ) / 2 times the encoding rate of the previous stage, and change the sending rate of the second transmission channel where overflow does not occur to (1+δ) / 2 times the original; if the second transmission channel overflows after adjusting the sending rate, the encoder is notified through a coding control instruction to adjust the encoding rate to 1 / (1+δ) times the encoding rate of the previous stage; if the first transmission channel no longer overflows after adjusting the sending rate, the encoding rate is maintained unchanged, and after an interval of time T, the encoder is notified through a coding control instruction to gradually increase the encoding rate until it reaches the initial rate; if the first transmission channel still overflows after adjusting the sending rate, the δ value is increased after an interval of time t, and the encoding rate continues to be reduced. When δ is greater than δo, the first transmission channel is notified to increase the encoding rate to 1 / (1+δ) times the encoding rate. The CPE corresponding to the channel sends a frequency switching instruction to cut off the first transmission channel; after sending the frequency switching instruction, the encoder is notified through the encoding control instruction to adjust the encoding code rate to n times the initial code rate, and the packet sequence number that has not been sent in the first local buffer is obtained from the to-be-sent packet sequence number linked list of the first transmission channel, and the packet data is read from the second local buffer according to the packet sequence number and sent using the second transmission channel, and the subsequent data is sent using the second transmission channel; if the frequency switching success instruction is received, the first transmission channel is re-established, and the encoder is notified through the encoding control instruction to gradually increase the encoding code rate until the initial code rate, and the data packets read from the first local buffer are sent alternately through the first transmission channel and the second transmission channel, wherein δ, δo and η are all greater than 0 and less than 1;

[0019] The CPE is specifically configured to negotiate with the network side to select a frequency point after receiving the frequency switching instruction from the controller, and return a frequency switching success instruction to the controller.

[0020] In some embodiments, when performing the dual channel congestion processing step,

[0021] The controller is specifically used to notify the encoder to modify the encoding rate to n times of the initial rate through an encoding control instruction; if the overflow phenomenon disappears, keep the encoding rate of the encoder unchanged, increase the encoding rate to the initial rate at an interval of T, if there is no more overflow, keep the encoding rate of the encoder at the initial rate, if the overflow continues, restore the encoding rate of the encoder to n times of the initial rate; if the overflow continues, send a frequency switching instruction to the standby CPE to cut off the first transmission channel where the overflow occurs; after sending the frequency switching instruction, notify the encoder to adjust the encoding rate to n times of the initial rate through an encoding control instruction, obtain the packet sequence number that has not been sent in the first local buffer from the linked list of packet sequence numbers to be sent of the standby CPE, read the packet data from the second local buffer according to the packet sequence number, use the second transmission channel to send it, and use the second transmission channel to send subsequent data; if the frequency switching success instruction of the standby CPE is received, re-establish the first transmission channel, notify the encoder to gradually increase the encoding rate until the initial rate through an encoding control instruction, and perform a single-channel congestion processing step, wherein n is greater than 0 and less than 1;

[0022] The standby CPE is specifically configured to negotiate with the network side to select a frequency point after receiving the frequency switching instruction from the controller, and return a frequency switching success instruction to the controller.

[0023] In some embodiments, the CPE is specifically configured not to perform frequency switching when no frequency switching instruction is received from the controller during movement.

[0024] The embodiment of the present invention further provides a high-definition live broadcast method, which is applied to the high-definition live broadcast device as described above, comprising:

[0025] Receiving media data encoded by the encoder;

[0026] The media data encoded by the encoder is sent to at least two CPEs, and the sending rate is adjusted according to the overflow status of the transmission channel between the CPEs, the encoding bit rate of the encoder is adjusted, and / or the CPE is controlled to perform frequency switching.

[0027] In some embodiments, the method further comprises:

[0028] Receive registration information of two CPEs, and divide the two CPEs into a main CPE and a backup CPE according to the registration time of the two CPEs, wherein the registration time of the backup CPE is later than the registration time of the main CPE; establish transmission channels with the main CPE and the backup CPE respectively, and set the sending buffer size of each CPE.

[0029] In some embodiments, the method comprises:

[0030] Initialize two circular linked lists Lp and Ls of the sequence numbers of packets to be sent, and record the sequence numbers of packets in the sending buffers of the main CPE and the backup CPE respectively. The number of nodes in each circular linked list is FGOP, and the sequence number of each data packet is stored in the linked list node; send the data packets read from the first local buffer alternately through two transmission channels, and update the packet sequence numbers to the corresponding Lp or Ls; when it is detected that the overflow time of one transmission channel exceeds a preset threshold, execute the single-channel congestion processing step; when it is detected that overflow occurs in two transmission channels, execute the dual-channel congestion processing step.

[0031] In some embodiments, the single channel congestion processing step includes:

[0032] The sending rate of the first transmission channel where overflow occurs is reduced to (1-δ) / 2 times the encoding rate of the previous stage, and the sending rate of the second transmission channel where overflow does not occur is reduced to (1+δ) / 2 times the original; if the second transmission channel overflows after adjusting the sending rate, the encoder is notified through a coding control instruction to adjust the encoding rate to 1 / (1+δ) times the encoding rate of the previous stage; if the first transmission channel no longer overflows after adjusting the sending rate, the encoding rate is maintained unchanged, and after an interval of time T, the encoder is notified through a coding control instruction to gradually increase the encoding rate until it reaches the initial rate; if the first transmission channel still overflows after adjusting the sending rate, the δ value is increased after an interval of time t, and the encoding rate continues to be reduced. When δ is greater than δo, a corresponding signal is sent to the first transmission channel The CPE sends a frequency switching instruction to cut off the first transmission channel; after sending the frequency switching instruction, the encoder is notified through the encoding control instruction to adjust the encoding code rate to η times the initial code rate, and the packet sequence number that has not been sent in the first local buffer is obtained from the linked list of packet sequence numbers to be sent on the first transmission channel, and the packet data is read from the second local buffer according to the packet sequence number and sent using the second transmission channel, and the subsequent data is sent using the second transmission channel; if the frequency switching success instruction is received, the first transmission channel is re-established, and the encoder is notified through the encoding control instruction to gradually increase the encoding code rate until the initial code rate, and the data packets read from the first local buffer are sent alternately through the first transmission channel and the second transmission channel, wherein δ, δo and η are all greater than 0 and less than 1.

[0033] In some embodiments, the dual-channel congestion processing step includes:

[0034] The encoder is notified to modify the encoding rate to n times of the initial rate through an encoding control instruction; if the overflow phenomenon disappears, the encoding rate of the encoder is kept unchanged, and the encoding rate is increased to the initial rate at an interval of T; if there is no more overflow, the encoding rate of the encoder is kept at the initial rate; if the overflow continues, the encoding rate of the encoder is restored to n times of the initial rate; if the overflow continues, a frequency switching instruction is sent to the standby CPE to cut off the first transmission channel where the overflow occurs; after sending the frequency switching instruction, the encoder is notified through an encoding control instruction to adjust the encoding rate to n times of the initial rate, obtain the packet sequence number that has not been sent in the first local buffer from the linked list of the to-be-sent packet sequence numbers of the standby CPE, read the packet data from the second local buffer according to the packet sequence number, use the second transmission channel to send it, and use the second transmission channel to send subsequent data; if the frequency switching success instruction of the standby CPE is received, the first transmission channel is re-established, the encoder is notified to gradually increase the encoding rate until the initial rate through an encoding control instruction, and a single-channel congestion processing step is performed, wherein n is greater than 0 and less than 1.

[0035] An embodiment of the present invention further provides a high-definition live broadcast device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the program, the high-definition live broadcast method as described above is implemented.

[0036] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the high-definition live broadcast method as described above are implemented.

[0037] The embodiments of the present invention have the following beneficial effects:

[0038] In the above scheme, the controller sends the media data encoded by the encoder to at least two CPEs, and can dynamically adjust the encoding bit rate and the sending rate according to the overflow status of the transmission channel between the CPEs. When overflow occurs and the CPE needs to be switched, the encoding bit rate can be reduced to ensure the continuity of transmission; when the CPE switch is about to end, the encoding bit rate can be increased to ensure the clarity and smoothness of the media data, thereby achieving high-definition live broadcast. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of the structure of a high-definition live broadcast device according to an embodiment of the present invention;

[0040] Figure 2 The figure is a flow chart of a high-definition live broadcast method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the technical problems, technical solutions and advantages to be solved by the embodiments of the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0042] The embodiment of the present invention provides a high-definition live broadcast device, such as Figure 1 As shown, including:

[0043] Encoder 11, used for encoding original media data;

[0044] A controller 12, configured to send the media data encoded by the encoder to at least two CPEs, and adjust the sending rate, adjust the encoding bit rate of the encoder and / or control the CPE to perform frequency switching according to the overflow state of the transmission channel between the CPEs;

[0045] The at least two CPEs 13 are used for uplink transmission of received media data.

[0046] In this embodiment, the controller sends the media data encoded by the encoder to at least two CPEs, and can dynamically adjust the encoding bit rate and the sending rate according to the overflow status of the transmission channel between the CPEs. When overflow occurs and the CPE needs to be switched, the encoding bit rate can be reduced to ensure the continuity of the transmission; when the CPE switch is about to end, the encoding bit rate can be increased to ensure the clarity and smoothness of the media data, thereby achieving high-definition live broadcast.

[0047] In some embodiments, the controller 12 is further configured to send an encoding control instruction to the encoder 11;

[0048] The encoder 11 is further configured to calculate an initial bit rate of the original media data according to a resolution and a frame rate of the input original media data, and to adjust the encoding bit rate for encoding the original media data according to an encoding control instruction of the controller during encoding.

[0049] In some embodiments, the controller 12 is also used to receive registration information of two CPEs 13, and divide the two CPEs into a main CPE and a backup CPE according to the registration time of the two CPEs, and the registration time of the backup CPE is later than the registration time of the main CPE; establish transmission channels with the main CPE and the backup CPE respectively, and set the sending buffer size of each CPE.

[0050] In some embodiments, the controller 12 includes a first local buffer and a second local buffer. The first local buffer caches the media data encoded by the encoder in a first-in-first-out queue manner, and the second local buffer stores data in a circular linked list manner. After the sending thread reads data from the first local buffer and puts it into the second local buffer and the CPE's sending buffer, the read data is deleted from the first local buffer.

[0051] In some embodiments, the controller 12 is specifically used to initialize two circular linked lists Lp and Ls of packet sequence numbers to be sent, and record the packet sequence numbers in the sending buffers in the main CPE and the backup CPE respectively. The number of nodes in each circular linked list is FGOP, and the packet sequence number of each data packet is stored in the linked list node; the data packets read from the first local buffer are sent alternately through two transmission channels, and the packet sequence numbers are updated to the corresponding Lp or Ls; when it is detected that the overflow time of a transmission channel exceeds a preset threshold, a single-channel congestion processing step is executed; when overflow is detected in two transmission channels, a dual-channel congestion processing step is executed.

[0052] In some embodiments, when performing the single channel congestion handling step,

[0053] The controller 12 is specifically used to reduce the sending rate of the first transmission channel where overflow occurs to (1-δ) / 2 times the encoding rate of the previous stage, and change the sending rate of the second transmission channel where overflow does not occur to (1+δ) / 2 times the original; if the second transmission channel overflows after the sending rate is adjusted, the encoder is notified through a coding control instruction to adjust the encoding rate to 1 / (1+δ) times the encoding rate of the previous stage; if the first transmission channel no longer overflows after the sending rate is adjusted, the encoding rate is maintained unchanged, and after an interval of time T, the encoder is notified through a coding control instruction to gradually increase the encoding rate until it reaches the initial rate; if the first transmission channel still overflows after the sending rate is adjusted, the δ value is increased after an interval of time t, and the encoding rate continues to be reduced. When δ is greater than δo, the first transmission channel is notified to increase the encoding rate to 1 / (1+δ) times the encoding rate. The CPE corresponding to the transmission channel sends a frequency switching instruction to cut off the first transmission channel; after sending the frequency switching instruction, the encoder is notified through the encoding control instruction to adjust the encoding code rate to n times the initial code rate, and the packet sequence number that has not been sent in the first local buffer is obtained from the to-be-sent packet sequence number linked list of the first transmission channel, and the packet data is read from the second local buffer according to the packet sequence number and sent using the second transmission channel, and the subsequent data is sent using the second transmission channel; if the frequency switching success instruction is received, the first transmission channel is re-established, and the encoder is notified through the encoding control instruction to gradually increase the encoding code rate until the initial code rate, and the data packets read from the first local buffer are sent alternately through the first transmission channel and the second transmission channel, wherein δ, δo and η are all greater than 0 and less than 1;

[0054] The CPE 13 is specifically configured to negotiate with the network side to select a frequency point after receiving the frequency switching instruction from the controller, and return a frequency switching success instruction to the controller.

[0055] In some embodiments, when performing the dual channel congestion processing step,

[0056] The controller 12 is specifically used to notify the encoder to modify the encoding rate to n times of the initial rate through the encoding control instruction; if the overflow phenomenon disappears, keep the encoding rate of the encoder unchanged, increase the encoding rate to the initial rate at an interval T, if there is no more overflow, keep the encoding rate of the encoder at the initial rate, if the overflow continues, restore the encoding rate of the encoder to n times of the initial rate; if the overflow continues, send a frequency switching instruction to the standby CPE to cut off the first transmission channel where the overflow occurs; after sending the frequency switching instruction, notify the encoder to adjust the encoding rate to n times of the initial rate through the encoding control instruction, obtain the packet sequence number that has not been sent in the first local buffer from the linked list of the to-be-sent packet sequence numbers of the standby CPE, read the packet data from the second local buffer according to the packet sequence number, use the second transmission channel to send it, and use the second transmission channel to send subsequent data; if the frequency switching success instruction of the standby CPE is received, re-establish the first transmission channel, notify the encoder to gradually increase the encoding rate until the initial rate through the encoding control instruction, and perform the single-channel congestion processing step, wherein n is greater than 0 and less than 1;

[0057] The standby CPE is specifically configured to negotiate with the network side to select a frequency point after receiving the frequency switching instruction from the controller, and return a frequency switching success instruction to the controller.

[0058] In some embodiments, the CPE is specifically configured not to perform frequency switching when no frequency switching instruction is received from the controller during movement.

[0059] The embodiment of the present invention also provides a high-definition live broadcast method, which is applied to the high-definition live broadcast device as described above. Figure 2 As shown, including:

[0060] Step 201: receiving media data encoded by an encoder;

[0061] Step 202: Send the media data encoded by the encoder to at least two CPEs, and adjust the sending rate, adjust the encoding bit rate of the encoder and / or control the CPE to perform frequency switching according to the overflow status of the transmission channel between the CPEs.

[0062] In this embodiment, the controller sends the media data encoded by the encoder to at least two CPEs, and can dynamically adjust the encoding bit rate and the sending rate according to the overflow status of the transmission channel between the CPEs. When overflow occurs and the CPE needs to be switched, the encoding bit rate can be reduced to ensure the continuity of the transmission; when the CPE switch is about to end, the encoding bit rate can be increased to ensure the clarity and smoothness of the media data, thereby achieving high-definition live broadcast.

[0063] In some embodiments, the method further comprises:

[0064] Receive registration information of two CPEs, and divide the two CPEs into a main CPE and a backup CPE according to the registration time of the two CPEs, wherein the registration time of the backup CPE is later than the registration time of the main CPE; establish transmission channels with the main CPE and the backup CPE respectively, and set the sending buffer size of each CPE.

[0065] In some embodiments, the method comprises:

[0066] Initialize two circular linked lists Lp and Ls of the sequence numbers of packets to be sent, and record the sequence numbers of packets in the sending buffers of the main CPE and the backup CPE respectively. The number of nodes in each circular linked list is FGOP, and the sequence number of each data packet is stored in the linked list node; send the data packets read from the first local buffer alternately through two transmission channels, and update the packet sequence numbers to the corresponding Lp or Ls; when it is detected that the overflow time of one transmission channel exceeds a preset threshold, execute the single-channel congestion processing step; when it is detected that overflow occurs in two transmission channels, execute the dual-channel congestion processing step.

[0067] In some embodiments, the single channel congestion processing step includes:

[0068] The sending rate of the first transmission channel where overflow occurs is reduced to (1-δ) / 2 times the encoding rate of the previous stage, and the sending rate of the second transmission channel where overflow does not occur is reduced to (1+δ) / 2 times the original; if the second transmission channel overflows after adjusting the sending rate, the encoder is notified through a coding control instruction to adjust the encoding rate to 1 / (1+δ) times the encoding rate of the previous stage; if the first transmission channel no longer overflows after adjusting the sending rate, the encoding rate is maintained unchanged, and after an interval of time T, the encoder is notified through a coding control instruction to gradually increase the encoding rate until it reaches the initial rate; if the first transmission channel still overflows after adjusting the sending rate, the δ value is increased after an interval of time t, and the encoding rate continues to be reduced. When δ is greater than δo, a corresponding signal is sent to the first transmission channel The CPE sends a frequency switching instruction to cut off the first transmission channel; after sending the frequency switching instruction, the encoder is notified through the encoding control instruction to adjust the encoding code rate to η times the initial code rate, and the packet sequence number that has not been sent in the first local buffer is obtained from the linked list of packet sequence numbers to be sent on the first transmission channel, and the packet data is read from the second local buffer according to the packet sequence number and sent using the second transmission channel, and the subsequent data is sent using the second transmission channel; if the frequency switching success instruction is received, the first transmission channel is re-established, and the encoder is notified through the encoding control instruction to gradually increase the encoding code rate until the initial code rate, and the data packets read from the first local buffer are sent alternately through the first transmission channel and the second transmission channel, wherein δ, δo and η are all greater than 0 and less than 1.

[0069] In some embodiments, the dual-channel congestion processing step includes:

[0070] The encoder is notified to modify the encoding rate to n times of the initial rate through an encoding control instruction; if the overflow phenomenon disappears, the encoding rate of the encoder is kept unchanged, and the encoding rate is increased to the initial rate at an interval of T; if there is no more overflow, the encoding rate of the encoder is kept at the initial rate; if the overflow continues, the encoding rate of the encoder is restored to n times of the initial rate; if the overflow continues, a frequency switching instruction is sent to the standby CPE to cut off the first transmission channel where the overflow occurs; after sending the frequency switching instruction, the encoder is notified through an encoding control instruction to adjust the encoding rate to n times of the initial rate, obtain the packet sequence number that has not been sent in the first local buffer from the linked list of the to-be-sent packet sequence numbers of the standby CPE, read the packet data from the second local buffer according to the packet sequence number, use the second transmission channel to send it, and use the second transmission channel to send subsequent data; if the frequency switching success instruction of the standby CPE is received, the first transmission channel is re-established, the encoder is notified to gradually increase the encoding rate until the initial rate through an encoding control instruction, and a single-channel congestion processing step is performed, wherein n is greater than 0 and less than 1.

[0071] The technical solution of the present invention is further described below in conjunction with specific embodiments:

[0072] The high-definition live broadcast device of this embodiment includes an encoder, a controller, a main CPE, and a backup CPE. The encoder is responsible for encoding the original media data; the controller is responsible for sending the media data encoded by the encoder to the main and backup CPEs. When the sending buffer of a CPE overflows, the sending rate to the CPE is reduced. When the sending rate is lower than the threshold and the sending buffer still overflows, a switching instruction is issued to the CPE to prevent ping-pong switching; after the controller is turned on, it accepts the registration of the main and backup CPEs. The first registered one is the main CPE, and the second registered one is the backup CPE. The main and backup CPEs are mainly responsible for the uplink transmission of media data.

[0073] The encoder calculates the initial bit rate according to the resolution and frame rate input by the user during initialization. During the encoding process, the encoder can receive encoding control instructions from the controller in real time to change the encoding bit rate.

[0074] The initial bitrate calculation process is as follows: For 4k live broadcast, the resolution is 3840*2160, the frame rate is 50fps, the depth is 10bit, and the bitrate before encoding is BR origin =3840*2160*10*3*50=12Gbps, calculated with a compression ratio of 60:1, the encoded video bit rate is approximately: BR encode =200Mbps.

[0075] The steps performed by the controller are as follows:

[0076] (1) After the controller is powered on, it accepts registrations from the primary and backup CPEs. The first registered CPE is the primary CPE, and the second registered CPE is the backup CPE.

[0077] (2) The controller establishes two transmission channels for the master and backup CPEs respectively, and the buffer size is set to bufferSize = BR encode *F GOP / 50*8, including BR encode is the bit rate of the encoded media data, F GOP The number of video frames in the buffer is the same as GOP, and the default value is 50. Then bufferSize = 200M*50 / (50*8) = 25M bytes.

[0078] (3) The controller initializes two local buffers, each with a size of bufferSize*60, i.e. 240M. Buffer Cq caches the data encoded by the encoder in a first-in-first-out queue. After the sending thread reads the data from Cq and puts it into the sending buffer, the data packet is deleted from Cq. The other buffer Cl is stored in a circular linked list. When the sending thread copies data from Cq, it also adds the data to Cl. The data organization in the buffer is shown in the following table, where the size of each data packet is fixed regardless of the bit rate. PTS (presentation timestamp) is given by the encoder, and each frame of data has the same PTS.

[0079] Package number Pts Frame data

[0080] (4) Initialize two circular linked lists Lp and Ls of the sequence numbers of packets to be sent, which are used to record the sequence numbers of packets in each sending buffer. The number of nodes in each circular linked list is F. GOP ,The packet sequence number of each data packet is stored in the linked list node.

[0081] (5) The data packets are placed alternately into the two sending buffers, and the packet sequence numbers are updated to the corresponding Lp or Ls.

[0082] (6) When it is detected that the write to the send buffer of one transmission channel fails, while the write to the other transmission channel is normal, and the duration exceeds 1s, the following i-Vi operation is performed:

[0083] i. Change the alternating transmission in step (5) to weighted transmission, specifically as follows: reduce the transmission rate of the current transmission channel (i.e., the transmission channel where writing fails) to (1-δ) / 2 times the encoding rate of the previous stage, and the corresponding transmission rate of the other transmission channel becomes (1+δ) / 2 times the original rate, where δ is calculated based on empirical values ​​and can be 0.1;

[0084] ii. If overflow also occurs in another transmission channel after adjusting the sending rate, the encoder is notified to adjust the encoding rate to 1 / (1+δ) times the original rate;

[0085] iii. If the overflow phenomenon of the transmission channel disappears after the weighted adjustment of the sending rate, the encoding bit rate is maintained unchanged, and after a certain time T (for example, 30s), the encoder is notified to gradually increase the encoding bit rate until it reaches the initial bit rate;

[0086] iv. If the transmission channel overflow still exists after the weighted adjustment of the transmission rate, the interval time t (default 2s) increases by δ, and the encoding bit rate of the encoder continues to be reduced. When δ is greater than δo, a frequency switching instruction is sent to the corresponding CPE to cut off the current transmission channel and stop sending media data to it. δo is calculated based on experience and defaults to 0.3;

[0087] After the V.CPE switching instruction is issued, the encoder is notified to change the encoding rate to n times the initial rate. n is determined based on experience and is 0.5 by default. The sequence number of the packet that has not been sent in the local buffer Cq is obtained from the list of packet sequence numbers to be sent on this transmission channel. The packet data is obtained from the local buffer Cl according to the packet sequence number and sent using another transmission channel. Subsequent data transmission is also sent through another transmission channel;

[0088] Vi. After receiving the successful frequency switching instruction, establish a media link (reselect and establish a transmission channel) and send media data to it. Inform the encoder to gradually increase the encoding bit rate to the initial bit rate and adjust the dual-channel transmission mode from weighted transmission to alternating transmission.

[0089] (7) When overflow is detected in the send buffers of the two transmission channels at the same time, the following i-vi operations are performed:

[0090] i. Notify the encoder to change the encoding rate to η times the initial rate, where η is determined based on experience and is 0.8 by default;

[0091] ii. If the overflow phenomenon disappears, keep the encoding rate of the encoder unchanged, and increase the code rate to the initial code rate after a certain interval T (30s by default). If there is no more overflow, keep the initial code rate. If there is still overflow, restore to n times of the initial code rate;

[0092] iii. If the overflow phenomenon still exists, a frequency switching command is sent to the standby CPE to cut off the media link of the current transmission channel and stop sending media data to it;

[0093] iv. Notify the encoder to change the encoding bit rate to η times the initial bit rate, where η is determined based on experience and is 0.3-0.4 by default;

[0094] V. Obtain the sequence number of the packet that has not been sent in the local buffer Cq from the list of the sequence number of the packets to be sent of the standby CPE channel, obtain the packet data from the local buffer Cl according to the sequence number of the packet, and send it using another transmission channel. Subsequent data transmission is also sent through another transmission channel;

[0095] Vi. After receiving the successful frequency switching instruction from the standby CPE, establish a media link and send media data to it, notify the encoder to adjust the encoding bit rate to the initial bit rate, return to the single channel overflow state described in step 6), and process according to step 6).

[0096] In this embodiment, a controller is used to control the switching of the main and backup CPEs to reduce ping-pong switching. When overflow occurs on both transmission channels at the same time, the encoder coding rate is first adjusted. When the coding rate is lower than the threshold and there is still buffer overflow, the backup CPE frequency switching is started. After the switching, the coding rate is switched to the initial rate and the single-channel overflow processing flow is entered. When congestion of the single-channel sending buffer is detected, the sending rates of the two transmission channels are first adjusted to weighted sending. If the adjustment of the sending rate causes overflow of both channels, the coding rate is adjusted appropriately. If the overflow of the transmission channel disappears, it is adjusted back to the original coding rate after a certain interval. The CPE switching process is only started when the coding rate is lower than the threshold and there is still buffer overflow. This mechanism only performs CPE frequency switching when necessary, which largely solves the ping-pong switching that may occur when moving in the cell switching area.

[0097] In addition, this embodiment uses two local buffers for data caching to prevent data loss during CPE switching. Two large local buffers Cq and Cl are initialized. The local buffer Cq caches the data encoded by the encoder in a first-in-first-out queue. After the sending thread reads data from Cq and puts it into the sending buffer, the data packet is deleted from Cq. The other local buffer Cl is stored in a circular linked list. When the sending thread copies data from Cq, it also adds the data packet to Cl. When network congestion is detected, the data packet at the congestion time is obtained from Cl for retransmission. The main and standby switching mechanism ensures that only one device is switched at the same time. When a transmission channel has a buffer overflow, the sending rate of the channel is first reduced. If the overflow phenomenon still exists when the sending rate is lower than the threshold, the frequency switching mechanism of the CPE is started. Before switching, the data packet sequence number in the sending buffer is obtained and taken out from Cl and handed over to another transmission channel for transmission. Only one CPE can switch channels at the same time. Even if the signal quality of the two transmission channels is poor, it can be guaranteed that the data in the switching process can be sent to the receiving end. If buffer overflow occurs in both transmission channels, the standby CPE will start switching the other transmission channel only after the switching is successful. In this way, even if the signal quality of both transmission channels is poor, the data during the switching process can be sent to the receiving end.

[0098] In addition, this embodiment dynamically adjusts the encoding bit rate of the media data according to the sending status to ensure the quality of the media data. When overflow occurs and the CPE needs to be switched, the encoding bit rate is reduced to ensure the continuity of the transmission. When the CPE switch is completed, the encoding bit rate is increased to ensure the clarity and smoothness of the media.

[0099] An embodiment of the present invention further provides a high-definition live broadcast device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the program, the high-definition live broadcast method as described above is implemented.

[0100] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the high-definition live broadcast method as described above are implemented.

[0101] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0102] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A high-definition live broadcast device, characterized in that: include: An encoder, used to encode raw media data; A controller, configured to send the media data encoded by the encoder to at least two CPEs, and adjust the sending rate, adjust the encoding bit rate of the encoder, and / or control the CPE to perform frequency switching according to the overflow state of the transmission channel between the CPEs; The at least two CPEs are used for uplink transmission of received media data; The controller is specifically used to execute a single-channel congestion processing step when it is detected that the overflow time of one transmission channel exceeds a preset threshold; and execute a dual-channel congestion processing step when it is detected that overflow occurs in two transmission channels; The controller is further used to receive registration information of two CPEs, and divide the two CPEs into a main CPE and a backup CPE according to the registration time of the two CPEs, wherein the registration time of the backup CPE is later than the registration time of the main CPE; The controller includes a first local buffer and a second local buffer, the first local buffer caches the media data encoded by the encoder in a first-in-first-out queue manner, the second local buffer stores data in a circular linked list manner, and after the sending thread reads data from the first local buffer and puts it into the second local buffer and the sending buffer of the CPE, the read data is deleted from the first local buffer; When executing the single channel congestion handling step, The controller is specifically used to reduce the sending rate of the first transmission channel where overflow occurs to (1-δ) / 2 times the encoding rate of the previous stage, and change the sending rate of the second transmission channel where overflow does not occur to (1+δ) / 2 times the original; if the second transmission channel overflows after adjusting the sending rate, the encoder is notified through a coding control instruction to adjust the encoding rate to 1 / (1+δ) times the encoding rate of the previous stage; if the first transmission channel no longer overflows after adjusting the sending rate, the encoding rate is maintained unchanged, and after an interval of time T, the encoder is notified through a coding control instruction to gradually increase the encoding rate until it reaches the initial rate; if the first transmission channel still overflows after adjusting the sending rate, the δ value is increased after an interval of time t, and the encoding rate continues to be reduced. When δ is greater than δo, the first transmission channel is notified to increase the encoding rate to 1 / (1+δ) times the encoding rate. The CPE corresponding to the channel sends a frequency switching instruction to cut off the first transmission channel; after sending the frequency switching instruction, the encoder is notified through the encoding control instruction to adjust the encoding code rate to n times the initial code rate, and the packet sequence number that has not been sent in the first local buffer is obtained from the to-be-sent packet sequence number linked list of the first transmission channel, and the packet data is read from the second local buffer according to the packet sequence number and sent using the second transmission channel, and the subsequent data is sent using the second transmission channel; if the frequency switching success instruction is received, the first transmission channel is re-established, and the encoder is notified through the encoding control instruction to gradually increase the encoding code rate until the initial code rate, and the data packets read from the first local buffer are sent alternately through the first transmission channel and the second transmission channel, wherein δ, δo and η are all greater than 0 and less than 1; When executing the dual channel congestion handling steps, The controller is specifically used to notify the encoder to modify the encoding rate to n times of the initial rate through an encoding control instruction; if the overflow phenomenon disappears, keep the encoding rate of the encoder unchanged, increase the encoding rate to the initial rate at an interval T, if there is no more overflow, keep the encoding rate of the encoder at the initial rate, if the overflow continues, restore the encoding rate of the encoder to n times of the initial rate; if the overflow continues, send a frequency switching instruction to the standby CPE to cut off the first transmission channel where the overflow occurs; after sending the frequency switching instruction, notify the encoder to adjust the encoding rate to n times of the initial rate through an encoding control instruction, obtain the packet sequence number that has not been sent in the first local buffer from the linked list of packet sequence numbers to be sent of the standby CPE, read the packet data from the second local buffer according to the packet sequence number and use the second transmission channel to send it, and use the second transmission channel to send subsequent data; if the frequency switching success instruction of the standby CPE is received, re-establish the first transmission channel, notify the encoder to gradually increase the encoding rate until the initial rate through an encoding control instruction, and perform a single-channel congestion processing step, wherein n is greater than 0 and less than 1.

2. The high-definition live broadcast device according to claim 1, characterized in that: The controller is also used to send encoding control instructions to the encoder; The encoder is also used to calculate the initial bit rate of the original media data according to the resolution and frame rate of the input original media data, and adjust the encoding bit rate for encoding the original media data according to the encoding control instruction of the controller during encoding.

3. The high-definition live broadcast device according to claim 1, characterized in that: A transmission channel is established with the primary CPE and the backup CPE respectively, and a sending buffer size of each CPE is set.

4. The high-definition live broadcast device according to claim 3, characterized in that: The controller is specifically used to initialize two circular linked lists Lp and Ls of packet sequence numbers to be sent, respectively recording the packet sequence numbers in the sending buffers in the primary CPE and the backup CPE, the number of nodes in each circular linked list is FGOP, and the packet sequence number of each data packet is stored in the linked list node; the data packets read from the first local buffer are sent alternately through two transmission channels, and the packet sequence numbers are updated to the corresponding Lp or Ls.

5. The high-definition live broadcast device according to claim 4, characterized in that: The CPE is specifically configured to negotiate with the network side to select a frequency point after receiving the frequency switching instruction from the controller, and return a frequency switching success instruction to the controller.

6. The high-definition live broadcast device according to claim 4, characterized in that: The standby CPE is specifically configured to negotiate with the network side to select a frequency point after receiving the frequency switching instruction from the controller, and return a frequency switching success instruction to the controller.

7. The high-definition live broadcast device according to claim 5 or 6, characterized in that: The CPE is specifically configured not to perform frequency switching when no frequency switching instruction from the controller is received during movement.

8. A high-definition live broadcast method, characterized in that: A high-definition live broadcast device as claimed in any one of claims 1 to 7, comprising: Receiving media data encoded by the encoder; The media data encoded by the encoder is sent to at least two CPEs, and the sending rate is adjusted according to the overflow status of the transmission channel between the CPEs, the encoding bit rate of the encoder is adjusted, and / or the CPE is controlled to perform frequency switching.

9. The high-definition live broadcast method according to claim 8, characterized in that: Also includes: receiving registration information of two CPEs, and dividing the two CPEs into a primary CPE and a backup CPE according to the registration time of the two CPEs, wherein the registration time of the backup CPE is later than the registration time of the primary CPE; A transmission channel is established with the primary CPE and the backup CPE respectively, and a sending buffer size of each CPE is set.

10. The high-definition live broadcast method according to claim 9, characterized in that: In the high-definition live broadcast device, the controller is further used to receive registration information of two CPEs, and divide the two CPEs into a main CPE and a backup CPE according to the registration time of the two CPEs, and the registration time of the backup CPE is later than the registration time of the main CPE; Establishing transmission channels with the primary CPE and the backup CPE respectively, and setting the sending buffer size of each CPE; The controller includes a first local buffer and a second local buffer, the first local buffer caches the media data encoded by the encoder in a first-in-first-out queue manner, the second local buffer stores data in a circular linked list manner, and after the sending thread reads data from the first local buffer and puts it into the second local buffer and the sending buffer of the CPE, the read data is deleted from the first local buffer; The controller is specifically used to initialize two circular linked lists Lp and Ls of packet sequence numbers to be sent, respectively recording the packet sequence numbers in the sending buffers of the primary CPE and the backup CPE, the number of nodes in each circular linked list is FGOP, and the packet sequence number of each data packet is stored in the linked list node; the data packets read from the first local buffer are sent alternately through two transmission channels, and the packet sequence numbers are updated to the corresponding Lp or Ls at the same time; The CPE is specifically configured to negotiate with the network side to select a frequency point after receiving the frequency switching instruction from the controller, and return a frequency switching success instruction to the controller; The high-definition live broadcast method comprises: Initialize two circular linked lists Lp and Ls of the sequence numbers of packets to be sent, and record the sequence numbers of packets in the sending buffers of the main CPE and the backup CPE respectively. The number of nodes in each circular linked list is FGOP, and the sequence number of each data packet is stored in the linked list node; send the data packets read from the first local buffer alternately through two transmission channels, and update the packet sequence numbers to the corresponding Lp or Ls; when it is detected that the overflow time of one transmission channel exceeds a preset threshold, execute the single-channel congestion processing step; when it is detected that overflow occurs in two transmission channels, execute the dual-channel congestion processing step.

11. The high-definition live broadcast method according to claim 10, characterized in that: The single channel congestion processing step comprises: The sending rate of the first transmission channel where overflow occurs is reduced to (1-δ) / 2 times the encoding rate of the previous stage, and the sending rate of the second transmission channel where overflow does not occur is reduced to (1+δ) / 2 times the original; if the second transmission channel overflows after adjusting the sending rate, the encoder is notified through a coding control instruction to adjust the encoding rate to 1 / (1+δ) times the encoding rate of the previous stage; if the first transmission channel no longer overflows after adjusting the sending rate, the encoding rate is maintained unchanged, and after an interval of time T, the encoder is notified through a coding control instruction to gradually increase the encoding rate until it reaches the initial rate; if the first transmission channel still overflows after adjusting the sending rate, the δ value is increased after an interval of time t, and the encoding rate continues to be reduced. When δ is greater than δo, a corresponding signal is sent to the first transmission channel The CPE sends a frequency switching instruction to cut off the first transmission channel; after sending the frequency switching instruction, the encoder is notified through the encoding control instruction to adjust the encoding code rate to η times the initial code rate, and the packet sequence number that has not been sent in the first local buffer is obtained from the linked list of packet sequence numbers to be sent on the first transmission channel, and the packet data is read from the second local buffer according to the packet sequence number and sent using the second transmission channel, and the subsequent data is sent using the second transmission channel; if the frequency switching success instruction is received, the first transmission channel is re-established, and the encoder is notified through the encoding control instruction to gradually increase the encoding code rate until the initial code rate, and the data packets read from the first local buffer are sent alternately through the first transmission channel and the second transmission channel, wherein δ, δo and η are all greater than 0 and less than 1.

12. The high-definition live broadcast method according to claim 10, characterized in that: The dual-channel congestion processing steps include: The encoder is notified to modify the encoding rate to n times of the initial rate through an encoding control instruction; if the overflow phenomenon disappears, the encoding rate of the encoder is kept unchanged, and the encoding rate is increased to the initial rate at an interval of T; if there is no more overflow, the encoding rate of the encoder is kept at the initial rate; if the overflow continues, the encoding rate of the encoder is restored to n times of the initial rate; if the overflow continues, a frequency switching instruction is sent to the standby CPE to cut off the first transmission channel where the overflow occurs; after sending the frequency switching instruction, the encoder is notified through an encoding control instruction to adjust the encoding rate to n times of the initial rate, obtain the packet sequence number that has not been sent in the first local buffer from the linked list of the to-be-sent packet sequence numbers of the standby CPE, read the packet data from the second local buffer according to the packet sequence number, use the second transmission channel to send it, and use the second transmission channel to send subsequent data; if the frequency switching success instruction of the standby CPE is received, the first transmission channel is re-established, the encoder is notified to gradually increase the encoding rate until the initial rate through an encoding control instruction, and a single-channel congestion processing step is performed, wherein n is greater than 0 and less than 1.

13. A high-definition live broadcast device, characterized in that: It comprises a memory, a processor and a computer program stored in the memory and executable on the processor; it is characterized in that when the processor executes the program, the high-definition live broadcast method as described in any one of claims 8 to 12 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the high-definition live broadcast method as described in any one of claims 8 to 12 are implemented.

Images