Live broadcast data processing method, device and system, computer equipment and storage medium

Abstract

The invention relates to a live broadcast data processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: through a target playing server, according to a request sent by a first playing terminal, sending a live broadcast query request carrying a first live broadcast stream identifier and an own playing server identifier to a target scheduling server; determining a target publishing server based on the first live broadcast stream identifier through the target scheduling server, determining a communication path between the target playing server and the target publishing server based on the playing server identifier and the publishing server identifier, and returning the communication path; when the communication quality between the target playing server and the target publishing server meets a preset condition, the communication path passes through at least one proxy server; and through the target playing server, sending a live broadcast data stream corresponding to the first live broadcast stream identifier obtained from the target publishing server based on the communication path to the first playing terminal. The method can improve the live broadcast quality.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a live broadcast data processing method, apparatus, system, computer equipment, storage medium, and computer program product. Background Art

[0002] With the development of computer technology, live streaming has emerged. This technology has gained popularity among users due to its combination of the advantages of images, sound, and text, and its ability to provide users with instant and interactive communication. The number of active users of live streaming continues to increase, and audience needs are also diversifying. More and more people are entering the live streaming industry not only as viewers but also as hosts.

[0003] Traditionally, the live broadcaster sends its audio and video data to the viewer, while the viewer receives the data from the live broadcaster. However, direct interaction between the viewer and the live broadcaster increases the pressure on the live broadcaster, which in turn affects the quality of the live broadcast. Summary of the Invention

[0004] Based on this, it is necessary to provide a live broadcast data processing method, device, system, computer equipment, computer-readable storage medium and computer program product that can improve the quality of live broadcast in response to the above technical problems.

[0005] This application provides a live broadcast data processing method, including:

[0006] Obtaining, through the target playback server, a live viewing request carrying a first live stream identifier sent by the first playback terminal, and sending a live query request carrying the first live stream identifier and the playback server identifier of the target playback server to the target scheduling server;

[0007] The target scheduling server determines a target publishing server based on the first live stream identifier, determines a communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server, and returns the communication path to the target playback server; when the communication quality between the target playback server and the target publishing server meets a preset condition, the communication path passes through at least one proxy server, and the proxy server is used to connect the target playback server and the target publishing server;

[0008] The target playback server obtains the live data stream corresponding to the first live stream identifier from the target publishing server based on the communication path, and sends the live data stream corresponding to the first live stream identifier to the first playback terminal.

[0009] The present application also provides a live broadcast data processing device, comprising:

[0010] The playback server processing module is configured to obtain, through the target playback server, a live viewing request carrying a first live stream identifier sent by the first playback terminal, and send a live query request carrying the first live stream identifier and the playback server identifier of the target playback server to the target scheduling server;

[0011] a scheduling server processing module, configured to determine, through the target scheduling server, a target publishing server based on the first live stream identifier, determine a communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server, and return the communication path to the target playback server; when the communication quality between the target playback server and the target publishing server meets a preset condition, the communication path passes through at least one proxy server, and the proxy server is configured to connect the target playback server and the target publishing server;

[0012] The playback server processing module is also used to obtain the live data stream corresponding to the first live stream identifier from the target publishing server through the target playback server based on the communication path, and send the live data stream corresponding to the first live stream identifier to the first playback terminal.

[0013] A live broadcast data processing system, characterized in that the system includes a target playback server and a target scheduling server:

[0014] The target playback server is configured to obtain a live viewing request carrying a first live stream identifier sent by a first playback terminal, and send a live query request carrying the first live stream identifier and a playback server identifier of the target playback server to the target scheduling server;

[0015] The target scheduling server is configured to determine a target publishing server based on the first live stream identifier, determine a communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server, and return the communication path to the target playback server; when the communication quality between the target playback server and the target publishing server meets a preset condition, the communication path passes through at least one proxy server, and the proxy server is configured to connect the target playback server and the target publishing server;

[0016] The target play server is further configured to acquire a live data stream corresponding to the first live stream identifier from the target publishing server based on the communication path, and send the live data stream corresponding to the first live stream identifier to the first play terminal.

[0017] The application further provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the live data processing method when executing the computer program.

[0018] The application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the live data processing method when executed by a processor.

[0019] The application further provides a computer program product, comprising a computer program, and the computer program implements the steps of the live data processing method when executed by a processor.

[0020] The live data processing method, device, system, computer device, storage medium and computer program product described above, through the target play server, acquires a live view request carrying a first live stream identifier sent by a first play terminal, and sends a live query request carrying the first live stream identifier and a play server identifier of the target play server to the target scheduling server; through the target scheduling server, the target publishing server is determined based on the first live stream identifier, the communication path between the target play server and the target publishing server is determined based on the play server identifier and the publishing server identifier of the target publishing server, and the communication path is returned to the target play server; when the communication quality between the target play server and the target publishing server meets a preset condition, the communication path passes through at least one proxy server, and the proxy server is used to connect the target play server and the target publishing server; through the target play server, the live data stream corresponding to the first live stream identifier is acquired from the target publishing server based on the communication path, and the live data stream corresponding to the first live stream identifier is sent to the first play terminal. In this way, the target play server responds to the live view request of the first play terminal, queries the target publishing server where the live data is located and the communication path between the target publishing server from the target scheduling server, the target play server acquires the live data from the target publishing server based on the queried communication path, and sends the live data to the first play terminal, so that the first terminal realizes live view. Through the orderly interaction between the first play terminal, the target play server, the target scheduling server and the target publishing server, the live view is realized, which can reduce the pressure of the live terminal and guarantee the live quality. Further, the target scheduling server determines the target publishing server where the live data is located according to the play server identifier, and then determines the communication path between the target play server and the target publishing server according to the play server identifier and the publishing server identifier. When the communication quality between the target play server and the target publishing server meets the preset condition, the communication path passes through at least one proxy server, and the proxy server can guarantee the stable transmission of the live data between the target play server and the target publishing server, further improving the live quality. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiments or related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0022] Figure 1 An application environment diagram of the live data processing method in an embodiment;

[0023] Figure 21 is a flow chart of a live broadcast data processing method according to an embodiment;

[0024] Figure 3 A schematic diagram of data interaction between various devices during a live broadcast in one embodiment;

[0025] Figure 4 is a schematic diagram of a communication protocol used between various servers in one embodiment;

[0026] Figure 5 A schematic diagram of data interaction between various device clusters during a live broadcast in one embodiment;

[0027] Figure 6 A schematic diagram of obtaining live broadcast data with the help of a proxy server in one embodiment;

[0028] Figure 7 A schematic diagram of data processing by a server during a live broadcast in one embodiment;

[0029] Figure 8 A schematic diagram of data processing between a live broadcast terminal and a playback terminal during a live broadcast process in one embodiment;

[0030] Figure 9 A schematic diagram of a live broadcast data processing system according to an embodiment;

[0031] Figure 10 A system schematic diagram of a live broadcast data processing system in another embodiment;

[0032] Figure 11 A system diagram of a live broadcast data processing system in another embodiment;

[0033] Figure 12 A system diagram of a live broadcast data processing system in another embodiment;

[0034] Figure 13 is a structural block diagram of a live data processing device in one embodiment;

[0035] Figure 14 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0037] The live data processing method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the playback terminal 102 communicates with the target playback server 104, the target playback server 104 communicates with the target scheduling server 106, the target scheduling server 106 communicates with the target publishing server 108, and the target publishing server 108 communicates with the live broadcast terminal 110. Under certain conditions, the target playback server 104 communicates with the target publishing server 108. Under certain conditions, the target playback server 104 communicates with the proxy server 112, and the target publishing server 108 communicates with the proxy server 112.

[0038] It can be understood that the playback terminal refers to the terminal for watching live broadcasts, that is, the audience terminal. The live broadcast terminal refers to the terminal that starts the live broadcast, that is, the anchor terminal. Live broadcast applications are set up on the live broadcast terminal and the playback terminal. The live broadcast application can refer to a client installed in the terminal. The client (also known as the application client, APP client) refers to a program installed and running in the terminal; the live broadcast application can also refer to an installation-free application, that is, an application that can be used without downloading and installing. This type of application can also be called a mini-program, which usually runs as a subprogram in the client; the live broadcast application can also refer to a web application opened through a browser; and so on.

[0039] Terminals include, but are not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices include smart speakers, smart TVs, smart air conditioners, and smart car devices. Portable wearable devices include smart watches, smart bracelets, and head-mounted devices. Servers can be standalone servers, server clusters composed of multiple physical servers, or distributed systems. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0040] It is understandable that when the live broadcast terminal is broadcasting live, it pushes the live broadcast data to the publishing server. When the playback terminal is watching the live broadcast, it can pull the live broadcast data from the playback server. The playback server can query the publishing server where the live broadcast data is located through the scheduling server, and the playback server pulls the live broadcast data from the queried publishing server. Under certain conditions, the proxy server is used to connect the playback server and the publishing server. Any of the playback server, publishing server, scheduling server, and proxy server can be an independent server, or a server cluster or distributed system composed of multiple physical servers. Of course, the playback server, publishing server, scheduling server, and proxy server can also be integrated into a server cluster or distributed system composed of multiple physical servers.

[0041] Specifically, the first playback terminal sends a live viewing request carrying a first live stream identifier to a target playback server. The target playback server responds to the live viewing request by sending a live query request carrying the first live stream identifier and its own playback server identifier to a target scheduling server. The target scheduling server responds to the live query request, determines a target publishing server based on the first live stream identifier, and determines a communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server. When the communication quality between the target playback server and the target publishing server meets preset conditions, the communication path passes through at least one proxy server, which is used to connect the target playback server and the target publishing server. The target scheduling server returns the communication path to the target playback server. Based on the communication path, the target playback server obtains the live data stream corresponding to the first live stream identifier from the target publishing server. The target playback server sends the live data stream corresponding to the first live stream identifier to the first playback terminal. The first playback terminal can decode and play the live data stream corresponding to the first live stream identifier.

[0042] It is understood that the live broadcast terminal will collect live broadcast data to generate a live broadcast data stream corresponding to the first live broadcast stream identifier during live broadcast. The first playback terminal can be any playback terminal.

[0043] In one embodiment, Figure 2 As shown, a live broadcast data processing method is provided, comprising the following steps.

[0044] Step S202: Obtain a live viewing request carrying a first live stream identifier sent by the first playback terminal through the target playback server, and send a live query request carrying the first live stream identifier and the playback server identifier of the target playback server to the target scheduling server.

[0045] When a live broadcast terminal starts a live broadcast, it can register relevant live broadcast information, such as the live stream ID, live broadcast type, and live data encoding type, with the scheduling server through the publishing server. During the live broadcast, the live broadcast terminal pushes the live data to the publishing server. When watching the live broadcast, the playback terminal uses the live stream ID to pull the live data from the playback server. The playback server can query the publishing server for the live data through the scheduling server and pull the live data from the queried publishing server.

[0046] When a player wants to watch a live broadcast, it sends a live streaming request to the server. This request carries the live stream ID of the desired broadcast and is used to request the live data corresponding to the live stream ID. The live stream ID identifies the live data for a particular broadcast. This ID allows for accurate identification and management of live data, enabling the transmission, playback, and monitoring of live content.

[0047] The first playback terminal can be any playback terminal. When the first playback terminal needs to watch a live broadcast, the first playback terminal obtains the first live stream identifier corresponding to the live broadcast to be watched, generates a live viewing request carrying the first live stream identifier, and sends the live viewing request to the target playback server. The target playback server can be any playback server or a specific playback server. For example, a playback server that matches the first playback terminal is obtained from the playback server cluster as the target playback server. The playback server that matches the first playback terminal can be a playback server that is geographically close to the first playback terminal, a playback server that is close to the operator of the first playback terminal, etc.

[0048] The playback server identifier is used to identify the playback server. For example, the playback server number is used as the playback server identifier; the playback server communication address is used as the playback server identifier; and so on. The target playback server responds to the live viewing request sent by the first playback terminal and sends a live query request to the target scheduling server. The live query request is used to request the publishing server where the live data corresponding to the live stream identifier is located. The target scheduling server can be any scheduling server or a specific scheduling server. For example, a scheduling server that matches the target playback server is obtained from the scheduling server cluster as the target scheduling server. The scheduling server that matches the target playback server can be a scheduling server that is geographically close to the target playback server, a scheduling server that is close to the target playback server operator, etc.

[0049] Specifically, when a first playback terminal needs to watch a live broadcast, the first playback terminal obtains the first live stream identifier corresponding to the live broadcast to be watched, generates a live broadcast viewing request carrying the first live stream identifier, and sends the live broadcast viewing request to the target playback server. For example, if a user enters a live broadcast room on the first playback terminal, the first playback terminal obtains the first live stream identifier corresponding to the live broadcast room, generates a live broadcast viewing request carrying the first live stream identifier, and sends the live broadcast viewing request to the target playback server.

[0050] The target playback server receives a live broadcast viewing request sent by the first playback terminal, the live broadcast viewing request carrying the first live broadcast stream identifier, and generates a live broadcast query request in response to the live broadcast viewing request, the live broadcast query request carrying the first live broadcast stream identifier and the playback server identifier of the target playback server. The target playback server sends the live broadcast query request to the target scheduling server.

[0051] Step S204, through the target scheduling server, determines the target publishing server based on the first live stream identifier, determines the communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server, and returns the communication path to the target playback server; when the communication quality between the target playback server and the target publishing server meets the preset conditions, the communication path passes through at least one proxy server, and the proxy server is used to connect the target playback server and the target publishing server.

[0052] The target publishing server is the publishing server where the live data corresponding to the first live stream identifier is located. The target publishing server can be any publishing server where the live data corresponding to the first live stream identifier is located, or it can be a publishing server where the live data corresponding to the first live stream identifier is located and that matches the target playback server. For example, a publishing server that matches the target playback server can be a publishing server that is geographically close to the target playback server, a publishing server that is close to the operator of the target playback server, etc. The publishing server identifier is used to identify the publishing server. For example, the publishing server number can be used as the publishing server identifier; the publishing server's communication address can be used as the publishing server identifier; and so on.

[0053] The target scheduling server not only needs to determine the target publishing server, but also needs to determine the communication path between the target playback server and the target publishing server. The communication path between the target playback server and the target publishing server refers to the network path used when transmitting data between the target playback server and the target publishing server. When the communication quality between the target playback server and the target publishing server meets the preset conditions, the communication path between the target playback server and the target publishing server passes through at least one proxy server. The proxy server is used to connect the target playback server and the target publishing server and transfer data between the target playback server and the target publishing server. For example, when the target playback server and the target publishing server cannot communicate with each other, the target playback server and the target publishing server are connected through the proxy server; when the communication pressure on the target publishing server is large, the communication pressure on the target publishing server is reduced through the proxy server. After the proxy server obtains the live data corresponding to the first live stream identifier from the target publishing server, it can provide the corresponding live data to the playback server that needs the live data corresponding to the first live stream identifier.

[0054] It is understandable that the preset conditions can be set according to actual needs. For example, the preset condition may be less than a preset communication quality threshold. When the communication quality between the target playback server and the target publishing server is less than the preset communication quality threshold, it is determined that the communication quality between the target playback server and the target publishing server meets the preset condition. The preset condition may be that the target playback server and the target publishing server cannot communicate with each other. The preset condition may be that the communication speed between the target playback server and the target publishing server is less than a preset speed. The preset condition is used to indicate that the communication quality between the playback server and the publishing server will have a certain degree of impact on the live broadcast. When the communication quality between the target playback server and the target publishing server meets the preset condition, by introducing a proxy server, the impact on the live broadcast can be alleviated and the quality of the live broadcast can be guaranteed.

[0055] The communication quality between the target playback server and the target publishing server can be obtained through dedicated testing, based on response information from the target playback server and the target publishing server when processing the live broadcast task, or can be pre-recorded. For example, data records indicating whether each playback server and each publishing server are interoperable can be pre-stored, and interoperability between the target playback server and the target publishing server can be queried from the data records. If the target playback server and the target publishing server are not interoperable, then the communication quality between the target playback server and the target publishing server meets the preset conditions.

[0056] Specifically, the target scheduling server responds to the live broadcast query request, determines a target publishing server based on the first live broadcast stream identifier, and searches for the publishing server where the live broadcast data corresponding to the first live broadcast stream identifier resides as the target publishing server. The target scheduling server determines the communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server. The target scheduling server can determine the communication quality between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier, and determine whether the communication quality between the target playback server and the target publishing server meets a preset condition. If the communication quality between the target playback server and the target publishing server meets the preset condition, the target scheduling server determines the required proxy server, and determines the communication path between the target playback server and the target publishing server based on the playback server identifier, the publishing server identifier, and the proxy server identifier of the proxy server, with the communication path traversing the proxy server. If the communication quality between the target playback server and the target publishing server does not meet the preset condition, the target scheduling server directly determines the communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier. The target scheduling server returns the communication path to the target playback server.

[0057] It is understood that when the communication quality between the target playback server and the target publishing server meets preset conditions, the communication path between the target playback server and the target publishing server may pass through at least one proxy server. The number of proxy servers can be set according to actual needs. For example, the worse the communication quality, the more proxy servers can be used; the more playback terminals, the more proxy servers can be used; the more playback terminals corresponding to the same live stream identifier, the more proxy servers can be used; and so on.

[0058] In one embodiment, the target scheduling server determines a target publishing server based on the playback server identifier and the first live stream identifier. Based on the playback server identifier and the first live stream identifier, the publishing server that hosts the live data corresponding to the first live stream identifier and matches the target playback server is selected as the target publishing server. For example, the publishing server that matches the target playback server can be a publishing server located in a geographically close proximity to the target playback server, a publishing server that is close to the operator of the target playback server, or the like. Thus, by selecting the publishing server that hosts the live data corresponding to the first live stream identifier and matches the target playback server as the target publishing server, and obtaining live data from the target publishing server, data acquisition efficiency can be improved, thereby ensuring live broadcast quality.

[0059] Step S206: The target playback server obtains the live data stream corresponding to the first live stream identifier from the target publishing server based on the communication path, and sends the live data stream corresponding to the first live stream identifier to the first playback terminal.

[0060] Among them, the live data stream corresponding to the first live stream identifier refers to the live data corresponding to the first live stream identifier.

[0061] Specifically, the target playback server obtains the live data stream corresponding to the first live stream identifier from the target publishing server based on the communication path fed back by the target scheduling server, and sends the live data stream corresponding to the first live stream identifier to the first playback terminal. The first playback terminal can decode and play the live data stream corresponding to the first live stream identifier.

[0062] It can be understood that each server along the communication path can process the received live data stream and then forward it to the next server, or can directly forward the received live data stream to the next server.

[0063] In the above-mentioned live broadcast data processing method, the target playback server responds to the live broadcast viewing request of the first playback terminal and queries the target publishing server where the live broadcast data is located, as well as the communication path between the target publishing server and the target playback server, from the target scheduling server. The target playback server obtains the live broadcast data from the target publishing server based on the queried communication path and sends the live broadcast data to the first playback terminal, thereby enabling the first terminal to watch the live broadcast. Live broadcast viewing is achieved through orderly interaction between the first playback terminal, the target playback server, the target scheduling server, and the target publishing server, which can reduce the pressure on the live broadcast terminal and ensure the quality of the live broadcast. Furthermore, the target scheduling server determines the target publishing server where the live broadcast data is located based on the playback server identifier, and then determines the communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier. When the communication quality between the target playback server and the target publishing server meets preset conditions, the communication path passes through at least one proxy server. The proxy server can ensure stable transmission of live broadcast data between the target playback server and the target publishing server, further improving the quality of the live broadcast.

[0064] In one embodiment, the live broadcast data processing method further includes:

[0065] The live data stream corresponding to the first live stream identifier is sent to the target publishing server through the live broadcast device.

[0066] Among them, the first communication protocol is adopted between the live broadcast terminal and the target publishing server, and between the first playback terminal and the target playback server, and the second communication protocol is adopted between the target publishing server and the target scheduling server, between the target playback server and the target scheduling server, and between the servers passed through the communication path; the reliability of the first communication protocol is lower than the reliability of the second communication protocol, and the transmission efficiency of the first communication protocol is higher than the transmission efficiency of the second communication protocol.

[0067] It can be understood that the higher the reliability of the communication protocol, the more the communication protocol can guarantee the delivery of data.

[0068] Specifically, during the live broadcast process of the live broadcast terminal, the live broadcast terminal will continuously generate live broadcast data, and the live broadcast terminal will continuously push the live broadcast data to the publishing server. The live broadcast device corresponding to the first live broadcast stream identifier sends the live broadcast data stream corresponding to the first live broadcast stream identifier to the target publishing server.

[0069] Live data is transmitted between the live terminal and the publishing server, and between the playback terminal and the playback server. Considering that the terminal network environment is relatively complex and the amount of live data is large, a certain degree of loss of live data is allowed. The first communication protocol with lower reliability but higher transmission efficiency can be used to transmit data between the live terminal and the publishing server, and the first communication protocol with lower reliability but higher transmission efficiency can be used to transmit data between the playback terminal and the playback server.

[0070] The target publishing server and the target scheduling server transmit command data, and the target playback server and the target scheduling server transmit command data. Considering the small amount of command data, the target publishing server and the target scheduling server can use a second communication protocol with higher reliability but lower transmission efficiency to transmit data, and the target playback server and the target scheduling server can use a second communication protocol with higher reliability but lower transmission efficiency to transmit data. Although the communication path between the playback server and the scheduling server transmits live data, to ensure data arrival, the second communication protocol with higher reliability but lower transmission efficiency can be used to transmit data between the servers along the communication path.

[0071] In one embodiment, the first communication protocol is the UDP protocol (User Datagram Protocol), and the second communication protocol is the TCP protocol (Transmission Control Protocol).

[0072] In one embodiment, the first communication protocol and the second communication protocol are both private encryption protocols with high security.

[0073] In the above embodiments, the use of a first communication protocol with higher transmission efficiency between the live broadcast terminal and the target publishing server, and between the first playback terminal and the target playback server, can improve data transmission efficiency, reduce live broadcast delay, and enhance live broadcast quality. The use of a second communication protocol with higher reliability between the target publishing server and the target scheduling server, between the target playback server and the target scheduling server, and between servers along the communication path can avoid or reduce data loss and ensure live broadcast quality.

[0074] In one embodiment, reference Figure 3 ,A live broadcast process involves data interaction between the live broadcast terminal (i.e., the anchor), the publishing server (pub), the scheduling server (scheduler), the database (such as redis), the playback server (player, play), and the playback terminal (i.e., the audience).

[0075] The database is used to record all stream ID information and server information. When the host sends a stream, it connects to the pub and informs the pub of the stream ID, type (audio / video / audio and video), audio and video encoding type, etc. (see Figure 3 ① in the figure). Pub will inform the scheduler of this information to register and inform the host of the registration result (see ②③④⑤ in the figure). If the registration is successful, the host will continue to push media data (media) to pub. When the audience watches the stream, it will use the stream ID to pull the stream from play (see ⑥ in the figure). Play will query the scheduler to find out which pub to pull the stream from, and then inform the audience of the query result (see ⑦⑧⑨⑩ in the figure). If the query is successful, play will pull the media data from pub and send the media data to the audience at the same time. The scheduler connects to redis, pub, and play at the same time to complete the corresponding scheduling. When sending the stream, the scheduler registers the stream information from pub to redis and informs pub of the result. When pulling the stream, the scheduler takes the stream information from play to query redis for the pub information where the stream is located. According to the corresponding policy configuration, it selects the most appropriate route and tells play to pull the stream from the specified pub.

[0076] Similarly, if it is a two-way interaction, all of the above are two-way. The playback terminal can also be used as a live broadcast terminal, and the live broadcast terminal can also be used as an audience terminal.

[0077] In one embodiment, reference Figure 4TCP is used between the scheduler (scheduling server) and the pub (publishing server) and player (playing server). No media data is transmitted between the scheduler, pub, and player; only command-related messages are small in size, ensuring message delivery. UDP is used between user terminals and pub and player. Due to the complex user network environment, this layer of interaction is subject to significant packet loss, out-of-order messages, and latency, which can trigger network congestion and other disasters. Furthermore, this layer transmits large amounts of media data, requiring low latency and a higher tolerance for packet loss and out-of-order messages than signaling. For these reasons, UDP is used between user terminals and pub and player. Furthermore, UDP can be combined with RTP (Real-time Transport Protocol) or RCP (RTP's control protocol) for reliable transmission. TCP is used between pub and player. Although media data is also transmitted between pub and player, pub and player can utilize cloud servers, eliminating the complexities of the user layer. Therefore, TCP is used between pub and player to ensure data delivery.

[0078] In one embodiment, determining a target publishing server based on the first live stream identifier includes:

[0079] Based on the first live stream identifier, a first publishing server is determined from the publishing server cluster; the first publishing server stores the live data stream corresponding to the first live stream identifier; based on the playback server identifier, the operator identifier and region identifier of the target playback server are determined; based on the operator identifier, a second publishing server is determined from each first publishing server; based on the region identifier, a target publishing server is determined from each second publishing server.

[0080] The operator identifier is used to identify the operator of the communication service used by the server, that is, to identify the telecommunications operator to which the server belongs. The region identifier is used to identify the deployment location of the server. It can be understood that the publishing server cluster includes multiple publishing servers.

[0081] Specifically, the target scheduling server determines a first publishing server from the publishing server cluster based on the first live stream identifier, and obtains the publishing server containing the live data corresponding to the first live stream identifier from the publishing server cluster as the first publishing server. The target scheduling server determines the operator identifier and region identifier of the target playback server based on the playback server identifier. For example, the target scheduling server queries the operator identifier and region identifier of the target playback server from a database based on the playback server identifier; the playback server identifier includes the operator identifier and region identifier, etc. Furthermore, the target scheduling server determines a second publishing server from each first publishing server based on the operator identifier of the target playback server, preferentially selecting as the second publishing server a publishing server with the same operator identifier as the target playback server from each first publishing server. Furthermore, the target scheduling server determines a target publishing server from each second publishing server based on the region identifier of the target playback server, preferentially selecting as the target publishing server a publishing server with the same region identifier as the target playback server from each second publishing server.

[0082] It is understood that if none of the first publishing servers has the same operator ID as the target playback server, then the first publishing server can be used as the second publishing server, or a random first publishing server can be selected as the second publishing server, or the first publishing server corresponding to a telecom operator with better historical live broadcast performance can be selected as the second publishing server. If none of the second publishing servers has the same region ID as the target playback server, then a second publishing server in a region close to the target playback server can be selected as the target publishing server.

[0083] In the above embodiment, a first publishing server is determined from a publishing server cluster based on a first live stream identifier, a second publishing server is determined from each of the first publishing servers based on an operator identifier, and a target publishing server is determined from each of the second publishing servers based on a region identifier. In this way, by first filtering out the publishing servers holding the live data required by the playback server, the playback server can accurately obtain the required live data from the queried publishing servers. Then, filtering based on the operator can improve the playback server's efficiency in obtaining live data from the queried publishing servers. Finally, filtering based on the region can further improve the playback server's efficiency in obtaining live data from the queried publishing servers.

[0084] In one embodiment, obtaining, through the target playback server, a live viewing request carrying a first live stream identifier sent by the first playback terminal, and sending a live query request carrying the first live stream identifier and a playback server identifier of the target playback server to the target scheduling server includes:

[0085] The first playback terminal determines the first playback server from the playback server cluster based on its own operator identifier, determines the target playback server from each first playback server based on its own region identifier, and sends a live viewing request carrying the first live stream identifier to the target playback server;

[0086] Through the target playback server, the target scheduling server is determined from the scheduling server cluster based on its own regional identifier, and a live query request carrying the first live stream identifier and the playback server identifier of the target playback server is sent to the target scheduling server.

[0087] The playback server cluster includes multiple publishing servers, and the scheduling server cluster includes multiple scheduling servers.

[0088] Specifically, when a playback terminal needs to watch a live broadcast, it can select the playback server it communicates with to improve communication efficiency. A first playback terminal determines a first playback server from a playback server cluster based on its own operator identifier, and prioritizes the playback server in the playback server cluster that has the same operator identifier as the first playback terminal as the first playback server. Furthermore, the first playback terminal determines a target playback server from each first playback server based on its own region identifier, and prioritizes the playback server in each first playback server that has the same region identifier as the first playback terminal as the target playback server. The first playback terminal sends a live broadcast viewing request carrying the first live stream identifier to the target playback server.

[0089] It is understood that if there is no playback server in the playback server cluster with the same operator identifier as the first playback terminal, then a playback server in the playback server cluster can be used as the first playback server, or a playback server in the playback server cluster corresponding to a telecom operator with better historical live broadcast performance can be used as the first playback server. If there is no playback server with the same region identifier as the first playback terminal among the first playback servers, then a first playback server close to the region of the first playback terminal can be used as the target playback server.

[0090] The playback server can also select the dispatch server with which it communicates to improve communication efficiency. The target playback server determines the target dispatch server from the dispatch server cluster based on its own regional identifier, prioritizing dispatch servers with the same regional identifier as the target playback terminal. If no dispatch server with the same regional identifier as the target playback terminal exists in the dispatch server cluster, a dispatch server in a region close to the target playback server can be selected as the target dispatch server. The target playback server sends a live broadcast query request carrying the first live broadcast stream identifier and its own playback server identifier to the target dispatch server.

[0091] In the above embodiment, the playback terminal first selects the playback server based on the operator, which can improve the efficiency of obtaining live data from the corresponding playback server. Then, the playback server is selected based on the region, which can further improve the efficiency of obtaining live data from the corresponding playback server. The playback server selects the scheduling server based on the region, which can improve the efficiency of querying data from the corresponding scheduling server.

[0092] In one embodiment, reference Figure 5 Redis (database), scheduler (scheduling server), pub (publishing server), and play (playing server) can all be expanded to clusters. Redis itself supports cluster deployment, with multiple schedulers, multiple pubs, and multiple play servers based on region, operator, and other factors.

[0093] When the user terminal is connected to the pub or play, the principle of proximity and consistency of operators shall be followed. If these cannot be met, the operator shall take precedence over the region. Figure 5 As shown in the figure, when a live broadcast terminal of operator A in region A selects pub, the priority is as follows: operator A in region A > operator A in region B > operator B in region A > operator B in region B. If a live broadcast terminal of operator A in region A publishes a stream, when a viewer terminal of operator A in region A views it, the media data links are ①, ②, and ③. When a viewer terminal of operator B in region A views it, the media data links are ①, ④, and ⑤. When a viewer terminal of operator A in region B views it, the media data links are ①, ⑥, and ⑦.

[0094] Furthermore, if pub and play are cloud servers, you can follow the principle of choosing cloud servers provided by the same cloud service provider. The cloud service provider will definitely guarantee the corresponding operator's regional issues.

[0095] In one embodiment, determining a communication path between a target playback server and a target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server includes:

[0096] The communication speed between the target playback server and the target publishing server is queried based on the playback server identifier and the publishing server identifier; when the communication speed is greater than or equal to the preset speed, the communication path between the target playback server and the target publishing server is obtained based on the communication address corresponding to the target publishing server; when the communication speed is less than the preset speed, the communication address corresponding to at least one proxy server is obtained, and the communication path between the target playback server and the target publishing server is obtained based on the communication address corresponding to the target publishing server and the communication address corresponding to the proxy server.

[0097] The preset speed is a preset speed threshold, which can be set according to actual needs.

[0098] Specifically, the target scheduling server can determine whether to add a proxy server during the live data transmission process based on the communication quality between the target playback server and the target publishing server. The communication quality between the target playback server and the target publishing server can be reflected by the communication speed between the target playback server and the target publishing server.

[0099] The target scheduling server can query the communication speed between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier, and compare the communication speed with a preset speed. If the communication speed is greater than or equal to the preset speed, indicating good communication between the target playback server and the target publishing server, the target scheduling server can obtain the communication path between the target playback server and the target publishing server based on the communication address corresponding to the target publishing server. The target playback server can directly access the target publishing server according to the communication address corresponding to the target publishing server to obtain live broadcast data. If the communication speed is less than the preset speed, indicating poor communication between the target playback server and the target publishing server, the target scheduling server can obtain the communication address corresponding to at least one proxy server and, based on the communication address corresponding to the target publishing server and the communication address corresponding to the proxy server, obtain the communication path between the target playback server and the target publishing server. The communication addresses corresponding to the proxy servers and the communication addresses corresponding to the target publishing server can be arranged in order, so that the target playback server can first access the proxy server according to the communication address corresponding to the proxy server, and the proxy server can then access the target publishing server according to the communication address corresponding to the target publishing server to obtain live broadcast data.

[0100] It is understandable that the communication speed between the target playback server and the target publishing server can be obtained by calculation. For example, the load of the target publishing server is detected, and the communication speed between the target playback server and the target publishing server is determined based on the load of the target publishing server. The communication speed is negatively correlated with the load, so that when the pressure on the target publishing server is high, the pressure on the target publishing server can be relieved through the proxy server. The communication speed between the target playback server and the target publishing server can also be pre-recorded. For example, if it is known that the target publishing server is located in China and the target playback server is located abroad, it is usually difficult for the target publishing server and the target playback server to communicate directly with each other. Then the communication speed between the target playback server and the target publishing server is obviously small or zero. Therefore, when the target playback server and the target publishing server cannot communicate with each other, the target playback server and the target publishing server can be connected through a proxy server.

[0101] In the above embodiment, when the communication speed is greater than or equal to the preset speed, the communication path between the target playback server and the target publishing server is obtained based on the communication address corresponding to the target publishing server, so that the target player can quickly obtain live broadcast data from the target publishing server, thereby ensuring the live broadcast speed. When the communication speed is less than the preset speed, the communication path between the target playback server and the target publishing server is obtained based on the communication address corresponding to the target publishing server and the communication address corresponding to the proxy server, so that the target player can obtain live broadcast data from the target publishing server via the proxy server, thereby ensuring the smooth progress of the live broadcast.

[0102] In one embodiment, there are cases where pub (publishing server) and play (playing server) are connected, and there are also cases where they are not connected, such as the campus network and the external network cannot communicate with each other, and the domestic network and the foreign network cannot communicate with each other. In addition, if a live broadcast end publishes the stream to a publishing server, all the audience ends of the live broadcast pull the stream from this publishing server. When the number of viewers is very large, the pressure on this publishing server will be very large, which may easily lead to the problem of uneven distribution of server resources. For example, a large conference with thousands of people will drag down some machines, but at the same time some machines are idle. In order to deal with the above situation, a proxy (proxy server) is added. The proxy can not only relieve the pressure on the pub, but also connect the pub and play. Reference Figure 6 The live broadcast terminal (i.e., the host) and the playback terminal (i.e., the audience) connect to the corresponding pub and play, respectively. If the pub and play are disconnected, the proxy can connect to both the pub and play. The pub will first send data to the proxy, which then sends it to the play.

[0103] It is understandable that in actual situations, there may be more than one proxy, and there may be two hops or even more. The relevant routing table needs to be written into redis (database) through the backend configuration. When the scheduling server is scheduled, the corresponding connection can be established hop by hop according to the routing table.

[0104] In one embodiment, the live broadcast data processing method further includes:

[0105] Through at least one server passed through the communication path, the first live data stream corresponding to the first live stream identifier sent by its own forward live broadcast device is obtained, jitter and packet loss processing are performed on the media data packets in the first live data stream, and the second live data stream corresponding to the first live stream identifier is obtained, and the second live data stream is sent to its own backward live broadcast device.

[0106] The servers that the communication path passes through include at least a publishing server and a playback server. Furthermore, the servers that the communication path passes through may also include a proxy server.

[0107] When transmitting live data, a server's forward live device refers to the preceding device communicating with the server, and a server's backward live device refers to the subsequent device communicating with the server. For example, if the communication path does not pass through a proxy server, for a publishing server, the publishing server's forward live device is the live terminal or another publishing server, and the backward live device is another publishing server or playback server. For a playback server, the playback server's forward live device is the publishing server or another playback server, and the backward live device is the playback terminal or another playback server. It is understood that in actual situations, there may be more than one publishing server, possibly two or more hops, and more than one playback server, possibly two or more hops.

[0108] The first live data stream corresponding to the first live stream identifier is the live data received by the server, and the second live data stream corresponding to the first live stream identifier is the live data sent by the server. The server processes the received live data to obtain new live data, and sends the new live data to the next live device.

[0109] The server handles jitter and packet loss for live broadcast data. Jitter and packet loss handling is data processing for network jitter. Jitter handling is used to ensure that data packets that are delayed due to network jitter are transmitted in the correct order. For example, received data packets are stored in a buffer, sorted according to timestamps, and then sent out in order. The purpose of buffering is to wait for data packets that are delayed due to network jitter to ensure that they can be transmitted or played in the correct order and time. For example, if data packets are too late due to excessive network delay or jitter, exceeding the set maximum delay threshold, these packets can be discarded to avoid freezes or delays in the live broadcast. Packet loss handling is used to compensate for data packets lost due to network jitter to reduce the impact of packet loss on the live broadcast. For example, interpolation or prediction techniques can be used to generate replacement data packets for lost data packets.

[0110] It can be understood that the media data packets in the live data stream include at least one of video data packets and audio data packets.

[0111] Specifically, the target playback server obtains the live data corresponding to the first live stream identifier from the target publishing server based on the queried communication path. During the transmission of the live data between servers, at least one server can perform jitter and packet loss processing on the received live data to improve the data quality transmitted to the next device.

[0112] The server receives the first live data stream corresponding to the first live stream identifier sent by its own forward live broadcast device, performs jitter and packet loss processing on the media data packets in the first live data stream, and obtains the second live data stream corresponding to the first live stream identifier. The live broadcast quality of the second live data stream corresponding to the first live stream identifier is higher than the live broadcast quality of the first live data stream corresponding to the first live stream identifier. The server sends the second live data stream to its own backward live broadcast device.

[0113] For example, the target publishing server receives the live data stream corresponding to the first live stream identifier sent by the live broadcast terminal, performs jitter and packet loss processing on the media data packets in the live data stream, and sends the processed live data stream to the target playback server. The target playback server receives the live data stream corresponding to the first live stream identifier sent by the live broadcast terminal, performs jitter and packet loss processing on the media data packets in the live data stream, and sends the processed live data stream to the first playback terminal.

[0114] It is understandable that after receiving the live broadcast data, the playback terminal may also perform jitter and packet loss processing on the live broadcast data before decoding and playing it.

[0115] In the above embodiment, at least one server through which the communication path passes no longer simply forwards live broadcast data, but performs jitter and packet loss processing on the received live broadcast data, and sends the processed live broadcast data to the next device. The processed live broadcast data helps to improve the quality of live broadcast and reduce problems such as freeze, delay, and disorder in live broadcast.

[0116] In one embodiment, jitter and packet loss processing is performed on media data packets in the first live data stream to obtain a second live data stream corresponding to the first live stream identifier, including:

[0117] The first live data stream is split to obtain multiple first media data packets; packet loss compensation is performed on the multiple first media data packets to obtain multiple second media data packets; the cache capacity of the jitter buffer is updated based on the transmission delay of the first media data packets, the multiple second media data packets are stored in the jitter buffer, and the second media data packets in the jitter buffer are added to the live data queue in an orderly manner; the data packets in the live data queue are encapsulated in an orderly manner to obtain a second live data stream corresponding to the first live stream identifier.

[0118] The first live data stream is split into multiple media data packets to obtain multiple first media data packets. Packet loss compensation is performed on the multiple first media data packets to obtain multiple second media data packets. Packet loss compensation refers to generating new media data packets to replace the lost or delayed media data packets when packet loss or delay is detected. Examples include inserting a filler packet; reconstructing a data packet using packets before and after the loss; creating a data packet similar to the lost packet through pattern matching and interpolation techniques, etc.

[0119] Data packets are stored in the jitter buffer and wait until their expected packetization time arrives. The packets are then added to the live data queue. The packets in the live data queue are sequentially encapsulated into a live data stream and sent out smoothly.

[0120] It can be understood that the jitter buffer can be further divided into an audio buffer and a video buffer. The audio buffer is used to cache audio data packets, and the video buffer is used to cache video data packets.

[0121] Specifically, the server can split the received first live data stream to obtain multiple first media data packets, perform packet loss compensation on the multiple first media data packets, and obtain multiple second media data packets. If packet loss or delay occurs, the multiple second media data packets include multiple first media data packets and new media data packets. The server updates the cache capacity of the jitter buffer based on the transmission delay of the first media data packets, enabling dynamic adjustment of the jitter buffer size based on network conditions detected through the data packets. The first media data packets carry a send time, and the transmission delay of the first media data packets is determined based on the time interval between the send and receive times of the first media data packets. A network jitter value is calculated based on the transmission delay of the first media data packets, and the jitter buffer size is adjusted based on the network jitter value. If a data packet is too late due to network problems, it may not be added to the jitter buffer. Appropriately discarding such data packets can avoid playback lag or delay. The server stores the multiple second media data packets in the jitter buffer. The server can remove data packets from the jitter buffer based on the data packet timestamp and the current system time and add them to the live data queue. The server encapsulates the data packets in the live data queue in order according to the timestamps to obtain the second live data stream corresponding to the first live stream identifier.

[0122] To further improve playback smoothness, the server can control the speed at which live data is sent. For example, when network delays and jitter are high, playback can be accelerated to catch up; when delays and jitter are low, playback can be slowed down to avoid premature data packet consumption.

[0123] In one embodiment, the first playback terminal performs jitter and packet loss processing on the received live broadcast data, and decodes and plays the processed live broadcast data.

[0124] In the above-mentioned embodiments, packet loss compensation can compensate for losses, reduce live broadcast interruptions, and ensure live broadcast quality. A jitter buffer can be used to wait for data packets that arrive late due to network jitter, ensuring that they are transmitted or played in the correct order and time. Sequentially encapsulating and sending data packets in the live broadcast data queue can also ensure live broadcast quality.

[0125] In one embodiment, the live broadcast data processing method further includes:

[0126] The system obtains feedback information of the backward live broadcast device for the second live broadcast data stream through at least one server passed through the communication path, and sends a supplementary live broadcast data stream to the backward live broadcast device based on the feedback information.

[0127] The feedback information is used to provide feedback on the reception of live data. For example, the feedback information includes packet loss information, latency information, etc. The supplementary live data stream is a supplementary live data stream that is provided in response to the feedback information.

[0128] Specifically, the server no longer simply forwards live data; instead, it processes the received live data and sends it to the next device. Because the server has data processing capabilities, it can promptly respond to feedback from the next device. The server can receive feedback from the back-to-back live device regarding the second live data stream and, based on this feedback, send a supplemental live data stream to the back-to-back live device. For example, the server can use this feedback to send back live data lost due to network issues; it can also control the speed of sending subsequent live data to the back-to-back live device based on this feedback; and so on.

[0129] In the above embodiment, the server can not only process jitter and packet loss of the received live broadcast data, but also respond to feedback information returned to the live broadcast device to improve the live broadcast quality.

[0130] In one embodiment, sending a supplementary live data stream backward to the live broadcast device based on the feedback information includes:

[0131] Based on the packet loss information in the feedback information, a data packet to be retransmitted is obtained; based on the data packet to be retransmitted, a supplementary live data stream is generated, and the supplementary live data stream is sent backward to the live broadcast device.

[0132] The feedback information includes packet loss information. Packet loss information indicates which data packets were lost. Data packets to be retransmitted are data packets that need to be retransmitted due to loss.

[0133] Specifically, the server can obtain packet loss information from the feedback information and, based on the packet loss information, obtain data packets to be retransmitted. For example, the server can obtain data packets to be retransmitted from the forward live broadcast device, and so on. The server can encapsulate the data packets to be retransmitted into a supplementary live broadcast data stream and send the supplementary live broadcast data stream to the backward live broadcast device, thereby compensating the backward live broadcast device for its lost data packets.

[0134] In the above embodiment, by using the packet loss information in the feedback information and retransmitting the lost data packets to the live broadcast device, it is possible to reduce the live broadcast freeze and improve the live broadcast quality.

[0135] In one embodiment, sending a supplementary live data stream backward to the live broadcast device based on the feedback information includes:

[0136] Based on the feedback information, the historical sending speed for the live data stream is updated to obtain the target sending speed for the live data stream; the backward live data stream of the second live data stream is obtained as a supplementary live data stream; based on the target sending speed, the supplementary live data stream is sent backward to the live device.

[0137] The historical sending speed refers to the sending speed previously used to send live data. The historical sending speed is updated to obtain the target sending speed. The target sending speed refers to the new sending speed used to send live data.

[0138] The backward live data stream of the second live data stream refers to the live data stream that needs to be transmitted after the second live data stream. It can be understood that during the live broadcast process, the live broadcast terminal continuously generates live data and needs to continuously send new live data to the playback terminal.

[0139] Specifically, the server can adjust the speed of subsequent live data transmission based on the feedback information. The server can obtain the historical transmission speed for the live data stream and update the historical transmission speed based on the feedback information to obtain a target transmission speed for the live data stream. For example, based on at least one of packet loss information and latency information in the feedback information, the server can adjust the historical transmission speed using a congestion control algorithm to obtain a target transmission speed. The server can transmit a backward live data stream of the second live data stream to the live device at the target transmission speed. In other words, the backward live data stream of the second live data stream serves as a supplementary live data stream and is transmitted to the live device based on the target transmission speed.

[0140] In the above embodiment, feedback information can reflect the network status. By using this feedback information to update the subsequent live broadcast data transmission speed, the data transmission rate and flow rate can be flexibly determined based on network conditions, thus avoiding or reducing problems such as packet loss and delay during data transmission, thereby improving live broadcast quality.

[0141] In one embodiment, obtaining feedback information from a backward live broadcast device regarding a second live broadcast data stream, and sending a supplementary live broadcast data stream to the backward live broadcast device based on the feedback information, includes:

[0142] Obtain feedback information from multiple backward live broadcast devices of the device for the second live broadcast data stream respectively; count the lost data packets corresponding to the packet loss information in each feedback information to obtain feedback statistical values ​​corresponding to each lost data packet; determine the data packets to be retransmitted from each lost data packet based on the feedback statistical values; generate a supplementary live broadcast data stream based on the data packets to be retransmitted, and send the supplementary live broadcast data stream to the backward live broadcast device corresponding to the data packet to be retransmitted.

[0143] The feedback information includes packet loss information. The lost data packets corresponding to the packet loss information are the data packets that were lost as indicated by the packet loss information. Statistics are collected on the loss of the lost data packets reported back by multiple backward live broadcast devices to obtain feedback statistics corresponding to the lost data packets. The feedback statistics corresponding to the lost data packets indicate the number of lost data packets, i.e., how many backward live broadcast devices lost the same data packet.

[0144] A lost data packet is selected from each lost data packet as a data packet to be retransmitted based on the feedback statistics. For example, the data packet to be retransmitted may be a lost data packet whose feedback statistics are greater than a preset statistics value; the data packet to be retransmitted may be a lost data packet with the largest feedback statistics value; and so on.

[0145] The backward live broadcast device corresponding to the data packet to be retransmitted refers to the backward live broadcast device that reports that the data packet to be retransmitted has been lost.

[0146] Specifically, a server may send the same live data to multiple downstream live streaming devices. For example, a publishing server may send live data for a single broadcast to multiple playback servers, which then transmit the live data to different viewers' playback terminals. As a result, the server can receive feedback from multiple downstream live streaming devices regarding the same live data.

[0147] The server obtains feedback information from multiple backward live broadcast devices for the second live broadcast data stream, and determines which backward live broadcast device should be given priority to feedback which live broadcast data based on each piece of feedback information. The server counts the lost data packets corresponding to the packet loss information in each piece of feedback information, and obtains feedback statistics corresponding to each lost data packet. The feedback statistics of the lost data packets can reflect whether the loss of the lost data packets is universal. Furthermore, based on the feedback statistics, the server determines the data packets to be retransmitted from each lost data packet, takes the commonly lost data packets as the data packets to be retransmitted, obtains the data packets to be retransmitted, generates a supplementary live broadcast data stream based on the data packets to be retransmitted, and sends the supplementary live broadcast data stream to the backward live broadcast device that has lost the data packets to be retransmitted.

[0148] In the above embodiment, statistics are collected on the packet loss of feedback information of multiple backward live broadcast devices for the same live broadcast stream, and feedback statistical values ​​corresponding to each lost data packet are obtained. Based on the feedback statistical values, data packets to be retransmitted are determined from each lost data packet, and data packets are retransmitted preferentially from each backward live broadcast device that has generally lost the same data packet. This can quickly meet the feedback of most backward live broadcast devices and ensure the live broadcast quality of most devices.

[0149] In one embodiment, sending a supplementary live data stream backward to the live broadcast device based on the feedback information includes:

[0150] The network status corresponding to the live broadcast device is updated based on the feedback information; when the network status is normal, a supplementary live broadcast data stream is sent back to the live broadcast device based on the feedback information.

[0151] Specifically, the server can update the network status corresponding to the backward live broadcast device based on the feedback information. The network status is divided into normal status and abnormal status. For example, the network jitter value is calculated based on the feedback information. When the network jitter value is greater than the preset threshold, the network status is determined to be abnormal, otherwise, the network status is normal; when the delay information exceeds the preset threshold for several consecutive times, the network status is determined to be abnormal, otherwise, the network status is normal; when the number of packet losses is greater than the preset threshold, the network status is determined to be abnormal, otherwise, the network status is normal; and so on. When the network status corresponding to the backward live broadcast device is normal, the server responds to the feedback information of the backward live broadcast device, and sends a supplementary live broadcast data stream to the backward live broadcast device based on the feedback information to ensure the effectiveness of the transmission of the supplementary live broadcast data stream. When the network status corresponding to the backward live broadcast device is abnormal, the server may not respond to the feedback information of the backward live broadcast device first to avoid wasting transmission resources.

[0152] In the above embodiment, the network status corresponding to the backward live broadcast device is updated based on the feedback information. When the network status is normal, a supplementary live broadcast data stream is sent to the backward live broadcast device based on the feedback information, which can avoid sending live broadcast data to the backward live broadcast device with network abnormalities and avoid wasting transmission resources.

[0153] In one embodiment, in conventional technology, the host encapsulates the encoded audio and video data into RTP packets and forwards them via UDP / TCP, ultimately reaching the viewer. The viewer decompresses the packets and encapsulates the feedback information into RTCP packets, which are then transmitted to the host via UDP / TCP. The host then responds accordingly based on the feedback information. However, if there are many viewers, the host would need to respond to all of them, which would put a lot of pressure on the host.

[0154] In this application method, the publishing server, playback server, and proxy server have the ability to process feedback information, not just forward data. For a particular stream, the playback server can handle viewer feedback, the publishing server handles feedback from the playback server or proxy server, the proxy server handles feedback from the playback server, and the host handles feedback from the publishing server. And so on, converging feedback to the nearest location and preventing it from spreading elsewhere, thereby improving feedback response speed.

[0155] Furthermore, for the same stream, if a playback server has 1,000 viewers providing feedback, these feedback can be managed according to a specific strategy. For example, if a single viewer experiences significant network issues, the response to the viewer with the most significant network issues cannot be delayed. Alternatively, if a large number of viewers report missing the same live data, the data can be prioritized for retransmission to these viewers.

[0156] In one embodiment, reference Figure 7 The server receives data sent from the live streaming device and depackets it, resulting in RTCP packets (i.e., RTCP packets) and RTP packets (i.e., RTP packets). The server performs forward error correction on the RTP packets, then stores the corresponding audio packets in the audio buffer and the corresponding video packets in the video buffer. Forward error correction can address audio and video packet loss. The server buffers received packets and then evenly extracts them from the other end of the buffer. This ensures relatively smooth data flow, addressing audio and video out-of-order and jitter issues. The audio packets in the audio buffer are added to the audio data queue, and the video packets in the video buffer are added to the video data queue. The packets in the audio and video data queues are then packaged and sent through smoothing. Smoothing controls the transmission rate and ensures network connectivity. Forward error correction can also be performed during packetization.

[0157] The server can perform flow control based on RTCP packets. Flow control reduces the sending bitrate when the output bitrate exceeds the bandwidth to prevent congestion. The sending bitrate can be determined by evaluating the bandwidth. RTCP packets contain packet loss and latency information. Bandwidth estimation can be performed based on packet loss. For example, if the packet loss rate is low, indicating normal packet loss with good network quality, it indicates that the bandwidth limit has not been reached and the estimated bandwidth value should be increased. Bandwidth estimation can also be performed based on latency. For example, if the network transmission latency of received packets continues to increase, it indicates that the network is deteriorating. When it reaches a certain level, the estimated bandwidth value should be reduced to prevent network congestion. Of course, lost packets should also be retransmitted as much as possible. It can be understood that latency-based bandwidth estimation methods can be divided into receiver-side bandwidth estimation methods and transmitter-side bandwidth estimation methods.

[0158] refer to Figure 8 Live broadcast terminals and playback terminals also have data processing capabilities similar to servers. In addition, live broadcast terminals can also collect and encode audio and video data. Playback terminals can decode, render, and play audio and video data.

[0159] In the present application method, the data processing methods between the server and the terminal are very similar, which ensures consistency in network processing and helps to improve data processing efficiency.

[0160] In one embodiment, the first playback terminal is a playback terminal in an interactive live broadcast scene.

[0161] The live data processing method further includes:

[0162] The content distribution server acquires a live viewing request carrying a second live stream identifier sent by a second playing terminal in a non-interactive live scenario, acquires a comprehensive live data stream corresponding to the second live stream identifier, and sends the comprehensive live data stream corresponding to the second live stream identifier to the second playing terminal.

[0163] The comprehensive live data stream is a live data stream generated by at least one live terminal corresponding to the second live stream identifier, which is acquired by the merging server, synthesized, and sent to the content distribution server.

[0164] The playing terminal in the interactive live scenario is a playing terminal that can perform interactive operations during live viewing. The playing terminal in the non-interactive live scenario is a playing terminal that cannot perform interactive operations or does not need to perform interactive operations during live viewing. The interactive operation is an interactive operation on a live picture, and specifically can be switching or adjusting the live picture during live viewing. When a live broadcast has multiple live terminals, each live terminal can generate its own stream. If a user can select which streams to receive during live viewing, the playing terminal of the user is a playing terminal in the interactive live scenario. If the user cannot select which streams to receive during live viewing, the playing terminal of the user is a playing terminal in the non-interactive live scenario. When a live broadcast has a single live terminal, if a user watches the live broadcast, the playing terminal of the user is a playing terminal in the non-interactive live scenario. For example, during a live broadcast with multiple anchors, if a user can select which anchor's picture to watch, the playing terminal of the user is a playing terminal in the interactive live scenario. If the user cannot select which anchor's picture to watch, the playing terminal of the user is a playing terminal in the non-interactive live scenario.

[0165] In one embodiment, the data processing capability of the web page is limited, the playing terminal that watches the live broadcast on the web page is a playing terminal in the non-interactive live scenario, and the playing terminal that watches the live broadcast on the client can be a playing terminal in the interactive live scenario.

[0166] The content distribution server is a server based on a content distribution network (CDN, Content Delivery Network). The merging server is used to merge live data generated by live devices in a live broadcast.

[0167] Specifically, for the playing terminal in the interactive live scenario, the live broadcast can be watched through the publishing server, the scheduling server, and the playing server. For the playing terminal in the non-interactive live scenario, the live broadcast can be watched through the merging server and the content distribution network.

[0168] For playback terminals in non-interactive live broadcast scenarios, when the playback terminal needs to watch the live broadcast, it sends a live viewing request to the content distribution server, and the live viewing request carries the second live stream identifier. The content distribution server can query the comprehensive live data stream corresponding to the second live stream identifier, and send the comprehensive live data stream corresponding to the second live stream identifier to the second playback terminal. The comprehensive live data stream corresponding to the second live stream identifier is sent by the merging server to the content distribution server. The merging server can obtain the live data stream generated by each live terminal corresponding to the second live stream identifier, synthesize each live data stream, and obtain a comprehensive data stream. For example, everyone in the meeting sends an audio and video stream to the corresponding publishing server, and the publishing server sends each stream to the merging server. After decoding all the streams, the merging server combines all the pictures into one picture, mixes all the sounds into one sound, and re-encodes the synthesized audio and video data into a new stream, and sends it to the content distribution server.

[0169] Of course, a playback terminal in a non-interactive live broadcast scene can also watch the live broadcast through the publishing server, the scheduling server, and the playback server. The first live stream identifier and the second live stream identifier can be the same live stream identifier or different live stream identifiers.

[0170] In one embodiment, each live broadcast terminal corresponding to the second live broadcast stream identifier can send its own live broadcast data to the corresponding publishing server. The publishing server sends the live broadcast data corresponding to the second live broadcast stream identifier to the merging server. The merging server merges the live broadcast data corresponding to the second live broadcast stream identifier to obtain a comprehensive live broadcast data stream, and sends the comprehensive live broadcast data stream to the content distribution server. For playback terminals in interactive live broadcast scenarios, live broadcast viewing can be achieved through the publishing server, scheduling server, and playback server. For playback terminals in non-interactive live broadcast scenarios, live broadcast viewing can be achieved through the publishing server, merging server, and content distribution network. The live broadcast processing architecture of the two interactive live broadcast scenarios is connected through the publishing server.

[0171] In the above embodiment, for playback terminals in interactive live broadcast scenarios, live broadcasts can be viewed through the publishing server, scheduling server, and playback server. The playback terminal can select which streams of a live broadcast to receive and decode and synthesize them locally, eliminating the need for individual servers to decode, synthesize, and re-encode all streams, reducing server overhead. For playback terminals in non-interactive live broadcast scenarios, live broadcasts can be viewed through the combined server and content distribution network, reducing terminal overhead.

[0172] In one embodiment, Figure 9 As shown, a live broadcast data processing system is provided, which includes a target playback server 902 and a target scheduling server 904.

[0173] The target playback server 902 is configured to obtain a live viewing request carrying a first live stream identifier sent by the first playback terminal, and send a live query request carrying the first live stream identifier and the playback server identifier of the target playback server to the target scheduling server.

[0174] The target scheduling server 904 is used to determine the target publishing server based on the first live stream identifier, determine the communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server, and return the communication path to the target playback server; when the communication quality between the target playback server and the target publishing server meets the preset conditions, the communication path passes through at least one proxy server, and the proxy server is used to connect the target playback server and the target publishing server.

[0175] The target playback server 902 is further configured to obtain the live data stream corresponding to the first live stream identifier from the target publishing server based on the communication path, and send the live data stream corresponding to the first live stream identifier to the first playback terminal.

[0176] Specifically, when the first playback terminal needs to watch a live broadcast, the first playback terminal obtains a first live stream identifier corresponding to the live broadcast to be watched, generates a live viewing request carrying the first live stream identifier, and sends the live viewing request to a target playback server.

[0177] The target playback server receives a live broadcast viewing request sent by the first playback terminal, the live broadcast viewing request carrying the first live broadcast stream identifier, and generates a live broadcast query request in response to the live broadcast viewing request, the live broadcast query request carrying the first live broadcast stream identifier and the playback server identifier of the target playback server. The target playback server sends the live broadcast query request to the target scheduling server.

[0178] The target scheduling server responds to the live broadcast query request, determines a target publishing server based on the first live broadcast stream identifier, and searches for the publishing server containing the live broadcast data corresponding to the first live broadcast stream identifier as the target publishing server. The target scheduling server determines a communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server. The target scheduling server can determine the communication quality between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier, and determine whether the communication quality between the target playback server and the target publishing server meets a preset condition. If the communication quality between the target playback server and the target publishing server meets the preset condition, the target scheduling server determines a required proxy server and determines a communication path between the target playback server and the target publishing server based on the playback server identifier, the publishing server identifier, and the proxy server identifier of the proxy server, with the communication path traversing the proxy server. If the communication quality between the target playback server and the target publishing server does not meet the preset condition, the target scheduling server directly determines a communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier. The target scheduling server returns the communication path to the target playback server.

[0179] The target playback server obtains the live data stream corresponding to the first live stream identifier from the target publishing server based on the communication path fed back by the target scheduling server, and sends the live data stream corresponding to the first live stream identifier to the first playback terminal. The first playback terminal can decode and play the live data stream corresponding to the first live stream identifier.

[0180] It is understood that the live data corresponding to the first live stream identifier is generated by the live terminal. The data processing content of the first playback terminal, the target playback server, the target scheduling server, the target publishing server, and the live terminal can refer to the embodiment of the live data processing method described above and will not be repeated here.

[0181] In the live broadcast data processing system described above, the target playback server responds to a live broadcast viewing request from a first playback terminal by querying the target publishing server where the live broadcast data is located, as well as the communication path between the target publishing server and the target playback server. The target playback server then obtains the live broadcast data from the target publishing server based on the queried communication path and sends the live broadcast data to the first playback terminal, thereby enabling the first terminal to view the live broadcast. Live broadcast viewing is achieved through orderly interaction between the first playback terminal, the target playback server, the target scheduling server, and the target publishing server, which can reduce the pressure on the live broadcast terminal and ensure the quality of the live broadcast. Furthermore, the target scheduling server determines the target publishing server where the live broadcast data is located based on the playback server identifier, and then determines the communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier. When the communication quality between the target playback server and the target publishing server meets preset conditions, the communication path passes through at least one proxy server. The proxy server can ensure stable transmission of live broadcast data between the target playback server and the target publishing server, further improving the quality of the live broadcast.

[0182] In one embodiment, Figure 10 As shown, the live data processing system further includes a target publishing server 1002.

[0183] The target publishing server 1002 is used to obtain a live broadcast initiation request carrying live broadcast introduction information sent by the live broadcast device, and send a live broadcast registration request carrying the live broadcast introduction information to the target scheduling server; the live broadcast introduction information includes a first live broadcast stream identifier.

[0184] The target scheduling server 904 is further used to associate the live broadcast introduction information and the publishing server identifier with the storage server and return the registration result to the target publishing server.

[0185] The target publishing server 1002 is further configured to return a registration result to the live broadcast device, and obtain a live broadcast data stream corresponding to the first live broadcast stream identifier sent by the live broadcast device after successful registration.

[0186] The live broadcast introduction information is used to introduce relevant information about a live broadcast. For example, this information includes the live stream identifier, live broadcast type, and the encoding type of the live broadcast data. The storage server is used to store the live broadcast introduction information corresponding to each live broadcast. Furthermore, the storage server can also store data such as server information and routing tables for various servers. The storage server can implement data storage based on Redis. The scheduling server connects to the publishing server, storage server, and playback server to complete the corresponding scheduling.

[0187] Specifically, when a live broadcast terminal initiates a live broadcast, it can send a live broadcast initiation request to the target publishing server. The live broadcast initiation request carries the live broadcast introduction information of the live broadcast to be initiated. The target publishing server can be any publishing server or a specific publishing server. For example, a publishing server that matches the live broadcast terminal is obtained from the publishing server cluster as the target publishing server. The publishing server that matches the live broadcast terminal can be a publishing server that is geographically close to the live broadcast terminal, a publishing server that is close to the operator of the live broadcast terminal, etc. The target publishing server responds to the live broadcast initiation request and sends a live broadcast registration request to the target scheduling server. The live broadcast registration request carries the live broadcast introduction information, and the live broadcast introduction information includes the first live stream identifier.

[0188] The target scheduling server registers the publishing server identifier and live broadcast introduction information of the target publishing server in association with the storage server, that is, the publishing server identifier and live broadcast introduction information of the target publishing server are stored in association with the storage server, so that when needed later, the publishing server where the live broadcast data corresponding to the first live broadcast stream identifier is located can be queried from the storage server.

[0189] The target scheduling server returns a registration result to the target publishing server, which then returns the registration result to the live streaming device. If the registration result is successful, the live streaming device collects audio data, video data, or audio and video data to generate a live data stream corresponding to the first live stream identifier and sends the live data stream corresponding to the first live stream identifier to the target publishing server. Subsequently, when needed, the playback terminal can retrieve the live data stream corresponding to the first live stream identifier from the target publishing server.

[0190] In the above embodiment, when the live broadcast terminal initiates a live broadcast, the publishing server informs the scheduling server to associate the live broadcast introduction information and its own server identifier and register it with the storage server, so that when the playback terminal needs to watch the live broadcast later, the scheduling server can query the publishing server where the live broadcast data is located from the storage server, so that the playback terminal can obtain the live broadcast data.

[0191] In one embodiment, the first playback terminal is a playback terminal in an interactive live broadcast scene. Figure 11 As shown, the live data processing system further includes a merging server 1102 and a content distribution server 1104.

[0192] The merging server 1102 is used to obtain the live data stream generated by at least one live terminal corresponding to the second live stream identifier, synthesize the various live data streams to obtain a comprehensive live data stream, and send the comprehensive live data stream to the content distribution server.

[0193] The content distribution server 1104 is used to obtain a live viewing request carrying a second live stream identifier sent by a second playback terminal in a non-interactive live broadcast scene, obtain the comprehensive live data stream corresponding to the second live stream identifier, and send the comprehensive live data stream corresponding to the second live stream identifier to the second playback terminal.

[0194] Specifically, for playback terminals in interactive live broadcast scenarios, they can watch live broadcasts through publishing servers, scheduling servers, and playback servers. For playback terminals in non-interactive live broadcast scenarios, they can watch live broadcasts through merging servers and content distribution networks.

[0195] When initiating a live broadcast, at least one live broadcast terminal corresponding to the second live broadcast stream identifier sends its respective live broadcast data stream directly to the merging server or through the publishing server. The merging server merges the live broadcast data streams of each live broadcast terminal to obtain a comprehensive live broadcast data stream, which the merging server sends to the content distribution server. The content distribution server sends the comprehensive live broadcast data stream to the playback terminal when the playback terminal needs it.

[0196] When a second playback terminal in a non-interactive live broadcast scenario needs to watch the live broadcast corresponding to the second live stream identifier, it sends a live broadcast viewing request carrying the second live stream identifier to the content distribution server. The content distribution server queries the comprehensive live data stream corresponding to the second live stream identifier and sends the comprehensive live data stream corresponding to the second live stream identifier to the second playback terminal. The second playback terminal can decode and play the comprehensive live data stream.

[0197] In the above embodiment, for playback terminals in interactive live broadcast scenarios, they can watch live broadcasts through the publishing server, scheduling server, and playback server. The playback terminal can select which streams of a live broadcast to receive and decode and synthesize them locally, eliminating the need for individual servers to decode, synthesize, and re-encode all streams, reducing server overhead. For playback terminals in non-interactive live broadcast scenarios, they can watch live broadcasts through the publishing server, merging server, and content distribution network, reducing terminal overhead.

[0198] In one embodiment, the live broadcast data processing system further includes an image analysis server configured to obtain a live broadcast data stream generated by a live broadcast terminal, perform image analysis on the live broadcast data stream, and obtain an image analysis result.

[0199] Specifically, the image analysis server can pull streams from the live broadcast terminal or the publishing server. The image analysis server is used to perform image analysis, such as violation detection. If the image analysis result indicates an image anomaly, the live broadcast terminal can be notified so that the user can make improvements.

[0200] In a specific embodiment, reference Figure 12 ,This application method provides two live broadcast architectures.

[0201] 1. A live broadcast terminal, a publishing server, a scheduling server, a database, a proxy server, a playback server, and a playback terminal constitute a live broadcast architecture.

[0202] When the live broadcast terminal sends a stream, it connects to the publishing server and informs the publishing server of the stream information of the stream it sends. The publishing server informs the scheduling server of the stream information to register in the database. If the registration is successful, the live broadcast terminal will continuously push live broadcast data to the publishing server. When the playback terminal watches the live broadcast, it will use the stream ID to pull the stream. The playback server will query the scheduling server based on the stream ID to find out which publishing server to pull the stream from. The scheduling server selects the most appropriate route based on the corresponding policy configuration and tells the playback server to pull the stream from the specified publishing server. The proxy server can connect the publishing server and the playback server when the publishing server is under high pressure or the playback server and the publishing server are not interoperable.

[0203] TCP is used between the dispatch server and the publishing and playback servers. UDP is used between the terminals and the publishing and playback servers. TCP is used between the publishing and playback servers. All communication protocols are proprietary, encrypted, and highly secure.

[0204] Publishing servers, playback servers, and proxy servers possess data processing capabilities. They can handle packet loss and jitter before forwarding received live data, and can respond to feedback from live devices. For example, for a particular stream, the playback server processes feedback from the playback terminal, the publishing server processes feedback from the playback server, and the live terminal processes feedback from the publishing server, and so on. This ensures that feedback is localized to the nearest location and prevents it from spreading elsewhere.

[0205] This architecture conserves uplink bandwidth on live streaming terminals, reducing client CPU and GPU utilization. Live streaming terminals only need to send live data to a single publishing server, eliminating the need to send data to all playback terminals. This architecture offers flexible terminal layout, allowing viewers to decide which streams they want to receive, where they are displayed, and at what quality. Testing has shown that latency can be maintained at 200ms on public networks, making it virtually imperceptible to users. This architecture utilizes a proprietary encryption protocol for transmission, ensuring the security of transmitted media data. This architecture is resistant to packet loss and weak network conditions. Testing has shown that audio and video communication remains unimpeded even in scenarios with high packet loss rates and high network jitter. This architecture offers high server concurrency and minimizes server pressure. A single server can handle high concurrency, and due to the cascade model (introducing proxy servers), the concurrency of the entire server cluster is also high.

[0206] 2. A live broadcast architecture consists of a live broadcast terminal, a publishing server, a merging server, a content distribution network (CDN), and a playback terminal.

[0207] The live broadcast terminal sends the live broadcast data to the publishing server, which sends the live broadcast data to the merging server. The merging server merges the live broadcast data generated by each live broadcast terminal for the same live broadcast to generate comprehensive live broadcast data and send the comprehensive live broadcast data to the content distribution network. The content distribution network can then send the comprehensive live broadcast data to the playback terminal when the playback terminal needs to watch the live broadcast.

[0208] The first live broadcast architecture is applicable to both interactive and non-interactive broadcast terminals. The second live broadcast architecture is applicable to non-interactive broadcast terminals.

[0209] It can be understood that the present application method can be applied to various types of live broadcasts, such as video conferencing, game live broadcasts, teaching live broadcasts, live broadcasts with goods, etc.

[0210] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0211] Based on the same inventive concept, the present application also provides a live data processing device for implementing the live data processing method mentioned above. The implementation solution provided by this device is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more live data processing device embodiments provided below can be found in the above-mentioned limitations of the live data processing method and will not be repeated here.

[0212] In one embodiment, Figure 13 As shown, a live broadcast data processing device is provided, including: a playback server processing module 1302 and a scheduling server processing module 1304, wherein:

[0213] The playback server processing module 1302 is used to obtain the live viewing request carrying the first live stream identifier sent by the first playback terminal through the target playback server, and send a live query request carrying the first live stream identifier and the playback server identifier of the target playback server to the target scheduling server.

[0214] The scheduling server processing module 1304 is used to determine the target publishing server based on the first live stream identifier through the target scheduling server, determine the communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server, and return the communication path to the target playback server; when the communication quality between the target playback server and the target publishing server meets the preset conditions, the communication path passes through at least one proxy server, and the proxy server is used to connect the target playback server and the target publishing server.

[0215] The playback server processing module 1302 is also used to obtain the live data stream corresponding to the first live stream identifier from the target publishing server through the target playback server based on the communication path, and send the live data stream corresponding to the first live stream identifier to the first playback terminal.

[0216] In one embodiment, the live data processing apparatus is further configured to:

[0217] Sending the live data stream corresponding to the first live stream identifier to the target publishing server through the live broadcast device;

[0218] Among them, the first communication protocol is adopted between the live broadcast terminal and the target publishing server, and between the first playback terminal and the target playback server, and the second communication protocol is adopted between the target publishing server and the target scheduling server, between the target playback server and the target scheduling server, and between the servers passed through the communication path; the reliability of the first communication protocol is lower than the reliability of the second communication protocol, and the transmission efficiency of the first communication protocol is higher than the transmission efficiency of the second communication protocol.

[0219] In one embodiment, the scheduling server processing module 1304 is further configured to:

[0220] Based on the first live stream identifier, determining a first publishing server from the publishing server cluster; the first publishing server stores a live data stream corresponding to the first live stream identifier;

[0221] Determine the operator identifier and region identifier of the target playback server based on the playback server identifier;

[0222] Determining a second publishing server from each first publishing server based on the operator identifier;

[0223] Based on the region identifier, a target publishing server is determined from the second publishing servers.

[0224] In one embodiment, the playback server processing module 1302 is further configured to:

[0225] The first playback terminal determines the first playback server from the playback server cluster based on its own operator identifier, determines the target playback server from each first playback server based on its own region identifier, and sends a live viewing request carrying the first live stream identifier to the target playback server;

[0226] Through the target playback server, the target scheduling server is determined from the scheduling server cluster based on its own regional identifier, and a live query request carrying the first live stream identifier and the playback server identifier of the target playback server is sent to the target scheduling server.

[0227] In one embodiment, the scheduling server processing module 1304 is further configured to:

[0228] Querying the communication speed between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier;

[0229] When the communication speed is greater than or equal to the preset speed, obtaining a communication path between the target playback server and the target publishing server based on the communication address corresponding to the target publishing server;

[0230] When the communication speed is lower than the preset speed, the communication address corresponding to at least one proxy server is obtained, and the communication path between the target playback server and the target publishing server is obtained based on the communication address corresponding to the target publishing server and the communication address corresponding to the proxy server.

[0231] In one embodiment, the live data processing apparatus is further configured to:

[0232] Through at least one server passed through the communication path, the first live data stream corresponding to the first live stream identifier sent by its own forward live broadcast device is obtained, jitter and packet loss processing are performed on the media data packets in the first live data stream, and the second live data stream corresponding to the first live stream identifier is obtained, and the second live data stream is sent to its own backward live broadcast device.

[0233] In one embodiment, the live data processing apparatus is further configured to:

[0234] Splitting the first live data stream to obtain multiple first media data packets;

[0235] performing packet loss compensation on the plurality of first media data packets to obtain a plurality of second media data packets;

[0236] updating a cache capacity of a jitter buffer based on a transmission delay of the first media data packet, storing a plurality of second media data packets in the jitter buffer, and sequentially adding the second media data packets in the jitter buffer to a live data queue;

[0237] The data packets in the live data queue are encapsulated in order to obtain a second live data stream corresponding to the first live stream identifier.

[0238] In one embodiment, the live data processing apparatus is further configured to:

[0239] The system obtains feedback information of the backward live broadcast device for the second live broadcast data stream through at least one server passed through the communication path, and sends a supplementary live broadcast data stream to the backward live broadcast device based on the feedback information.

[0240] In one embodiment, the live data processing apparatus is further configured to:

[0241] Based on the packet loss information in the feedback information, obtain the data packet to be retransmitted;

[0242] A supplementary live data stream is generated based on the data packet to be retransmitted, and the supplementary live data stream is sent backward to the live device.

[0243] In one embodiment, the live data processing apparatus is further configured to:

[0244] Based on the feedback information, the historical sending speed for the live data stream is updated to obtain the target sending speed for the live data stream;

[0245] Acquire a backward live data stream of the second live data stream as a supplementary live data stream;

[0246] Based on the target sending speed, a supplementary live data stream is sent backward to the live device.

[0247] In one embodiment, the live data processing apparatus is further configured to:

[0248] Obtain feedback information of the second live broadcast data stream from the plurality of backward live broadcast devices of the device;

[0249] Collecting statistics on lost data packets corresponding to the packet loss information in each feedback information to obtain feedback statistics corresponding to each lost data packet;

[0250] Determining a data packet to be retransmitted from each lost data packet based on the feedback statistics;

[0251] A supplementary live data stream is generated based on the data packet to be retransmitted, and the supplementary live data stream is sent to the subsequent live broadcast device corresponding to the data packet to be retransmitted.

[0252] In one embodiment, the live data processing apparatus is further configured to:

[0253] Update the network status corresponding to the live broadcast device based on the feedback information;

[0254] When the network status is normal, a supplementary live data stream is sent back to the live broadcast device based on the feedback information.

[0255] In one embodiment, the first playback terminal is a playback terminal in an interactive live broadcast scene. The live broadcast data processing device is further configured to:

[0256] Obtaining, through a content distribution server, a live viewing request carrying a second live stream identifier sent by a second playback terminal in a non-interactive live broadcast scenario, obtaining a comprehensive live data stream corresponding to the second live stream identifier, and sending the comprehensive live data stream corresponding to the second live stream identifier to the second playback terminal;

[0257] The integrated live data stream is obtained by merging the live data stream generated by at least one live terminal corresponding to the second live stream identifier obtained by the merging server, synthesizing the various live data streams and sending them to the content distribution server.

[0258] In the live broadcast data processing device described above, the target playback server responds to a live broadcast viewing request from a first playback terminal and queries the target publishing server where the live broadcast data is located, as well as the communication path between the target publishing server and the target playback server, from a target scheduling server. The target playback server obtains the live broadcast data from the target publishing server based on the queried communication path and sends the live broadcast data to the first playback terminal, thereby enabling the first terminal to view the live broadcast. Live broadcast viewing is achieved through orderly interaction between the first playback terminal, the target playback server, the target scheduling server, and the target publishing server, which can reduce the pressure on the live broadcast terminal and ensure the quality of the live broadcast. Furthermore, the target scheduling server determines the target publishing server where the live broadcast data is located based on the playback server identifier, and then determines the communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier. When the communication quality between the target playback server and the target publishing server meets preset conditions, the communication path passes through at least one proxy server. The proxy server can ensure stable transmission of live broadcast data between the target playback server and the target publishing server, further improving the quality of the live broadcast.

[0259] Each module in the live data processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0260] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 14 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data such as live broadcast introduction information and server information of various servers. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a live broadcast data processing method is implemented.

[0261] Those skilled in the art will understand that Figure 14The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0262] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0263] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0264] In one embodiment, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0265] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0266] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0267] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0268] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A live broadcast data processing method, characterized in that: The method comprises: Obtaining, through the target playback server, a live viewing request carrying a first live stream identifier sent by the first playback terminal, and sending a live query request carrying the first live stream identifier and the playback server identifier of the target playback server to the target scheduling server; The target scheduling server determines a target publishing server based on the first live stream identifier, determines a communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server, and returns the communication path to the target playback server; when the communication quality between the target playback server and the target publishing server meets a preset condition, the communication path passes through at least one proxy server, and the proxy server is used to connect the target playback server and the target publishing server; The target playback server obtains the live data stream corresponding to the first live stream identifier from the target publishing server based on the communication path, and sends the live data stream corresponding to the first live stream identifier to the first playback terminal.

2. The method according to claim 1, characterized in that The method further comprises: Sending the live data stream corresponding to the first live stream identifier to the target publishing server through the live broadcast device; Among them, a first communication protocol is adopted between the live broadcast terminal and the target publishing server, and between the first playback terminal and the target playback server, and a second communication protocol is adopted between the target publishing server and the target scheduling server, between the target playback server and the target scheduling server, and between the servers through which the communication path passes; the reliability of the first communication protocol is lower than the reliability of the second communication protocol, and the transmission efficiency of the first communication protocol is higher than the transmission efficiency of the second communication protocol.

3. The method according to claim 1, characterized in that The determining a target publishing server based on the first live stream identifier includes: Based on the first live stream identifier, determining a first publishing server from a publishing server cluster; the first publishing server stores a live data stream corresponding to the first live stream identifier; Determining the operator identifier and the region identifier of the target playback server based on the playback server identifier; Determining a second publishing server from each first publishing server based on the operator identifier; Based on the region identifier, a target publishing server is determined from the second publishing servers.

4. The method according to claim 1, wherein The method of acquiring, through the target playback server, a live viewing request carrying a first live stream identifier sent by a first playback terminal, and sending a live query request carrying the first live stream identifier and a playback server identifier of the target playback server to the target scheduling server includes: The first playback terminal determines the first playback server from the playback server cluster based on its own operator identifier, determines the target playback server from each first playback server based on its own region identifier, and sends a live viewing request carrying the first live stream identifier to the target playback server; Through the target playback server, the target scheduling server is determined from the scheduling server cluster based on its own regional identifier, and a live query request carrying the first live stream identifier and the playback server identifier of the target playback server is sent to the target scheduling server.

5. The method according to claim 1, wherein The determining of the communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server includes: querying the communication speed between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier; When the communication speed is greater than or equal to a preset speed, obtaining a communication path between the target playback server and the target publishing server based on a communication address corresponding to the target publishing server; When the communication speed is less than a preset speed, the communication address corresponding to at least one proxy server is obtained, and the communication path between the target playback server and the target publishing server is obtained based on the communication address corresponding to the target publishing server and the communication address corresponding to the proxy server.

6. The method according to claim 1, characterized in that The method further comprises: Through at least one server passed through the communication path, the first live data stream corresponding to the first live stream identifier sent by its own forward live broadcast device is obtained, jitter and packet loss processing are performed on the media data packets in the first live data stream, and the second live data stream corresponding to the first live stream identifier is obtained, and the second live data stream is sent to its own backward live broadcast device.

7. The method according to claim 6, characterized in that The performing jitter and packet loss processing on the media data packets in the first live data stream to obtain the second live data stream corresponding to the first live stream identifier includes: Splitting the first live data stream to obtain multiple first media data packets; performing packet loss compensation on the plurality of first media data packets to obtain a plurality of second media data packets; updating a cache capacity of a jitter buffer based on a transmission delay of the first media data packet, storing the plurality of second media data packets in the jitter buffer, and sequentially adding the second media data packets in the jitter buffer to a live data queue; The data packets in the live data queue are encapsulated in order to obtain a second live data stream corresponding to the first live stream identifier.

8. The method according to claim 6, characterized in that The method further comprises: Obtain feedback information of the backward live broadcast device for the second live broadcast data stream through at least one server passed through the communication path, and send a supplementary live broadcast data stream to the backward live broadcast device based on the feedback information.

9. The method according to claim 8, characterized in that The sending of the supplementary live data stream to the backward live broadcast device based on the feedback information includes: Acquire a data packet to be retransmitted based on the packet loss information in the feedback information; A supplementary live data stream is generated based on the data packet to be retransmitted, and the supplementary live data stream is sent to the backward live broadcast device.

10. The method according to claim 8, characterized in that The sending of the supplementary live data stream to the backward live broadcast device based on the feedback information includes: Based on the feedback information, updating the historical sending speed for the live data stream to obtain a target sending speed for the live data stream; Obtaining a backward live data stream of the second live data stream as a supplementary live data stream; Based on the target sending speed, the supplementary live data stream is sent to the backward live device.

11. The method according to claim 8, characterized in that The obtaining feedback information of the backward live broadcast device regarding the second live broadcast data stream, and sending a supplementary live broadcast data stream to the backward live broadcast device based on the feedback information, comprises: Obtaining feedback information from multiple backward live broadcast devices of the device for the second live broadcast data stream respectively; Collecting statistics on lost data packets corresponding to the packet loss information in each feedback information to obtain feedback statistics corresponding to each lost data packet; Determining a data packet to be retransmitted from each of the lost data packets based on the feedback statistics; A supplementary live data stream is generated based on the data packet to be retransmitted, and the supplementary live data stream is sent to the backward live broadcast device corresponding to the data packet to be retransmitted.

12. The method according to claim 8, characterized in that The sending of the supplementary live data stream to the backward live broadcast device based on the feedback information includes: Updating the network status corresponding to the backward live broadcast device based on the feedback information; When the network state is normal, a supplementary live data stream is sent to the backward live broadcast device based on the feedback information.

13. The method according to any one of claims 1 to 12, characterized in that The first playback terminal is a playback terminal in an interactive live broadcast scene; The method further comprises: obtaining, through a content distribution server, a live viewing request carrying a second live stream identifier sent by a second playback terminal in a non-interactive live broadcast scenario, obtaining a comprehensive live data stream corresponding to the second live stream identifier, and sending the comprehensive live data stream corresponding to the second live stream identifier to the second playback terminal; The comprehensive live broadcast data stream is obtained by merging the live broadcast data stream generated by at least one live broadcast terminal corresponding to the second live broadcast stream identifier obtained by the merging server, synthesizing the various live broadcast data streams and sending them to the content distribution server.

14. A live broadcast data processing system, characterized in that: The system includes a target playback server and a target scheduling server: The target playback server is configured to obtain a live viewing request carrying a first live stream identifier sent by a first playback terminal, and send a live query request carrying the first live stream identifier and a playback server identifier of the target playback server to the target scheduling server; The target scheduling server is configured to determine a target publishing server based on the first live stream identifier, determine a communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server, and return the communication path to the target playback server; when the communication quality between the target playback server and the target publishing server meets a preset condition, the communication path passes through at least one proxy server, and the proxy server is configured to connect the target playback server and the target publishing server; The target playback server is further configured to obtain the live data stream corresponding to the first live stream identifier from the target publishing server based on the communication path, and send the live data stream corresponding to the first live stream identifier to the first playback terminal.

15. The system according to claim 14, wherein: The system further includes a target publishing server: The target publishing server is configured to obtain a live broadcast initiation request carrying live broadcast introduction information sent by a live broadcast device, and send a live broadcast registration request carrying the live broadcast introduction information to the target scheduling server; The live broadcast introduction information includes a first live broadcast stream identifier; The target scheduling server is further configured to associate the live broadcast introduction information with the publishing server identifier and register it in a storage server, and return a registration result to the target publishing server; The target publishing server is further configured to return the registration result to the live broadcast device, and obtain the live broadcast data stream corresponding to the first live broadcast stream identifier sent by the live broadcast device after successful registration.

16. The system according to claim 14, wherein: The first playback terminal is a playback terminal in an interactive live broadcast scene; the system also includes a merging server and a content distribution server: The merging server is configured to obtain a live data stream generated by at least one live terminal corresponding to the second live stream identifier, synthesize the individual live data streams to obtain a comprehensive live data stream, and send the comprehensive live data stream to the content distribution server; The content distribution server is used to obtain a live viewing request carrying the second live stream identifier sent by a second playback terminal in a non-interactive live broadcast scene, obtain the comprehensive live data stream corresponding to the second live stream identifier, and send the comprehensive live data stream corresponding to the second live stream identifier to the second playback terminal.

17. A live broadcast data processing device, characterized in that: The device comprises: The playback server processing module is configured to obtain, through the target playback server, a live viewing request carrying a first live stream identifier sent by the first playback terminal, and send a live query request carrying the first live stream identifier and the playback server identifier of the target playback server to the target scheduling server; a scheduling server processing module, configured to determine, through the target scheduling server, a target publishing server based on the first live stream identifier, determine a communication path between the target playback server and the target publishing server based on the playback server identifier and the publishing server identifier of the target publishing server, and return the communication path to the target playback server; when the communication quality between the target playback server and the target publishing server meets a preset condition, the communication path passes through at least one proxy server, and the proxy server is configured to connect the target playback server and the target publishing server; The playback server processing module is also used to obtain the live data stream corresponding to the first live stream identifier from the target publishing server through the target playback server based on the communication path, and send the live data stream corresponding to the first live stream identifier to the first playback terminal.

18. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 13 are implemented.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

20. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images