Video broadcast processing system, method, device and equipment

Abstract

The present application discloses a video broadcast processing system, method device and equipment, which relates to the field of signal processing. The system includes a relay client, a commentary client, a streaming client and a first cloud server. The first cloud server is used to receive the live video stream sent by the streaming client; the commentary client is used to pull the live video stream from the first cloud server and obtain the commentary video stream; the first cloud server is also used to receive the commentary video stream sent by the commentary client; the relay client is used to pull the live video stream and the commentary video stream from the first cloud server, splice the live video stream and the commentary video stream to obtain a multi-screen video stream, and generate a broadcast video stream and push it to the audience terminal to display the live screen. In the above manner, it is possible to use multiple clients to jointly produce and broadcast the video stream, effectively overcoming the limitation of being in the same scene. The present application can be applied to various scenarios such as cloud technology, artificial intelligence, and smart transportation.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of signal processing, and in particular to a video broadcast processing system, method, apparatus and device. Background Art

[0002] With the rapid development of wireless network transmission technology, this technology is applied to more and more signal transmission fields. For example, some major events are usually broadcast by multiple countries and different TV stations at the same time, thus providing viewing channels for people in different regions around the world.

[0003] In related technologies, it is usually necessary to broadcast the public signals of the event offline, that is, the director, broadcast director, subtitle operator, sound control operator, data operator, playback operator and many other staff members are required to be in a fixed director's room, and the commentator is in a studio adjacent to the director's room, with the support of professional broadcasting equipment such as video switchers, mixing consoles, recording and playback equipment, graphic packaging production equipment, and call matrices, so as to complete the broadcast of an e-sports event.

[0004] However, offline event broadcasting solutions have extremely high requirements for venues, which involve the connection and configuration of expensive and complex professional broadcasting equipment. However, due to travel restrictions, inconvenient equipment transfer and other event broadcasting issues, offline event broadcasting solutions have high restrictions and cannot flexibly adapt to current signal broadcasting requirements. Summary of the invention

[0005] The embodiment of the present application provides a video broadcast processing system, method, apparatus and device, which can use multiple clients to jointly produce, process and broadcast video streams, effectively overcome the offline scene limitation of being in the same scene, and has the advantages of high flexibility and low equipment cost. The technical solution is as follows.

[0006] On the one hand, a video broadcast processing system is provided, the system comprising a relay client, a commentary client, a streaming client and a first cloud server;

[0007] The first cloud server is used to receive the live video stream sent by the push stream client through the first push stream address, wherein the first cloud server stores a correspondence between the first push stream address and the first pull stream address corresponding to the broadcast event, and the first pull stream address is an address configured by the relay client and the commentary client for pulling the live video stream;

[0008] The commentary client is used to pull the live video stream from the first cloud server through the first stream pulling address; obtain the commentary video stream; and send the commentary video stream to the first cloud server through the second stream pushing address;

[0009] The first cloud server is further used to receive the commentary video stream sent by the commentary client through the second push stream address, the first cloud server stores a correspondence between the second push stream address and the second pull stream address, and the second pull stream address is an address configured by the transit client for pulling the commentary video stream;

[0010] The relay client is used to pull the live video stream from the first cloud server through the first stream pulling address, and to pull the commentary video stream from the first cloud server through the second stream pulling address; splicing the live video stream and the commentary video stream to obtain a multi-screen video stream; generating a broadcast video stream based on the multi-screen video stream and pushing it to the audience terminal for live screen display.

[0011] On the other hand, a video broadcast processing method is provided, the method comprising:

[0012] Pulling the live video stream from the first cloud server through the first stream pulling address; the first stream pulling address is an address configured by the transit client for receiving the live video stream;

[0013] Pulling the commentary video stream from the first cloud server through a second stream pulling address, where the second stream pulling address is an address configured by the transit client for receiving the commentary video stream;

[0014] The live video stream and the commentary video stream are spliced ​​to obtain a multi-screen video stream; a relay video stream is generated based on the multi-screen video stream, and the relay video stream is pushed to the viewer terminal for live screen display;

[0015] Among them, the first cloud server is used to store the correspondence between the first push stream address and the first pull stream address corresponding to the broadcast event, and the first push stream address is the address configured in the push stream client for sending the live video stream; and to store the correspondence between the second push stream address and the second pull stream address corresponding to the commentary video stream, and the second push stream address is the address configured in the commentary client for sending the commentary video stream.

[0016] On the other hand, a video broadcast processing method is provided, the method comprising:

[0017] Pulling the live video stream from the first cloud server through the first stream pulling address, where the first stream pulling address is an address configured by the commentary client for receiving the live video stream; the first cloud server is used to store the correspondence between the first stream pushing address corresponding to the broadcast event and the first stream pulling address, where the first stream pushing address is an address configured in the stream pushing client for sending the live video stream;

[0018] Get the commentary video stream;

[0019] The commentary video stream is sent to the first cloud server via a second push stream address, wherein the second push stream address is an address configured in the commentary client for sending the commentary video stream; the first cloud server is also used to store the correspondence between the second push stream address and the second pull stream address corresponding to the commentary video stream, wherein the second pull stream address is an address configured in the transit client for receiving the commentary video stream.

[0020] In another aspect, a video broadcast processing device is provided, the device comprising:

[0021] A first pulling module, used to pull the live video stream from the first cloud server through a first pulling stream address; the first pulling stream address is an address configured by the transit client for receiving the live video stream;

[0022] A second pulling module, used to pull the commentary video stream from the first cloud server through a second pulling stream address, where the second pulling stream address is an address configured by the transfer client for receiving the commentary video stream;

[0023] A splicing module is used to splice the live video stream and the commentary video stream to obtain a multi-screen video stream; generate a relay video stream based on the multi-screen video stream, and push the relay video stream to the audience terminal for live screen display;

[0024] Among them, the first cloud server is used to store the correspondence between the first push stream address and the first pull stream address corresponding to the broadcast event, and the first push stream address is the address configured in the push stream client for sending the live video stream; and to store the correspondence between the second push stream address and the second pull stream address corresponding to the commentary video stream, and the second push stream address is the address configured in the commentary client for sending the commentary video stream.

[0025] In another aspect, a video broadcast processing device is provided, the device comprising:

[0026] A pulling module, used to pull a live video stream from a first cloud server through a first pulling stream address, wherein the first pulling stream address is an address configured by the commentary client for receiving the live video stream; the first cloud server is used to store a correspondence between a first push stream address corresponding to a broadcast event and the first pulling stream address, wherein the first push stream address is an address configured in the push stream client for sending the live video stream;

[0027] An acquisition module is used to obtain the commentary video stream;

[0028] A sending module is used to send the commentary video stream to the first cloud server via a second push stream address, wherein the second push stream address is an address configured in the commentary client for sending the commentary video stream; the first cloud server is also used to store the correspondence between the second push stream address and the second pull stream address corresponding to the commentary video stream, wherein the second pull stream address is an address configured in the transit client for receiving the commentary video stream.

[0029] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the video broadcast processing method as described in any of the above-mentioned embodiments of the present application.

[0030] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the video broadcast processing method as described in any of the above-mentioned embodiments of the present application.

[0031] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the video relay processing method described in any of the above embodiments.

[0032] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0033] In the video broadcast processing system, the first cloud server receives the live video stream sent by the push stream client through the first push stream address, and enables the commentary client and the transfer client to pull the live video stream from the first cloud server by means of the first pull stream address corresponding to the first push stream address; after receiving the commentary video stream, the commentary client sends the commentary video stream to the first cloud server through the second push stream address, so that the transfer client pulls the commentary video stream from the first cloud server by means of the second pull stream address corresponding to the second push stream address, and splices the live video stream and the commentary video stream to obtain a multi-screen video stream, so that the transfer client generates a broadcast video stream sent to the audience terminal based on the multi-screen video stream. Even if multiple clients are located in different areas, the first cloud server can be used to realize the broadcast and production processing between video streams, effectively overcoming the offline scene limitation of being in the same scene, and with the help of different clients and cloud servers, it achieves the characteristics of high flexibility, low labor cost, and low equipment cost, and can be widely used in various broadcast scenes, improving the efficiency of human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 It is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0036] Figure 2 is an interactive schematic diagram of a video broadcast processing system provided by an exemplary embodiment of the present application;

[0037] Figure 3 is a schematic diagram of an interface of a multi-image video stream provided by an exemplary embodiment of the present application;

[0038] Figure 4 is an interactive schematic diagram of a video broadcast processing system provided by another exemplary embodiment of the present application;

[0039] Figure 5 It is a flowchart of a method for a transit client to perform video relay processing provided by an exemplary embodiment of the present application;

[0040] Figure 6 It is a flowchart of a method for a narration client to perform video broadcast processing provided by an exemplary embodiment of the present application;

[0041] Figure 7is a flowchart of sending multiple video streams using SRT technology provided by an exemplary embodiment of the present application;

[0042] Figure 8 is a schematic diagram of an explanation interface provided by an exemplary embodiment of the present application;

[0043] Fig. 9 is a flow chart of a remote voice call process provided by an exemplary embodiment of the present application;

[0044] Fig.10 is a schematic diagram of a configuration interface of a video playback program provided by an exemplary embodiment of the present application;

[0045] Fig.11 It is a schematic diagram of an interface for configuring content under an audio option provided by an exemplary embodiment of the present application;

[0046] Fig.12 is a schematic diagram of a setting interface of a server provided by an exemplary embodiment of the present application;

[0047] Fig.13 It is a flowchart of a method for producing a client to perform video broadcast processing provided by an exemplary embodiment of the present application;

[0048] Fig.14 It is a schematic diagram of an interface of a production client performing a video production process provided by an exemplary embodiment of the present application;

[0049] Fig.15 It is a schematic diagram of an interface under a button function option provided by an exemplary embodiment of the present application;

[0050] Fig.16 is an enlarged schematic diagram of button content provided by an exemplary embodiment of the present application;

[0051] Fig.17 is a schematic diagram of a physical button provided by an exemplary embodiment of the present application;

[0052] Fig.18 It is a schematic diagram of an interface for adjusting a graphic effect provided by an exemplary embodiment of the present application;

[0053] Fig.19 It is a schematic diagram of an interface of a content introduction poster provided by an exemplary embodiment of the present application;

[0054] Fig. 20 is a schematic diagram of an interface of a target music program provided by an exemplary embodiment of the present application;

[0055] Fig.21 It is a schematic diagram of an interface for video production provided by an exemplary embodiment of the present application;

[0056] Fig. 22 is a schematic diagram of obtaining control data of different effects provided by an exemplary embodiment of the present application;

[0057] Fig.23 It is a schematic diagram of an interface of a public signal content of a competition provided by an exemplary embodiment of the present application;

[0058] Fig.24 It is a schematic diagram of an interface for receiving and processing event public signal content provided by an exemplary embodiment of the present application;

[0059] Fig.25 is a flowchart of video stream push provided by an exemplary embodiment of the present application;

[0060] Fig.26 It is a schematic diagram of a setting interface on a production client provided by an exemplary embodiment of the present application;

[0061] Fig. 27 is an overall schematic diagram of a video broadcast processing system provided by an exemplary embodiment of the present application;

[0062] Fig.28 is a flowchart of the operation sequence in the video broadcasting process provided by an exemplary embodiment of the present application;

[0063] Fig.29 It is a structural block diagram of a terminal provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application clearer, the following will further describe the implementation mode of the present application in detail with reference to the accompanying drawings. First, a brief introduction is given to the terms involved in the embodiments of the present application.

[0065] Secure Reliable Transport (SRT): is an open source Internet transmission protocol based on the UDP-based Data Transfer Protocol (UDT) that can achieve low-latency network video transmission.

[0066] Real Time Messaging Protocol (RTMP): A protocol family developed based on the Transmission Control Protocol (TCP), including the RTMP basic protocol and multiple variants such as RTMPT, RTMPS, and RTMPE. RTMP is a network protocol designed for real-time data communication, mainly used for audio, video, and data communication between Flash / AIR platforms and streaming media and interactive servers that support the RTMP protocol.

[0067] The fifth major revision of HyperText Markup Language 5 (HTML5): HTML5 is the specification of HTML, the core language of the Web. When users browse web pages by any means, the content they see is originally in HTML format, and some technical processing is used in the browser to convert it into recognizable information. HTML5 consists of different technologies and has been widely used in the Internet, providing more standard machines for enhancing network applications.

[0068] Virtual Private Network (VPN): It is defined as a temporary, secure connection established through a public Internet. It is a secure, stable tunnel that passes through a chaotic public network. This tunnel can encrypt data several times to achieve the purpose of safe use of the Internet. It is widely used in corporate offices. VPN can also be an extension of the corporate intranet, which can help remote users, company branches, business partners and suppliers establish a reliable and secure connection with the company's intranet. It is used to economically and effectively connect to the secure extranet virtual private network of business partners and users, facilitate remote office, etc.

[0069] In the related technology, it is usually necessary to broadcast the public signal of the event offline, that is, the director, director, subtitle operator, sound control operator, data operator, playback operator and many other staff are required to be in a fixed director's room, and the commentator is in the studio adjacent to the director's room, with the support of professional broadcasting equipment such as video switchers, mixers, recording and playback equipment, graphic packaging production equipment, and call matrices, so as to complete the broadcast of an e-sports event. However, the offline event broadcasting plan has extremely high requirements for the venue, which involves the connection and configuration of expensive and complex professional broadcasting equipment. However, due to travel restrictions, inconvenient equipment transfer and other event broadcasting issues, the offline event broadcasting plan has high restrictions and cannot flexibly adapt to the current signal broadcasting requirements.

[0070] In an embodiment of the present application, a video broadcast processing system is provided, which can perform a common production processing process on the video stream with the help of the broadcast process of the video stream by multiple clients, effectively overcoming the offline scene limitation of being in the same scene, and achieving the characteristics of high flexibility, low labor cost, and low equipment cost. The video broadcast processing system trained for this application includes at least one of the various scenarios such as game event broadcasting scenarios, sports event broadcasting scenarios, remote office scenarios, and remote teaching scenarios when applied. It is worth noting that the above application scenarios are only illustrative examples. The video broadcast processing system provided in this embodiment can also be applied to other scenarios, and the embodiment of the present application does not limit this.

[0071] It should be noted that before collecting relevant data of users and during the process of collecting relevant data of users, this application can display a prompt interface, pop-up window or output voice prompt information, and the prompt interface, pop-up window or voice prompt information is used to prompt the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining relevant data of users after obtaining the confirmation operation issued by the user to the prompt interface or pop-up window, otherwise (that is, when the confirmation operation issued by the user to the prompt interface or pop-up window is not obtained), the relevant steps of obtaining relevant data of users are terminated, that is, the relevant data of users are not obtained. In other words, all user data collected by this application are collected with the consent and authorization of the user, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0072] Secondly, the implementation environment involved in the embodiments of the present application is described. The video broadcast processing system provided in the embodiments of the present application includes multiple terminals, and the video broadcast process is realized through the interaction between the multiple terminals; or, the video broadcast processing system provided in the embodiments of the present application includes multiple terminals (or: clients) and servers, and the video broadcast process is realized through the interaction between the terminals and the server, etc., which is not limited in the embodiments of the present application. Optionally, the video broadcast process is realized by the interaction between multiple terminals as an example for description.

[0073] For illustration, please refer to Figure 1 The implementation environment involves a streaming client 110, a first cloud server 120, a commentary client 130, and a transfer client 140. The streaming client 110 is a client that pushes the live video stream corresponding to the specified live event; the first cloud server 120 is a cloud server that forwards various video streams, signals and other data; the commentary client 130 is a client that receives the commentary of the live video stream and generates commentary content; the transfer client 140 is a client that processes various video streams, signals and other data.

[0074] In the first cloud server 120, the live video stream sent by the push stream client 110 through the first push stream address is received, wherein the first cloud server 120 stores the correspondence between the first push stream address and the first pull stream address corresponding to the broadcast event, and the first pull stream address is the address for pulling the live video stream configured by the transit client 140 and the commentary client 130. For example: the push stream client 110 is a client for broadcasting a certain event, and the push stream client 110 processes the video stream corresponding to the live event to generate a live video stream, and pushes the stream to multiple cloud servers and clients including the first cloud server 120.

[0075] In the commentary client 130, the live video stream is pulled from the first cloud server 120 through the first pull stream address; the commentary video stream is obtained; and the commentary video stream is sent to the first cloud server 120 through the second push stream address. For example: after pulling the live video stream, the commentary client 130 displays the live video stream on the screen corresponding to the commentary client 130, and based on the microphone, camera and other devices configured by the commentary client 130, receives the commentary video stream generated by the commentator's commentary on the live video stream; and pushes the commentary video stream to the first cloud server 120.

[0076] Therefore, the first cloud server 120 also receives the commentary video sent by the commentary client 130 through the second push stream address. The first cloud server 120 stores the corresponding relationship between the second push stream address and the second pull stream address, which is the address configured by the transit client 140 for pulling the commentary video stream.

[0077] In the relay client 140, the live video stream is pulled from the first cloud server 120 through the first pull stream address, and the commentary video stream is pulled from the first cloud server through the second pull stream address; the live video stream and the commentary video stream are spliced ​​to obtain a multi-screen video stream; a broadcast video stream is generated based on the multi-screen video stream and pushed to the audience terminal for live screen display. For example: after the relay client receives the live commentary video stream and the commentary video stream, the live commentary video stream and the commentary video stream are spliced ​​to obtain a multi-screen video stream, and then a broadcast video stream is generated based on the multi-screen video stream and pushed to the audience terminal for display, so as to realize the cooperation of multiple clients and complete the video broadcast processing process with the help of the cloud server.

[0078] It is worth noting that the above-mentioned terminals include but are not limited to mobile terminals such as mobile phones, tablet computers, portable laptops, intelligent voice interaction devices, smart home appliances, and vehicle-mounted terminals, and can also be implemented as desktop computers, etc.; the above-mentioned servers can be independent physical servers, or server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. Among them, cloud technology refers to a hosting technology that unifies a series of resources such as hardware, applications, and networks in a wide area network or a local area network to realize the calculation, storage, processing, and sharing of data. In some embodiments, the above-mentioned servers can also be implemented as nodes in a blockchain system.

[0079] Combined with the above-mentioned noun introduction and application scenarios, the video broadcast processing system provided by this application is described. Figure 2 As shown, there are streaming client 210, first cloud server 220, explanation client 230 and relay client 240 in the communication network. The following describes the client and cloud server in the video broadcast processing system respectively.

[0080] Streaming Client 210

[0081] Illustratively, the streaming client 210 is implemented as at least one of a variety of clients such as a mobile client, a computer client, a tablet client, a car client, and a radio client.

[0082] The streaming client 210 is used to perform a video stream production process for a video stream related to a broadcast event. In an illustrative manner, when a broadcast event is played, recorded, broadcasted, etc., a video stream corresponding to the broadcast event is generated. For example, when a game event is broadcasted, a game event video stream corresponding to the game event is generated; or when a sports event is played, a sports event video stream corresponding to the sports event is generated.

[0083] Optionally, the video stream generated when playing, recording, and rebroadcasting the rebroadcast event is the same video stream, that is, the video stream generated when playing the rebroadcast event is used as the live video stream when rebroadcasting the video. Schematically, the push stream client records the rebroadcast event to generate a live video stream, and rebroadcasts the generated live video stream.

[0084] The live video stream can be used to know the current live content of the broadcast event. In an illustrative example, the live video stream M generated by the live broadcast of the broadcast event can be used to know the current broadcast event. For example, the current game event A is a virtual game between team 3 and team 5.

[0085] Optionally, the streaming client 210 is a client that streams the live video stream corresponding to a specified live event. For example, the streaming client 210 is a client that broadcasts a certain event. The streaming client 210 processes the video stream corresponding to the live event to generate a live video stream, and streams it to multiple cloud servers and / or clients including the first cloud server 220. In some embodiments, based on the broadcast authorization of the streaming client 210, the video stream pushed by the streaming client 210 is pulled.

[0086] Illustratively, before any client or cloud server needs to pull the video stream generated by the streaming client 210, it sends a video stream acquisition request to the streaming client 210 to request the streaming client 210 to push the video stream to it.

[0087] Optionally, the push stream client 210 can push the live video stream 1 corresponding to event A and the live video stream 2 corresponding to event B; if any client or cloud server needs to pull the live video stream 1 and / or the live video stream 2, it can send the corresponding video stream acquisition request to the push stream client 210. For example: taking any client or cloud server as the first cloud server 220, and the first cloud server 220 sending a request for the live video stream 1 as an example, the first cloud server 220 sends a request for the acquisition of the live video stream 1 to the push stream client 210, and then decides whether to give the push stream client 210 the broadcast authorization of the live video stream 1 based on the push stream client 210. Optionally, the broadcast authorization of the push stream client 210 can also be determined by a pre-agreed method. For example: determining the broadcast authorization situation by artificial agreement, etc.

[0088] In an optional embodiment, when any client or cloud server is granted broadcast authorization (i.e., when any client or cloud server has the right to broadcast), the streaming client 210 assigns a first streaming pull address to the client or cloud server, as well as a correspondence between the first streaming push address and the first streaming pull address, so that when the video stream is pushed to the client or cloud server, the video stream push process is implemented based on the first streaming push address and the first streaming pull address.

[0089] Indicatively, the first push stream address is an address pre-configured by the push stream client for the push process of the live video stream, which is used to push the live video stream to the first cloud server, and is a specific identifier for pushing the live video stream to the first cloud server, which is used to identify the push direction of the live video stream. For example, the first push stream address is the first SRT push stream address, that is, the address for pushing the live video stream using the SRT technology.

[0090] Indicatively, the first stream pull address is an address pre-configured for the transfer client and the commentary client, corresponding to the first stream push address, through which the transfer client and the commentary client can pull the live video stream from the first cloud server, so that the transfer client and the commentary client can obtain the live video stream. For example: the first stream pull address is the first SRT stream pull address, that is, the address for pulling the live video stream using the SRT technology, and the first SRT stream pull address corresponds to the first SRT stream push address.

[0091] First cloud server 220

[0092] Cloud server (Elastic Compute Service, ECS) is a form of server, which is a simple, efficient, safe, reliable, and elastic computing service with scalable processing power. Compared with physical servers, the management method of cloud servers is simpler and more efficient. Users do not need to purchase hardware in advance, and can quickly create or release any number of cloud servers. The first cloud server 220 is a pre-selected cloud server.

[0093] Illustratively, the first cloud server is a pre-selected cloud server for broadcasting the event video; or, the first cloud server is a pre-selected cloud server with specific configuration content, etc.

[0094] In the first cloud server 220, the live video stream sent by the streaming client 210 through the first streaming address is received.

[0095] The first cloud server 220 stores a correspondence between the first push stream address and the first pull stream address corresponding to the broadcast event. The broadcast event is an event to be broadcasted that is predetermined by the first cloud server 220. Optionally, the broadcast event is a game event; or, the broadcast event is a sports event; or, the broadcast event is a classroom teaching event, etc.

[0096] Schematically, the first cloud server 220 has the rebroadcast authorization granted by the push stream client 210, that is, the first cloud server 220 has the right to rebroadcast. Therefore, the first cloud server 220 stores the first stream pull address assigned by the push stream client 210 to the first cloud server 220, and the correspondence between the first push stream address and the first stream pull address. Therefore, when the push stream client 210 sends a live video stream, the push stream client 210 sends the live video stream to the first cloud server 220 with the first stream pull address through the first push stream address, effectively avoiding the process of sending the live video stream to other clients or cloud servers that do not have the right to rebroadcast. Among them, the first stream pull address is the address configured by the transit client 240 and the commentary client 230 for pulling the live video stream.

[0097] In some embodiments, if the transit client 240 and the commentary client 230 need to obtain the live video stream pulled by the first cloud server 220 , they also need to obtain the authorization license of the first cloud server 220 .

[0098] Optionally, the transfer client 240 and the commentary client 230 respectively send authorization requests to the first cloud server 220 to request the first cloud server 220 to send the pulled live video stream to them. For example: the transfer client 240 sends an authorization request to the first cloud server 220 to request the first cloud server 220 to send the live video stream to the transfer client 240; similarly, the commentary client 230 sends an authorization request to the first cloud server 220 to request the first cloud server 220 to send the live video stream to the commentary client 230.

[0099] Optionally, if the first cloud server 220 grants the transit client 240 and the commentary client 230 the right to pull the live video stream based on their respective authorization requests, the first stream pulling address is sent to the transit client 240 and the commentary client 230, so that the transit client 240 and the commentary client 230 pull the live video stream pulled by the first cloud server according to the first stream pulling address respectively obtained.

[0100] In some embodiments, when the relay client 240 and the commentary client 230 send an authorization request to the first cloud server 220, they send the corresponding Internet Protocol (IP) address, so that the first cloud server 220 can send the live video stream to their respective clients based on the IP address and the first stream pulling address.

[0101] Explanation Client 230

[0102] The commentary client 230 is a client that receives the commentary video stream. Schematically, the commentary client 230 is a client controlled by a commentator.

[0103] In the commentary client 230 , the live video stream is pulled from the first cloud server 220 through the first stream pulling address.

[0104] Illustratively, when the commentary client 230 obtains the right to pull the live video stream granted by the first cloud server 220, the commentary client 230 receives the first stream pulling address distributed by the first cloud server 220, so that when pulling the live video stream, the live video stream is pulled from the first cloud server 220 through the first stream pulling address.

[0105] Get the commentary video stream.

[0106] The commentary video stream is a video stream obtained based on the live video stream. Schematically, the commentary client 230 is a client controlled by the commentator, and the commentary client 230 receives the commentary content of the commentator and produces the commentary content, thereby achieving the process of obtaining the commentary video stream.

[0107] In an exemplary embodiment, the explanation content includes at least one of the explanation audio content and the explanation video content. Optionally, the explanation video content is produced by a device with a camera and a mobile client, etc., and the explanation audio content is produced by a device with an audio collection function, such as a microphone and a mobile client, etc.

[0108] For example, the commentary client 230 is implemented as a computer client. After the commentary client 230 receives the live video stream, the live video stream is displayed on the screen of the computer client. The commentator gives a video commentary on the live video stream. During the video commentary process, the microphone device connected to the computer client collects the commentary audio content, and the computer client connected to the microphone device produces the commentary audio content, thereby obtaining commentary audio data; in addition, the camera device configured by the computer client collects the commentary video content, and the computer client produces the commentary video content, thereby obtaining commentary video data, and the commentary audio content and the commentary video content constitute the commentary video stream. That is, the commentary video stream is a video stream produced based on the live video stream.

[0109] The commentary video stream is sent to the first cloud server 220 via the second streaming address.

[0110] The second push stream address is an address corresponding to the commentary video stream and used to send the commentary video stream. After the commentary client 230 obtains the commentary video stream, it sends the commentary video stream to the first cloud server 220 through the second push stream address.

[0111] First cloud server 220

[0112] The content of the first cloud server 220 is supplemented with explanation.

[0113] The first cloud server 220 also receives the commentary video stream sent by the commentary client 230 through the second streaming address. The first cloud server 220 stores the corresponding relationship between the second streaming address and the second streaming address.

[0114] Indicatively, the second push stream address is different from the first push stream address, and is an address pre-configured by the commentary client for the push process of the commentary video stream, which is used to push the commentary video stream to the first cloud server. It is a specific identifier for pushing the commentary video stream to the first cloud server, and is used to identify the push flow direction of the commentary video stream. For example, the second push stream address is the second SRT push stream address, that is, the address for pushing the commentary video stream using SRT technology.

[0115] In a schematic diagram, the second stream pull address is an address pre-configured for the transfer client, corresponding to the second stream push address, through which the transfer client can pull the commentary video stream from the first cloud server, so that the transfer client can obtain the commentary video stream. For example, the second stream pull address is the second SRT stream pull address, that is, the address for pulling the commentary video stream using SRT technology, and the second SRT stream pull address corresponds to the second SRT stream push address.

[0116] The second stream pulling address is an address configured by the transit client 240 for pulling the commentary video stream.

[0117] In some embodiments, if the transit client 240 needs to obtain the commentary video stream pulled by the first cloud server 220 , it also needs to obtain the authorization license of the first cloud server 220 .

[0118] Optionally, the transfer client 240 sends an authorization request to the first cloud server 220 to request the first cloud server 220 to send the pulled commentary video stream to it. In an exemplary manner, if the first cloud server 220 grants the transfer client 240 the right to pull the commentary video stream based on the authorization request, the second pull stream address is sent to the transfer client 240, so that the transfer client 240 pulls the commentary video stream pulled by the first cloud server according to the obtained second pull stream address.

[0119] In some embodiments, when the transit client 240 sends an authorization request to the first cloud server 220, it sends the IP address of the transit client 240, so that the first cloud server 220 sends the commentary video stream to the transit client 240 based on the IP address and the second stream pulling address; or, the first cloud server 220 sends the commentary video stream to the transit client 240 based on the IP address and the second stream pulling address previously sent by the transit client 240, etc.

[0120] Transfer Client 240

[0121] Illustratively, the transfer client 240 is implemented as at least one of multiple clients such as a mobile client, a computer client, a tablet client, and a car client.

[0122] In the transit client 240 , the live video stream is pulled from the first cloud server 220 through the first stream pulling address, and the commentary video stream is pulled from the first cloud server 220 through the second stream pulling address.

[0123] Illustratively, in addition to the first stream pulling address distributed by the first cloud server 220 for pulling the live video stream, the transit client 240 also has a second stream pulling address distributed by the first cloud server 220 for pulling the commentary video stream, so that the transit client 240 pulls the live video stream from the first cloud server 220 through the first stream pulling address, and pulls the commentary video stream from the first cloud server 220 through the first stream pulling address.

[0124] The live video stream and the commentary video stream are spliced ​​together to obtain a multi-screen video stream.

[0125] In an optional embodiment, the relay client 240 receives different video streams from different clients or the first cloud server 220 , and the different video streams are video streams generated around the relay event.

[0126] Illustratively, the relay client 240 receives at least one live video stream generated by live broadcasting of the relayed event from the first cloud server 220; or / and, the relay client receives at least one commentary video stream generated by content commentary of the relayed event from the first cloud server 220; or / and, the relay client receives at least one other video stream related to the relayed event (such as a character interview video stream introducing a character in a specified live event) from any client, etc.

[0127] Schematically, after obtaining the live video stream and the commentary video stream, the live video stream and the commentary video stream are spliced ​​to obtain a multi-screen video stream. The multi-screen video stream is a video stream composed of multiple video screens, which can simultaneously represent the temporal relationship between the live video stream and the commentary video stream. Figure 3 The figure shows a schematic diagram of the interface corresponding to the multi-screen video stream. Figure 3It is a stitched picture corresponding to a multi-screen video stream at any moment (timestamp), and the stitched picture includes multiple event pictures (event picture 311, event picture 312, event picture 313 and event picture 314). In addition, the stitched picture also includes multiple commentary pictures (commentary picture 321, commentary picture 322, commentary picture 323 and commentary picture 324), a production picture 330 and an event preparation picture 340.

[0128] When the video screen in the spliced ​​screen is black, white, etc., it means that the signal of the video screen has been disconnected; or when the video screen in the spliced ​​screen is gray or has a specific buffer mark, etc., it means that the signal connection of the video screen is unstable. For example, if the event preparation screen 340 is black, it means that the signal of the event preparation screen 340 is disconnected at the current moment, such as because the event has started, the screen is automatically disconnected, or the event preparation screen 340 is manually cut off. Generate a broadcast video stream based on the multi-screen video stream and push it to the audience terminal for live screen display.

[0129] In an optional embodiment, after obtaining the multi-picture video stream, the broadcast video stream is obtained based on the adjustment of the multi-picture video stream; or, after obtaining the multi-picture video stream, at least one video stream is selected from the multi-picture video stream as the broadcast video stream, etc.

[0130] In some embodiments, the broadcast video stream is sent to the viewer terminal for video screen display.

[0131] In an illustrative manner, after obtaining the relay video stream, the relay video stream is sent to the viewer terminal. The viewer terminal is used to receive the relay video stream and display the relay video stream on a screen corresponding to the viewer terminal. For example, after receiving the relay video stream, the relay video stream is rendered on a screen corresponding to the viewer terminal, thereby realizing the display of the relayed video on the viewer terminal.

[0132] The above-mentioned video broadcast processing system includes the streaming client 210, the first cloud server 220, the commentary client 230 and the transit client 240, which is only an illustrative system composition. The video broadcast processing system can also include a production client, a server, other cloud servers, etc.; in addition, the streaming client 210, the first cloud server 220, the commentary client 230 and the transit client 240 are also only illustrative names, and the embodiments of the present application do not limit this.

[0133] To sum up, even if multiple clients are located in different areas, the first cloud server can be used to realize the broadcasting and production processing between video streams, effectively overcoming the offline scene limitation of requiring them to be in the same scene. With the help of different clients and cloud servers, it achieves the characteristics of high flexibility, low labor cost, and low equipment cost. It can be widely used in various broadcasting scenarios, improving the efficiency of human-computer interaction.

[0134] In an optional embodiment, the video broadcast processing system provided by the present application is described, taking the application of the system in a communication network as an example. Figure 4 As shown, in the communication network, in addition to the existence of Figure 2 In addition to the streaming client 210, the first cloud server 220, the commentary client 230 and the transfer client 240 shown in the figure, it also includes a production client 410 for controlling and adjusting the multi-screen video stream. Figure 2 The client and cloud server in the video broadcast processing system shown are supplemented with explanations respectively, and the production client 410 is explained.

[0135] Transfer Client 240

[0136] In the transfer client 240, the multi-screen video stream is sent to the first cloud server 220 via the third streaming address.

[0137] The third streaming address is an address configured by the transit client 240 for sending a multi-screen video stream.

[0138] Illustratively, after the relay client 240 obtains a multi-screen video stream by splicing the commentary video stream and the live video stream, the multi-screen video stream is sent to the first cloud server 220 via the third streaming address configured by the relay client 240 for sending the multi-screen video stream.

[0139] In an optional embodiment, the transit client 240 obtains a multi-screen video stream by splicing the live screen corresponding to the live video stream and the commentary screen corresponding to the commentary video stream.

[0140] In the transfer client 240, the live video stream and the commentary video stream are separated to obtain the live screen corresponding to the live video stream and the commentary screen corresponding to the commentary video stream; the live screen and the commentary screen are spliced ​​based on the video track to obtain a multi-screen video stream. Schematically, the video stream includes video data and audio data. The video data is used to indicate the data corresponding to the video screen frame in the video stream; the audio data is used to indicate the data corresponding to the audio content in the video stream.

[0141] Illustratively, with respect to the live video stream generated when the game competition event A is broadcast live, it includes not only the video picture content (video data) when the game competition event A occurs, but also the audio content (audio data) when the game competition event A occurs, such as: the voices of the contestants, the background sounds of the game, the cheers of the audience, etc.

[0142] Illustratively, for the commentary video stream generated for the commentary of game event A, it includes not only the commentary screen content (video data) when the commentator is commenting on game event A, but also the audio content (audio data) when commentating on game event A, such as: the commentary voice of the commentator, the background sound of the commentary environment, etc.

[0143] Schematically, the video track is used to indicate that the live video stream and the commentary video stream are aligned with each other using the timestamp as the alignment standard, that is, the time axes corresponding to the live video stream and the commentary video stream are aligned to determine multiple timestamps on the time axis, and the video images corresponding to the live video stream and the commentary video stream at each timestamp are determined; the video images corresponding to the live video stream and the commentary video stream at the same timestamp are spliced ​​to obtain the spliced ​​images corresponding to each timestamp; based on the timing relationship between multiple timestamps on the time axis, the spliced ​​images corresponding to different timestamps are combined to obtain a multi-image video stream with a timing relationship.

[0144] In an optional embodiment, in the transfer client 240, the live video stream and the commentary video stream are separated to obtain audio data corresponding to the live video stream and the commentary video stream respectively; and multiple audio data are sent to the first cloud server 220 through the third push stream address. The audio data includes the corresponding audio identifier.

[0145] Illustratively, the third streaming address is used not only to send a multi-screen video stream, but also to send a plurality of audio data, where the audio data includes live audio data corresponding to the live video stream and commentary audio data corresponding to the commentary video stream.

[0146] First cloud server 220

[0147] In the first cloud server 220, a multi-image video stream sent by the transit client 240 via the third streaming address is received.

[0148] The first cloud server 220 stores a correspondence between the third push stream address and the third pull stream address; the third pull stream address is an address configured by the production client 410 for pulling the multi-screen video stream. Schematically, the third push stream address and the third pull stream address are both addresses associated with the multi-screen video stream. After the transit client 240 sends the multi-screen video stream through the third push stream address, the first cloud server 220 determines the production client 410 corresponding to the third pull stream address based on the relationship between the third push stream address and the third pull stream address, thereby determining the sending direction of the multi-screen video stream.

[0149] In an optional embodiment, the first cloud server 220 is further configured to receive multiple audio data sent by the transit client 240 through the third streaming address, wherein the multiple audio data include live audio data corresponding to the live video stream and commentary audio data corresponding to the commentary video stream.

[0150] Make client 410

[0151] Illustratively, the production client 410 is implemented as at least one of a variety of clients such as a mobile client, a computer client, a tablet client, a car client, and a radio client.

[0152] In the production client 410, the multi-screen video stream is pulled from the first cloud server 220 through the third stream pulling address.

[0153] Illustratively, a third streaming address is configured in the production client 410, so that when the first cloud server 220 sends a multi-screen video stream based on the relationship between the third streaming push address and the third streaming address, the production client 410 pulls the multi-screen video stream from the first cloud server 220 through the third streaming address to determine the video picture obtained by splicing the live video stream and the commentary video stream.

[0154] The control signal is sent to the transfer client 240 based on the multi-screen video stream. Optionally, the production client 410 is a client controlled by the production staff. The production client 410 generates a control signal by receiving the control operation of the production staff, and sends the generated control signal to the transfer client 240, and adjusts the control signal with the help of the transfer client 240.

[0155] Illustratively, the production client 410 is implemented as a computer client. After the production client 410 receives the multi-image video stream, the multi-image video stream is displayed on the screen of the computer client.

[0156] Optionally, the production staff determines the display conditions of the live video stream and the commentary video stream through the multi-screen video stream, such as: the live video stream accounts for a larger proportion in the multi-screen video stream and has brighter colors; the commentary video stream accounts for a smaller proportion in the multi-screen video stream and has a noisy background, etc. And based on the display conditions, the production staff receives a manual picture adjustment operation to generate a control signal. For example: receiving an adjustment operation to enlarge the live video stream, and generating a corresponding control signal to instruct the transit client 240 to adjust the display effect of the live video stream in the multi-screen video stream, etc.

[0157] Optionally, the production staff adds graphic effects, filter effects, creates graphic posters, etc. to the multi-screen video stream, and generates a control signal based on the adjustment process to instruct the transit client 240 to adjust the display of the multi-screen video stream, etc.

[0158] In an optional embodiment, the production client is configured with a first stream pulling address corresponding to the live video stream.

[0159] In the production client 410, the live video stream is pulled from the first cloud server through the first pull stream address; and the live video stream is played. Schematically, the production client 410 is implemented as a computer client, and after the production client 410 receives the live video stream, the live video stream is displayed on the screen of the computer client.

[0160] Optionally, the production staff determines the reception status of the live video stream through the live video stream, such as: the live video stream is stuck, the live video stream is played smoothly, etc. Optionally, the reception information is manually marked based on the reception status, so as to generate a control signal. For example: double-clicking the live video stream indicates that the live video stream is stuck, and generates a corresponding control signal to instruct the transit client 240 to adjust the reception of the live video stream, etc.

[0161] In an optional embodiment, the production client 410 is also used to receive the picture adjustment control operation of the production staff and generate effect control data.

[0162] In the production client 410, a picture effect control operation on at least one video stream in the multi-picture video stream is received, and effect control data corresponding to at least one video picture is generated.

[0163] The effect control data is data obtained by configuring the video pictures of the multi-picture video stream.

[0164] Illustratively, when the production client adjusts multiple video streams based on the multi-picture video streams and audio data, the effect control data is generated based on the received picture effect control operation.

[0165] Optionally, the picture effect control operation includes a display adjustment operation for a multi-picture video stream. Schematically, the display adjustment operation is used to indicate the adjustment of the display conditions of different video pictures in the multi-picture video stream. For example, the size of video picture O is increased, and the size of video picture P is decreased; or, graphic effects are added to video picture O, etc.

[0166] In an optional embodiment, an effect adjustment operation on at least one video picture in a multi-picture video stream is received, and effect control data corresponding to the at least one video picture is generated.

[0167] Illustratively, a multi-image video stream includes several video images from multiple video streams, wherein each video stream includes multiple video images. When receiving an effect adjustment operation on at least one video image in the multi-image video stream, an effect adjustment operation on a video image of a certain video stream at a certain moment is received; or an effect adjustment operation on a video image of a certain video stream in a certain continuous time period is received; or an effect adjustment operation on a video image of a certain video stream in a certain video length is received, etc.

[0168] For example: a multi-picture video stream includes video stream A, video stream B and video stream C. When receiving an effect adjustment operation on at least one video picture in the multi-picture video stream, an effect adjustment operation on multiple video pictures (multiple picture frames) of video stream A in the multi-picture video stream at 2 minutes and 34 seconds is received.

[0169] In some embodiments, the effect adjustment operation is an operation implemented by a video producer through a trigger operation. Schematically, the production client 410 is a client controlled by a video producer. After receiving a multi-screen video stream, the video producer adjusts the display effect of at least one video screen in the multi-screen video stream, thereby implementing the effect adjustment operation on at least one video screen in the multi-screen video stream.

[0170] Optionally, based on the received effect adjustment operation, effect control data corresponding to at least one video picture is generated, and the effect control data is used to adjust the at least one video picture.

[0171] Illustratively, if the effect adjustment operation is a display size adjustment operation, display size configuration data corresponding to the display size adjustment operation is generated, so that when the video picture is subsequently adjusted, the display size of the video picture is adjusted.

[0172] In an illustrative manner, a graphic effect adjustment operation for a specified video picture in a multi-picture video stream is received, and graphic effect configuration data corresponding to the specified video picture is generated. The graphic effect configuration data is used to add a graphic effect to the selected specified video picture.

[0173] The graphic effect configuration data includes at least one of filter effect configuration data, graphic introduction configuration data, and graphic animation configuration data. The filter effect configuration data is data generated by configuring filters for a specified video screen.

[0174] The graphic introduction configuration data is data generated for content introduction of the video content in the specified video screen. For example, the production staff introduces the content of the video content in the specified video screen and generates graphic introduction configuration data. For example, the graphic introduction configuration data includes the character introduction of the video character; or the graphic introduction configuration data includes a brief description of the video event title, etc.

[0175] Graphics and text animation configuration data is data generated for configuring graphics and text special effects for a specified video screen, for example: data generated for configuring artistic fonts for a specified video animation; or data generated for configuring special effect logos for a specified video animation, etc.

[0176] Optionally, in response to the presence of multiple production clients 410 receiving the multi-picture video stream sent by the transit client 240, the multiple production clients 410 respectively receive the picture effect control operation on the multi-picture video stream and respectively generate effect control data.

[0177] The multiple production clients 410 are respectively controlled by different video production personnel, and different production clients can receive different effect adjustment operations and generate effect control data corresponding to the different effect adjustment operations.

[0178] For example: production client 1 receives the effect adjustment operation for video screen a and video screen b, thereby generating first effect control data obtained by adjusting video screen a and video screen b; production client 2 receives the effect adjustment operation for video screen t and video screen j, thereby generating second effect control data obtained by adjusting video screen t and video screen j, and so on.

[0179] In an optional embodiment, the production client 410 is further used to adjust the audio data.

[0180] In the production client 410, audio data is obtained from the first cloud server 220 through the third streaming address; an audio selection operation among multiple audio data is received, and an audio control signal is sent to the transit client.

[0181] The audio control signal includes an audio identifier corresponding to at least one selected audio data.

[0182] Illustratively, an audio selection operation among a plurality of audio data is received, at least one selected audio data is used as background audio data corresponding to a multi-picture video stream, and an audio identifier corresponding to the background audio data is used as an audio control signal.

[0183] Optionally, the audio control signal and the effect control data are sent to the relay client 240 .

[0184] Transfer Client 240

[0185] Supplement the content of the above-mentioned transfer client 240.

[0186] In the transit client 240 , a control signal sent by the production client 410 is received.

[0187] Illustratively, after the production client 410 generates the control signal, the control signal is sent to the transit client 240, so that the transit client 240 receives the control signal. A broadcast video stream is generated based on the control signal and the multi-screen video stream.

[0188] Optionally, the control signal includes adjustment information for the multi-picture video stream. After the relay client 240 receives the control signal, the multi-picture video stream is adjusted through the control signal, so that the adjusted video stream is used as the broadcast video stream.

[0189] In an optional embodiment, compared with the production client 410, the transit client 240 is a client with stronger processing capabilities. Therefore, the control signal generated by the production client 410 is transmitted to the transit client 240, and the transit client 240 processes the multi-screen video stream based on the control signal, which is more conducive to improving the accuracy of processing.

[0190] Optionally, the above audio control signal and effect control data are used as control signals sent to the relay client 240, and the control signal provides a basis for adjusting the multi-image video stream for the relay client 240. The relay client 240 adjusts the multi-image video stream based on the audio control signal and effect control data.

[0191] The video images in the multi-image video stream are adjusted based on the effect control data in the control signal, and the audio data used as the background audio of the relayed video stream is determined based on the audio control signal in the control signal, thereby obtaining the relayed video stream.

[0192] In an illustrative manner, after the effect control data is obtained, the multi-image video stream is adjusted by the effect control data. Based on the difference of the effect control data, the multi-image video stream is adjusted differently.

[0193] (I) Effect control data is implemented as display size configuration data

[0194] Optionally, when the effect control data received by the transit client 240 is implemented as display size configuration data, the display size of the specified video screen is adjusted. The display size configuration data is data generated for the display size adjustment operation of the specified video screen in the multi-screen video stream.

[0195] Illustratively, when the display size of the designated video screen is adjusted, it includes: increasing the display size of the designated video screen; or decreasing the display size of the designated video screen, etc.

[0196] Optionally, when the display size of a specified video screen changes, the display sizes of other video screens in the multi-screen video stream may also change. For example: a multi-screen video stream includes a specified video screen and another video screen, and when the display size of the specified video screen is increased, the display size of the other video screen is proportionally reduced; or, when the display size of the specified video screen changes, the display size of other video screens in the multi-screen video stream does not change. For example: a multi-screen video stream includes a specified video screen and another video screen, and when the display size of the specified video screen is increased, the display size of the other video screen does not change, etc.

[0197] (II) Effect control data is implemented as display area configuration data

[0198] Optionally, when the effect control data received by the transit client 240 is implemented as display area configuration data, the display area of ​​the specified video screen is adjusted. The display area configuration data is data generated for the display area adjustment operation of the specified video screen in the multi-screen video stream.

[0199] Illustratively, when the display area of ​​the designated video picture is adjusted, it includes: configuring the designated video picture in the designated display area; or removing the designated video picture from the multi-picture video stream, etc.

[0200] (III) Effect control data is implemented as graphic effect configuration data

[0201] Optionally, when the effect control data received by the transit client 240 is implemented as graphic effect configuration data, the display effect of the specified video screen is adjusted. The graphic effect configuration data is used to edit the display effect of the specified video screen in the multi-screen video stream; or the graphic effect configuration data is used to edit the display content of the specified video screen in the multi-screen video stream.

[0202] The graphic effect configuration data includes at least one of filter effect configuration data, graphic introduction configuration data, and graphic animation configuration data.

[0203] (1) The filter effect configuration data is data generated by configuring a filter for a specified video screen. In an exemplary manner, when the relay client receives the filter effect configuration data and adjusts the multi-screen video stream, the filter corresponding to the filter effect configuration data is added to the specified video screen in the multi-screen video stream, such as rendering the filter effect configuration data in the form of a filter layer onto the specified video screen.

[0204] (2) The graphic introduction configuration data is data generated for content introduction of the video content in the specified video screen. Schematically, when the transit client receives the graphic introduction configuration data and adjusts the multi-screen video stream, the graphic introduction content is inserted into the specified video screen based on the graphic introduction content indicated by the graphic introduction configuration data. For example: the graphic introduction content is taken as a layer and superimposed on the screen layer corresponding to the specified video screen. Optionally, the graphic introduction content is implemented as a character introduction of the video character, and the character introduction of the video character is inserted next to the video character in the specified video screen to assist in understanding the video character, etc.

[0205] (3) Graphics and text animation configuration data is data generated for configuring graphics and text special effects for a specified video screen. Schematically, when the transit client receives the graphics and text animation configuration data and adjusts the multi-screen video stream, the graphics and text logo corresponding to the graphics and text animation configuration data is edited in the specified video screen. For example: the graphics and text animation configuration data is an emoticon package data, and the emoticon package data is added to the specified video screen; or, the graphics and text animation configuration data is a text content, and the text content is added to the specified video screen; or, the graphics and text animation configuration data is a graphic logo, and the graphic logo is added to a specified area in the specified video screen, etc.

[0206] It is worth noting that the above are merely illustrative examples and are not limited to the embodiments of the present application.

[0207] In some embodiments, the video broadcast processing system includes at least one production client 410. That is, the video stream can be produced independently by one production client 410, or the video stream can be produced together by multiple production clients 410. After the multiple production clients 410 generate their respective corresponding effect control data, the multiple production clients 410 send the generated effect control data to the transit client, so that the transit client can globally adjust the multi-screen video stream based on the multiple effect control data.

[0208] Optionally, first effect control data sent by a first production client and second effect control data sent by a second production client are received; in response to a configuration conflict between the first effect control data and the second effect control data, a configuration conflict condition and processing priorities corresponding to the first production client and the second production client are determined; in response to the processing priority of the first production client being higher than the processing priority corresponding to the second production client, the multi-screen video stream is adjusted through the configuration data corresponding to the configuration conflict condition in the first effect control data to obtain a broadcast data stream.

[0209] Optionally, when there are multiple production clients 410 sending their own effect control data to the transit client 240, the spliced ​​data stream is adjusted using the effect control data corresponding to the multiple production clients 410. However, there may be configuration conflicts in the effect control data sent by multiple production clients. The configuration conflict is used to indicate that there is at least one conflict in the adjustment methods of multiple effect control data, for example: different effect control data make opposite adjustments to the same video picture.

[0210] The first production client and the second production client are two schematic clients in the multiple production clients 410. The first effect control data is the effect control data generated by the first production client based on receiving the global adjustment operation; the second effect control data is the effect control data generated by the second production client based on receiving the global adjustment operation.

[0211] Illustratively, the first effect control data generated by the first production client is used to instruct to zoom in on the video picture U in the multi-picture video stream, and to add a graphic mark "XXX" to the video picture T; the second effect control data generated by the second production client is used to instruct to zoom out on the video picture U in the multi-picture video stream, and to add a graphic mark "III" to the video picture P. After receiving the first effect control data and the second effect control data, the transit client adjusts the multi-picture video stream, adds a graphic mark "XXX" to the video picture T in the multi-picture video stream based on the first effect control data, and adds a graphic mark "III" to the video picture P based on the second effect control data.

[0212] However, there is a partial configuration conflict between the first effect control data and the second effect control data, that is, there is a conflict in the configuration of enlarging the video picture U in the multi-picture video stream in the first effect control data and reducing the video picture U in the multi-picture video stream in the second effect control data. Based on the partial configuration conflict, the transit client determines the configuration conflict condition and the processing priorities corresponding to the first production client and the second production client, respectively.

[0213] Among them, the configuration conflict condition is used to indicate that there is data with configuration conflict between the first effect control data and the second effect control data. For example: the configuration conflict condition between the above-mentioned first effect control data and the second effect control data is implemented as "zooming in / out of the video screen in the multi-screen video stream". Among them, the processing priority is used to indicate the priority of applying the effect control data. Schematically, if the processing priority of the first production client is higher than the processing priority of the second production client, the multi-screen video stream is adjusted by the configuration data corresponding to the configuration conflict condition in the first effect control data, and the data corresponding to the configuration conflict condition in the second effect control data is no longer used to adjust the multi-screen video stream.

[0214] For example: there is a conflict between the configurations in the first effect control data requiring to enlarge the video screen U in the multi-screen video stream and the configuration in the second effect control data requiring to reduce the video screen U in the multi-screen video stream, and the configuration conflict condition is determined to be "enlarging / reducing the video screen in the multi-screen video stream". Based on the fact that the processing priority of the first production client is higher than the processing priority corresponding to the second production client, the multi-screen video stream is adjusted through the configuration data corresponding to the configuration conflict condition in the first effect control data, that is, the video screen U in the multi-screen video stream is enlarged through the data content of "enlarging the video screen U in the multi-screen video stream" in the first effect control data; and the data corresponding to the configuration conflict condition in the second effect control data is no longer used to adjust the multi-screen video stream, that is, the video screen U in the multi-screen video stream is no longer reduced through the data content of "reducing the video screen U in the multi-screen video stream" in the second effect control data.

[0215] In an optional embodiment, in response to a configuration conflict between first effect control data and second effect control data, processing priorities corresponding to the first production client and the second production client are determined respectively; in response to the processing priority of the first production client being higher than the processing priority corresponding to the second production client, the multi-screen video stream is globally adjusted through the first effect control data corresponding to the first production client to obtain a broadcast data stream.

[0216] In schematic form, if there is a configuration conflict (full conflict or partial conflict) between the first effect control data and the second effect control data, the multi-picture video stream is globally adjusted by processing the first effect control data with a higher processing priority, and the multi-picture video stream is no longer adjusted using the second effect control data with a lower processing priority.

[0217] It is worth noting that the above are merely illustrative examples and are not limited to the embodiments of the present application.

[0218] In an optional embodiment, the effect control data is also used to select at least one video stream from the multi-picture video stream for display. Optionally, after adjusting the multi-picture video stream based on the effect control data, a broadcast video stream is obtained, in which at least one video flow of the plurality of video streams after adjusting the video pictures of the multi-picture video stream exists, and background audio data to be used as background audio exists.

[0219] In some embodiments, the relay video stream is sent to the viewer terminal for video screen display. Schematically, after the relay video stream is obtained, the relay video stream is sent to the viewer terminal. The viewer terminal is used to receive the relay video stream and display the relay video stream on the screen corresponding to the viewer terminal. For example: after receiving the relay video stream, the relay video stream is rendered on the screen corresponding to the viewer terminal, thereby realizing the process of displaying the relay video on the display client.

[0220] In an optional embodiment, a virtual private network channel is established between the transit client 240 and the production client 410. The production client 410 is further configured to send a control signal to the transit client 240 through the virtual private network channel; the transit client 240 is further configured to receive the control signal sent by the production client 410 through the virtual private network channel.

[0221] In the embodiment of the present application, the production client is used to adjust the multi-screen video stream and generate a control signal, and the control signal is sent to the transfer client; the transfer client adjusts the multi-screen video stream with the control signal to obtain a broadcast video stream that can be displayed by the viewer terminal. By processing the video stream differently by multiple clients, not only the scene limitation of the offline requirement of being in the same scene is effectively overcome, but also the display effect of the broadcast video stream is enriched, and the audience's experience is improved.

[0222] In an optional embodiment, the video broadcast processing system provided by the present application is described, taking the application of the system in a communication network as an example. Schematically, there are a streaming client, a first cloud server, a commentary client, and a transfer client in the communication network. Taking the transfer client that processes the live video stream and the commentary video stream as an example, the process of video broadcast processing by the transfer client is described. Schematically, Figure 5 As shown, the above video broadcasting process is implemented as the following steps 510 to 540.

[0223] Step 510: Pull the live video stream from the first cloud server through the first stream pulling address.

[0224] The first streaming address is an address configured by the transit client for receiving the live video stream.

[0225] The first cloud server is used to store the correspondence between the first streaming address and the first streaming address corresponding to the broadcast event, and the first streaming address is an address configured in the streaming client for sending a live video stream.

[0226] Optionally, the live video stream is pulled from the first cloud server via a secure and reliable transmission protocol SRT. For example, the event video SRT stream is pulled from the first cloud server via SRT.

[0227] Step 520: Pull the commentary video stream from the first cloud server through the second stream pull address.

[0228] The second stream pulling address is an address configured by the relay client for receiving the commentary video stream.

[0229] In addition, the first cloud server also stores the correspondence between the second push stream address and the second pull stream address corresponding to the commentary video stream, and the second push stream address is the address configured in the commentary client for sending the commentary video stream. Optionally, the commentary video stream is pulled from the first cloud server via SRT. For example: Pull the event commentary video SRT stream from the first cloud server via SRT.

[0230] Step 530, splicing the live video stream and the commentary video stream to obtain a multi-screen video stream.

[0231] Illustratively, the live video stream and the commentary video stream include video data and audio data respectively.

[0232] In an optional embodiment, the live video stream and the commentary video stream are respectively separated to obtain live video data and live audio data corresponding to the live video stream; and the commentary video data and commentary audio data corresponding to the commentary video stream are obtained. The video data is used to indicate the data corresponding to the video picture frame in the video stream; the audio data is used to indicate the data corresponding to the audio content in the video stream. Optionally, after aligning the time axes corresponding to the live video stream and the commentary video stream, the live video data and the commentary video data are spliced ​​to obtain a multi-picture video stream.

[0233] Step 540: Generate a broadcast video stream based on the multi-screen video stream, and push the broadcast video stream to the viewer terminal for live screen display.

[0234] Optionally, the multi-screen video stream is directly used as a broadcast video stream and pushed to the viewer terminal for live screen display.

[0235] In an optional embodiment, the multi-screen video stream is sent to the first cloud server via a third streaming address.

[0236] Among them, the third streaming address is the address configured by the transit client for sending multi-screen video streams; the correspondence between the third streaming address and the third streaming address is stored in the first cloud server, and the third streaming address is the address configured by the production client for receiving multi-screen video streams.

[0237] In an optional embodiment, a control signal sent by a production client is received.

[0238] The production client is used to pull the multi-screen video stream from the first cloud server through the third stream pulling address.

[0239] In an optional embodiment, a relay video stream is generated based on the control signal and the multi-view video stream.

[0240] Optionally, the multi-screen video stream is sent to a production client, which controls and adjusts the multi-screen video stream to generate a broadcast video stream, and pushes the broadcast video stream to a viewer terminal for live screen display.

[0241] In an embodiment of the present application, with the help of a transit client, the live video stream and the commentary video stream are spliced ​​based on the timeline to obtain a multi-screen video stream with a time sequence relationship. This is beneficial for the production staff to make more targeted adjustments to the video screen based on the multiple video screens in the multi-screen video stream after receiving the multi-screen video stream, thereby effectively improving the efficiency of video stream processing.

[0242] In an optional embodiment, the video broadcast processing system provided by the present application is described, taking the application of the system in a communication network as an example. Schematically, there are a streaming client, a first cloud server, a commentary client, and a relay client in the communication network. The process of performing video broadcast processing on the commentary client is described. Schematically, Figure 6 As shown, the above video broadcasting process is implemented as the following steps 610 to 630.

[0243] Step 610: Pull the live video stream from the first cloud server through the first stream pulling address.

[0244] Among them, the first stream pulling address is the address configured by the commentary client for receiving the live video stream; the first cloud server is used to store the correspondence between the first stream pushing address corresponding to the broadcast event and the first stream pulling address, and the first stream pushing address is the address configured in the stream pushing client for sending the live video stream.

[0245] The live video stream is a video stream generated by live broadcasting of a relay event.

[0246] In an illustrative manner, a broadcast event is a pre-determined live event to be broadcasted by video, for example, a broadcast event is a game event, a sports event, a classroom teaching event, etc.

[0247] Optionally, the live video stream is a video stream produced by the push client, and is used to indicate a video stream generated for live broadcasting of a relay event, that is, the push client is used to perform a video stream production process for a video stream related to the relay event.

[0248] In some embodiments, the commentary client receives a live video stream sent by a push client; or, the commentary client receives a live video stream that is rebroadcasted by a cloud server from a live video stream sent by a push client, etc.

[0249] Step 620, obtaining the commentary video stream.

[0250] Optionally, the explanation client is controlled by the explanation staff, and the explanation production process is realized by receiving the explanation content of the explanation staff and producing the explanation content.

[0251] In an exemplary embodiment, the explanation content includes at least one of the explanation audio content and the explanation video content. Optionally, the explanation video content is produced by a device with a camera and a mobile client, etc., and the explanation audio content is produced by a device with an audio collection function, such as a microphone and a mobile client, etc.

[0252] For example: the commentary client is implemented as a computer client. After the commentary client receives the live video stream, the live video stream is displayed on the screen of the computer client, and the commentator provides video commentary on the live video stream. During the video commentary process, the microphone device connected to the computer client collects the commentary audio content, and the computer client connected to the microphone device produces the commentary audio content, thereby obtaining commentary audio data; in addition, the camera device configured by the computer client collects the commentary video content, and the computer client produces the commentary video content, thereby obtaining commentary video data. The commentary audio content and the commentary video content constitute the commentary video stream.

[0253] Therefore, the commentary video stream is a video stream produced based on the live video stream.

[0254] Step 630: Send the commentary video stream to the first cloud server via the second streaming address.

[0255] The second push stream address is an address configured in the commentary client for sending the commentary video stream; the first cloud server is also used to store the correspondence between the second push stream address and the second pull stream address corresponding to the commentary video stream, and the second pull stream address is an address configured in the transit client for receiving the commentary video stream.

[0256] In some embodiments, the commentary video stream is sent to the relay client via a secure and reliable transmission protocol SRT.

[0257] Schematically, the transit client is used to receive multiple video streams related to a specified live event, the video streams including video data and audio data; the multiple video streams include a live video stream generated by live broadcasting of the specified live event, a commentary video stream generated by commenting on the specified live event (i.e., a video stream obtained based on the live video stream), and other video streams.

[0258] Indicative, such as Figure 7 As shown, the following description is given by taking the case where the multiple video streams sent to the transit client are live video streams and commentary video streams, the SRT technology is used to send the multiple video streams, and the live event is designated as a competition event as an example.

[0259] The streaming client 710 that produces the live video stream sends the event public signal SRT stream (live video stream sent using SRT technology) to the cloud server 720 in zone C. In addition, the streaming client 710 will also send the event public signal SRT stream to other zones.

[0260] After the cloud server 720 in the C zone receives the event common signal SRT stream, the event common signal SRT stream will be broadcast to the commentary client 730, the transfer client 740 and the production client 750. After the commentary client 730 receives the event common signal SRT stream, it receives the commentary content of the commentator through the camera and microphone, and produces the commentary content, so that the produced commentary video stream is broadcasted to the transfer client 740 via the cloud server 720 in the C zone. After receiving multiple video streams (i.e., the event common signal SRT stream and the commentary video stream), the transfer client 740 produces the event common signal SRT stream and the commentary video stream, for example, based on the timeline, the event common signal SRT stream and the commentary video stream are spliced ​​to obtain a multi-screen video stream, and the audio data corresponding to the multi-screen video stream and the multiple video streams (i.e., the event audio corresponding to the event common signal SRT stream and the commentary audio corresponding to the commentary video stream) are sent to the production client 750, that is: Figure 7 The process shown in FIG. 7 is that the transit client 740 sends the multi-screen monitoring SRT stream to the production client 750 .

[0261] Indicative, such as Figure 8 FIG. 1 is a schematic diagram of the explanation interface, which includes an explanation screen a810 corresponding to the commentator A and an explanation screen b820 corresponding to the commentator B.

[0262] In an optional embodiment, the production client and the commentary client implement the video commentary process in which the video production staff assists the commentator through a voice call process.

[0263] Optionally, the production client is also used to send voice prompt information to the transfer client, and the voice prompt information is used to prompt to adjust the interpretation content.

[0264] In a live video broadcast, the video producer may need to remind the commentator to explain a certain part of the content; or the video producer may need to remind the commentator how to operate; or the video producer may need to remind the commentator to explain the signal transmission problem, etc. Based on the above needs, the video producer can communicate with the commentator through a voice call.

[0265] There is voice prompt information during the voice call, and the voice prompt information includes the voice content of the video producer prompting the commentator, that is, the voice prompt information is used to prompt the commentator to adjust the commentary content.

[0266] The transfer client is also used to receive the voice prompt information sent by the production client and send the voice prompt information to the interpretation client. The interpretation client is also used to receive the voice prompt information and adjust the interpretation content based on the voice prompt information. Optionally, the interpretation client displays the voice prompt information to prompt the interpreter to adjust the interpretation content.

[0267] Indicative, such as Fig. 9 As shown, it is a flowchart of the remote voice call process. It includes two production clients (production client 910 and production client 920), and the two production clients can simultaneously send instant call audio (i.e., voice call process) to the transfer client 930 via the server unity server; or, any production client (such as: production client 910 or production client 920) sends instant call audio to the transfer client 930 via the server unity server, and the server unity server will send the received instant call audio to the other production client (such as: production client 920 or production client 910).

[0268] Optionally, a video broadcasting program and a video playing program are installed on the production client.

[0269] The video broadcast program is an application program for video broadcast production, which is installed on the transfer client and the production client, and the video broadcast program is used to perform the video stream broadcast process and the video stream processing process.

[0270] The video player is an application for video and audio playback, and the video player is installed on the production client and the commentary client. Schematically, on the production client, the video broadcast program and the video player are connected via a VBCable virtual audio line.

[0271] Alternatively, if Fig. 9 As shown, the instant call audio is received by the video broadcast program installed on the relay client, and the instant call audio is played and relayed by the video playback program connected to the video broadcast program through the VBCable virtual audio line. In addition, the video playback program (relay server) sends the instant call audio to the commentary client via the cloud server; optionally, the commentary client receives the PGM video auxiliary monitoring stream relayed by the cloud server and sent by the production client.

[0272] The commentary client displays the voice prompt information carried by the instant call audio. The commentator obtains the commentary content based on the PGM video auxiliary monitoring stream and the voice prompt information. The commentary client produces the commentary content into a commentary video stream, which is fed back to the production client through the instant call audio, thereby realizing the remote instant call process.

[0273] In an optional embodiment, the production client sends voice prompt information to the interpretation client, and the voice prompt information is used to prompt the interpretation content to be adjusted; the interpretation client receives the voice prompt information; and the interpretation content is adjusted based on the voice prompt information. Fig.10 As shown, it is a schematic diagram of the configuration interface of the video player installed on the transfer client. It includes a setting function bar 1010, and the setting function bar 1010 includes multiple setting function options. Based on the trigger operation of the setting function option-video option 1020, the configuration content under the video option is displayed, as shown on the right side of the interface. Among them, by triggering the selection device option 1021, the device that can be used is selected. In addition, the configuration interface of the video option 1020 also displays a video screen preview area 1022; and other configuration contents, such as: selecting whether to have a mirror effect; selecting whether it is high-definition image quality; selecting whether to perform video noise reduction, etc.

[0274] In an optional embodiment, based on Fig.10 The configuration interface diagram shown in the figure shows the trigger operation of the audio option in the setting function option. Fig.11 The configuration contents under Audio Options are shown.

[0275] By triggering the speaker option 1111, a usable speaker device is selected; by triggering the microphone option 1112, a usable microphone device is selected. In addition, a volume adjustment area is also displayed on the configuration interface corresponding to the audio option, which is used to adjust the volume of the speaker and the microphone respectively.

[0276] In some embodiments, Figure 8 The following describes the settings of the Unity server. Fig.12 As shown, it is a schematic diagram of the setting interface of the server unity server, wherein the current display group is selected first, that is: first, the current display group (currently displayed group) option 1210 is selected, such as: selecting the test group as the current display group; secondly, multiple use objects under the use object name option 1220 are selected, such as: selecting the production client (production host); in addition, the test channel 1230 of the selected use object is adjusted, wherein the T under the test channel 1230 represents "talking", and the L under the test channel 1230 represents "listening", such as: selecting the "T" and "L" of the production client, then the audio content can be received and sent with the help of the server unity server.

[0277] In the embodiment of the present application, the commentary client also included in the video broadcast processing system is described. The commentary client can be based on the received live video stream, by receiving the commentary content of the live video stream by the commentator, and use the commentary video stream generated based on the commentary content as at least one of the multiple video streams, so as to facilitate the transit client to also broadcast the commentary video stream in a video stream processing manner and in an appropriate form to the display client, so as to present it to the user object, enrich the broadcast form of the video broadcast, and improve the stickiness of the user object; in addition, the commentary client can also instruct the commentator to adjust the commentary content based on the voice prompt information, enriching the accuracy of the commentary content.

[0278] In an optional embodiment, the video broadcast processing system provided by the present application is described, taking the application of the system in a communication network as an example. Schematically, there are a streaming client, a first cloud server, a commentary client, and a transfer client in the communication network. The process of performing video broadcast processing on the production client is described. Schematically, Fig.13 As shown, the above video broadcast processing process is implemented as the following steps 1310 to 1330.

[0279] Step 1310: Pull the multi-screen video stream from the first cloud server through the third stream pulling address.

[0280] Among them, the first cloud server is used to receive the multi-screen video stream sent by the transit client through the third push stream address. The first cloud server stores the correspondence between the third push stream address and the third pull stream address. The third pull stream address is the address configured by the production client for pulling the multi-screen video stream.

[0281] The relay client is used to send the multi-screen video stream to the first cloud server through the third streaming address, and the third streaming address is an address configured by the relay client for sending the multi-screen video stream.

[0282] The multi-image video stream is a video stream obtained based on the live video stream and the commentary video stream. In an optional embodiment, the relay client and the production client realize the relay association relationship for the relay event through a video relay program.

[0283] In an illustrative manner, the video broadcast program is an application program for video broadcast production, which is installed on the relay client and the production client, and the video broadcast program is also installed on the production client. The video stream broadcast process and the video stream processing process are performed through the video broadcast program.

[0284] Optionally, the video broadcast production process is performed offline through a director station installed with a video broadcast program. When the video broadcast processing process is performed through the video broadcast processing system provided by the embodiment of the present application, a communication relationship between the production client and the director station is established through remote control.

[0285] Optionally, the production client remotely controls the director's station by building a VPN service across the public network. Schematically, the director's station and the production client are both installed with the remote control software Companion, which uses the remote control software Companion to control the server to transmit the TCP / IP control protocol, so that the video production staff can remotely control the functions of the director's station installed with the video broadcast program in an area outside the director's station by operating the production client, so that the director's station loads the same video production project as the production client. Fig.14 As shown, after the production client is connected to the physical control panel, the video production process based on remote control can begin.

[0286] Indicative, such as Fig.14As shown, it is a schematic diagram of the interface of the video production process through remote control displayed on the production client. The function selection bar includes multiple function options such as connection and button. The connection function option includes a connection option 1410. The connection option 1410 includes at least one client, which is used to indicate the client that can be remotely controlled, such as: including the director station 1420. The status 1430 is used to indicate the remote control status corresponding to different clients at present, such as: the status 1430 corresponding to the director station 1420 is "OK", which means that the current director station 1420 is connected to the production client, and the production client can remotely control the director station 1420; the active status 1430 is used to indicate the operation status that can be performed on the director station 1420. For example: delete the director station 1420 from the connection option 1410; or disconnect the remote connection status between the production client and the director station 1420, etc.

[0287] Step 1320: Generate a control signal based on the multi-image video stream.

[0288] Optionally, a picture effect control operation on at least one video stream in a multi-picture video stream is received, and effect control data corresponding to at least one video picture is generated, where the effect control data is a representation of a control signal.

[0289] The effect control data is data obtained by configuring the video pictures of the multi-picture video stream.

[0290] In an optional embodiment, a picture effect control operation on at least one video picture in a multi-picture video stream is received, and effect control data corresponding to a timestamp of the at least one video picture is generated.

[0291] Indicatively, the video production staff used Fig.14 The video production interface shown in the figure is used to produce the video. Fig.14 As shown, when the button function option in the function selection bar is triggered, the following is displayed Fig.15 The interface content under the button function options shown in the figure can enable video producers to control the picture effects of multi-screen video streams through multiple button controls under the button function options.

[0292] Optionally, based on the fact that the production client receives the multi-image video stream and audio data sent by the transit client, the following is presented on the production client interface: Figure 3 Based on the multi-screen video stream, the video producer can Fig.15 The interface content under the button function options shown performs picture effect control operations on multi-screen video streams, thereby generating effect control data corresponding to the picture effect control operations on the production client.

[0293] like Fig.15 As shown, different picture effect control operations can be implemented through multiple trigger buttons 1510 under the button function options. For example, after the "COUNT DOWN" button is triggered, the countdown function can be implemented.

[0294] like Fig.16 As shown in FIG. 1 , it is an enlarged schematic diagram of multiple trigger buttons. By triggering the page turning control 1610, more trigger buttons can be found to achieve more screen effect control operations. Fig.17 As shown in FIG. 1 , a schematic diagram of a physical button of multiple trigger buttons is shown, wherein the multiple trigger buttons 1710 included in the physical button are as shown in FIG. Fig.15 The multiple trigger buttons under the button function options shown are corresponding. That is, the functions that can be realized by the physical button can be realized by Fig.15 The interface shown is implemented.

[0295] In an optional embodiment, a display size adjustment operation for a specified video picture in a multi-picture video stream is received, and display size configuration data corresponding to the specified video picture is generated.

[0296] In an optional embodiment, a graphic effect adjustment operation for a specified video frame in a multi-frame video stream is received, and graphic effect configuration data corresponding to the specified video frame is generated. The graphic effect configuration data includes at least one of filter effect configuration data, graphic introduction configuration data, and graphic animation configuration data.

[0297] Among them, the filter effect configuration data is data generated by configuring filters for the specified video screen; the graphic introduction configuration data is data generated by introducing the content of the video content in the specified video screen; the graphic animation configuration data is data generated by configuring graphic special effects for the specified video screen. Schematically, the graphic effect configuration data is implemented as graphic introduction configuration data as an example to illustrate that the video production staff intends to introduce the video characters A and B in the video screen with graphics.

[0298] like Fig.18As shown, it is a schematic diagram of the interface for adjusting the graphic effect of video character A, wherein the basic information 1810 (such as image, name, etc.) of video character A is displayed on the right side of the interface, and the virtual battle information 1820 of video character A and video character B is included on the left side of the interface. Based on the basic information 1810 of video character A and the virtual battle information 1820 of video character A and video character B, the video production staff produces a content introduction poster obtained by introducing the video character A with graphics, and generates graphic introduction configuration data corresponding to the content introduction poster; similarly, in another schematic diagram of the interface for adjusting the graphic effect of video character B, the basic information of video character B and the virtual battle information of video character A and video character B are displayed. Based on the basic information of video character B and the virtual battle information of video character A and video character B, the video production staff produces a content introduction poster obtained by introducing the video character B with graphics, and generates graphic introduction configuration data corresponding to the content introduction poster.

[0299] Optionally, the video production staff merges the content introduction poster corresponding to the video character A and the content introduction poster corresponding to the video character B into one content introduction poster, and generates graphic introduction configuration data corresponding to the content introduction poster, so that when the content is broadcast, the merged content introduction poster is displayed, such as Fig.19 The video characters A 1910 and B 1920 are shown. Different video characters also include game information selected by the video production staff, such as average damage information (ACS), average kill information (AVG KILLS), damage information (DMG / RND), etc.

[0300] In an optional embodiment, a target music program logged into the target music account is installed on the production client, wherein the target music program is used to synchronize the currently played music to at least one other client logged into the target music account.

[0301] Optionally, a synchronization instruction is received to synchronize the currently playing music to at least one client logged in to the target music account; based on the synchronization instruction, the currently playing music is synchronized to a transit client logged in to the target music account.

[0302] The currently playing music is used as the background audio data corresponding to the multi-screen video stream, and the transfer client is installed with a target music program that logs in to the target music account. When the currently playing music is used as the background audio data corresponding to the multi-screen video stream, you can choose to delete the selected audio data and use the currently playing music as the background audio data; or you can choose to superimpose the currently playing music as the background audio data in addition to the selected audio data, that is, include multiple audio data as the background audio data. Schematically, as Fig. 20 As shown, it is a schematic diagram of the interface of the target music program. In the interface corresponding to the target music program, a synchronous playback control 2010 is included. Based on the triggering operation of the synchronous playback control 2010 by the video production personnel, the process of generating the synchronization instruction is realized. Since the synchronization instruction is used to instruct to synchronize the currently played music to the transfer client logged in with the target music account, after the video production personnel triggers the synchronous playback control 2010, the music played on the production client will be synchronized to the transfer client, thereby realizing the process of music synchronization.

[0303] Indicative, such as Fig.21 As shown, it is a schematic diagram of the video production interface displayed by the video producer during the video production process, which includes the event video screen 2110, the commentary video screen 2120 and the production area 2130. The video producer adjusts the different video screens in the video stream through the production area 2130 and generates effect control data corresponding to the production client.

[0304] Step 1330: Send a control signal to the transit client.

[0305] Illustratively, after the production client generates effect control data corresponding to the screen effect control operation, the effect control data is sent to the transit client.

[0306] In an optional embodiment, the video broadcast processing system further includes a second cloud server. The second cloud server corresponds to a graphic configuration address. The second cloud server is used to collect event data corresponding to the broadcast event. The production client is also used to access the second cloud server through the graphic configuration address and display a graphic configuration webpage, which includes event data; based on the event data, the graphic configuration operation in the graphic configuration webpage is received to generate a graphic configuration result.

[0307] Illustratively, the second cloud server is a cloud server different from the first cloud server, and is a cloud server pre-selected for performing the graphic configuration process.

[0308] The graphic and text configuration result includes event data; and the graphic and text configuration result is sent to the second cloud server.

[0309] The transit client is also used to receive the image and text configuration result from the second cloud server; and perform image and text configuration on the multi-screen video stream based on the image and text configuration result.

[0310] Indicative, such as Fig. 22As shown, the designated live event is a game event. The cloud server 2210 corresponding to the broadcast event sends the real-time video stream of the game to the second cloud server, and the second cloud server receives the graphic effect configuration data generated by the video production staff through the HTML5 online graphic packaging technology, and sends the graphic effect configuration data to the transfer client.

[0311] Optionally, the respective data are sent with the help of other cloud servers. For example, background audio data generated by the video producer through music synchronization with the target music program is received with the help of a music cloud server, and the background audio data is sent to the transit client; in addition, effect control data generated by the video producer through picture effect control operations on the control panel is received with the help of an effect cloud server, and the effect control data is sent to the transit client, etc.

[0312] In the embodiment of the present application, the processing of the multi-screen video stream by the production client is described. The production client can receive the picture effect control operation of the multi-screen video stream based on the differential adjustment of the video screen by the production personnel, thereby generating effect control data, which indicates the adjustment content such as the video screen that needs to be adjusted and the adjustment method in the multi-screen video stream, so that after sending the effect control data to the transit client, the transit client can make more targeted adjustments to the multi-screen video stream based on the effect control data, enrich the presentation method of the broadcast video stream that needs to be broadcast, and improve the production efficiency.

[0313] In an optional embodiment, the video broadcast of a game event is illustrated by the above-mentioned video broadcast processing system. The processing scheme for game broadcasting using the above-mentioned system is called "an online distributed cloud production and broadcasting scheme". Schematically, the video broadcasting process mainly uses a high-performance PC host set up in a stable temperature and power network environment such as an equipment room, and installed with a video broadcasting program (virtual director software) as the production core, and a pre-set cloud server as the transmission and broadcasting core for public signal forwarding (SRT technology) and broadcast (RTMP technology) signal distribution, and an independently set up VPN network as the control core. Two online producers and two online commentators can use conventionally configured computers in their respective homes to complete an online distributed cloud production process for the public signal broadcast of the event. Online distributed cloud production is completed by a combination of multiple solutions based on the content and process of the public signal production of the event. The following is an example of broadcasting a foreign game event. The solution includes the following four parts: (i) production signal SRT stream forwarding; (ii) broadcast signal RTMP stream distribution; (iii) production content remote control solution; (iv) remote instant call solution.

[0314] (I) Production signal SRT stream forwarding

[0315] like Fig.18 The figure shows a flow chart of forwarding and producing signals using the SRT technology. The contents involved are introduced as follows.

[0316] (1) Event public signal SRT stream

[0317] Indicatively, the public signal of the foreign game event is produced by the International Signal Production Center, pushed to the cloud in the form of SRT stream and forwarded to each competition area. Competition area C receives the event public signal SRT stream through the foreign node corresponding to the cloud server, and then forwards it to the country. The domestic Chinese stream production team receives the event public signal SRT stream through the video broadcast program and produces it. Usually there are at least 2 channels of event public signals and they back up each other. The relay client (the production host with the video broadcast program installed) receives it and uses it for program production signals; the commentator (commentary client) receives the event public signal SRT stream through the video broadcast program and mainly uses it for monitoring and comparison comments; the production staff (production client) receives the event public signal SRT stream mainly to monitor the status of the event public signal SRT stream. If there is any problem, it will be promptly fed back to the person in charge of the public signal distribution center so that the signal can be adjusted. The content of the event public signal mainly includes at least one of the following forms: pre-match player entry and preparation screen, game scene screen, ban or selection (Ban&Pick) screen, game match screen, instant replay in the game match, and post-match highlight replay, etc. Indicatively, such as Fig.23 As shown, it is a schematic diagram of the interface of the public signal content of the event. Interface 2310 is a schematic diagram of the player entry interface before the game; interface 2320 is a schematic diagram of the preparation screen; interface 2330 is a schematic diagram of the game screen; interface 2340 is a schematic diagram of the wonderful replay screen after the game, etc.

[0318] Indicative, such as Fig.24 , which is a schematic diagram of an interface for receiving and processing the content of a public signal of a competition, including a display area 2410 for the content of the public signal of the competition and a processing area 2420 for processing the content of the public signal of the competition.

[0319] (2) Commentator camera signal SRT stream

[0320] Indicatively, the commentator's image and sound are also forwarded to the relay client via the cloud server in the form of SRT stream through the video broadcast program, and then the two commentators are allowed to appear in the same frame through multi-image synthesis through the relay client. Figure 8 The explanation screen shown.

[0321] (3) Multi-screen monitoring signal SRT stream

[0322] Multi-screen monitoring is mainly provided to production personnel, such as broadcast directors and technicians, to instruct auxiliary production personnel to monitor all signal images at the same time. Optionally, after the multi-screen monitoring signal SRT stream is produced by the relay client, it is pushed to the cloud server in the form of a multi-screen monitoring signal SRT stream, and then forwarded to the production client by the cloud server, so that the production personnel can use the computer client to open the network stream through the video playback program to receive it. Figure 2 As shown, it is the multi-screen video stream corresponding to the multi-screen monitoring signal.

[0323] (II) RTMP streaming distribution of broadcast signals

[0324] In an illustrative manner, after the relay client completes the production and obtains the broadcast video stream, the broadcast video stream is distributed through the RTMP technology. Fig.25 As shown, the PGM broadcast signal (broadcast video stream) finally output by the relay client will be pushed to other cloud servers (i.e., cloud servers that receive video streams) in the form of RTMP streams through the built-in push broadcast function of the video broadcast program installed on the relay client 2510, and then distributed to various foreign live broadcast platforms for broadcast via the foreign nodes corresponding to other cloud servers.

[0325] 3. Remote control solution for content production

[0326] (1) Remote control of virtual director station

[0327] Schematically, remote control of the director's desk is achieved by building a VPN service across the public network, using the Companion control server to transmit the TCP / IP control protocol, allowing production staff to remotely control the director's desk functions at home. At the same time, Companion is also installed on the production staff's computer, and the same Companion project as on the transit client is loaded. Remote control can be achieved by connecting to the Streamdeck physical control panel.

[0328] (2) HTML5 online graphics and text packaging remote broadcast control

[0329] Graphics and text packaging plays an important role in the broadcasting of game events. It provides important support for enriching the production content and presenting the competition data. Usually, a high-performance graphics and text packaging host is required for offline production. In the above solution, HTML5 online graphics and text packaging is adopted. The production staff can operate and control the Barracks online graphics and text packaging HTML5 background page at home. The Barracks cloud server collects the real-time data on the game event server and presents it to the online graphics and text packaging HTML5 page foreground. Then the transfer client receives the HTML5 web page signal source as the broadcast signal.

[0330] (3) Remote control of music playback

[0331] Indicatively, music, as an indispensable part of the broadcast of game events, plays a role in setting off the atmosphere of the event and improving the auditory experience of the audience during the production of the game content. Optionally, a music application with a music synchronization playback function is used, which can achieve the function of simultaneously playing music on multiple terminal devices with the same target music account. It only needs to install the music playback program on the transfer client and the production client (such as a computer client) corresponding to the production personnel, log in to the same target music account, turn on the music playback function of the music playback program, and remotely operate the music playback.

[0332] (IV) Remote Instant Call Solution

[0333] The instant call function can help the production staff to issue process switching commands in time, and give the commentator oral reminders. Usually, offline production needs to be equipped with professional call matrices and call panels. Optionally, in the above scheme, the call needs of online production can be solved with the help of Unityintercom remote call software, which has the characteristics of low latency, multi-channel and multi-terminal. In addition, in the above scheme, through the VBCable virtual audio line, Unity intercom is connected to the video player routing, so that the commentator can use the video player to assist in monitoring the PGM data board information sent by the transfer client to the video player, while receiving the Unity intercom remote call command from the production staff through the video player, so as to minimize the software learning and operation costs of the commentator and make him more focused on the event commentary.

[0334] In addition, if Fig.26 As shown, it is a schematic diagram of the setting interface on the production client. Among them, based on the triggering operation of the setting control 2610, the setting area 2620 shown on the right is displayed, and based on various setting contents, the contents of receiving the process switching password and forwarding the process switching password are realized.

[0335] In an optional embodiment, if Fig. 27 As shown in FIG. 2 , it is an overall schematic diagram of the video broadcast processing system 2700. The overall flow chart is as follows: Figure 7 The flowchart shown in Fig. 9 The flowchart shown in Fig. 22 The flowchart shown and Fig.25 The flowchart shown.

[0336] In an optional embodiment, if Fig.28As shown, it is a flowchart of the operation sequence in the process of video broadcasting, which includes the overseas production center 2810, the cloud server C zone 2820, the transfer client 2830, the cloud server overseas node 2840, the live broadcast platform 2850 and the target client 2860.

[0337] The overseas production center 2810 forwards the SRT stream related to the game and production to the cloud server C zone 2820; the cloud server C zone 2820 forwards the SRT stream related to the game and production to the transit client 2830; the transit client 2830 broadcasts and pushes the produced video stream through cloud production, that is, broadcasts and pushes the RTMP stream; the cloud server overseas node 2840 distributes the received RTMP stream to the live broadcast platform 2850; thereby, the live broadcast platform 2850 sends it to multiple target clients 2860 in the form of Chinese stream program signals, thereby allowing multiple target clients 2860 to watch the Chinese stream program signals. It is worth noting that the above is only an illustrative example, and the embodiments of the present application are not limited to this.

[0338] To summarize, by processing video streams differently through multiple clients, even if the multiple clients are located in different areas, the video streams can be jointly produced and processed with the help of the broadcast process between the video streams, effectively overcoming the offline scene limitation of requiring the same scene. With the help of different clients, it achieves the characteristics of high flexibility, low labor cost, and low equipment cost, and can be widely used in various broadcast scenarios, improving the efficiency of human-computer interaction.

[0339] Fig.29 The block diagram of the structure of an electronic device 2900 provided by an exemplary embodiment of the present application is shown. The electronic device 2900 may be a portable mobile terminal, and may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, etc. Generally, the electronic device 2900 includes: a processor 2901 and a memory 2902.

[0340] In some embodiments, the electronic device 2900 further includes one or more sensors, including but not limited to: a proximity sensor, a gyroscope sensor, and a pressure sensor. In some embodiments, the electronic device 2900 further includes other components, which can be understood by those skilled in the art. Fig.29 The structure shown in the figure does not constitute a limitation on the electronic device 2900, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0341] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A video broadcast processing system, characterized in that: The system includes a transfer client, a commentary client, a streaming client, a production client and a first cloud server; The first cloud server is used to receive the live video stream sent by the push stream client through the first push stream address, wherein the first cloud server stores a correspondence between the first push stream address and the first pull stream address corresponding to the broadcast event, and the first pull stream address is an address configured by the relay client and the commentary client for pulling the live video stream; The commentary client is used to pull the live video stream from the first cloud server through the first stream pulling address; obtain the commentary video stream; and send the commentary video stream to the first cloud server through the second stream pushing address; The first cloud server is further used to receive the commentary video stream sent by the commentary client through the second push stream address, the first cloud server stores a correspondence between the second push stream address and the second pull stream address, and the second pull stream address is an address configured by the transit client for pulling the commentary video stream; The transfer client is used to pull the live video stream from the first cloud server through the first stream pulling address, and pull the commentary video stream from the first cloud server through the second stream pulling address; splice the live video stream and the commentary video stream to obtain a multi-screen video stream; and send the multi-screen video stream to the first cloud server; The first cloud server is further configured to receive the multi-image video stream sent by the transfer client; The production client is used to pull the multi-screen video stream from the first cloud server; and send a control signal to the transfer client based on the multi-screen video stream; The relay client is also used to receive the control signal sent by the production client; generate a broadcast video stream based on the control signal and the multi-screen video stream, and push it to the audience terminal for live screen display.

2. The system according to claim 1, characterized in that The transfer client is further used to send the multi-image video stream to the first cloud server through a third streaming address, where the third streaming address is an address configured by the transfer client for sending the multi-image video stream; The first cloud server is further used to receive the multi-image video stream sent by the transit client through the third streaming address, the first cloud server stores a correspondence between the third streaming address and the third streaming address, and the third streaming address is an address configured by the production client for pulling the multi-image video stream; The production client is used to pull the multi-screen video stream from the first cloud server through the third stream pulling address.

3. The system according to claim 2, characterized in that A virtual private network channel is established between the transit client and the production client; The production client is further used to send the control signal to the transit client through the virtual private network channel; The transit client is further configured to receive the control signal sent by the production client through the virtual private network channel.

4. The system according to claim 2, characterized in that The production client is also used to receive a picture effect control operation on at least one video stream in the multi-picture video stream, generate effect control data corresponding to the at least one video picture; and send the effect control data to the transit client.

5. The system according to any one of claims 2 to 4, characterized in that: The system also includes a second cloud server; the second cloud server corresponds to a graphic configuration address; The second cloud server is used to collect event data corresponding to the broadcast event; The production client is further used to access the second cloud server through the graphic configuration address and display a graphic configuration webpage, wherein the graphic configuration webpage includes the event data; based on the event data, receive a graphic configuration operation in the graphic configuration webpage, and generate a graphic configuration result, wherein the graphic configuration result includes the event data; Sending the graphic configuration result to the second cloud server; The transfer client is also used to receive the graphic configuration result from the second cloud server; and perform graphic configuration on the multi-screen video stream based on the graphic configuration result.

6. The system according to any one of claims 2 to 4, characterized in that: The transfer client and the production client are respectively installed with a remote communication program, the remote communication program corresponds to a remote communication server, and the transfer client and the commentary client are respectively installed with a video call program; the remote communication program on the transfer client and the video call program are connected via a virtual audio line; The production client is further used to send voice prompt information to the transfer client through the remote communication server, and the voice prompt information is used to prompt to adjust the interpretation content; The transfer client is further used to receive the voice prompt information sent by the production client through the remote communication server; send the voice prompt information to the video call program through the virtual audio line; send the voice prompt information to the first cloud server through the video call program; The first cloud server is further configured to receive the voice prompt information sent by the transfer client; and forward the voice prompt information to the explanation client; The explanation client is also used to receive the voice prompt information sent by the first cloud server; and play the voice prompt information.

7. A video broadcast processing method, characterized in that: The method comprises: Pulling the live video stream from the first cloud server through the first stream pulling address; the first stream pulling address is an address configured by the transit client for receiving the live video stream; Pulling the commentary video stream from the first cloud server through a second stream pulling address, where the second stream pulling address is an address configured by the transit client for receiving the commentary video stream; The live video stream and the commentary video stream are spliced ​​to obtain a multi-screen video stream; a control signal sent by a production client is received; a relay video stream is generated based on the control signal and the multi-screen video stream, and the relay video stream is pushed to a viewer terminal for live screen display; Among them, the first cloud server is used to store the correspondence between the first push stream address and the first pull stream address corresponding to the broadcast event, and the first push stream address is the address configured in the push stream client for sending the live video stream; and, store the correspondence between the second push stream address and the second pull stream address corresponding to the commentary video stream, and the second push stream address is the address configured in the commentary client for sending the commentary video stream; the production client is used to pull the multi-screen video stream sent by the transit client from the first cloud server, and send the control signal to the transit client based on the multi-screen video stream.

8. A video broadcast processing method, characterized in that: The method comprises: Pulling a live video stream from a first cloud server through a first stream pulling address, where the first stream pulling address is an address configured by a commentary client for receiving the live video stream; the first cloud server is used to store a correspondence between a first stream pushing address corresponding to a broadcast event and the first stream pulling address, where the first stream pushing address is an address configured in a stream pushing client for sending a live video stream; Get the commentary video stream; The commentary video stream is sent to the first cloud server via a second push stream address, where the second push stream address is an address configured in the commentary client for sending the commentary video stream; the first cloud server is also used to store the correspondence between the second push stream address and the second pull stream address corresponding to the commentary video stream, where the second pull stream address is an address configured by the transit client for receiving the commentary video stream; the commentary video stream is used to participate in generating a multi-screen video stream, where the multi-screen video stream is obtained by splicing the live video stream and the commentary video stream by the transit client; the transit client is also used to receive a control signal sent by a production client, generate a relay video stream based on the control signal and the multi-screen video stream, and push the relay video stream to the audience terminal for live screen display; the production client is used to pull the multi-screen video stream sent by the transit client from the first cloud server, and send the control signal to the transit client based on the multi-screen video stream.

9. A video broadcast processing device, characterized in that: The device comprises: A first pulling module is used to pull the live video stream from the first cloud server through a first pulling stream address; the first pulling stream address is an address configured by the transit client for receiving the live video stream; A second pulling module, used to pull the commentary video stream from the first cloud server through a second pulling stream address, where the second pulling stream address is an address configured by the transfer client for receiving the commentary video stream; A splicing module is used to splice the live video stream and the commentary video stream to obtain a multi-screen video stream; receive a control signal sent by a production client; generate a relay video stream based on the control signal and the multi-screen video stream, and push the relay video stream to a viewer terminal for live screen display; Among them, the first cloud server is used to store the correspondence between the first push stream address and the first pull stream address corresponding to the broadcast event, and the first push stream address is the address configured in the push stream client for sending the live video stream; and, store the correspondence between the second push stream address and the second pull stream address corresponding to the commentary video stream, and the second push stream address is the address configured in the commentary client for sending the commentary video stream; the production client is used to pull the multi-screen video stream sent by the transit client from the first cloud server, and send the control signal to the transit client based on the multi-screen video stream.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the video broadcast processing method as described in claim 7 or 8.

Images