Live video push control method and apparatus, and corresponding terminals

[0050] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, in which the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary, and are only used to explain the present invention, and cannot be construed as limiting the present invention.

[0051] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the described features, integers, steps, and operations, but does not exclude the presence or addition of one or more other features, integers, steps, and operations.

[0052] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings as those commonly understood by those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms such as those defined in a general dictionary should be understood as having a meaning consistent with the meaning in the context of the prior art, and unless specifically defined as here, they will not be idealized or overly Explain the formal meaning.

[0053] Those skilled in the art should understand that the concepts of “application”, “application”, “application software” and similar expressions referred to in the present invention are the same concepts well known to those skilled in the art, and refer to a series of computer instructions and related concepts. Data resources are organically constructed computer software suitable for electronic operation. Unless specifically specified, this naming itself is not restricted by the type and level of programming language, nor the operating system or platform on which it runs. Of course, such concepts are not restricted by any form of terminal.

[0054] Those skilled in the art should understand that the user interface and display interface referred to in the present invention generally refer to the display interface that can be used to send the control instruction to the smart terminal. For example, it may be the setting page of the Android/IOS/Windows Phone system. An option (or button, added by the application, the same below), can also be an option in the notification bar or interactive page called from the desktop, or a page constructed by an active component of the application One of the options.

[0055] In one embodiment, the present invention provides a video push control method. The execution subject of this embodiment is the director, and the live video push control method is executed by software installed on the director, such as figure 1 Shown, including:

[0056] S100: Generate a live video stream.

[0057] The director terminal receives the audio and video streams collected by various video and audio collection equipment, that is, receives multiple source video streams, and then edits and processes the source video streams to generate a live video stream. Specifically, the director terminal receives the trigger terminal triggered by the user operation Processing instructions, according to the processing instructions to generate a live video stream, the live video stream includes images and audio, the processing instructions include image and audio processing instructions, in response to the processing instructions to combine multiple source video streams into one Live video streaming. For example, in a large-scale live broadcast scene, it is necessary to display the two channels of images of the host computer and the secondary camera of the interviewing audience on the same screen, and the two channels of video stream images collected by the two cameras are synthesized through the synthesis command at the director end.

[0058] S200: Transmit the live video stream to a server built in the director, and configure a streaming address in the local area network for the live video stream.

[0059] In this embodiment, a server is configured inside the director. Specifically, a server is built on the director based on Nginx and rtmp modules, and the rtmp service is monitored on port 1935. At this time, the server is equivalent to being built in the director, and then the director The software on the terminal can push the live video stream generated above to the server, that is, transmit the live video stream to a specific storage path at the director, and configure and generate a pull in the local area network for the live video stream. Streaming address, through which other terminals in the local area network can access the server and request to obtain the live video stream data. Preferably, the server built in the director in this embodiment is pre-built and configured on the director. When the director generates a live video stream, the live video stream can be immediately transmitted to the server built in the director to improve live broadcast The transmission efficiency of the video stream reduces the playback delay of the live video.

[0060] In this embodiment, the server is preferably configured in the director, that is, the director is configured to have the function of a server. It is understandable that the server can be set up separately from the director. At this time, the server and the director are two independent devices. Similarly, the director can also push the live video stream to the server.

[0061] S300: Receive a streaming request initiated by a player in the same local area network as the director terminal based on the streaming address, and send the live video stream to the player through the server.

[0062] After configuring the streaming address in the local area network for the live video stream, the player in the same local area network as the director can initiate a streaming request based on the streaming address, such as the generated streaming in the local area network. The address is rtmp://192.168.1.10:1935/live/1, the player uses a video player such as VLC and then enters the pull stream address, the guide end initiates a pull request, and the director receives the pull request , Which is equivalent to that the server receives a request to pull the stream and sends the live video stream to the playback end through the server, which is equivalent to that the playback end requests the live video stream from the server and then pulls the live video stream from the server.

[0063] The present invention configures the server at the director end and configures the streaming address in the local area network for the live video stream, and the player in the same local area network can pull the live video stream according to the streaming address, so as to realize the communication to the player in the same local area network. The video stream is pulled for distribution, which reduces the delay of playing live video on the broadcast end in the same local area network, so as to push the generated live video stream to the live broadcast end in a large-scale live broadcast scene, especially in the scene of the program that needs a large screen display. There is no need to lay a dedicated HDMI or SDI video transmission cable, and the live video will be displayed on multiple large screens on the scene simultaneously for the live audience to watch. With the minimum equipment cost and labor cost, the large screen display of the program can be completed efficiently.

[0064] According to an embodiment of the present invention, after the director receives a streaming request initiated by a player in the same local area network as the director based on the streaming address, the method further includes:

[0065] S31: Obtain the priorities of all playback terminals that currently initiate a streaming request;

[0066] The sending the live video stream to the playback end through the server includes:

[0067] S32: Control the server to preferentially send the live video stream to a high-priority player.

[0068] In this embodiment, after receiving the streaming request initiated by multiple players in the same local area network, the director obtains the priority of the player that initiates the current request. For example, in a large-scale live broadcast program, there are 4 players on site. A1, A2, A3, and A4 are each connected to a large screen to display the live video screen. The player A1 is connected to the live main screen, and its priority is the highest. When the player A1, A2, and A3 simultaneously guide the broadcaster to initiate streaming Upon request, after obtaining the priority of each playback terminal, the broadcast director preferentially sends the live video stream to the playback terminal A1 with the highest priority to ensure that the live broadcast video playback delay of the main screen is the lowest.

[0069] In another embodiment of the present invention, after the director receives the streaming request initiated by the player in the same local area network as the director based on the streaming address, the method further includes:

[0070] S310: Obtain the priorities of all playback terminals that currently initiate a streaming request;

[0071] S320: Allocate a preset high proportion of bandwidth to the high priority playback terminal;

[0072] In this embodiment, the sending the live video stream to the playback end through the server includes:

[0073] S330: Send the live video stream to the playback terminal through the server according to the bandwidth allocated for each playback terminal.

[0074] In this embodiment, after receiving a streaming request initiated by multiple players in the same local area network, the director obtains the priority of the player that initiates the current request. For example, in a large-scale live broadcast program, there are 4 broadcasters on site. Terminals A1, A2, A3, and A4 are each connected to a large screen to display video images. Among them, the A1 player terminal is connected to the live main screen, and its priority is the highest, and the priority of A2, A3, and A4 decreases sequentially. When the player A1, When A2 and A3 simultaneously initiate a request to pull a stream, the director obtains the priority of each player, and allocates 40% of the bandwidth to A1, and 30% of the bandwidth to A2 and A3, and then according to the allocation of each player Bandwidth sends the live video stream to the playback terminal through the server. Even if the network transmission speed in the local area network is relatively fast, assigning a higher proportion of bandwidth to the playback terminal with high priority through this embodiment can better ensure the main screen The video plays smoothly.

[0075] In the embodiment of the present invention, after the broadcast director generates the live video stream, the method further includes:

[0076] The director terminal transmits the live video stream to the director background server, so that the live room viewing terminal of the external network obtains the live video stream from the director background server.

[0077] Further, in the embodiment of the present invention, the broadcast director not only sends the generated live video stream to the live broadcast end to meet the viewing needs of the live audience, but also pushes the live video stream to the live broadcast platform so that the user can watch the scene through the live broadcast platform. Live, specifically, the director pushes the live video stream to the director backend server. The director backend server receives the live video stream pushed by the director on the one hand, and on the other hand, it receives the live video stream on the external network (that is, it is not in the same local area network as the director. ) Is requested by the viewer of the live broadcast platform to send the live video stream to the viewer on the external network; therefore, users on the live broadcast platform can pull the live video from the director backend server after entering the live room of the live broadcast platform Watch the live broadcast after streaming.

[0078] Further, in the foregoing embodiment, after the directing end generates the live video stream, the method further includes:

[0079] Check the current network transmission rate;

[0080] If the current network transmission rate is less than the preset value, reduce the configuration information of the live video stream;

[0081] Transmit the live video stream with reduced configuration information to the director backend server.

[0082] In this embodiment, the director detects the current network transmission rate before transmitting the live video stream to the director backend server. A too low transmission rate will result in unstable transmission of the live video stream. If the current network transmission rate is less than the preset Value, the director automatically reduces the configuration information of the live video stream, the configuration information includes the bit rate, frame rate, and resolution of the live video stream. Reducing the configuration information of the live video stream helps to reduce the bandwidth occupied by the live video stream. , Even when watching on the viewing side, a certain amount of picture quality will be lost. When the network transmission rate is less than the preset value, the user will be broadcast live from a third party by reducing the configuration information of the live video stream and then transmitting the live video stream to the director backend server The ability to watch the live video on the platform completely and smoothly is the first consideration, and the quality display effect of the live video is the second consideration to ensure the playback of the live video.

[0083] Further, when the network transmission rate is restored to be greater than the preset value, the configuration information of the live video stream is restored and the live video stream is transmitted to the director background server, so that users of the third-party platform can watch the live video stream clearly and smoothly.

[0084] Preferably, in the embodiment of the present invention, the streaming address is an RTMP streaming address in a local area network.

[0085] The currently used streaming media push protocols mainly include RTMP, WebRTC, and private protocols based on UDP. Among them, RTMP is the acronym for Real Time Messaging Protocol (Real Time Messaging Protocol), which is based on TCP and is a protocol family , Including RTMP basic protocol and RTMPT/RTMPS/RTMPE and other variants, RTMP is a network protocol designed for real-time data communication, mainly used between Flash/AIR platform and streaming media/interactive server supporting RTMP protocol For audio, video and data communications, RTMP is the current mainstream streaming media transmission protocol and is widely used in the field of live broadcast. The RTMP streaming address based on RTMP starts with rtmp://, and in this embodiment, the director and the on-site player are connected to the same local area network. The local area network (LAN) refers to the A group of computers interconnected by multiple computers. Each computer (or its equipment) on the same LAN has one or more LAN IP addresses. Each LAN IP address is allocated inside the LAN, and the IP starts with 192 The IP addresses of different LANs can be repeated without affecting each other. The streaming address in a local area network in this embodiment is rtmp://192.168.1.10:1935/live/1, and the player can pull the corresponding live video stream through the streaming address, and configure the streaming address as the local area network The internal RTMP streaming address is compatible with the streaming protocols of most common video players, which improves the layout efficiency of large-scale live broadcasts.

[0086] In another embodiment, the present invention provides a live video push control device, the device stores a plurality of instructions, the instructions are suitable for being loaded by a processor and executed the above video push control method, including:

[0087] Generate live video stream;

[0088] Transmitting the live video stream to a server built in the director, and configuring a streaming address in the local area network for the live video stream;

[0089] Receiving a streaming request initiated based on the streaming address by a player in the same local area network as the director, and sending the live video stream to the player through the server.

[0090] For ease of description, the live video push control device is split into a functional module architecture, such as figure 2 Shown, including:

[0091] Synthesis module 100: used to generate a live video stream;

[0092] Configuration module 200: used to transmit the live video stream to a server built in the director, and configure a streaming address in the local area network for the live video stream;

[0093] The streaming module 300 is configured to receive a streaming request initiated by a player in the same local area network as the director terminal based on the streaming address, and send the live video stream to the player through the server.

[0094] The synthesis module 100 receives audio and video streams collected by various video and audio capture devices, that is, receives multiple source video streams, and then edits and processes the source video streams to generate a live video stream, specifically, receives processing instructions triggered by the user operation terminal , Generating a live video stream according to the processing instruction, the live video stream including image and audio, the processing instruction includes processing instructions for the image and audio, and in response to the processing instruction, synthesize a multi-source video stream into a live video flow.

[0095] The configuration module 200 configures a server inside the director. Specifically, a server is built based on the Nginx and rtmp modules on the director and listens to the rtmp service on port 1935. At this time, the server is equivalent to being built in the director, and then generates the above Push the live video stream to the server, that is, transmit the live video stream to a specific storage path of the director, and configure and generate a streaming address in the local area network for the live video stream.

[0096] After the configuration module 200 configures the streaming address in the local area network for the live video stream, the playback end in the same local area network as the director can initiate a pull request based on the pull address and guide the broadcaster, and the streaming module 300 receives In the streaming request, the live video stream is sent to the playback terminal through the server.

[0097] In yet another embodiment, the present invention also provides a terminal for implementing a method for controlling live video push, such as image 3 What is shown is a block diagram of a part of the structure of a terminal provided in an embodiment of the present invention. For ease of description, only parts related to the embodiment of the present invention are shown. For specific technical details that are not disclosed, please refer to the method part of the embodiment of the present invention. The terminal can be any terminal device including a mobile phone, a PC (Personal computer, personal computer), a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales, sales terminal), a vehicle-mounted computer, etc., in this embodiment , The terminal is a director.

[0098] reference image 3 The terminal includes: a baseband processing module 310, a memory 320, an input unit 330, a display unit 340, a sensor 350, an audio circuit 360, a wireless fidelity (WiFi) module 370, a processor 380, and a power supply 390. . Those skilled in the art can understand, image 3 The terminal structure shown in does not constitute a limitation on the terminal, and may include more or fewer components than shown in the figure, or combine some components, or arrange different components.

[0099] The baseband processing module 310 may be used to synthesize the baseband signal to be transmitted or decode the received baseband signal. Specifically, when transmitting, the audio signal is compiled into a baseband code for transmission; when receiving, the received baseband code is interpreted as an audio signal. At the same time, it is also responsible for the compilation of address information (mobile phone number, website address), text information (short message text, website text), and picture information (MMS).

[0100] The memory 320 may be used to store software programs and modules. The processor 380 executes various functional applications and data processing of the terminal by running the software programs and modules stored in the memory 320. The memory 320 may mainly include a program storage area and a data storage area. The program storage area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.), etc.; The data (such as audio data, phone book, etc.) created by the use of the terminal, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0101] The input unit 330 may be used to receive inputted number or character information, and generate key signal input related to user settings and function control of the terminal. Specifically, the input unit 330 may include a touch panel 331 and other input devices 332. The touch panel 331, also called a touch screen, can collect user touch operations on or near it (for example, the user uses any suitable objects or accessories such as fingers, stylus, etc.) on the touch panel 331 or near the touch panel 331. Operation), and drive the corresponding connection device according to the preset program. Optionally, the touch panel 331 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch position, and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it To the processor 380, and can receive commands sent by the processor 380 and execute them. In addition, the touch panel 331 can be implemented in multiple types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 331, the input unit 330 may also include other input devices 332. Specifically, the other input device 332 may include, but is not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackball, mouse, and joystick.

[0102] The display unit 340 may be used to display information input by the user or information provided to the user and various menus of the terminal. The display unit 340 may include a display panel 341. Optionally, the display panel 341 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 331 can cover the display panel 341. When the touch panel 331 detects a touch operation on or near it, it transmits it to the processor 380 to determine the type of the touch event, and then the processor 380 responds to the touch event. The type provides corresponding visual output on the display panel 341. Although in image 3 In the above, the touch panel 331 and the display panel 341 are used as two independent components to realize the input and input functions of the terminal. However, in some embodiments, the touch panel 331 and the display panel 341 can be integrated to realize the input of the terminal. And output function.

[0103] The terminal may also include at least one sensor 350, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor can include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 341 according to the brightness of the ambient light. The proximity sensor can close the display panel 341 and/or when the terminal is moved to the ear. Or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (usually three-axis), and can detect the magnitude and direction of gravity when it is stationary. It can be used to identify terminal posture applications (such as horizontal and vertical screen switching, related Games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, percussion), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which can be configured on the terminal, I will not here Go into details.

[0104] The audio circuit 360, the speaker 361, and the microphone 362 can provide an audio interface between the user and the terminal. The audio circuit 360 can transmit the electric signal converted from the received audio data to the speaker 361, and the speaker 361 converts it into a sound signal for output; on the other hand, the microphone 362 converts the collected sound signal into an electric signal, and the audio circuit 360 After being received, it is converted into audio data, and then processed by the audio data output processor 380, and sent to, for example, another terminal via the RF circuit 310, or the audio data is output to the memory 320 for further processing.

[0105] WiFi is a short-distance wireless transmission technology. Through the WiFi module 370, the terminal can help users send and receive e-mails, browse web pages, and access streaming media. It provides users with wireless broadband Internet access. although image 3 The WiFi module 370 is shown, but it is understandable that it is not an essential component of the terminal, and can be omitted as needed without changing the essence of the invention.

[0106] The processor 380 is the control center of the terminal. It uses various interfaces and lines to connect various parts of the entire terminal. It executes by running or executing software programs and/or modules stored in the memory 320, and calling data stored in the memory 320. Various functions of the terminal and processing data, so as to monitor the terminal as a whole. Optionally, the processor 380 may include one or more processing units; preferably, the processor 380 may integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, application programs, etc. , The modem processor mainly deals with wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 380.

[0107] The terminal also includes a power source 390 (such as a battery) for supplying power to various components. Preferably, the power source can be logically connected to the processor 380 through a power management system, so that functions such as charging, discharging, and power consumption management can be managed through the power management system.

[0108] Although not shown, the terminal may also include a camera, a Bluetooth module, etc., which will not be repeated here.

[0109] In addition, the functional units in the various embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer readable storage medium.

[0110] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc.

[0111] The present invention configures the server at the director end and configures the streaming address in the local area network for the live video stream, and the player in the same local area network can pull the live video stream according to the streaming address, so as to realize the communication to the player in the same local area network. The video stream is pulled for distribution, which reduces the delay of playing live video on the broadcast end in the same local area network, so as to push the generated live video stream to the live broadcast end in a large-scale live broadcast scene, especially in the scene of the program that needs a large screen display. There is no need to lay a dedicated HDMI or SDI video transmission cable, and the live video will be displayed on multiple large screens on the scene simultaneously for the live audience to watch. With the minimum equipment cost and labor cost, the large screen display of the program can be completed efficiently.

[0112] At the same time, the present invention controls the live video stream to be preferentially pushed to the high-priority playback terminal by setting the priority of the live playback terminal. The high-priority playback terminal is connected to the live main screen to ensure that the live video playback delay of the main screen is the lowest.

[0113] Furthermore, the present invention allocates a higher proportion of bandwidth to the high-priority playback terminal by setting the priority of the live playback terminal, and then sends the live video stream to all the broadcast terminals through the server according to the bandwidth allocated for each playback terminal. As for the playback terminal, the high-priority playback terminal is connected to the live main screen to ensure smooth video playback on the main screen.

[0114] The above are only part of the embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications are also It should be regarded as the protection scope of the present invention.