A live streaming method and device

Abstract

The present application provides a live streaming method and device, which relates to the field of audio and video data processing and can also be used in the financial field, including: establishing a live streaming room according to the live streaming information set by the anchor client and receiving the live streaming video data uploaded by the anchor client; and live streaming the live streaming video data under health monitoring according to the live streaming control instructions of the anchor client. The present application can reduce the requirements for the performance of the anchor client device and the network bandwidth, thereby providing viewers with a stable background streaming live streaming viewing experience.

Description

Technical Field

[0001] The present application relates to the field of audio and video data processing and can be used in the financial field, specifically a live streaming method and device. Background Art

[0002] Among the current live streaming systems, some systems only support the host's online live broadcast, which collects audio and video streams in real time through the host's client or peripherals and pushes them to the server. The live broadcast quality and network delay of this kind of live streaming system are heavily dependent on the host's network bandwidth, client device performance, etc. There are also some client software that supports recorded video streaming, which requires the video file to be pre-uploaded to the host's client device before the live streaming can be completed. In this kind of live streaming system, the video files will occupy a large amount of client local storage resources, and are highly dependent on the host's client device, and the performance requirements are more stringent. At present, there is no live streaming method that can effectively solve the above problems. Summary of the invention

[0003] In response to the problems in the prior art, the present application provides a live streaming method and device, which can reduce the requirements on the host's client device performance and network bandwidth, thereby providing viewers with a stable background streaming live viewing experience.

[0004] In order to solve the above technical problems, this application provides the following technical solutions:

[0005] In a first aspect, the present application provides a live streaming method, comprising:

[0006] Establishing a live broadcast room according to the live broadcast information set by the anchor client and receiving the live broadcast video data uploaded by the anchor client;

[0007] The live streaming of the live video data is performed under health monitoring according to the live streaming control instructions of the anchor client.

[0008] Furthermore, a live broadcast room is established according to the live broadcast information set by the anchor client, including:

[0009] Receive live broadcast information sent by the anchor client; wherein the live broadcast information includes live broadcast start time, live broadcast end time, anchor identity, recording and broadcasting identification and recording and broadcasting video address;

[0010] Performing live broadcast time verification according to the live broadcast start time and the live broadcast end time;

[0011] Performing anchor identity authentication according to the anchor identity identifier;

[0012] The live broadcast room is established according to the recording and broadcasting identifier and the recording and broadcasting video address.

[0013] Further, receiving the live video data uploaded by the anchor client includes:

[0014] Sending preset segmentation configuration parameters to the anchor client, so that the anchor client performs segmentation processing on the live video data according to the segmentation configuration parameters to obtain live video data segments;

[0015] Receiving each live video data segment in parallel, and comparing the file size of the received live video data segment with the file size of the live video data segment sent by the anchor client; wherein, if the two are consistent, the reception of the live video data segment is completed;

[0016] Perform integrity check on the live video data segments that have been received.

[0017] Furthermore, the live broadcast control instruction is a live broadcast start instruction; the live broadcast streaming of the live broadcast video data under health monitoring according to the live broadcast control instruction of the anchor client includes:

[0018] Read the corresponding live video data according to the recorded video address;

[0019] Asynchronously start the streaming process of the live video data, and stream the live video data to the content distribution network under health monitoring.

[0020] Further, the live broadcast control instruction is a live broadcast pause instruction; the live broadcast streaming of the live broadcast video data under health monitoring according to the live broadcast control instruction of the anchor client includes:

[0021] Asynchronously pausing the streaming process of the live video data, and recording the playing time of the live video data when the streaming is paused.

[0022] Further, the live broadcast control instruction is a live broadcast resumption instruction; and the live broadcast streaming under health monitoring of the live broadcast video data according to the live broadcast control instruction of the anchor client includes:

[0023] According to the pause duration, the continuous streaming process of the live video data is asynchronously started to continue streaming the live video data to the content distribution network; wherein the continuous streaming process uses the playback duration of the live video data when streaming is paused as the start time of this live streaming.

[0024] Further, the live broadcast control instruction is a live broadcast end instruction; the live broadcast streaming of the live broadcast video data under health monitoring according to the live broadcast control instruction of the anchor client includes:

[0025] Asynchronously end the streaming process of the live video data.

[0026] Furthermore, the streaming of the live video data to the content distribution network under health monitoring includes:

[0027] Monitor whether abnormal keywords are returned;

[0028] If yes, obtain the resource usage status and system performance parameters of each streaming node;

[0029] Calculate the health score of each streaming node according to the resource usage status and system performance parameters;

[0030] According to the health score of each streaming node, the live video data is dispatched to the streaming node whose health score exceeds the health threshold for live streaming based on the breakpoint resume technology.

[0031] Furthermore, after the integrity check is performed on the received live video data segments, the method further includes:

[0032] The encoding format and encapsulation format of the live video data segments that have been received are converted.

[0033] In a second aspect, the present application provides a live streaming device, comprising:

[0034] A live broadcast data receiving unit, used to establish a live broadcast room according to the live broadcast information set by the anchor client and receive the live broadcast video data uploaded by the anchor client;

[0035] A health monitoring streaming unit is used to perform live streaming of the live video data under health monitoring according to the live broadcast control instructions of the anchor client.

[0036] Furthermore, the live data receiving unit includes:

[0037] A live broadcast information receiving module, used to receive the live broadcast information sent by the anchor client; wherein the live broadcast information includes the live broadcast start time, the live broadcast end time, the anchor identity, the recording and broadcasting identity and the recording and broadcasting video address;

[0038] A time verification module, used to verify the live broadcast time according to the live broadcast start time and the live broadcast end time;

[0039] An identity authentication module, used to perform host identity authentication according to the host identity identifier;

[0040] The live broadcast room establishment module is used to establish the live broadcast room according to the recording and broadcasting identifier and the recording and broadcasting video address.

[0041] Furthermore, the live data receiving unit includes:

[0042] A fragmentation parameter sending module, used to send preset fragmentation configuration parameters to the anchor client, so that the anchor client performs fragmentation processing on the live video data according to the fragmentation configuration parameters to obtain live video data fragments;

[0043] The data segment receiving module is used to receive each live video data segment in parallel, and compare the file size of the received live video data segment with the file size of the live video data segment sent by the anchor client; wherein, if the two are consistent, the reception of the live video data segment is completed;

[0044] The integrity check module is used to perform integrity check on the live video data segments that have been received.

[0045] Furthermore, the live broadcast control instruction is to start the live broadcast; the health monitoring streaming unit includes:

[0046] A video data reading module, used to read the corresponding live video data according to the recorded video address;

[0047] The push stream start module is used to asynchronously start the push stream process of the live video data and push the live video data to the content distribution network under health monitoring.

[0048] Furthermore, the live broadcast control instruction is to pause the live broadcast; the health monitoring streaming unit includes:

[0049] The streaming pause module is used to asynchronously pause the streaming process of the live video data and record the playing time of the live video data when the streaming is paused.

[0050] Furthermore, the live broadcast control instruction is live broadcast resumption; the health monitoring streaming unit includes:

[0051] The streaming recovery module is used to asynchronously start the continuous streaming process of the live video data according to the paused playback duration, and continue to stream the live video data to the content distribution network; wherein the continuous streaming process uses the playback duration of the live video data when the streaming is paused as the start time of this live streaming.

[0052] Furthermore, the live broadcast control instruction is to end the live broadcast; the health monitoring streaming unit includes:

[0053] The streaming end module is used to asynchronously end the streaming process of the live video data.

[0054] Furthermore, the health monitoring streaming unit includes:

[0055] The exception return module is used to monitor whether the exception keyword is returned;

[0056] The status parameter acquisition module is used to obtain the resource usage status and system performance parameters of each streaming node;

[0057] A health calculation module is used to calculate the health score of each streaming node according to the resource usage status and system performance parameters;

[0058] The streaming scheduling module is used to schedule the live video data to the streaming nodes whose health scores exceed the health threshold according to the health scores of each streaming node, and perform live streaming based on the breakpoint resume technology.

[0059] Furthermore, after the integrity check is performed on the received live video data segments, the method further includes:

[0060] The encoding format and encapsulation format of the live video data segments that have been received are converted.

[0061] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the live streaming method when executing the program.

[0062] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the live streaming method when executed by a processor.

[0063] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which implements the steps of the live streaming method when executed by a processor.

[0064] In response to the problems in the prior art, the live streaming method and device provided by the present application can overcome the technical defects of the live streaming client pushing the recorded video stream, and utilize the system for pushing the recorded video stream in the background, so that the host client does not need to participate in the pushing process of the audio and video stream, and can interact with the audience or control the live streaming process only by operating the page buttons; in this case, the live streaming does not consume the network traffic of the host client, and there are no special requirements for the host's device performance; for the audience, the live viewing effect of the recorded video is completely consistent with the real-time live broadcast, which is transparent and imperceptible; it can not only support the host's non-online live broadcast, reduce the requirements for the host client device performance and network bandwidth, but also improve the audience's experience and enhance the live viewing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 This is a flow chart of the live streaming method in the embodiment of the present application;

[0067] Figure 2 A flowchart of establishing a live broadcast room in an embodiment of the present application;

[0068] Figure 3 This is a flowchart of receiving live video data in an embodiment of the present application;

[0069] Figure 4 This is one of the flowcharts of live streaming in the embodiment of this application;

[0070] Figure 5 This is the second flowchart of the live streaming in the embodiment of this application;

[0071] Figure 6 This is a structural diagram of a live streaming device in an embodiment of the present application;

[0072] Figure 7 This is one of the structural diagrams of the live data receiving unit in the embodiment of the present application;

[0073] Figure 8 This is the second structural diagram of the live broadcast data receiving unit in the embodiment of the present application;

[0074] Fig. 9 This is one of the structural diagrams of the health monitoring streaming unit in the embodiment of the present application;

[0075] Fig.10 This is the second structural diagram of the health monitoring streaming unit in the embodiment of the present application;

[0076] Fig.11 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0077] Fig.12 This is the overall architecture diagram of the embodiment of the present application;

[0078] Fig.13 This is a schematic diagram of the video uploading process in the embodiment of the present application;

[0079] Fig.14 This is a schematic diagram of the video transcoding process in an embodiment of the present application;

[0080] Fig.15This is a schematic diagram of the streaming and status monitoring process in the embodiment of the present application;

[0081] Fig.16 This is a schematic diagram of the monitoring and scheduling process in an embodiment of the present application. DETAILED DESCRIPTION

[0082] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0083] It should be noted that the live streaming method and device provided in the present application can be used in the financial field, and can also be used in any field other than the financial field. The application field of the live streaming method and device provided in the present application is not limited.

[0084] The acquisition, storage, use and processing of data in the technical solution of this application comply with the relevant provisions of laws and regulations.

[0085] In one embodiment, see Figure 1 In order to reduce the requirements on the host client device performance and network bandwidth, thereby providing viewers with a stable background live streaming viewing experience, this application provides a live streaming method, including:

[0086] S101: establishing a live broadcast room according to the live broadcast information set by the anchor client and receiving live broadcast video data uploaded by the anchor client;

[0087] S102: Perform live streaming of the live video data under health monitoring according to the live broadcast control instruction of the anchor client.

[0088] It is understood that the following terms are involved in the embodiments of the present application and have the following meanings:

[0089] Content Delivery Network (CDN): A network content service system that is built on an IP network and provides content distribution and services based on the efficiency requirements, quality requirements, and content order of content access and application. Using a content delivery network, users can obtain content resources nearby, reduce network latency, and improve access response speed.

[0090] Live streaming: The host collects audio and video streams through client hardware devices or peripherals and pushes them to the content distribution network.

[0091] Live streaming: The audience pulls the aforementioned audio and video streams from the content distribution network.

[0092] Recorded video transcoding: Convert pre-recorded video files into a unified format.

[0093] Background streaming: Different from the front-end streaming method of the client application software, the background server realizes the automatic streaming of the video. In the embodiment of the present application, the live streaming process is completed by the execution subject of the embodiment of the present application, that is, a server that can implement the live streaming method.

[0094] See also Fig.12 The server used to implement the live streaming method in the embodiment of the present application includes the following components: a live room creation component, a video upload component, a video transcoding component, a live streaming control component, a streaming and status monitoring component, a monitoring scheduling component, and a data storage component (corresponding to the data storage layer and the file server). Among them, the file server is used to store the audio and video stream files uploaded by the anchor. The data storage layer is used to store various attributes and parameters related to the audio and video stream files, such as but not limited to the video file address.

[0095] In the specific implementation, the anchor can first define the basic information of the live broadcast room. If the anchor chooses to push the recorded video, the video upload component needs to upload the pre-recorded video file to the file server. Correspondingly, the file server needs to receive the video file. The Create Live Broadcast Room component can also create a live broadcast room based on the above basic information.

[0096] Since video file formats are diverse and their encodings are different, in order to better be compatible with the viewing end (also known as the audience client), the video transcoding component also needs to perform unified transcoding processing on the video files; after transcoding, the video files are written to the file server to generate an access address, and then persisted through the data storage layer.

[0097] After the anchor starts live streaming through the live streaming control component, the streaming and status monitoring component queries the video file address from the data storage layer and asynchronously starts the streaming process, pushing the audio and video stream corresponding to the video file address to the content distribution network.

[0098] The monitoring and scheduling component calculates the health scores of all streaming nodes through preset monitoring indicators and health scoring algorithms, and then writes the health score calculation results to the data storage layer, records the live broadcast room information where the abnormal streaming interruption occurs, and selects and schedules new streaming nodes to continue streaming from the breakpoint time point, thereby ensuring the stability and continuity of streaming.

[0099] The data storage component is responsible for the storage and verification of live broadcast related data. The data storage component uses a relational database to persistently store the basic information of the live broadcast room and stores the hot data in the live broadcast in a distributed cache. The underlying components of the data storage component provide read and write capabilities for other components to call. In addition, this component can also verify the legitimacy of the anchor identity, live broadcast time, and live broadcast sessions.

[0100] From the above description, it can be seen that the live streaming method provided by the present application can overcome the technical defects of the live streaming client pushing the recorded video stream, and utilizes the system for pushing the recorded video stream in the background, so that the host client does not need to participate in the pushing process of the audio and video stream, and can interact with the audience or control the live streaming process only by operating the page buttons; in this case, the live streaming does not consume the network traffic of the host client, and there are no special requirements for the host's device performance; for the audience, the live viewing effect of the recorded video is completely consistent with the real-time live broadcast, which is transparent and imperceptible; it can not only support the host's non-online live broadcast, reduce the requirements for the host client device performance and network bandwidth, but also improve the audience's experience and enhance the live viewing effect.

[0101] Steps S101 to S102 are described in detail below.

[0102] Step S101: establishing a live broadcast room according to the live broadcast information set by the anchor client and receiving the live broadcast video data uploaded by the anchor client.

[0103] Figure 2 This is a specific embodiment of the live streaming method implemented in the embodiment of the present application.

[0104] In one embodiment, see Figure 2 , establish a live broadcast room according to the live broadcast information set by the anchor client, including:

[0105] S201: receiving live broadcast information sent by the anchor client; wherein the live broadcast information includes live broadcast start time, live broadcast end time, anchor identity, recording and broadcasting identity and recording and broadcasting video address;

[0106] S202: Verify the live broadcast time according to the live broadcast start time and the live broadcast end time;

[0107] S203: Performing host identity authentication according to the host identity identifier;

[0108] S204: Establish the live broadcast room according to the recording and broadcasting identifier and the recording and broadcasting video address.

[0109] It is understandable that during the creation of the live broadcast room, the anchor can define the basic information of the live broadcast room. The main server should receive the above basic information accordingly, and then establish the live broadcast room according to the above basic information.

[0110] Specifically, before the anchor initiates a live broadcast, the basic information of the live broadcast room must be defined first, which may include but is not limited to the live broadcast start time startTime, live broadcast end time endTime, whether it is recorded (also known as the recording flag) recordLiveFlag, anchor id uccid and recorded video address pushRtmpUrl. For the above basic information, the anchor end uploads it through the service interface and then submits it to the data storage component for persistence. The data storage component also needs to verify the anchor's live broadcast room information, such as judging whether the same anchor has overlapping live broadcast time based on the start and end time of the live broadcast, and whether the user id that starts the broadcast is the real anchor id, as shown in the following table.

[0111] When the live broadcast starts Live broadcast end time Recording Broadcaster ID Recorded video address 2022-02-18 16:00 2022-02-18 17:00 yes 10159 NULL

[0112] From the above description, it can be seen that the live streaming method provided by this application can establish a live broadcast room according to the live broadcast information set by the anchor client.

[0113] Figure 3 This is a specific embodiment of the live streaming method implemented in the embodiment of the present application.

[0114] In one embodiment, see Figure 3 , receiving live video data uploaded by the anchor client, including:

[0115] S301: Sending preset segmentation configuration parameters to the anchor client, so that the anchor client performs segmentation processing on the live video data according to the segmentation configuration parameters to obtain live video data segments;

[0116] S302: receiving each live video data segment in parallel, and comparing the file size of the received live video data segment with the file size of the live video data segment sent by the anchor client; wherein, if the two are consistent, the reception of the live video data segment is completed;

[0117] S303: Perform integrity check on the live video data segments that have been received.

[0118] Understandably, see Fig.13 The process of receiving live video data uploaded by the anchor client corresponds to the process of the anchor quickly uploading the pre-recorded audio and video stream to the background file server.

[0119] In order to provide a higher-definition live broadcast, the audio and video stream files to be uploaded may reach several GB. In order to increase the upload rate, the technology of fragmented upload can be adopted: the anchor client first obtains the fragment size configuration from the data storage layer, and then implements local fragmentation of the video file according to the fragment size; when the file server receives each fragment file, it appends them to the same file in the same storage directory. Among them, the fragment size configuration in the data storage layer can be set by the execution subject of this application and notified to the anchor, which corresponds to the process of sending the preset fragment configuration parameters to the anchor client.

[0120] Compare the size of the file selected by the anchor client for uploading with the file size calculated after writing the fragments in the background; if the sizes are the same, it can be determined that the audio and video stream file has been uploaded. At this time, compare the MD5 value of the video file uploaded by the anchor client with the MD5 value of the file written in the background; if the comparison is consistent, it can be determined that the video file has not been lost or tampered with during the transmission process, thereby proving the integrity of the file. After the audio and video stream file is uploaded, the video file is moved to the directory to be transcoded and the video transcoding component takes over the processing.

[0121] From the above description, it can be seen that the live streaming method provided by the present application is capable of receiving the live video data uploaded by the anchor client.

[0122] Figure 4 This is a specific embodiment of the live streaming method implemented in the embodiment of the present application.

[0123] In one embodiment, see Figure 4 , the live broadcast control instruction is a live broadcast start instruction; the live broadcast streaming of the live broadcast video data under health monitoring according to the live broadcast control instruction of the anchor client includes:

[0124] S401: Reading corresponding live video data according to the recorded video address;

[0125] S402: asynchronously start the streaming process of the live video data, and stream the live video data to a content distribution network under health monitoring.

[0126] Specifically, see Figure 5 , the live video data is pushed to the content distribution network under health monitoring, including: monitoring whether abnormal keywords are returned (S501); if so, obtaining the resource usage status and system performance parameters of each push node (S502); calculating the health score of each push node according to the resource usage status and system performance parameters (S503); scheduling the live video data to the push node whose health score exceeds the health threshold according to the health score of each push node for live streaming based on breakpoint resume technology (S504).

[0127] Understandably, see Fig.15 The streaming and status monitoring component is responsible for the background streaming of recorded videos, execution control, streaming to the content distribution network, and status monitoring during the streaming process to the content distribution network.

[0128] The push component in the push and status monitoring component uses the transcoded video resource pushRtmpUrl of the file server as the output source and the content distribution network as the input source. The parameter configuration required by the push component in the push and status monitoring component is provided by the data storage layer. If the parameters need to be adjusted, they can be refreshed in real time through the data storage layer. The push status will be monitored synchronously during the push process. After the push is completed, the output information of the console can be parsed. If it does not contain the abnormal error keyword "error", it means that the recorded video stream has ended pushing, and the background will automatically end the live broadcast room. If it contains the error keyword "error", it means that the push has ended abnormally and needs to be taken over by the monitoring and scheduling component.

[0129] See also Fig.16 ,The monitoring and scheduling component is responsible for the health check of the ,streaming nodes and system performance and capacity monitoring, and realizes automatic ,scheduling based on the health score value.

[0130] The monitoring component in the streaming and status monitoring component can periodically collect the resource usage status and system performance parameters of the background streaming server (generally a connection is established during streaming and the connection is maintained until the end of streaming), such as CPU usage, memory occupancy, network bandwidth, and the running status of the streaming component process. For different monitoring indicators, the streaming and status monitoring component uses a health scoring algorithm to perform summary statistics and calculate the health score of the background streaming process (corresponding to the streaming node) of each streaming server. After sorting the streaming nodes by health score, the health score and sorting are written to the data storage layer. The larger the health score, the busier the current streaming node is.

[0131] The embodiments of the present application support the customization of health monitoring indicators and health score statistical algorithms, which can be flexibly implemented according to the usage scenario. For example, the set health monitoring indicators include "CPU usage", "memory occupancy", and "network bandwidth occupancy ratio", and the score statistical algorithm is ∑ indicator value i×weight i. For example, the CPU usage weight is set to 0.4, the memory occupancy weight is set to 0.4, and the network bandwidth occupancy weight is set to 0.2. When the CPU usage rate is 70%, the memory occupancy rate is 80%, and the network bandwidth occupies 20% of the total bandwidth, the health score of the streaming node is calculated as: 0.4×0.7+0.4×0.8+0.2×0.2=0.64.

[0132] For each streaming node, its health score is calculated based on the weight and the proportion of the index. When the health score exceeds the preset threshold (such as 0.7), the current streaming node can be determined as a "busy node" and will temporarily stop accepting new live streaming tasks. Instead, the "most idle" (with the smallest health score) node among all streaming server nodes is obtained from the data storage layer to be responsible for the background streaming tasks of other live broadcasts.

[0133] In the case of "streaming interruption" due to system failure during the streaming process, the monitoring component writes the interrupted live broadcast room and the interruption time information into the data storage layer, and then the scheduling component selects a new "most idle" node to obtain the live broadcast room information with abnormal interruption from the data storage layer. The new node asynchronously continues the streaming process, thereby realizing breakpoint resumption of interrupted video files and ensuring the stability and continuity of the background live streaming.

[0134] From the above description, it can be seen that the live streaming method provided by the present application can perform live streaming under health monitoring of the live video data according to the live streaming control instructions of the anchor client.

[0135] In one embodiment, the live broadcast control instruction is a live broadcast pause instruction; and the live broadcast streaming of the live broadcast video data under health monitoring according to the live broadcast control instruction of the anchor client includes:

[0136] Asynchronously pausing the streaming process of the live video data, and recording the playing time of the live video data when the streaming is paused.

[0137] In one embodiment, the live broadcast control instruction is a live broadcast resumption instruction; and the live broadcast streaming of the live broadcast video data under health monitoring according to the live broadcast control instruction of the anchor client includes:

[0138] According to the pause duration, the continuous streaming process of the live video data is asynchronously started to continue streaming the live video data to the content distribution network; wherein the continuous streaming process uses the playback duration of the live video data when streaming is paused as the start time of this live streaming.

[0139] In one embodiment, the live broadcast control instruction is a live broadcast end instruction; and the live broadcast streaming of the live broadcast video data under health monitoring according to the live broadcast control instruction of the anchor client includes:

[0140] Asynchronously end the streaming process of the live video data.

[0141] It is understandable that the live broadcast control component is used by the anchor to control various states of the live broadcast room during the live broadcast. The live broadcast room states that the anchor can control include: start, end, pause and resume. The conversion between these states can be provided to the anchor through the service interface of the execution subject of the embodiment of the present application, for example, through an APP or WEB tool.

[0142] ① Live broadcast start status: First, read the basic information of the live broadcast room (including the address of the recorded video) from the data storage layer, and then the streaming process is asynchronously started by the streaming and status monitoring component to read the recorded video and distribute the streaming to the CDN.

[0143] ② Live broadcast pause status: The streaming and status monitoring component asynchronously ends the streaming process and records the playback time of the video when it is paused to the data storage layer.

[0144] ③ Live broadcast recovery status: First, obtain the pause playback duration of the live broadcast room from the data storage layer, and then asynchronously start a new streaming process through the streaming and status monitoring component. The new streaming process will use the pause playback duration as the start time of this streaming.

[0145] ④Live broadcast end status: The streaming process is asynchronously ended by the streaming and status monitoring components.

[0146] From the above description, it can be seen that the live streaming method provided by the present application can perform live streaming under health monitoring of the live video data according to the live streaming control instructions of the anchor client.

[0147] In one embodiment, after the integrity check is performed on the received live video data segments, the method further includes:

[0148] The encoding format and encapsulation format of the live video data segments that have been received are converted.

[0149] Understandably, see Fig.14 ,After the recorded video is successfully uploaded to the file server, the video transcoding component is used to convert the original video file into a video transcoding file with a unified encoding format and packaging format.

[0150] The video files uploaded by the anchor may correspond to different packaging formats (such as mp4, flv, etc.), different video encoding formats (such as H264, MPEG-1, etc.), and different audio encoding formats (such as MP3, AAC, etc.). In order to adapt to the player on the viewing end and improve the audience's live viewing experience, the video files need to be uniformly transcoded. The video transcoding component scans the directory to be transcoded. If there is a video file to be transcoded, it first obtains the business lock of the video file from the data storage component. If the business lock is successfully obtained, it means that the video file will be processed separately by the designated transcoding server, and other servers will no longer transcode the same video file. The video transcoding component will start the transcoding service thread to uniformly convert the audio and video encoding format and packaging format of the video content. After the transcoding is completed, the file server will generate the access address of the video transcoding file, and update the recorded video address in the basic information of the live broadcast room through the data storage layer.

[0151] The transcoding process can be performed one by one according to the fragments, or the fragments can be combined into a complete file before transcoding.

[0152] From the above description, it can be seen that the live streaming method provided by the present application can perform encoding format conversion and encapsulation format conversion on the live video data fragments that have been received.

[0153] In summary, the present invention makes full use of the hardware resource configuration of the background server to realize the background streaming of recorded video files. By regularly monitoring the streaming process, the running health status of the streaming node can be determined. The unified scheduling ensures the balance of the streaming load, realizes the self-protection of streaming overload and the self-repair of abnormalities. The live broadcast effect no longer depends on the device performance and network bandwidth of the anchor client, which reduces the live broadcast cost of the anchor and improves the live broadcast experience of the viewing end.

[0154] Based on the same inventive concept, the embodiments of the present application also provide a live streaming device, which can be used to implement the method described in the above embodiments, as described in the following embodiments. Since the principle of solving the problem by the live streaming device is similar to that of the live streaming method, the implementation of the live streaming device can refer to the implementation of the method based on the software performance benchmark determination, and the repeated parts will not be repeated. As used below, the terms "unit" or "module" can be a combination of software and / or hardware that implements predetermined functions. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0155] In one embodiment, see Figure 6In order to reduce the requirements on the host client device performance and network bandwidth, thereby providing viewers with a stable background live streaming viewing experience, the present application provides a live streaming device, including: a live data receiving unit 601 and a health monitoring streaming unit 602.

[0156] The live broadcast data receiving unit 601 is used to establish a live broadcast room according to the live broadcast information set by the anchor client and receive the live broadcast video data uploaded by the anchor client;

[0157] The health monitoring streaming unit 602 is used to perform live streaming of the live video data under health monitoring according to the live broadcast control instructions of the anchor client.

[0158] In one embodiment, see Figure 7 , the live data receiving unit 601 comprises:

[0159] The live broadcast information receiving module 701 is used to receive the live broadcast information sent by the anchor client; wherein the live broadcast information includes the live broadcast start time, the live broadcast end time, the anchor identity, the recording and broadcasting identity and the recording and broadcasting video address;

[0160] A time verification module 702 is used to verify the live broadcast time according to the live broadcast start time and the live broadcast end time;

[0161] An identity authentication module 703 is used to perform host identity authentication according to the host identity identifier;

[0162] The live broadcast room establishment module 704 is used to establish the live broadcast room according to the recording and broadcasting identifier and the recording and broadcasting video address.

[0163] In one embodiment, see Figure 8 , the live data receiving unit 601 comprises:

[0164] The fragmentation parameter sending module 801 is used to send preset fragmentation configuration parameters to the anchor client, so that the anchor client performs fragmentation processing on the live video data according to the fragmentation configuration parameters to obtain live video data fragments;

[0165] The data segment receiving module 802 is used to receive each live video data segment in parallel, and compare the file size of the received live video data segment with the file size of the live video data segment sent by the anchor client; if the two are consistent, the reception of the live video data segment is completed;

[0166] The integrity check module 803 is used to perform integrity check on the live video data segments that have been received.

[0167] In one embodiment, see Fig. 9 , the live broadcast control instruction is to start the live broadcast; the health monitoring streaming unit 602 includes: a video data reading module 901 and a streaming start module 902.

[0168] The video data reading module 901 is used to read the corresponding live video data according to the recorded video address;

[0169] The streaming start module 902 is used to asynchronously start the streaming process of the live video data and stream the live video data to the content distribution network under health monitoring.

[0170] In one embodiment, the live broadcast control instruction is to pause the live broadcast; the health monitoring streaming unit 602 includes:

[0171] The streaming pause module is used to asynchronously pause the streaming process of the live video data and record the playing time of the live video data when the streaming is paused.

[0172] Furthermore, the live broadcast control instruction is live broadcast resumption; the health monitoring streaming unit includes:

[0173] The streaming recovery module is used to asynchronously start the continuous streaming process of the live video data according to the paused playback duration, and continue to stream the live video data to the content distribution network; wherein the continuous streaming process uses the playback duration of the live video data when the streaming is paused as the start time of this live streaming.

[0174] Furthermore, the live broadcast control instruction is to end the live broadcast; the health monitoring streaming unit includes:

[0175] The streaming end module is used to asynchronously end the streaming process of the live video data.

[0176] In one embodiment, see Fig.10 The health monitoring streaming unit 502 includes: an abnormal return module 1001, a state parameter acquisition module 1002, a health calculation module 1003 and a streaming scheduling module 1004.

[0177] The abnormal return module 1001 is used to monitor whether abnormal keywords are returned;

[0178] The state parameter acquisition module 1002 is used to obtain the resource usage status and system performance parameters of each streaming node;

[0179] A health calculation module 1003 is used to calculate the health score of each streaming node according to the resource usage status and system performance parameters;

[0180] The streaming scheduling module 1004 is used to schedule the live video data to the streaming nodes whose health scores exceed the health threshold according to the health scores of each streaming node, so as to perform live streaming based on the breakpoint resume technology.

[0181] In one embodiment, after the integrity check is performed on the received live video data segments, the method further includes:

[0182] The encoding format and encapsulation format of the live video data segments that have been received are converted.

[0183] From the hardware level, in order to reduce the requirements on the performance and network bandwidth of the anchor client device, thereby providing viewers with a stable background live streaming viewing experience, the present application provides an embodiment of an electronic device for implementing all or part of the content in the live streaming method, and the electronic device specifically includes the following content:

[0184] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the live streaming device and related devices such as the core business system, user terminal and related database; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the live streaming method and the embodiment of the live streaming device in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0185] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0186] In practical applications, part of the live streaming method can be performed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0187] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0188] Fig.11 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig.11 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig.11 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0189] In one embodiment, the live streaming method function may be integrated into the central processing unit 9100. The central processing unit 9100 may be configured to perform the following control:

[0190] S101: establishing a live broadcast room according to the live broadcast information set by the anchor client and receiving live broadcast video data uploaded by the anchor client;

[0191] S102: Perform live streaming of the live video data under health monitoring according to the live broadcast control instruction of the anchor client.

[0192] From the above description, it can be seen that the live streaming method and device provided by the present application can overcome the technical defects of the live streaming client pushing the recorded video stream, and utilize the system of pushing the recorded video stream in the background, so that the host client does not need to participate in the pushing process of the audio and video stream, and can interact with the audience or control the live streaming process only by operating the page buttons; in this case, the live streaming does not consume the network traffic of the host client, and there are no special requirements for the host's device performance; for the audience, the live viewing effect of the recorded video is completely consistent with the real-time live broadcast, which is transparent and imperceptible; it can not only support the host's non-online live broadcast, reduce the requirements for the host client device performance and network bandwidth, but also improve the audience's experience and enhance the live viewing effect.

[0193] In another embodiment, the live streaming device can be configured separately from the central processor 9100. For example, the data composite transmission device live streaming device can be configured as a chip connected to the central processor 9100, and the functions of the live streaming method can be realized through the control of the central processor.

[0194] like Fig.11 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig.11 In addition, the electronic device 9600 may also include Fig.11 For components not shown, reference may be made to the prior art.

[0195] like Fig.11 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0196] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0197] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0198] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0199] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0200] The communication module 9110 is a transmitter / receiver 9110 that sends and receives signals via an antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0201] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0202] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the live streaming method in the above embodiments, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps of the live streaming method in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0203] S101: establishing a live broadcast room according to the live broadcast information set by the anchor client and receiving live broadcast video data uploaded by the anchor client;

[0204] S102: Perform live streaming of the live video data under health monitoring according to the live broadcast control instruction of the anchor client.

[0205] From the above description, it can be seen that the live streaming method and device provided by the present application can overcome the technical defects of the live streaming client pushing the recorded video stream, and utilize the system of pushing the recorded video stream in the background, so that the host client does not need to participate in the pushing process of the audio and video stream, and can interact with the audience or control the live streaming process only by operating the page buttons; in this case, the live streaming does not consume the network traffic of the host client, and there are no special requirements for the host's device performance; for the audience, the live viewing effect of the recorded video is completely consistent with the real-time live broadcast, which is transparent and imperceptible; it can not only support the host's non-online live broadcast, reduce the requirements for the host client device performance and network bandwidth, but also improve the audience's experience and enhance the live viewing effect.

[0206] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0207] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0208] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0209] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0210] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A live streaming method, characterized in that: include: Receive live broadcast information sent by the anchor client; wherein the live broadcast information includes the live broadcast start time, the live broadcast end time, the anchor identity, the recording and broadcasting identity and the recording and broadcasting video address; Performing live broadcast time verification according to the live broadcast start time and the live broadcast end time; Performing anchor identity authentication according to the anchor identity identifier; A live broadcast room is established according to the recording and broadcasting identifier and the recording and broadcasting video address, and live broadcast video data uploaded by the anchor client is received, and then the live broadcast video data is uniformly transcoded by a video transcoding component, and then the transcoded live broadcast video data is written to a file server to generate an access address; According to the live broadcast control instruction of the anchor client, the live broadcast video data after the transcoding process is pushed live under health monitoring; wherein, the live broadcast push under health monitoring refers to pushing live according to the health status of the push node; Wherein, if the live broadcast control instruction is a live broadcast start instruction; the live broadcast streaming under health monitoring of the transcoded live broadcast video data according to the live broadcast control instruction of the anchor client includes: Read the corresponding live video data according to the recorded video address; Asynchronously starting a streaming process of the live video data, and streaming the live video data to a content distribution network under health monitoring; The receiving of live video data uploaded by the anchor client includes: Sending preset segmentation configuration parameters to the anchor client, so that the anchor client performs segmentation processing on the live video data according to the segmentation configuration parameters to obtain live video data segments; Receiving each live video data segment in parallel, and comparing the file size of the received live video data segment with the file size of the live video data segment sent by the anchor client; wherein, if the two are consistent, the reception of the live video data segment is completed; Perform integrity check on the live video data segments that have been received; wherein the integrity check is performed in the form of MD5 value comparison.

2. The live streaming method according to claim 1, characterized in that: If the live broadcast control instruction is a live broadcast pause instruction; the live broadcast streaming under health monitoring of the transcoded live video data according to the live broadcast control instruction of the anchor client also includes: Asynchronously pausing the streaming process of the live video data, and recording the playing time of the live video data when the streaming is paused.

3. The live streaming method according to claim 2, characterized in that: If the live broadcast control instruction is a live broadcast resumption instruction; the live broadcast streaming under health monitoring of the live broadcast video data after the transcoding processing according to the live broadcast control instruction of the anchor client also includes: According to the pause duration, the continuous streaming process of the live video data is asynchronously started to continue streaming the live video data to the content distribution network; wherein the continuous streaming process uses the playback duration of the live video data when streaming is paused as the start time of this live streaming.

4. The live streaming method according to claim 1, characterized in that: If the live broadcast control instruction is a live broadcast end instruction; the live broadcast streaming under health monitoring of the transcoded live video data according to the live broadcast control instruction of the anchor client also includes: Asynchronously end the streaming process of the live video data.

5. The live streaming method according to claim 1, characterized in that: The streaming of the live video data to the content distribution network under health monitoring includes: Monitor whether abnormal keywords are returned; If yes, obtain the resource usage status and system performance parameters of each streaming node; Calculate the health score of each streaming node according to the resource usage status and system performance parameters; According to the health score of each streaming node, the live video data is dispatched to the streaming node whose health score exceeds the health threshold for live streaming based on the breakpoint resume technology.

6. The live streaming method according to claim 1, characterized in that: After the integrity check is performed on the received live video data segments, the method further includes: The encoding format and encapsulation format of the live video data segments that have been received are converted.

7. A live streaming device, characterized in that: include: The live broadcast data receiving unit includes: a live broadcast information receiving module, a time verification module, an identity authentication module and a live broadcast room establishment module; The live broadcast information receiving module is used to receive the live broadcast information sent by the anchor client; wherein the live broadcast information includes the live broadcast start time, the live broadcast end time, the anchor identity, the recording and broadcasting identity and the recording and broadcasting video address; The time verification module is used to verify the live broadcast time according to the live broadcast start time and the live broadcast end time; The identity authentication module is used to perform host identity authentication according to the host identity identifier; The live broadcast room establishment module is used to establish a live broadcast room according to the recording and broadcasting identifier and the recording and broadcasting video address, and receive the live broadcast video data uploaded by the anchor client, and then perform unified transcoding processing on the live broadcast video data through the video transcoding component, and then write the transcoded live broadcast video data to the file server to generate an access address; A health monitoring streaming unit is used to perform live streaming under health monitoring on the live video data after transcoding according to the live broadcast control instruction of the anchor client; wherein the live streaming under health monitoring refers to performing live streaming according to the health status of the streaming node; Wherein, if the live broadcast control instruction is a live broadcast start instruction; the health monitoring streaming unit includes: a video data reading module and a streaming start module; The video data reading module is used to read the corresponding live video data according to the recorded video address; The streaming start module is used to asynchronously start the streaming process of the live video data, and stream the live video data to the content distribution network under health monitoring; Wherein, the live data receiving unit further includes: A fragmentation parameter sending module, used to send preset fragmentation configuration parameters to the anchor client, so that the anchor client performs fragmentation processing on the live video data according to the fragmentation configuration parameters to obtain live video data fragments; The data segment receiving module is used to receive each live video data segment in parallel, and compare the file size of the received live video data segment with the file size of the live video data segment sent by the anchor client; wherein, if the two are consistent, the reception of the live video data segment is completed; The integrity verification module is used to perform integrity verification on the live video data segments that have been received; wherein the integrity verification is carried out in the form of MD5 value comparison.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the live streaming method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the live streaming method described in any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the live streaming method described in any one of claims 1 to 6 are implemented.

Images