Live video processing method and device, equipment and storage medium

Abstract

The invention relates to a live video processing method and device, equipment and a storage medium. The live video processing method comprises the following steps: in response to a submission instruction of a first file list, creating a code stream merging container; wherein the first file list comprises a first storage path of a first video clip, and the first video clip is obtained by editing at least one live broadcast data stream fragment generated in a live broadcast process; performing code stream merging processing on the first video clip read from the first storage path through a code stream merging container to generate a playable target video clip; and transmitting the target video clip to a cloud device for storage, so that the cloud device transmits the target video clip to the terminal device when receiving an on-demand request sent by the terminal device. According to the method provided by the invention, the video can be processed in real time in the live broadcast process, the delay problem of video production is reduced, and more efficient and flexible cloud video processing capability is realized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for processing live video. Background Art

[0002] In the current cloud video transcoding processing, the common practice is to adopt a single-task single-node mode. Specifically, when a user initiates a video processing task, the system will create a new independent node for the task to execute the task, and the node will be destroyed after the task is completed. However, this mode can only transcode the live video that has ended, and cannot perform real-time transcoding and processing during the live broadcast, that is, it has obvious limitations in real-time follow-up processing. And because the transcoding task is not processed until the live broadcast ends, the final generated video file will have a large delay. Summary of the invention

[0003] In order to solve the above technical problems, the embodiments of the present disclosure provide a method, apparatus, device and storage medium for processing live video.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for processing a live video, including:

[0005] In response to a submission instruction of the first file list, a stream merging container is created; wherein the first file list includes a first storage path of a first video segment, and the first video segment is obtained by clipping at least one live data stream segment generated during the live broadcast process;

[0006] Performing stream merging processing on the first video segment read from the first storage path through the stream merging container to generate a playable target video segment;

[0007] The target video segment is transmitted to the cloud device for storage, so that the cloud device transmits the target video segment to the terminal device when receiving a video-on-demand request from the terminal device.

[0008] Optionally, in response to the submission instruction of the first file list, before creating the stream merging container, the method includes:

[0009] During the live broadcast process in the live broadcast room, at least one live data stream fragment is clipped by a production container to generate a first video segment, and a first file list is generated according to a first storage path of the first video segment, and a submit instruction is triggered at the same time;

[0010] After creating the stream merging container, the method further includes:

[0011] A stateful communication service is started between the bitstream merging container and the production container; wherein the stateful communication service is used to transmit a second file list generated by the production container to the merging container, the bitstream merging container refers to a stateful task operation container, the second file list includes a second storage path of a second video clip, and the second video clip includes at least a portion of the first video clip.

[0012] Optionally, performing stream merging processing on the first video segment read from the first storage path to generate a playable target video segment includes:

[0013] Reading a first video segment from a first storage path, and determining whether a video duration of the first video segment is greater than a set duration;

[0014] When the video duration is greater than the set duration, a target video segment of the set duration is cut out from the video duration, and the remaining video segments in the first video segment except the target video segment are stored in the target memory corresponding to the stream merging container; or

[0015] When the video duration is less than the set duration, the first video segment is stored in the target memory until the video durations of all video segments stored in the target memory reach the set duration.

[0016] Optionally, when the video duration is less than the set duration, after storing the first video segment in the target memory, the method further includes:

[0017] Obtain the submitted second file list, and read the second video segment from the second storage path included in the second file list; wherein the first video segment includes a live video segment from a starting time point to a first ending time point, and the second video segment includes a live video segment from a starting time point to a second ending time point;

[0018] Merging the second video segment with the first video segment stored in the target memory to obtain a merged video segment;

[0019] In the case that the combined video segment is longer than the set duration or a multiple of the set duration, at least one captured target video segment of the set duration is transmitted to a cloud device for storage.

[0020] Optionally, merging the second video segment with the first video segment stored in the target memory to obtain a merged video segment includes:

[0021] In the case where the second video segment has modified the content of the first video segment, comparing the first video segment and the second video segment to determine the same video segment; wherein the same video segment refers to a live video with the same content from the starting time point to the common time point in the first video segment and the second video segment;

[0022] The incremental video segment after the common time point in the second video segment and the same video segment are merged to generate a merged video segment.

[0023] Optionally, before transmitting the captured at least one target video segment of a set duration to a cloud device for storage, the method further includes:

[0024] Obtain the third end time point of the previous video clip stored in the cloud device;

[0025] When the third end time point is greater than the common time point, a segment of live video from the common time point to the third end time point stored in the cloud device is deleted.

[0026] Optionally, merging the second video segment with the first video segment stored in the target memory to obtain a merged video segment includes:

[0027] Determining an incremental video segment of the second video segment relative to the first video segment when the second video segment does not modify the content of the first video segment;

[0028] The incremental video segments are merged to obtain a merged video segment.

[0029] Optionally, in response to the submission instruction of the first file list, before creating the stream merging container, the method further includes:

[0030] Displaying a production page; wherein the production page includes a preview area and a shard area, the preview area is used to display at least one live data stream shard, and each live data stream shard has a non-repeating serial number;

[0031] In response to a trigger operation of submitting an identifier in a sharding area, obtaining sharding information set in the sharding area; wherein the sharding information includes a first sequence number set at a sharding entry point and a second sequence number set at a sharding exit point;

[0032] Determine all serial numbers between the first serial number and the second serial number, and generate a first video segment according to all live data stream segments corresponding to all serial numbers.

[0033] Optionally, the first video segment includes at least one video fragment, each video fragment is composed of all live data stream fragments corresponding to consecutive sequence numbers, and the first video segment read from the first storage path is subjected to stream merging processing to generate a playable target video segment, including:

[0034] Reading at least one video segment from a first storage path;

[0035] Perform bitstream merging processing on at least one video segment to obtain at least one target video segment of a set duration and transmit it to a cloud device for storage.

[0036] In a second aspect, an embodiment of the present disclosure provides a live video processing device, the device comprising:

[0037] A creating unit, configured to create a stream merging container in response to a submission instruction of a first file list; wherein the first file list includes a first storage path of a first video segment, and the first video segment is obtained by clipping at least one live data stream segment generated during a live broadcast process;

[0038] A processing unit, configured to merge the first video segments read from the first storage path by using a merge container to obtain a playable target video segment;

[0039] The storage unit is used to transmit the target video segment to the cloud device for storage, so that the cloud device can transmit the target video segment to the terminal device when receiving the on-demand request sent by the terminal device.

[0040] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0041] Memory;

[0042] Processor; and

[0043] Computer programs;

[0044] The computer program is stored in the memory and is configured to be executed by the processor to implement the method of the first aspect.

[0045] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method of the first aspect are implemented.

[0046] The live video processing method provided by the present disclosure includes: creating a stream merging container in response to a submission instruction of a first file list; wherein the first file list includes a first storage path of a first video clip, and the first video clip is obtained after editing at least one live data stream fragment generated during the live broadcast process; performing stream merging processing on the first video clip read from the first storage path through the stream merging container to generate a playable target video clip; transmitting the target video clip to a cloud device for storage, so that the cloud device transmits the target video clip to the terminal device when receiving a video-on-demand request from the terminal device. The method provided by the present application can perform real-time video processing such as transcoding and editing during the live broadcast process, reduce the delay problem of video production, and achieve more efficient and flexible cloud video processing capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 A flowchart of a method for processing live video provided by an embodiment of the present disclosure;

[0050] Figure 2 A schematic diagram of making a page provided in an embodiment of the present disclosure;

[0051] Figure 3 A timing diagram of processing a live video provided by an embodiment of the present disclosure;

[0052] Figure 4 A live video processing architecture diagram provided by an embodiment of the present disclosure;

[0053] Figure 5 A diagram of an application scenario of live video provided by an embodiment of the present disclosure;

[0054] Figure 6 for Figure 1 A detailed flow chart of S102 in a method for processing live video is shown;

[0055] Figure 7 A data diagram of live video processing provided by an embodiment of the present disclosure;

[0056] Figure 8 A schematic diagram of the structure of a live video processing device provided by an embodiment of the present disclosure;

[0057] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0059] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0060] In view of the above technical problems, the present disclosure provides a method for processing live video, which is described in detail through one or more of the following embodiments.

[0061] The live video processing method provided by the embodiment of the present disclosure can be applied to the video production scene in the live broadcast process. The method can be performed by a live video processing device, which can be implemented by software and / or hardware, and the device can be integrated in an electronic device. Among them, the electronic device can include but is not limited to mobile terminals such as smart phones, laptops, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA), tablet computers (Tablet Personal Computer, Tablet PC), PMP (portable multimedia player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable devices, etc., and fixed terminals such as digital televisions, desktop computers, smart home devices, etc.

[0062] Figure 1 A flowchart of a method for processing live video provided by an embodiment of the present disclosure specifically includes the following steps: Figure 1 The following steps are shown:

[0063] S101: In response to a submission instruction of a first file list, create a stream merging container.

[0064] The first file list includes a first storage path of a first video segment, and the first video segment is obtained by editing at least one live data stream segment generated during the live broadcast process.

[0065] It is understandable that after the live broadcast begins, live data stream fragments will be generated in real time. Live data stream fragmentation refers to dividing the continuous live video stream into multiple small, fixed-length segments or segments. Each segment is usually an independent media file, containing a few seconds to tens of seconds of video content. Multiple live data stream fragments will constitute a real-time live video file. A real-time live video file refers to a complete and uninterrupted video file. For example, when the broadcast starts for 20 minutes, a 20-minute real-time live video file will be generated. During the live broadcast, the live data stream fragments are edited to obtain live video clips. For example, all live data stream fragments in the first 10 minutes of the live broadcast are edited for the first time to obtain the first live video clip, which is recorded as the first video clip. Editing refers to extracting, editing and processing specific clips from real-time streaming media or recorded live videos to create shorter and more targeted content. The first video clip is then stored in a distributed file system (Ceph File System, cephfs), and after generating a first file list according to the first storage path of the first video clip in cephfs, the first file list is submitted, wherein the first file list is a list corresponding to the first live video clip submitted, and multiple file lists will be submitted later until a submission instruction indicating the end of the live broadcast or the end of the video processing is received. In response to the submission instruction of the first file list, a stream merging container is created. The stream merging container can be understood as a pod node, and the specific creation process is not described in detail. Alternatively, before the first file list is submitted, at least one stream merging container is pre-created, and then in response to the submission instruction, a stream merging container dedicated to the live broadcast room is determined from at least one pre-created stream merging container. Alternatively, at the beginning of the live broadcast, a submission instruction indicating the creation of a stream merging container is generated.

[0066] Optionally, in response to the submission instruction of the first file list, before creating the stream merging container, the method includes:

[0067] During the live broadcast process, at least one live broadcast data stream fragment is edited by a production container to generate a first video segment, and a first file list is generated according to a first storage path of the first video segment, and a submission instruction is triggered at the same time.

[0068] It is understandable that the production container is an exclusive container created for the target user, and the target user can exclusively use the resources of the production container. Specifically, the target user can perform real-time editing and processing on at least one live data stream segment generated during the live broadcast process based on the production container. For example, during the live broadcast process, at least one live data stream segment is edited and processed through the production container to generate a first video segment, such as clipping processing, subtitle recognition processing, etc. Subsequently, the first file list is generated according to the first storage path of the first video segment in cephfs through the production container, and the first file list is submitted at the same time, triggering the submission instruction.

[0069] Optionally, after creating the stream merging container, the method further includes:

[0070] A stateful communication service is started between a bitstream merging container and a production container; wherein the stateful communication service is used to transmit a second file list generated by the production container to the merging container, the bitstream merging container refers to a stateful task operation container, the second file list includes a second storage path of a second video clip, and the second video clip includes at least a portion of the first video clip.

[0071] It is understandable that after the creation of the stream merging container is completed, a stateful communication service, such as a stateful http (HyperText Transfer Protocol) service, is enabled inside the stream merging container and the production container, and subsequent live video clips are continuously sent to the stream merging container through real-time http communication, so as to achieve the purpose of real-time production of live video. Among them, the stateful http service refers to maintaining the status information of the merging process between multiple requests, such as the progress of the video merging, the number of processed video clips, and other contextual information, so as to facilitate the subsequent stream merging container to realize the stream merging processing between multiple video clips submitted multiple times.

[0072] Optionally, in response to the submission instruction of the first file list, before creating the stream merging container, the method further includes:

[0073] Display a production page; wherein the production page includes a preview area and a segment area, the preview area is used to display at least one live data stream segment of a real-time live video file, and each live data stream segment has a non-repeating serial number; in response to a trigger operation of submitting an identifier in the segment area, obtain the segment information set in the segment area; wherein the segment information includes a first serial number set at a segment entry point and a second serial number set at a segment exit point; determine all serial numbers between the first serial number and the second serial number, and generate a first video segment according to all live data stream segments corresponding to all serial numbers.

[0074] It is understandable that the production page refers to the page for producing a production platform, and the production platform includes a production container and a stream merging container. Among them, the production page includes a preview area and a slicing area. The preview area displays the live data stream slicing. Each live data stream slicing has a serial number. The serial numbers of multiple live data stream slicing are continuous and non-repetitive. The serial number can be specifically determined according to the live broadcast time corresponding to each live data stream slicing. This form of previewing multiple live data stream slicing is convenient for users to intuitively and comprehensively understand the live video content. For example, each live data stream slicing is set to have a time of 2 seconds. There are 30 live data stream slicings in a 1-minute real-time live video file. The 30 live data stream slicings can be set with serial numbers 0-29 in sequence. The slicing area refers to the area where the slicing information is determined according to the serial number of each live data stream slicing. The slicing area specifically includes the slicing entry point setting area and the slicing exit point setting area. The serial number can be edited in each setting area. After the sequence number editing is completed in the segment area, the submission mark is triggered. In response to the triggering operation of the submission mark, the segment information set in the segment area is obtained, and the segment information includes a first sequence number set at the segment entry point and a second sequence number set at the segment exit point. All sequence numbers between the first sequence number and the second sequence number are counted, such as the first sequence number is 5, the second sequence number is 7, and there are a total of 3 sequence numbers between sequence number 5 and sequence number 7. After all the sequence numbers are determined, the first video segment is generated according to all the live data stream segments corresponding to all the sequence numbers.

[0075] The first video segment includes at least one video fragment, and each video fragment is composed of all live data stream fragments corresponding to continuous sequence numbers.

[0076] It is understandable that the first video segment includes at least one video fragment, and each video fragment is composed of all live data stream fragments corresponding to continuous serial numbers, that is, multiple fragment information can be set based on the fragment area to generate multiple video fragments.

[0077] For example, see Figure 2 , Figure 2 A schematic diagram of a production page provided in an embodiment of the present disclosure, the production page includes a preview area and a segment area, the preview area displays multiple live data stream segments and video information of the multiple live data stream segments, the video information includes the serial number of each live data stream segment and the live broadcast duration corresponding to each live data stream segment, such as serial number 5 to serial number 12, taking each live data stream segment as an example of 2 seconds, the live broadcast duration corresponding to serial number 5 is 00:00:12, that is, serial number 5 is the live data stream segment when the live broadcast reaches 12 seconds, such as Figure 2As shown in the figure, the live broadcast duration corresponding to serial number 12 is 00:00:26, and the live broadcast duration corresponding to serial number 12 is 00:00:28. The segment area displays two editing areas and a submission mark. Each editing area can set the serial number of the segment entry point and segment exit point, and each editing area corresponds to a video segment. Figure 2 As shown, the first fragment information obtained by editing the editing area 1 includes information that the first serial number is 5 and the second serial number is 7, and the three live data stream fragments corresponding to the three consecutive serial numbers of serial number 5, serial number 6 and serial number 7 can constitute the first video fragment, and the second fragment information obtained by editing the editing area 2 includes information that the first serial number is 10 and the second serial number is 12, and the three live data stream fragments corresponding to the three consecutive serial numbers of serial number 10, serial number 11 and serial number 12 can constitute the second video fragment, and the first video segment generated after triggering the submission mark includes video fragment 1 of 12 seconds to 16 seconds and video fragment 2 of 22 seconds to 26 seconds. It can be understood that the number of editing areas set in the fragment area is not limited and can be set according to user needs.

[0078] S102: Merge the first video segments read from the first storage path using a stream merging container to generate a playable target video segment.

[0079] The target video segment includes at least a portion of the first video segment.

[0080] It is understandable that, based on the above S101, the stream merging container reads the first video segment from the cephfs based on the first storage path, and then performs merging processing on the first video segment to generate a playable target video segment.

[0081] S103: Upload the target video clip to the cloud device for storage, so that the cloud device can transmit the target video clip to the terminal device when receiving a video-on-demand request from the terminal device.

[0082] It is understandable that, based on the above S102, after uploading the target video clip to the cloud device, the real-time generated video clip can be released during the live broadcast process. Subsequently, when the cloud device receives a video-on-demand request from the terminal device, it transmits the stored target video clip to the terminal device to realize synchronous viewing of the live video on the terminal device, such as wherein the video-on-demand request (Video on Demand, VOD) refers to a service mode in which users can choose to watch video content at any time according to their needs.

[0083] Optionally, performing stream merging processing on the first video segment read from the first storage path to generate a playable target video segment may be specifically implemented through the following steps:

[0084] At least one video segment is read from the first storage path, and a code stream merging process is performed on the at least one video segment to obtain at least one target video segment of a set duration.

[0085] Understandably, the set duration can be 6 minutes, that is, the video clip is transmitted to the cloud device with a 6-minute video clip as the transmission unit. For example, if the total duration of the first video clip is 10 minutes, the first 6-minute video clip is intercepted and uploaded to the cloud device, and the last 4-minute video clip is stored in the memory of the stream merging container until the 4-minute video clip can be synthesized into a 6-minute video clip based on the video clip submitted next time and uploaded to the cloud device, that is, the target video clip includes at least part of the first video clip. The cloud device can be a cloud server that supports a public cloud content delivery network (Content Delivery Network, CDN).

[0086] It is understandable that, in the case where the first video segment includes at least one video fragment, the code stream merging container performs code stream merging processing on the at least one video fragment, obtains at least one target video segment of a set duration and transmits it to the cloud device. For example, if the first video segment includes a first video segment of 2 minutes and a second video segment of 5 minutes, the code stream merging container performs code stream merging processing on the two video segments, and extracts a 6-minute target video segment from the 7-minute video segment and uploads it to the cloud device, and the remaining 1-minute video segment is stored in the memory of the code stream merging container, waiting for the next code stream merging processing.

[0087] For example, see Figure 3 , Figure 3 A processing sequence diagram of a live video provided in an embodiment of the present disclosure, the production container issues a submission instruction after the live broadcast starts on the user side to create a stream merging container, the stream merging container performs stream merging processing on the live data stream fragments according to the file list submitted by the production container, and uploads the real-time operation stream video files (video clips) to the public cloud CDN, and continuously submits production video clips as the live broadcast progresses, and finally generates a complete live video. After receiving the live broadcast end instruction submitted by the user side, the production container recycles the stream merging container, which can be understood as a task work container. If the production container has a task that needs to be processed by the stream merging container, it will create a dedicated stream merging container for the task, and release the dedicated stream merging container when the task ends.

[0088] For example, see Figure 4 , Figure 4A processing architecture diagram of a live video provided in an embodiment of the present disclosure is provided. The live data stream fragments are previewed based on the production page. At this time, the live data stream fragments have been collected and transcoded from the live signal and stored in cephfs; the live data stream fragments are edited and production instructions are generated; a production container is built based on the production instructions; a stream merging container is built based on the submission instructions generated by the production container; the stream merging container reads the produced video clips from cephfs to perform stream merging processing; and the merged target video clips are uploaded to the public cloud CDN.

[0089] For example, see Figure 5 , Figure 5 An application scenario diagram of a live video provided by an embodiment of the present disclosure, wherein satellite TV signals, cable TV signals and / or network live broadcast signals are transmitted to a production platform through a variety of protocols after signal conversion, and the production platform is used to generate video clips, wherein the production platform includes a scheduling node, a collection node, a transcoding node, a production node and cephfs, and uses the services provided by the scheduling node, the collection node and the transcoding node through the collection list page to process the collected signals, generate live data stream fragments, and store the live data stream fragments in cephfs. Use the services provided by the production node through the production page to clip, edit and process the live data stream fragments stored in cephfs, generate video clips, and upload the video clips to the program library and material library of the cloud device, wherein the production node includes a production container and a stream merging container.

[0090] The live video processing method provided by the present disclosure can complete the preview, pre-production, pre-production, pre-distribution and other video strip processing through the constructed production container and stream merging container during the live broadcast process. There is no need to wait until the live broadcast ends before stripping the video files that have ended the live broadcast, which improves the live video processing efficiency and realizes the cloud processing of real-time follow-up production. In addition, the production of subsequent live videos is continuously sent to the stream merging container through real-time http communication, achieving the purpose of real-time production. When the live broadcast ends, the stream merging container exits after the work is completed, and the resources are recycled, realizing the reasonable allocation and rational use of resources, and improving the live broadcast production efficiency and video online efficiency.

[0091] Based on the above embodiments, Figure 6 for Figure 1 The detailed flow diagram of S102 in a live video processing method shown in FIG. 1 is to perform a stream merging process on the first video segment read from the first storage path to generate a playable target video segment. The stream merging process is performed by using a stream merging container and specifically includes the following steps: Figure 6 The following steps are shown:

[0092] S601: Read a first video segment from a first storage path, and determine whether a video duration of the first video segment is greater than a set duration.

[0093] It can be understood that the first video clip is a live video clip submitted after video editing. After reading the first video clip, it is determined whether the video length of the first video clip is greater than the set length. Based on the above example, it is determined whether the total length of the first video clip is greater than 6 minutes to determine whether to upload it directly to the cloud device and / or wait to be merged with other video clips before uploading.

[0094] S602: When the video duration is greater than the set duration, a target video segment of the set duration is cut out from the video duration, and the remaining video segments in the first video segment except the target video segment are stored in a target memory corresponding to the stream merging container.

[0095] Among them, the video duration of the remaining video clips is less than the set duration.

[0096] It is understandable that, based on the above S601, if the video duration is greater than the set duration, at least one target video segment of the set duration is cut out from the video duration. For example, if the video duration is 13 minutes, which is twice as long as 6 minutes, two target video segments can be cut out. After determining the target video segment, the remaining video segments in the first video segment except the target video segment are stored in the memory of the stream merging container. For example, in the above example, the duration of the remaining video segments is 1 minute, and at this time, there are more video segments than multiples of 6 minutes in the memory.

[0097] It is understandable that if there are no remaining video segments of the first video segment, or the video duration of the first video segment is equal to the set duration, there is no need to perform the storage operation of the remaining video segments in the memory.

[0098] S603: When the video duration is less than the set duration, the first video segment is stored in the target memory until the video duration of all video segments stored in the target memory reaches the set duration.

[0099] It can be understood that, based on the above S601, if the video length is less than the set length, the first video segment is directly stored in the memory, waiting for the next video segment to be submitted, until the total length of the video segment stored in the memory and the next video segment reaches the set length, and it is transmitted to the cloud device, or, after the live broadcast ends, it is transmitted to the cloud device.

[0100] Optionally, when the video duration is less than the set duration, after storing the first video segment in the target memory, the method further includes:

[0101] Obtain a submitted second file list, and read a second video segment from a second storage path included in the second file list; wherein the first video segment includes a live video segment from a starting time point to a first ending time point, and the second video segment includes a live video segment from a starting time point to a second ending time point; merge the second video segment and the first video segment stored in the target memory to obtain a merged video segment; when the merged video segment is greater than a set duration or a multiple of a set duration, transmit at least one captured target video segment of a set duration to a cloud device for storage.

[0102] It is understandable that when the video duration is less than the set duration and the live broadcast has not ended, the second file list submitted by the production container is obtained. The second file list can be understood as the file list submitted for the second time, that is, the next file list of the first file list. The second file list includes the second storage path of the second video clip in cephfs, wherein the first video clip includes a live video from the start time point to the first end time point, the start time point is the live broadcast start time point or the submission time point of the merge task, and the first end time point is a certain time point in the live broadcast process, such as the first video clip is a video clip of the live broadcast period of 0 to 10 minutes, the start time point is 00:00:00, the first end time point is 00:10:00, that is, the first video clip is a video clip of 10 minutes of live broadcast. The second video clip refers to a live video from the start time point to the second end time point, such as the second video clip is a video clip of the live broadcast period of 0 to 15 minutes, the second end time point is 00:15:00, that is, the second video clip is a video clip of 15 minutes of live broadcast, in this case, the second end time is greater than the first end time in timing. Another possible situation is that the second video segment is a video segment of the live broadcast period from 0 to 5 minutes after the start of the live broadcast, and the second end time point is 00:5:00, that is, the second video segment is a video segment of 5 minutes of live broadcast. In this case, the second end time is shorter than the first end time in terms of timing, that is, there is no temporal limitation relationship between the duration and the end time point of each submitted video segment, and it can be set according to the user's editing needs. After reading the second video segment, the second video segment and the first video segment stored in the memory are merged to generate a merged video segment, and at least one target video segment is cut out from the merged video segment. If the video duration of the merged video segment does not reach the set duration, the merged video segment continues to be stored in the memory. If the video duration of the merged video segment reaches the set duration, the target video segment of the video duration is uploaded to the cloud device, and the remaining video segments continue to be stored in the memory.

[0103] Optionally, merging the second video segment with the first video segment stored in the target memory to obtain a merged video segment may be specifically implemented through the following steps:

[0104] In the case where the second video segment has modified the content of the first video segment, the first video segment and the second video segment are compared to determine the same video segment; wherein the same video segment refers to a live video with the same content from the start time point to the common time point in the first video segment and the second video segment; the incremental video segment after the common time point in the second video segment and the same video segment are merged to generate a merged video segment. Alternatively, in the case where the second video segment has not modified the content of the first video segment, the incremental video segment of the second video segment relative to the first video segment is determined, and the incremental video segments are merged to obtain a merged video segment.

[0105] It is understandable that when merging the second video segment with the first video segment, it is necessary to determine the incremental video segment between the two video segments, that is, each submitted video segment starts from the starting time point, but the respective end time points are mostly different, and there are common video segments and incremental video segments between the subsequently submitted video segments and the previously submitted video segments. For the incremental video segments, if the second video segment only adds other video segments on the basis of the first video segment, that is, the second video segment completely covers the first video segment, in this case, it is sufficient to directly merge the incremental video segments of the first video segment and the second video segment. If the second video segment has modified the content submitted for the first video segment, it needs to be merged again. In this case, the first video segment and the second video segment are compared to determine the same video segment. The same video segment is a live video starting from the starting time point and ending at the common time point. The content after the common time point is modified. It is necessary to delete the content after the common time point in the first video segment, and use the content after the common time point in the second video segment as the incremental video segment. For example, if the first video segment is a 10-minute live video, and the second video segment is a 15-minute live video, starting from the 5th minute, the content of the first video segment is modified in the second video segment, and the 5th minute is the common time point, the video segments from 0 to 5 minutes are the same video segments, and the video segments from 5 to 15 minutes in the second video segment are the incremental video segments. The video segments from 5 to 10 minutes in the first video segment need to be deleted, and then the video segments from 0 to 5 minutes (the same video segment) and the video segments from 5 to 15 minutes in the second video segment (the incremental video segment) are merged to generate a 15-minute merged video segment. Then continue to submit new video segments until the live broadcast ends, the task ends, or the last file list ends.

[0106] Optionally, before transmitting the captured at least one target video segment of a set duration to a cloud device for storage, the method further includes:

[0107] Obtain the third end time point of the previous video segment stored in the cloud device; when the third end time point is greater than the common time point, delete a segment of live video from the common time point to the third end time point stored in the cloud device.

[0108] Understandably, the third end time point of the previous video clip stored in the cloud device is obtained. The previous video clip refers to a video clip stored at the most recent time before the target video clip is stored. The third end time and the common time point are compared in terms of timing. If the third end time point is greater than the common time point, it means that the video clip that has been executed for cloud storage is later than the common time point. It is necessary to delete the common time point and the third end time point from the cloud storage data. Based on the above example, the previous video clip stored in the cloud refers to the video clip from the 0th minute to the 8th minute, the third end time point is the 8th minute, the common time point is the 5th minute, and the third end time point is greater than the common time point. Since the video clip from the 5th minute to the 8th minute has been modified, it is necessary to delete this part of the video clip from the cloud storage, and then re-store the video clip from the common time point to the second end time point in the second video clip to the cloud device. Alternatively, the deletion operation is not performed on the cloud device, and the video clip stored in the cloud device is directly overwritten. Other possible steps for processing in the cloud are not limited.

[0109] For example, see Figure 7 , Figure 7 A data diagram of a live video processing provided by an embodiment of the present disclosure is shown in FIG. 1 . The production container submits a file list to the stream merging container in a manner of multiple shard submissions, such as Figure 7 As shown, segment submission 1 submits file list 1 to the stream merging container. The stream merging container reads video clips from cephfs based on the storage path in file list 1, and uploads video clips with a video length of 6 minutes to the public cloud CDN. After the merge processing of file list 1 is completed, the merge process in the stream merging container cyclically waits for file list 2 submitted by segment submission 2, determines the incremental video clips of file list 1 and file list 2, continues to upload 6 minutes of video clips and waits for the next file list, and ends the merge process when the end mark is received. The stream merging process of file list N by the stream merging container is not described in detail here.

[0110] A method for processing live video provided by an embodiment of the present disclosure can not only perform real-time strip splitting during the live broadcast process, but also complete the stream merging processing of video segments submitted multiple times through the merging process in the stream merging container, and can also modify the previously merged video segments during the merging process, thereby improving the video production efficiency and making the production process more flexible.

[0111] Based on the above embodiments, Figure 8 The structure diagram of a live video processing device provided by an embodiment of the present disclosure is as follows. The live video processing device provided by an embodiment of the present disclosure can execute the processing flow provided by the live video processing method embodiment, such as Figure 8 As shown, the live video processing device 800 includes a creation unit 801, a processing unit 802 and a storage unit 803, wherein:

[0112] The creation unit 801 is configured to create a stream merging container in response to a submission instruction of a first file list; wherein the first file list includes a first storage path of a first video segment, and the first video segment is obtained by clipping at least one live data stream segment generated during a live broadcast process;

[0113] The processing unit 802 is configured to perform a stream merging process on the first video segment read from the first storage path through a stream merging container to generate a playable target video segment;

[0114] The storage unit 803 is used to transmit the target video segment to the cloud device for storage, so that the cloud device can transmit the target video segment to the terminal device when receiving a video-on-demand request from the terminal device.

[0115] Optionally, the live video processing device 800 is further used for:

[0116] During the live broadcast process, at least one live broadcast data stream fragment is clipped by a production container to generate a first video segment, and a first file list is generated according to a first storage path of the first video segment, and a submit instruction is triggered at the same time;

[0117] Optionally, the live video processing device 800 is further used for:

[0118] A stateful communication service is started between the bitstream merging container and the production container; wherein the stateful communication service is used to transmit a second file list generated by the production container to the merging container, the bitstream merging container refers to a stateful task operation container, the second file list includes a second storage path of a second video clip, and the second video clip includes at least a portion of the first video clip.

[0119] Optionally, the processing unit 802 is configured to:

[0120] Reading a first video segment from a first storage path, and determining whether a video duration of the first video segment is greater than a set duration;

[0121] When the video duration is longer than the set duration, a target video segment of a multiple of the set duration is cut out from the video duration, and the target video segment is transmitted to the cloud for storage, and the remaining video segments in the first video segment except the target video segment are stored in the target memory corresponding to the stream merging container; or

[0122] When the video duration is less than the set duration, the first video segment is stored in the target memory until the video durations of all video segments stored in the target memory reach the set duration.

[0123] Optionally, the processing unit 802 is further configured to:

[0124] Obtain the submitted second file list, and read the second video segment from the second storage path included in the second file list; wherein the first video segment includes a live video segment from a starting time point to a first ending time point, and the second video segment includes a live video segment from a starting time point to a second ending time point;

[0125] Merging the second video segment with the first video segment stored in the target memory to obtain a merged video segment;

[0126] In the case that the combined video segment is longer than the set duration or a multiple of the set duration, at least one captured target video segment of the set duration is transmitted to a cloud device for storage.

[0127] Optionally, the processing unit 802 is further configured to:

[0128] In the case where the second video segment has modified the content of the first video segment, comparing the first video segment and the second video segment to determine the same video segment; wherein the same video segment refers to a live video with the same content from the starting time point to the common time point in the first video segment and the second video segment;

[0129] The incremental video segment after the common time point in the second video segment and the same video segment are merged to generate a merged video segment.

[0130] Optionally, the processing unit 802 is further configured to:

[0131] Obtain the third end time point of the previous video clip stored in the cloud device;

[0132] When the third end time point is greater than the common time point, a segment of live video from the common time point to the third end time point stored in the cloud device is deleted.

[0133] Optionally, the processing unit 802 is further configured to:

[0134] Determining an incremental video segment of the second video segment relative to the first video segment when the second video segment does not modify the content of the first video segment;

[0135] The incremental video segments are merged to obtain a merged video segment.

[0136] Optionally, the live video processing device 800 is further used for:

[0137] Displaying a production page; wherein the production page includes a preview area and a segment area, the preview area is used to display multiple live data stream segments included in the real-time live video file, and each of the multiple live data stream segments has a non-repeating serial number;

[0138] In response to a trigger operation of submitting an identifier in a sharding area, obtaining sharding information set in the sharding area; wherein the sharding information includes a first sequence number set at a sharding entry point and a second sequence number set at a sharding exit point;

[0139] Determine all serial numbers between the first serial number and the second serial number, and generate a first video segment according to all live data stream segments corresponding to all serial numbers.

[0140] The first video segment includes at least one video fragment, and each video fragment is composed of all live data stream fragments corresponding to continuous sequence numbers.

[0141] Optionally, the processing unit 802 is further configured to:

[0142] At least one video segment is read from the first storage path; and a bitstream merging process is performed on the at least one video segment to obtain at least one target video segment of a set duration and transmit it to a cloud device for storage.

[0143] Figure 8 The live video processing device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0144] Fig. 9 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Fig. 9, which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure. The electronic device 900 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0145] like Fig. 9 As shown, the electronic device 900 may include a processing device 901 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903 to implement the live video processing method of the embodiment described in the present disclosure. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0146] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0147] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the live video processing method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0148] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0149] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0150] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0151] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0152] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0153] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0154] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

[0155] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0156] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0157] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or gateway that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or gateway. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or gateway that includes the elements.

[0158] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing live video, characterized in that: The method comprises: In response to a submission instruction of a first file list, creating a stream merging container; wherein the first file list includes a first storage path of a first video segment, and the first video segment is obtained by clipping at least one live data stream segment generated during a live broadcast process; Performing code stream merging processing on the first video segment read from the first storage path by using the code stream merging container to generate a playable target video segment; The target video segment is transmitted to a cloud device for storage, so that the cloud device transmits the target video segment to the terminal device when receiving a video-on-demand request from the terminal device.

2. The method according to claim 1, characterized in that Before creating the stream merging container in response to the submit instruction of the first file list, the method further includes: During the live broadcast process, at least one live broadcast data stream fragment is clipped by a production container to generate a first video segment, and a first file list is generated according to a first storage path of the first video segment, and a submit instruction is triggered at the same time; After creating the stream merging container, the method further includes: A stateful communication service is started between the bitstream merging container and the production container; wherein the stateful communication service is used to transmit a second file list generated by the production container to the bitstream merging container, the bitstream merging container is a stateful task operation container, the second file list includes a second storage path of a second video clip, and the second video clip includes at least part of the first video clip.

3. The method according to claim 1, characterized in that Performing code stream merging processing on the first video segment read from the first storage path to generate a playable target video segment includes: Reading the first video segment from the first storage path, and determining whether the video duration of the first video segment is greater than a set duration; In the case where the video duration is greater than the set duration, a target video segment of the set duration is cut out from the video duration, and the remaining video segments of the first video segment except the target video segment are stored in the target memory corresponding to the stream merging container; or In a case where the video duration is less than the set duration, the first video segment is stored in the target memory until the video durations of all video segments stored in the target memory reach the set duration.

4. The method according to claim 3, characterized in that In the case where the video duration is less than the set duration, after storing the first video segment in the target memory, the method further includes: Obtain the submitted second file list, and read the second video segment from the second storage path included in the second file list; wherein the first video segment includes a live video segment from a starting time point to a first ending time point, and the second video segment includes a live video segment from the starting time point to a second ending time point; Merging the second video segment with the first video segment stored in the target memory to obtain a merged video segment; In the case that the combined video segment is longer than the set duration or a multiple of the set duration, at least one captured target video segment of the set duration is transmitted to the cloud device for storage.

5. The method according to claim 4, characterized in that The step of merging the second video segment with the first video segment stored in the target memory to obtain a merged video segment includes: In the case where the second video segment has modified the content of the first video segment, comparing the first video segment and the second video segment to determine the same video segment; wherein the same video segment refers to a live video segment with the same content from the starting time point to the common time point in the first video segment and the second video segment; The incremental video segment after the common time point in the second video segment and the same video segment are merged to generate a merged video segment.

6. The method according to claim 5, characterized in that Before transmitting the captured at least one target video segment of the set duration to the cloud device for storage, the method further includes: Obtaining a third end time point of a previous video clip stored in the cloud device; When the third end time point is greater than the common time point, a live video segment from the common time point to the third end time point stored in the cloud device is deleted.

7. The method according to claim 4, wherein merging the second video segment with the first video segment stored in the target memory to obtain a merged video segment comprises: In a case where the second video segment does not modify the content of the first video segment, determining an incremental video segment of the second video segment relative to the first video segment; The incremental video segments are merged to obtain a merged video segment.

8. The method according to claim 1, characterized in that Before creating the stream merging container in response to the submit instruction of the first file list, the method further includes: Displaying a production page; wherein the production page includes a preview area and a fragment area, the preview area is used to display the at least one live data stream fragment, and each live data stream fragment has a non-repeating serial number; In response to a trigger operation of submitting an identifier in the sharding area, acquiring sharding information set in the sharding area; wherein the sharding information includes a first sequence number set at a sharding entry point and a second sequence number set at a sharding exit point; Determine all sequence numbers between the first sequence number and the second sequence number, and generate a first video segment based on all live data stream fragments corresponding to all the sequence numbers.

9. The method according to claim 8, characterized in that The first video segment includes at least one video fragment, each video fragment is composed of all live data stream fragments corresponding to consecutive sequence numbers, and the first video segment read from the first storage path is subjected to stream merging processing to generate a playable target video segment, including: Reading the at least one video segment from the first storage path; The at least one video segment is subjected to code stream merging processing to obtain at least one target video segment of a set duration.

10. A live video processing device, characterized in that: The device comprises: A creating unit, configured to create a stream merging container in response to a submission instruction of a first file list; wherein the first file list includes a first storage path of a first video segment, and the first video segment is obtained by clipping at least one live data stream segment generated during a live broadcast process; a processing unit, configured to perform a code stream merging process on the first video segment read from the first storage path through the code stream merging container to generate a playable target video segment; The storage unit is used to transmit the target video segment to the cloud device for storage, so that the cloud device transmits the target video segment to the terminal device when receiving a video-on-demand request from the terminal device.

11. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the live video processing method as described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the live video processing method as described in any one of claims 1 to 9 are implemented.

Images