A video stream processing method and device, electronic equipment and storage medium

Abstract

A video stream processing method and device, electronic equipment and storage medium are disclosed. The method is applied to a server or a video encoding end, and includes: receiving a mixed video stream sent by a shooting device; unpacking the mixed video stream to obtain at least one intra-coded frame, at least one first forward prediction coded frame, at least one second forward prediction coded frame, and at least one bidirectional prediction coded frame; copying the intra-coded frame, and packaging the original intra-coded frame and the at least one first forward prediction coded frame as a live video stream for video live broadcast; and packaging the copied intra-coded frame, the at least one second forward prediction coded frame, and the at least one bidirectional prediction coded frame as a storage video stream for local storage. The technical solution of the present application can ensure the real-time performance of video playing while reducing the storage capacity of video data.

Description

Technical Field

[0001] This invention relates to the field of multimedia technology, and in particular to a method, apparatus, electronic device, and storage medium for processing video streams. Background Technology

[0002] In the field of multimedia systems, the shooting device can upload video streams to a server or video encoding terminal, which then forwards the video streams to other terminals for live playback. At the same time, the server or video encoding terminal stores the video data.

[0003] Considering the real-time and smooth playback of live video, the shooting device typically only encodes I-frames and P-frames (intra-coded frames and forward predictive coded frames) when uploading the video stream, while omitting B-frames (bidirectional predictive coded frames). However, omitting B-frame encoding results in a higher bitrate for the video stream, consuming more storage capacity when storing video data on the server or at the video encoding end. Summary of the Invention

[0004] This invention provides a video stream processing method, apparatus, electronic device, and storage medium to reduce the data storage capacity required for video while ensuring real-time video playback.

[0005] In a first aspect, embodiments of the present invention provide a method for processing a video stream, which is applied to a server or a video encoding end, and the method includes:

[0006] Receive the mixed video stream sent by the shooting device;

[0007] The hybrid video stream is decapsulated to obtain at least one intra-coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame.

[0008] The intra-coded frame is copied, and the original intra-coded frame and at least one first forward predictive coded frame are encapsulated into a live video stream for live video playback.

[0009] The copied intra-coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame are encapsulated into a storage video stream for local storage.

[0010] Secondly, embodiments of the present invention provide a video stream processing method, which is applied to a shooting device, and the method includes:

[0011] The original video frame images are encoded to obtain at least one intra-coded frame;

[0012] Based on the intra-coded frames, first mode coding is performed to obtain at least one first forward predictive coded frame; wherein, the first mode coding is inter-frame compression coding without using bidirectional predictive coded frame coding.

[0013] Based on the intra-coded frame, perform second-mode coding to obtain at least one second forward predictive coded frame and at least one bidirectional predictive coded frame; wherein, the second-mode coding is inter-frame compression coding.

[0014] At least one intra-frame coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame are mixed and encapsulated to obtain a mixed video stream, and the mixed video stream is sent to a server or video encoding end.

[0015] Thirdly, embodiments of the present invention also provide a video stream processing apparatus, the apparatus comprising:

[0016] A mixed video stream receiving module is used to receive mixed video streams sent by the shooting device;

[0017] A hybrid video stream decapsulation module is used to decapsulate the hybrid video stream to obtain at least one intra-coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame.

[0018] The live video stream encapsulation module is used to copy the intra-coded frame and encapsulate the original intra-coded frame and at least one first forward predictive coded frame into a live video stream for live video playback.

[0019] The storage video stream encapsulation module is used to encapsulate the copied intra-coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame into a storage video stream for local storage.

[0020] Fourthly, embodiments of the present invention also provide a video stream processing apparatus, the apparatus comprising:

[0021] The intra-frame coded frame acquisition module is used to encode the original video frame image to obtain at least one intra-frame coded frame;

[0022] The first mode coding module is used to perform first mode coding based on the intra-coded frame to obtain at least one first forward predictive coded frame; wherein, the first mode coding is inter-frame compression coding without using bidirectional predictive coded frame coding.

[0023] The second mode coding module is used to perform second mode coding based on the intra-coded frame to obtain at least one second forward predictive coded frame and at least one bidirectional predictive coded frame; wherein, the second mode coding is inter-frame compression coding.

[0024] The hybrid encapsulation module is used to encapsulate at least one intra-frame coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame to obtain a hybrid video stream, and send the hybrid video stream to a server or video encoding end.

[0025] Fifthly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the video stream processing method as described in any of the embodiments of the present invention.

[0026] In a sixth aspect, embodiments of the present invention also provide a storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform a video stream processing method as described in any of the embodiments of the present invention.

[0027] The technical solution of this invention involves encoding the original video frame image using a shooting device in a first mode and a second mode, then mixing and encapsulating the mixed video stream. The mixed video stream is then sent to a server or video encoding terminal. After the server or video encoding terminal decapsulates the mixed video stream, it copies the intra-frame encoded frames, encapsulates the original intra-frame encoded frames and the first forward predictive encoded frames into a live video stream for live video playback, and encapsulates the copied intra-frame encoded frames, the second forward predictive encoded frames, and the bidirectional predictive encoded frames into a storage video stream for local storage. This solves the problems in existing technologies where encoding I-frames, P-frames, and B-frames affects the real-time performance of video playback, and abandoning B-frame encoding results in a high bitrate and excessive storage capacity. This solution reduces the storage capacity required for video data storage while ensuring real-time video playback.

[0028] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of a video stream processing method provided in Embodiment 1 of the present invention;

[0031] Figure 2a This is a flowchart of another video stream processing method provided in Embodiment 2 of the present invention;

[0032] Figure 2b This is a schematic diagram of a hybrid video stream structure provided in Embodiment 2 of the present invention;

[0033] Figure 3 This is a flowchart of a video stream processing method provided in Embodiment 3 of the present invention;

[0034] Figure 4 This is a flowchart of another video stream processing method provided in Embodiment 4 of the present invention;

[0035] Figure 5 This is a schematic diagram of the structure of a video stream processing device provided in Embodiment 5 of the present invention;

[0036] Figure 6 This is a schematic diagram of the structure of a video stream processing device provided in Embodiment Six of the present invention;

[0037] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment 7 of the present invention. Detailed Implementation

[0038] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0039] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0040] Example 1

[0041] Figure 1The flowchart of a video stream processing method provided in Embodiment 1 of the present invention is applicable to situations where the video data storage capacity is reduced without affecting the real-time performance of video playback. The method can be executed by a video stream processing device, which can be implemented in hardware and / or software. The video stream processing device can be configured in a server or video encoding terminal and used in conjunction with a shooting device. The video encoding terminal is a terminal with end-to-end video encoding function.

[0042] like Figure 1 As shown, the method includes:

[0043] S110: Receives the mixed video stream sent by the shooting device.

[0044] In this embodiment, the shooting device is pre-connected to the server or video encoding terminal, and the shooting device is one that supports multi-channel video stream encapsulation. For example, the shooting device can be a camera, a panoramic camera, etc., and this embodiment does not limit the specific type of shooting device.

[0045] The hybrid video stream is obtained by combining and encapsulating the original video frame images after the capturing device performs inter-frame compression coding separately and inter-frame compression coding without using bidirectional predictive coding. The original video frame images are the individual frames in the video captured by the capturing device. Inter-frame compression coding, also known as IPB coding, encodes the original video frame images into I-frames, P-frames, and B-frames. Inter-frame compression coding without using bidirectional predictive coding, also known as IP coding, encodes the original video frame images into I-frames and P-frames.

[0046] In this embodiment of the invention, the shooting device performs IPB encoding and IP encoding on the original video frame images and then encapsulates them together. Optionally, a video stream copying and distribution module can be deployed in the server or video encoding end, and the video stream copying and distribution module receives the mixed video stream sent by the shooting device.

[0047] S120. Decapsulate the mixed video stream to obtain at least one intra-coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame.

[0048] The method of unpacking the mixed video stream is matched with the method of mixing and encapsulating the shooting device. This embodiment does not restrict the method of unpacking the mixed video stream and mixing and encapsulating it.

[0049] For example, the specific method of hybrid encapsulation can be RTP (Real-time Transport Protocol) + PS (Program Stream), RTP + ES (Elementary Streams), or TS (Transport Stream). Correspondingly, during decapsulation, the hybrid encapsulation method is determined based on the payload type of the received hybrid video stream, thereby selecting the matching decapsulation method. Specifically, if the hybrid encapsulation is determined to be RTP + PS, the video stream is decapsulated according to the PS packet format; if the hybrid encapsulation is determined to be RTP + ES, the video stream is decapsulated according to the ES packet format; and if the hybrid encapsulation is determined to be TS, the video stream is decapsulated according to the TS packet format.

[0050] Intra-coded frames are also known as I-frames. The first forward predictive coded frame is a P-frame generated when IP encoding is performed on an intra-coded frame. The second forward predictive coded frame is a P-frame generated when IPB encoding is performed on an intra-coded frame. Bidirectional predictive coded frames are also known as B-frames.

[0051] S130. Copy the intra-coded frame and encapsulate the original intra-coded frame and at least one first forward predictive coded frame into a live video stream for live video playback.

[0052] In this embodiment of the invention, the shooting device uses the same I-frame as a reference frame when performing IP encoding and IPB encoding, and the I-frame is not repeatedly encapsulated during hybrid encapsulation. Therefore, in this embodiment, after the hybrid video stream is decapsulated, when re-encapsulating, the I-frame needs to be copied. The original I-frame and the copied I-frame are used for live video stream encapsulation and stored video stream encapsulation, respectively.

[0053] After the I-frame is copied, the original I-frame and the P-frame generated by IP encoding based on the I-frame are re-encapsulated and used as a live video stream for live video playback.

[0054] Compared to sending the IPB-encoded video stream separately by the shooting device and playing the video live based on the IPB-encoded video stream, the technical solution of this invention has lower latency and improves the real-time performance of video playback.

[0055] S140. The copied intra-coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame are encapsulated into a storage video stream for local storage.

[0056] After the I-frame is copied, the copied I-frame, the P-frame generated by IPB encoding based on the I-frame, and the B-frame generated by IPB encoding based on the I-frame are re-encapsulated to generate a storage video stream for storage.

[0057] The technical solution of this invention, compared to the shooting device separately sending the IP-encoded video stream and storing it, occupies less storage capacity. Furthermore, in this embodiment, the I-frame in the mixed video stream is encapsulated only once. After the video stream copying and distribution module decapsulates the mixed video stream, the I-frame is copied and re-encapsulated to obtain the live video stream and the stored video stream. Compared to the shooting device directly sending the IPB-encoded and IP-encoded video streams to the server or video encoding end, where the server or video encoding end stores the IPB-encoded video stream and plays it live, this method consumes less network bandwidth and reduces network latency.

[0058] The technical solution of this invention involves encoding the original video frame image using a shooting device in a first mode and a second mode, then mixing and encapsulating the mixed video stream. The mixed video stream is then sent to a server or video encoding terminal. After the server or video encoding terminal decapsulates the mixed video stream, it copies the intra-frame encoded frames, encapsulates the original intra-frame encoded frames and the first forward predictive encoded frames into a live video stream for live video playback, and encapsulates the copied intra-frame encoded frames, the second forward predictive encoded frames, and the bidirectional predictive encoded frames into a storage video stream for local storage. This solves the problems in existing technologies where encoding I-frames, P-frames, and B-frames affects the real-time performance of video playback, and abandoning B-frame encoding results in a high bitrate and excessive storage capacity. This solution reduces the storage capacity required for video data storage while ensuring real-time video playback.

[0059] Example 2

[0060] Figure 2a This is a flowchart of another video stream processing method provided in Embodiment 2 of the present invention. Based on the above embodiments, the present invention further specifies the process of deblocking the mixed video stream and adds the process of forwarding the live video stream.

[0061] like Figure 2a As shown, the method includes:

[0062] S210: Receives the mixed video stream sent by the shooting device.

[0063] Optionally, a video stream copying and distribution module deployed on a server or video encoding terminal receives the mixed video stream sent by the shooting device.

[0064] S220. Decapsulate the mixed video stream to obtain a mixed video frame image.

[0065] After the video stream copying and distribution module decapsulates the mixed video stream, it obtains multiple mixed video frame images, which can be distinguished based on the identifier of each mixed video frame image.

[0066] S230. Based on the identifier of each mixed video frame image, divide each mixed video frame image into an intra-coded frame, a first forward predictive coded frame, a second forward predictive coded frame, and a bidirectional predictive coded frame.

[0067] After encoding, the capturing device can add identifiers to each frame before mixing and encapsulating them. Therefore, the video stream copying and distribution module can determine the encoding method of each mixed video frame image based on the identifiers of the mixed video frame images obtained after decapsulating the mixed video stream.

[0068] For example, Figure 2b A schematic diagram of a hybrid video stream structure is provided, such as... Figure 2b As shown, each intra-frame coded frame can be represented by I1, I2..., and by Pa1, Pa2, Pa3...Pa... n This represents the first forward predictive coded frame obtained by IP coding using I1 as the reference frame. (Pb3…Pb...) n This represents the second forward predictive coded frame obtained by IPB coding with I1 as the reference frame, and Bb1 and Bb2 represent the bidirectional predictive coded frames obtained by using the intra-coded frame and the second forward predictive coded frame as reference frames. This embodiment does not impose restrictions on the matching relationship between the images and identifiers of each hybrid video frame.

[0069] S240. Copy the intra-coded frame and encapsulate the original intra-coded frame and at least one first forward predictive coded frame into a live video stream.

[0070] Based on the above Figure 2b For example, before re-encapsulating, I1, I2... are copied, and I1, Pa1, Pa2, Pa3... are re-encapsulated into a live video stream.

[0071] S250. The live video stream is sent to at least one terminal so that the terminal can play the live video stream live.

[0072] The terminal is equipped with a video player, and can be a smartphone, personal computer, tablet, or other device. Optionally, a video stream copying and distribution module can forward the repackaged live video stream to the terminal, enabling the terminal to play the video live based on the live video stream. Playing the video live based on the repackaged live video stream obtained from I1, Pa1, Pa2, Pa3, etc., can improve the real-time performance of video playback and reduce network latency.

[0073] S260. The copied intra-coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame are encapsulated into a storage video stream for local storage.

[0074] Based on the above Figure 2b For example, the copied I1, Bb1, Bb2, Pb3… are re-encapsulated to obtain the storage video stream, which is then stored locally. Since the bitrate of the video stream decreases after adding B-frame encoding, storing the video data based on the storage video stream obtained by re-encapsulating I1, Bb1, Bb2, Pb3… can reduce the storage capacity required.

[0075] Example 3

[0076] Figure 3 This is a flowchart of a video stream processing method provided in Embodiment 3 of the present invention. This embodiment is applicable to situations where the video data storage capacity is reduced without affecting the real-time performance of video playback. The method can be executed by a video stream processing device, which can be implemented in hardware and / or software and can be configured in a shooting device.

[0077] like Figure 3 As shown, the method includes:

[0078] S310. Encode the original video frame image to obtain at least one intra-coded frame.

[0079] Intra-coded frames are keyframes; they are the first frame in a video frame sequence and also the reference frame for IPB and IP coding.

[0080] S320. Based on the intra-coded frame, perform first mode coding to obtain at least one first forward predictive coded frame.

[0081] The first mode encoding is an inter-frame compression encoding that does not use bidirectional predictive coding frame encoding.

[0082] Using an I-frame as a reference frame, IP encoding is performed to obtain at least one P-frame. Specifically, using... Figure 2bFor example, using I1 as the reference frame, frame Pa1 is encoded; using I1 and Pa1 as reference frames, frame Pa2 is encoded; using I1, Pa1, and Pa2 as reference frames, frame Pa3 is encoded… In this embodiment, the reference frame for encoding each P frame is not restricted.

[0083] S330. Based on the intra-coded frame, perform second-mode coding to obtain at least one second forward predictive coded frame and at least one bidirectional predictive coded frame.

[0084] The second mode encoding is inter-frame compression encoding;

[0085] Using an I-frame as a reference frame, IPB encoding is performed to obtain at least one P-frame and at least one B-frame. Specifically, using... Figure 2b For example, using I1 as the reference frame, frame Pb3 is encoded; using I1 and Pb3 as reference frames, frames Bb1 and Bb2 are encoded. It should be noted that in this embodiment, when using intra-frame coded frames and the second forward predictive coded frame as reference frames for bidirectional predictive coded frame encoding, this embodiment takes encoding one bidirectional predictive coded frame as an example. However, this embodiment does not limit the number of bidirectional predictive coded frames.

[0086] S340. At least one intra-frame coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame are mixed and encapsulated to obtain a mixed video stream, and the mixed video stream is sent to a server or video encoding end.

[0087] The imaging device performs hybrid encapsulation on intra-coded frames and frames obtained by first-mode encoding and second-mode encoding. The specific method of hybrid encapsulation has been exemplified in the above embodiments; this embodiment does not limit the specific method of hybrid encapsulation used by the imaging device.

[0088] The technical solution in this embodiment performs IPB encoding and IP encoding on I-frames separately, and encapsulates I-frames only once during hybrid encapsulation. Compared to transmitting only the video stream obtained by IPB encoding, the video playback has higher real-time performance and lower latency. Compared to transmitting only the video stream obtained by IP encoding, it occupies less storage capacity. Compared to transmitting both the video stream obtained by IPB encoding and the video stream obtained by IP encoding, it occupies less network bandwidth.

[0089] The technical solution of this invention encodes the original video frame image using a shooting device to obtain intra-frame coded frames. First-mode encoding and second-mode encoding are then performed on the intra-frame coded frames. The resulting intra-frame coded frames, first forward predictive coded frames, second forward predictive coded frames, and bidirectional predictive coded frames are then mixed and encapsulated. The mixed video stream is then sent to a server or video encoding end. The server or video encoding end decapsulates the mixed video stream, reuses the intra-frame coded frames, and re-encapsulates them to obtain a live video stream and a stored video stream. This solves the problems in existing technologies where encoding I-frames, P-frames, and B-frames affects the real-time performance of video playback, and abandoning B-frame encoding results in a high bitrate and excessive storage capacity. This solution reduces the storage capacity required for video data storage while ensuring real-time video playback.

[0090] Example 4

[0091] Figure 4 This is a flowchart of another video stream processing method provided in Embodiment 4 of the present invention. Based on the above embodiments, the present invention further specifies the second mode encoding process and the hybrid encapsulation process.

[0092] like Figure 4 As shown, the method includes:

[0093] S410. Encode the original video frame image to obtain at least one intra-coded frame.

[0094] S420. Based on the intra-coded frame, perform first mode coding to obtain at least one first forward predictive coded frame.

[0095] The above embodiments have described and illustrated the specific process of encoding the first mode based on the intra-coded frame, and will not be repeated here.

[0096] S430. Based on the intra-coded frames, perform inter-frame predictive coding to obtain at least one second forward predictive coding frame.

[0097] S440. Based on the intra-coded frame and the second forward predictive coded frame, perform bidirectional predictive coded frame encoding to obtain at least one bidirectional predictive coded frame.

[0098] Specifically, P-frames are encoded based on I-frames to obtain P-frames. Then, B-frames are encoded using I-frames and P-frames as reference frames to obtain B-frames.

[0099] The above embodiments have described and illustrated the specific process of second-mode encoding based on intra-coded frames, and will not be repeated here.

[0100] S450. Add matching identifiers to each intra-coded frame, each first forward predictive coded frame, each second forward predictive coded frame, and each bidirectional predictive coded frame.

[0101] Adding identifiers to each frame aims to enable the video stream copying and distribution module to determine the encoding method of each frame based on the identifiers, thereby determining which video stream each frame belongs to when it is re-encapsulated.

[0102] S460, The intra-frame coded frames with added identifiers, the first forward predictive coded frames, the second forward predictive coded frames, and the bidirectional predictive coded frames are mixed and encapsulated.

[0103] The specific methods of hybrid packaging are illustrated and explained in the above embodiments, and will not be repeated here.

[0104] S470. Obtain the mixed video stream and send the mixed video stream to the server or video encoding terminal.

[0105] After the shooting device obtains the mixed video stream through hybrid encapsulation, it sends the mixed video stream to the server or video encoding end. Optionally, it can send it to the video stream copying and distribution module deployed in the server or video encoding end.

[0106] Example 5

[0107] Figure 5 This is a schematic diagram of a video stream processing device provided in Embodiment 5 of the present invention. The device is deployed in a server or video encoding terminal and used in conjunction with a shooting device. Figure 5 As shown, the device includes: a mixed video stream receiving module 510, a mixed video stream decapsulation module 520, a live video stream encapsulation module 530, and a stored video stream encapsulation module 540. Wherein:

[0108] The mixed video stream receiving module 510 is used to receive the mixed video stream sent by the shooting device;

[0109] The hybrid video stream decapsulation module 520 is used to decapsulate the hybrid video stream to obtain at least one intra-coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame.

[0110] The live video stream encapsulation module 530 is used to copy the intra-coded frame and encapsulate the original intra-coded frame and at least one first forward predictive coded frame into a live video stream for live video playback.

[0111] The storage video stream encapsulation module 540 is used to encapsulate the copied intra-coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame into a storage video stream for local storage.

[0112] Based on the above embodiments, the hybrid video stream deblocking module 520 includes:

[0113] A hybrid video stream decapsulation unit is used to decapsulate the hybrid video stream to obtain hybrid video frame images;

[0114] The hybrid video frame image partitioning unit is used to partition each hybrid video frame image into an intra-coded frame, a first forward predictive coded frame, a second forward predictive coded frame, and a bidirectional predictive coded frame based on the identifier of each hybrid video frame image.

[0115] Based on the above embodiments, the device further includes:

[0116] A live video stream sending module is used to send the live video stream to at least one terminal so that the terminal can play the live video stream live.

[0117] The video stream processing apparatus provided in the embodiments of the present invention can execute the video stream processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0118] Example 6

[0119] Figure 6 This is a schematic diagram of a video stream processing device provided in Embodiment Six of the present invention. This device is deployed within a shooting device. Figure 6 As shown, the device includes: an intra-frame encoded frame acquisition module 610, a first-mode encoding module 620, a second-mode encoding module 630, and a hybrid encapsulation module 640. Wherein:

[0120] Intra-coded frame acquisition module 610 is used to encode the original video frame image to obtain at least one intra-coded frame;

[0121] The first mode coding module 620 is used to perform first mode coding based on the intra-coded frame to obtain at least one first forward predictive coded frame; wherein the first mode coding is inter-frame compression coding without using bidirectional predictive coded frame coding.

[0122] The second mode coding module 630 is used to perform second mode coding based on the intra-coded frame to obtain at least one second forward predictive coded frame and at least one bidirectional predictive coded frame; wherein, the second mode coding is inter-frame compression coding.

[0123] The hybrid encapsulation module 640 is used to encapsulate at least one intra-frame coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame to obtain a hybrid video stream, and send the hybrid video stream to a server or a video encoding end.

[0124] Based on the above embodiments, the second mode encoding module 630 includes:

[0125] Inter-frame predictive coding frame coding unit is used to perform inter-frame predictive coding frame coding based on intra-frame coded frames to obtain at least one second forward predictive coding frame.

[0126] A bidirectional predictive coding frame encoding unit is used to perform bidirectional predictive coding frame encoding based on an intra-coded frame and a second forward predictive coding frame to obtain at least one bidirectional predictive coding frame.

[0127] Based on the above embodiments, the hybrid packaging module 640 includes:

[0128] The identifier addition unit is used to add matching identifiers to each intra-coded frame, each first forward predictive coded frame, each second forward predictive coded frame, and each bidirectional predictive coded frame.

[0129] The hybrid encapsulation unit is used to encapsulate each intra-frame coded frame with added identifiers, each first forward predictive coded frame, each second forward predictive coded frame, and each bidirectional predictive coded frame in a hybrid manner.

[0130] The video stream processing apparatus provided in the embodiments of the present invention can execute the video stream processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0131] Example 7

[0132] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment Seven of the present invention, as shown below. Figure 7 As shown, the electronic device includes a processor 70, a memory 71, an input device 72, and an output device 73; the number of processors 70 in the electronic device can be one or more. Figure 7 Taking a processor 70 as an example; the processor 70, memory 71, input device 72, and output device 73 in the electronic device can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0133] The memory 71, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as modules corresponding to the video stream processing method in this embodiment of the invention (e.g., the hybrid video stream receiving module 510, hybrid video stream decapsulation module 520, live video stream encapsulation module 530, and stored video stream encapsulation module 540 in the video stream processing device, or the intra-frame encoded frame acquisition module 610, first mode encoding module 620, second mode encoding module 630, and hybrid encapsulation module 640). The processor 70 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 71, thereby implementing the above-described video stream processing method. This method includes:

[0134] Receive the mixed video stream sent by the shooting device;

[0135] The hybrid video stream is decapsulated to obtain at least one intra-coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame.

[0136] The intra-coded frame is copied, and the original intra-coded frame and at least one first forward predictive coded frame are encapsulated into a live video stream for live video playback.

[0137] The copied intra-coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame are encapsulated into a storage video stream for local storage.

[0138] Alternatively, the method may also include:

[0139] The original video frame images are encoded to obtain at least one intra-coded frame;

[0140] Based on the intra-coded frames, first mode coding is performed to obtain at least one first forward predictive coded frame; wherein, the first mode coding is inter-frame compression coding without using bidirectional predictive coded frame coding.

[0141] Based on the intra-coded frame, perform second-mode coding to obtain at least one second forward predictive coded frame and at least one bidirectional predictive coded frame; wherein, the second-mode coding is inter-frame compression coding.

[0142] At least one intra-frame coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame are mixed and encapsulated to obtain a mixed video stream, and the mixed video stream is sent to a server or video encoding end.

[0143] The memory 71 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 71 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 71 may further include memory remotely located relative to the processor 70, which can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0144] Input device 72 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the electronic device. Output device 73 may include display devices such as a display screen.

[0145] Example 8

[0146] Embodiment 8 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a video stream processing method, the method comprising:

[0147] Receive the mixed video stream sent by the shooting device;

[0148] The hybrid video stream is decapsulated to obtain at least one intra-coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame.

[0149] The intra-coded frame is copied, and the original intra-coded frame and at least one first forward predictive coded frame are encapsulated into a live video stream for live video playback.

[0150] The copied intra-coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame are encapsulated into a storage video stream for local storage.

[0151] Alternatively, the method may also include:

[0152] The original video frame images are encoded to obtain at least one intra-coded frame;

[0153] Based on the intra-coded frames, first mode coding is performed to obtain at least one first forward predictive coded frame; wherein, the first mode coding is inter-frame compression coding without using bidirectional predictive coded frame coding.

[0154] Based on the intra-coded frame, perform second-mode coding to obtain at least one second forward predictive coded frame and at least one bidirectional predictive coded frame; wherein, the second-mode coding is inter-frame compression coding.

[0155] At least one intra-frame coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame are mixed and encapsulated to obtain a mixed video stream, and the mixed video stream is sent to a server or video encoding end.

[0156] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the method operations described above, but can also perform related operations in the video stream processing method provided in any embodiment of the present invention.

[0157] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0158] It is worth noting that in the embodiments of the video stream processing device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0159] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for processing a video stream, characterized in that, Applied to a server or video encoding end, the method includes: Receive the mixed video stream sent by the shooting device; The hybrid video stream is decapsulated to obtain at least one intra-coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame; the first forward predictive coded frame and the second forward predictive coded frame are generated using different encoding methods. The video stream copying and distribution module copies the intra-coded frames and encapsulates the original intra-coded frames and at least one first forward predictive coded frame into a live video stream for live video playback. The video stream copying and distribution module encapsulates the copied intra-coded frames, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame into a storage video stream for local storage. The process of decapsulating the mixed video stream to obtain at least one intra-coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame includes: The mixed video stream is decapsulated to obtain mixed video frame images; Based on the identifiers of each hybrid video frame image, each hybrid video frame image is divided into intra-coded frames, first forward predictive coded frames, second forward predictive coded frames, and bidirectional predictive coded frames.

2. The method according to claim 1, characterized in that, After encapsulating the original intra-coded frames and at least one first forward predictive coded frame into a live video stream, it also includes: The live video stream is sent to at least one terminal so that the terminal can play the live video stream live.

3. A method for processing a video stream, characterized in that, Applied to a shooting device, the method includes: The original video frame images are encoded to obtain at least one intra-coded frame; Based on the intra-coded frames, first mode coding is performed to obtain at least one first forward predictive coded frame; wherein, the first mode coding is inter-frame compression coding without using bidirectional predictive coded frame coding. Based on the intra-coded frame, perform second-mode coding to obtain at least one second forward predictive coded frame and at least one bidirectional predictive coded frame; wherein, the second-mode coding is inter-frame compression coding. At least one intra-frame coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame are mixed and encapsulated to obtain a mixed video stream, and the mixed video stream is sent to a server or video encoding end. The method of co-encapsulating at least one intra-frame coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame includes: Add matching identifiers to each intra-coded frame, each first forward predictive coded frame, each second forward predictive coded frame, and each bidirectional predictive coded frame. Each intra-frame coded frame with added identifiers, each first forward predictive coded frame, each second forward predictive coded frame, and each bidirectional predictive coded frame are then encapsulated together.

4. The method according to claim 3, characterized in that, Based on the intra-coded frame, second-mode coding is performed to obtain at least one second forward predictive coded frame and at least one bidirectional predictive coded frame, including: Based on the intra-coded frames, perform inter-predictive coded frames to obtain at least one second forward predictive coded frame; Based on the intra-coded frame and the second forward predictive coded frame, bidirectional predictive coded frame encoding is performed to obtain at least one bidirectional predictive coded frame.

5. A video stream processing apparatus, characterized in that, Deployed on a server or video encoding terminal, the device includes: A mixed video stream receiving module is used to receive mixed video streams sent by the shooting device; A hybrid video stream decapsulation module is used to decapsulate the hybrid video stream to obtain at least one intra-coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame; the first forward predictive coded frame and the second forward predictive coded frame are generated using different encoding methods; The hybrid video stream deblocking module specifically includes: A hybrid video stream decapsulation unit is used to decapsulate the hybrid video stream to obtain hybrid video frame images; The hybrid video frame image partitioning unit is used to partition each hybrid video frame image into an intra-coded frame, a first forward predictive coded frame, a second forward predictive coded frame, and a bidirectional predictive coded frame according to the identifier of each hybrid video frame image. The live video stream encapsulation module is used to copy the intra-coded frame through the video stream copying and distribution module, and encapsulate the original intra-coded frame and at least one first forward predictive coded frame into a live video stream for live video playback. The storage video stream encapsulation module is used to encapsulate the copied intra-coded frames, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame into a storage video stream for local storage through the video stream copying and distribution module.

6. A video stream processing apparatus, characterized in that, Deployed on a shooting device, the device includes: The intra-frame coded frame acquisition module is used to encode the original video frame image to obtain at least one intra-frame coded frame; The first mode coding module is used to perform first mode coding based on the intra-coded frame to obtain at least one first forward predictive coded frame; wherein, the first mode coding is inter-frame compression coding without using bidirectional predictive coded frame coding. The second mode coding module is used to perform second mode coding based on the intra-coded frame to obtain at least one second forward predictive coded frame and at least one bidirectional predictive coded frame; wherein, the second mode coding is inter-frame compression coding. The hybrid encapsulation module is used to encapsulate at least one intra-frame coded frame, at least one first forward predictive coded frame, at least one second forward predictive coded frame, and at least one bidirectional predictive coded frame to obtain a hybrid video stream, and send the hybrid video stream to a server or video encoding end. The hybrid packaging module specifically includes: The identifier addition unit is used to add matching identifiers to each intra-coded frame, each first forward predictive coded frame, each second forward predictive coded frame, and each bidirectional predictive coded frame. The hybrid encapsulation unit is used to encapsulate each intra-frame coded frame with added identifiers, each first forward predictive coded frame, each second forward predictive coded frame, and each bidirectional predictive coded frame in a hybrid manner.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the video stream processing method as described in any one of claims 1-4.

8. A storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the video stream processing method as described in any one of claims 1-4.

Images