High-definition playing method based on video background extraction

Abstract

The invention discloses a high-definition playing method based on video background extraction, which comprises the following steps: S1, a stream pulling module pulls original data and sends the data to a transcoding device; s2, performing secondary compression on the received original stream by the transcoding device, extracting background content in the original stream, and making and integrating the background content into an SEI segment; s3, the transcoding device transmits the secondarily compressed data to a stream pulling module, wherein the stream pulling module comprises an original stream and a compressed stream; s4, if the user carries out real-time streaming watching, the compressed stream at the moment is transmitted to the client through the streaming media device, the client detects the SEI, SEI fragments are reintegrated and recovered into clear YUV background data, the clear YUV background data are superposed on each frame of data of the compressed stream, and the compressed stream is recovered to be in a clear state to be played; the compressed stream enters a video recording module to be stored, in the video recording process, an index module is set to detect SEI frames carrying background data in real time, and indexes are recorded in a disk storage and correspond to video segments.

Description

Technical Field

[0001] The invention belongs to the technical field of video encoding and decoding, and in particular relates to a high-definition playback method based on video background extraction. Background Art

[0002] In the field of security video surveillance, whether it is live or on-demand, for the video being watched, if the original video quality is relatively high and there are more details in the video, then the bit rate is high, which occupies more memory resources when storing and requires higher bandwidth when transmitting; on the contrary, if the video is compressed more severely, the video storage occupies less memory, and when watching in real time, although storage and bandwidth are saved, the picture is relatively blurry, affecting the viewing of details. In general, the video surveillance picture does not move often, so always transmitting clear and high-quality pictures actually wastes a lot of bandwidth; in a weak network environment, data packet loss and delay will also occur. Summary of the invention

[0003] In view of the above problems, the present invention provides a high-definition playback method based on video background extraction, which is used to present a clearer live or on-demand video viewing experience with a smaller video data storage volume.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] A high-definition playback method based on video background extraction is applied to a high-definition playback system based on video background extraction, which includes a client device, a media server device and a transcoding device, wherein the client device includes a playback end logic control module, the streaming media server device includes a stream pulling module, a recording module, an index recording module and a video-on-demand control module, and the transcoding device includes a transcoding module and a background extraction module, and includes the following steps:

[0006] S1, the stream pulling module pulls the original data and sends it to the transcoding device;

[0007] S2, the transcoding device performs secondary compression on the received original stream, extracts the background content in the original stream and produces an SEI segment;

[0008] S3, the transcoding device transmits the secondary compressed data to the stream pulling module, and the stream pulling module includes the original stream and the compressed stream;

[0009] S4: If the user is watching the real-time stream, the compressed stream is transmitted to the client through the streaming device. The client detects the SEI, re-integrates the SEI fragments, restores them to clear YUV background data, and superimposes them on each frame of the compressed stream, so that the compressed stream can be played back in a clear state.

[0010] The compressed stream enters the video recording module for storage. During the video recording process, the index module is set to detect the SEI frame carrying background data in real time, and record the index in the disk storage to correspond to the video clip. When the user initiates a video on demand, the video on demand module first finds the starting time point of the video on demand, accesses the background frame SEI position recorded by the index module to find the index, and extracts the SEI into frame data separately in the video clip that records the SEI position, and outputs it together with the video on demand data.

[0011] In a possible implementation, the transcoding device in S3 performs secondary compression on the received original stream, extracts the background content in the original stream and produces a complete SEI segment, including: separately encoding the YUV raw data of the background of the video screen in the original stream into multiple different SEI segments, and after the clear video frame is compressed, outputting a video frame sequence in which a clear background YUV data is dispersed in multiple SEI segments.

[0012] In a possible implementation, the YUV raw data of the background of the video screen in the original stream is separately encoded into multiple different SEI fragments, including: extracting the foreground from the YUV raw data through the dynamic key area, and the rest is the background picture that basically does not change; saving the background picture separately as SEI, and then for each frame in the video stream, compressing the background area with a higher compression ratio and the foreground with a lower compression ratio to obtain a video code stream; and inserting the generated SEI into the video code stream; performing the above processing once every first time.

[0013] In a possible implementation, the recording index includes: establishing an index table to record forward SEI index information, wherein the recorded information in the index table includes a video clip and a video clip corresponding to the SEI index.

[0014] In a possible implementation, the accessing index module records the background frame SEI position search index, including: when starting to play on demand, confirming the video clip corresponding to the start time of the play on demand, the play on demand module will first search the index table recorded by the index module, and determine the video clip corresponding to its SEI index. If the video clip to be played does not have SEI data, it is necessary to borrow the video clip where the SEI index is located. The play on demand module first reads the video clip where the SEI index is located, parses the SEI segment inside, and then sends it together with the data of the video clip corresponding to the start time of the play on demand.

[0015] In a possible implementation, the original data is audio and video data with high definition, high bit rate and low compression ratio.

[0016] In a possible implementation, the original data source includes a network camera IPC or a network video recorder NVR.

[0017] The present invention has the following beneficial effects: by extracting the video data background through transcoding and combining the streaming media storage index mechanism, a solution with a small amount of transmitted data but clear video display is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of the steps of a high-definition playback method based on video background extraction according to an embodiment of the present invention;

[0019] Figure 2 It is a principle block diagram of a high-definition playback system based on video background extraction applied in a high-definition playback method based on video background extraction according to an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of an index in an embodiment of the present invention;

[0021] Figure 4 Schematic diagram of access index in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Reference Figure 1 , which is a flowchart of the steps of a high-definition playback method based on video background extraction according to an embodiment of the present invention, and is applied to Figure 2 As shown, a high-definition playback system based on video background extraction includes a client device, a media server device and a transcoding device, wherein the client device includes a playback end logic control module, the streaming media server device includes a stream pulling module, a recording module, an index recording module and a video-on-demand control module, and the transcoding device includes a transcoding module and a background extraction module, and includes the following steps:

[0024] S1, the stream pulling module pulls the original data and sends it to the transcoding device; the original data is high-definition, high-bitrate, low-compression audio and video data, for example, the bit rate of video data with a resolution of 1080P is generally 4Mbps. The source of the original data includes the network camera IPC or the network video recorder NVR.

[0025] S2, the transcoding device performs secondary compression on the received original stream, extracts the background content in the original stream and produces an SEI segment;

[0026] S3, the transcoding device transmits the secondary compressed data to the stream pulling module, and the stream pulling module includes the original stream and the compressed stream;

[0027] S4: If the user is watching the real-time stream, the compressed stream is transmitted to the client through the streaming device. The client detects the SEI, re-integrates the SEI fragments, restores them to clear YUV background data, and superimposes them on each frame of the compressed stream, so that the compressed stream can be played back in a clear state.

[0028] The compressed stream enters the video recording module for storage. During the video recording process, the index module is set to detect the SEI frame carrying background data in real time, and record the index in the disk storage to correspond to the video clip. When the user initiates a video on demand, the video on demand module first finds the starting time point of the video on demand, accesses the background frame SEI position recorded by the index module to find the index, and extracts the SEI into frame data separately in the video clip that records the SEI position, and outputs it together with the video on demand data.

[0029] Specifically, in another embodiment of the present invention, a high-definition playback method based on video background extraction, the transcoding device in S3 performs secondary compression on the received original stream, extracts the background content in the original stream and makes it into SEI segments, including: separately encoding the YUV raw data of the video screen background in the original stream into multiple different SEI segments, and after the clear video frame is compressed, outputting a video frame sequence in which a clear background YUV data is dispersed in multiple SEI segments. For general security video data, because its point position is relatively fixed, except for the changing factors such as people, cars, and flying objects in the air, the background such as the environmental position is unchanged.

[0030] Furthermore, the YUV raw data of the background of the video screen in the original stream is separately encoded into a plurality of different SEI fragments, including: extracting the foreground (extracting frequently changing information such as people and cars) from the YUV raw data through the dynamic key area, and the remaining is the background screen that does not change basically (and occupies a large part of the screen); saving the background screen separately as SEI (the data of the background screen with clear SEI here can be uncompressed or compressed at a lower compression ratio, which can reduce the bit rate to a certain extent while maintaining clarity), and then for each frame of the video stream, the background area is compressed at a higher compression ratio (because the background has been saved on SEI, the background in each frame of the code stream can be compressed at a very high compression ratio), and the foreground is compressed at a lower compression ratio to obtain a video code stream, for example, the background can be compressed 4-10 times, 1080P (assuming that the original input code rate is 4Mbps) can be compressed to 400kbps-1Mbps, and the compression ratio of the foreground part remains unchanged; and inserting the generated SEI into the video code stream; performing the above processing once every first time. It can be generated every few minutes or more than ten minutes. This is equivalent to extracting and saving the background separately to maximize the compression of the video stream.

[0031] After the above operations, media data with a higher compression ratio is obtained, which can then be transmitted to the subsequent recording and live broadcast modules. In short, the background picture at a moment is used in the next few minutes; it is equivalent to only clearly saving the background once in the bit stream, and the content of the changing part of each frame.

[0032] In a high-definition playback method based on video background extraction according to another embodiment of the present invention, recording the index in S4 includes: establishing an index table to record forward SEI index information, wherein the information recorded in the index table includes video clips and the video clips corresponding to their SEI indexes.

[0033] like Figure 3 As shown in the figure, the real-time stream has been re-encoded after passing through the video module, and the background has been extracted. At this time, the data of the clear background picture is distributed in three pieces of SEI data (SEI1, 2, 3), which are transmitted together with the audio and video data; in order to ensure the reduction of the bit rate, the SEI does not change frequently (once every few minutes or ten minutes); SEI is saved in the video file together with the audio and video data, and some video clips will not have SEI, so playing this video clip alone will result in a blurred picture; in order to ensure that no matter which video clip is watched when on demand, a clear picture can be displayed immediately, so an index file is needed to record the "forward SEI index", and the information recorded in the index table is as follows:

[0034] Video clip 1: SEI index (this document, video clip 1); Video clip 2: SEI index (video clip 1); Video clip 3: SEI index (video clip 1); Video clip 4: SEI index (video clip 4); Video clip 5: SEI index (video clip 4).

[0035] Specifically, accessing the background frame SEI position search index recorded by the index module includes: when starting to play on demand, confirming the video clip corresponding to the start time of the play on demand, the play on demand module will first search the index table recorded by the index module to determine the video clip corresponding to its SEI index. If the starting video clip has no SEI data, it is necessary to borrow the video clip where the SEI index is located. The play on demand module first reads the video clip where the SEI index is located, parses it into the SEI segment inside, and then sends it together with the data of the video clip corresponding to the start time of the play on demand.

[0036] Continue as Figure 4 As shown, when the on-demand video is started, if the start time of the on-demand video is the time of video segment 3, in order to display the video content of segment 3 clearly and at a low bit rate, the on-demand module will first search the record of the index module and find that the SEI index of segment 3 is video segment 1. This means that there is no SEI data in the time period of segment 3 and it needs to be borrowed forward. Therefore, the on-demand module will first read video segment 1, parse the SEI segment inside, and then send it out together with the data of video segment 3, so that the user can see the clear video segment 3.

[0037] Since the video background in the security field does not change frequently, SEI background extraction does not occur frequently, and a SEI transmission can restore high definition within tens of minutes; this allows us to compress the video dozens of times, reduce the transmission bit rate and bandwidth usage, and ultimately still see clear video images through SEI frames.

[0038] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A high-definition playback method based on video background extraction, characterized in that: The invention is applied to a high-definition playback system based on video background extraction, which includes a client device, a media server device and a transcoding device, wherein the client device includes a playback end logic control module, the streaming media server device includes a stream pulling module, a recording module, an index recording module and a video-on-demand control module, and the transcoding device includes a transcoding module and a background extraction module, and includes the following steps: S1, the stream pulling module pulls the original data and sends it to the transcoding device; S2, the transcoding device performs secondary compression on the received original stream, extracts the background content in the original stream and produces an SEI segment; S3, the transcoding device transmits the secondary compressed data to the stream pulling module, and the stream pulling module includes the original stream and the compressed stream; S4: If the user is watching the real-time stream, the compressed stream is transmitted to the client through the streaming device. The client detects the SEI, re-integrates the SEI fragments, restores them to clear YUV background data, and superimposes them on each frame of the compressed stream, so that the compressed stream can be played back in a clear state. The compressed stream enters the video recording module for storage. During the video recording process, the index module is set to detect the SEI frame carrying background data in real time, and record the index in the disk storage to correspond to the video clip. When the user initiates a video on demand, the video on demand module first finds the starting time point of the video on demand, accesses the background frame SEI position recorded by the index module to find the index, and extracts the SEI into frame data separately in the video clip that records the SEI position, and outputs it together with the video on demand data.

2. The high-definition playback method based on video background extraction as claimed in claim 1, characterized in that: The transcoding device in S3 performs secondary compression on the received original stream, extracts the background content in the original stream and produces a complete SEI segment, including: separately encoding the YUV raw data of the background of the video screen in the original stream into multiple different SEI segments, and after the clear video frame is compressed, outputting a video frame sequence in which a clear background YUV data is dispersed in multiple SEI segments.

3. The high-definition playback method based on video background extraction as claimed in claim 2, characterized in that: The YUV raw data of the background of the video screen in the original stream is separately encoded into multiple different SEI fragments, including: extracting the foreground from the YUV raw data through the dynamic key area, and the rest is the background picture that basically does not change; saving the background picture separately as SEI, and then for each frame in the video stream, compressing the background area with a higher compression ratio and the foreground with a lower compression ratio to obtain a video code stream; and inserting the generated SEI into the video code stream; performing the above processing once every first time.

4. The high-definition playback method based on video background extraction as claimed in claim 1, characterized in that: The recording index includes: establishing an index table to record forward SEI index information, wherein the recorded information in the index table includes a video clip and a video clip corresponding to its SEI index.

5. The high-definition playback method based on video background extraction as claimed in claim 1, characterized in that: The access index module records the background frame SEI position search index, including: when starting to play on demand, confirming the video clip corresponding to the start time of the play on demand, the play on demand module will first search the index table recorded by the index module to determine the video clip corresponding to its SEI index. If the video clip to be played does not have SEI data, it is necessary to borrow the video clip where the SEI index is located. The play on demand module first reads the video clip where the SEI index is located, parses the SEI clip inside, and then sends it together with the data of the video clip corresponding to the start time of the play on demand.

6. The high-definition playback method based on video background extraction according to any one of claims 1 to 5, characterized in that: The original data is audio and video data with high definition, high bit rate and low compression ratio.

7. The high-definition playback method based on video background extraction as claimed in claim 6, characterized in that: The original data source includes an IP camera IPC or a network video recorder NVR.

Images