A video bitstream analysis method and system

Abstract

The application discloses a video code stream analysis method and system, and relates to the technical field of image data processing, and comprises the following steps: acquiring a video code stream; judging whether the video code stream is a program stream or an audio stream; if yes, obtaining a corresponding code stream format; if not, extracting NAL unit types belonging to H.264 streams and H.265 streams from the NAL unit header of the video code stream and respectively storing the NAL unit types in H.264 queues and H.265 queues; and judging that the format of the video code stream is the code stream format to which the NAL unit types of the queue in which all the NAL unit types exist belong only when all the NAL unit types of one queue exist and the NAL unit types of the other queue do not completely exist. The code stream format is identified by code stream characteristics instead of being simply agreed by a transmission protocol, the real format of code stream coding is quickly and accurately judged, and multiple loop analyses are not required, so that the cost is low and the speed is fast.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, in particular to a video bitstream analysis method and system. BACKGROUND

[0002] Bitrate refers to the data flow used by a video file in a unit of time, also called code rate, which is the most important part in picture quality control in video coding. With the same resolution, the larger the code rate of a video file, the smaller the compression ratio, and the better the picture quality.

[0003] Currently, when video streams are transmitted, audio data and video data are usually encapsulated into ES streams (basic code streams, continuous code streams containing video, audio or data) and PS streams (program streams), and then transmitted through the RTP / RTCP protocol.

[0004] However, some manufacturers do not implement the encapsulation process in a standard manner, and there are deviations in their understanding of standard protocols, resulting in incorrect audio and video coding information received at the receiving end, such as deviations in the encapsulation format of ES streams and PS streams, inconsistencies between ACC audio streams and ADTS audio streams, and coding errors in H.264 and H.265, etc., which prevents the receiving end from normally processing the data stream and receiving and forwarding the video stream without a certificate.

[0005] Furthermore, in the current video bitstream analysis method, H.264 and H.265 are analyzed by an H.264 feature recognizer for one-pass bitstream analysis to generate a variable value, and then H.265 is analyzed for one-pass bitstream analysis to also generate a variable value, and the bitstream format is determined by comparing the two variable values. This method requires multiple loop bitstream analysis, which is inefficient. SUMMARY

[0006] To solve the above problems, the present application provides a video bitstream analysis method and system, which identifies multiple bitstream formats through bitstream features, rather than simply relying on transmission protocols to determine the true format of the bitstream coding quickly and accurately, without the need for multiple loop analysis, low cost and fast speed.

[0007] To achieve the above purpose, the present application adopts the following technical solutions:

[0008] In a first aspect, the present application provides a video bitstream analysis method, comprising:

[0009] obtaining a video bitstream;

[0010] determining whether the video bitstream is a program stream or an audio stream; if so, obtaining the corresponding bitstream format;

[0011] If not, the NAL unit types belonging to the H.264 stream and the H.265 stream are extracted from the NAL unit header of the video bitstream, and are respectively stored in the H.264 queue and the H.265 queue.

[0012] If and only if the NAL unit types of one of the queues are all present, and the NAL unit types of the other queue are not all present, the format of the video bitstream is determined as the format of the bitstream to which the queue with all present NAL unit types belongs.

[0013] As an alternative embodiment, the video bitstream is determined as a program stream or an audio stream by byte matching.

[0014] As an alternative embodiment, the byte matching includes:

[0015] The first five bytes of the video bitstream are extracted, and if the first five bytes match 00 00 00 01 BA, the video bitstream is determined as a PS stream.

[0016] Otherwise, the first two bytes of the video bitstream are extracted, and if the first two bytes match FFF*, the video bitstream is determined as an AAC stream, and whether the video bitstream is an AAC stream is determined again by parsing the AAC ADTS header. If yes, the video bitstream is finally determined as an AAC stream, otherwise, the H.264 stream and the H.265 stream are identified.

[0017] As an alternative embodiment, the NAL unit types of the H.264 stream are selected to include IDR, SPS and PPS, and the NAL unit types belonging to the H.264 stream and their values extracted from the NAL unit header are stored in the H.264 queue.

[0018] As an alternative embodiment, the NAL unit types of the H.265 stream are selected to include VPS, SPS and PPS, and the NAL unit types belonging to the H.265 stream and their values extracted from the NAL unit header are stored in the H.265 queue.

[0019] As an alternative embodiment, if the NAL unit types of the SPS, the PPS and the IDR of the H.264 stream in the H.264 queue are all present, and the NAL unit types of the VPS, the SPS and the PPS of the H.265 stream in the H.265 queue are not all present, the format of the video bitstream is determined as the H.264 stream.

[0020] As an alternative embodiment, if the NAL unit types of the VPS, the SPS and the PPS of the H.265 stream in the H.265 queue are all present, and the NAL unit types of the SPS, the PPS and the IDR of the H.264 stream in the H.264 queue are not all present, the format of the video bitstream is determined as the H.265 stream.

[0021] In a second aspect, the present application provides a video stream analysis system, comprising:

[0022] a obtaining module configured to obtain a video stream;

[0023] a first identifying module configured to determine whether the video stream is a program stream or an audio stream, and if so, obtain a corresponding stream format;

[0024] a second identifying module configured to, if not, extract NAL unit types belonging to H.264 stream and H.265 stream respectively from the NAL unit header of the video stream, and store them in H.264 queue and H.265 queue respectively;

[0025] a determining module configured to determine the format of the video stream as the stream format to which the NAL unit types of the queue in which all the NAL unit types exist belong, only when all the NAL unit types of one of the queues exist and the NAL unit types of the other queue do not exist completely.

[0026] In a third aspect, the present application provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions are run by the processor to complete the method of the first aspect.

[0027] In a fourth aspect, the present application provides a computer readable storage medium for storing computer instructions, wherein the computer instructions are executed by a processor to complete the method of the first aspect.

[0028] Compared with the prior art, the present application has the following beneficial effects:

[0029] The present application innovatively proposes a video stream analysis method and system, designs a simple stream analysis technology, identifies program stream, audio stream, H.264 stream or H.265 stream through video stream characteristics, rather than simply making an agreement through transmission protocol, can quickly and accurately determine the real format of the stream coding, avoids the situation that the application cannot be normally watched due to the non-standard implementation of manufacturers, and has low implementation cost and does not need to consume expensive CPU to realize decoding, but only needs to identify stream characteristics.

[0030] The present application innovatively proposes a video stream analysis method and system, identifies H.264 stream and H.265 stream through NAL unit types, fully considers the NAL unit type value interval of H.264 stream and H.265 stream, does not need to traverse the video stream multiple times, can identify the accurate stream format at one time, has low cost and high speed, and guarantees the identification accuracy through the secondary determination of the queue.

[0031] Advantages of the additional aspects of the application will become apparent in the following description. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate exemplary embodiments of the application and together with the description serve to explain the application.

[0033] Figure 1 A flow chart of a video stream analysis method is provided for embodiment 1 of the application. DETAILED DESCRIPTION

[0034] The application will be further described with reference to the drawings and embodiments.

[0035] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0036] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0037] The embodiments in the application and the features in the embodiments can be combined with each other in the case of no conflict.

[0038] Embodiment 1

[0039] As shown in the figure, the embodiment provides a video stream analysis method, comprising: Figure 1

[0040] Obtaining a video stream;

[0041] Judging whether the video stream is a program stream or an audio stream; if yes, obtaining a corresponding stream format;

[0042] If not, extracting NAL unit types belonging to H.264 and H.265 respectively from the NAL unit header of the video stream, and storing them in H.264 queue and H.265 queue respectively;

[0043] ​The format of the video code stream is determined as the code stream format of the queue with all NAL unit types existing only when all NAL unit types of one queue exist and NAL unit types of the other queue do not exist completely.

[0044] In the embodiment, the PS stream and the AAC stream have independent characteristics and the prefixes of the two streams are unique in the process of determining whether the video code stream is a program stream (PS stream) or an audio stream (AAC stream), so the embodiment determines by matching the bytes.

[0045] Specifically, the embodiment includes the following steps.

[0046] The first five bytes of the video code stream are extracted and identified.

[0047] If the first five bytes match 00 00 00 01 BA, the video code stream is determined as a PS stream, and then the audio and video streams can be obtained by decapsulating the PS stream.

[0048] Otherwise, the video code stream is determined as an ES stream, and the first two bytes of the video code stream are extracted and identified.

[0049] If the first two bytes match FFF*, the video code stream is determined as an AAC stream, and the AAC ADTS header is further analyzed to determine whether the video code stream is an AAC stream again. If yes, the video code stream is finally determined as an AAC stream. Otherwise, the H.264 stream and the H.265 stream are identified.

[0050] Since the existing solutions are analyzed by multiple cycles, the efficiency is relatively low. Therefore, the embodiment identifies the values of the key NAL unit types in the H.264 stream and the H.265 stream by analyzing the code rate characteristics of the H.264 stream and the H.265 stream, and directly identifies the H.264 stream and the H.265 stream by comparing the values of the key NAL unit types, without multiple cycles of the code stream, but only one identification.

[0051] In the frame structure of the H.264 stream and the H.265 stream, the functions are divided into NAL units (network extraction layer) and VCL units (video coding layer). One frame of coded image is divided into one or more NAL units. Each NAL unit includes a NAL unit header (NAL Unit Header) and a NAL unit body. The NAL unit header is composed of a start code and a NAL unit type (NAL Unit Type). The NAL unit body is usually raw byte sequence payload data.

[0052] The NAL unit types belonging to the H.264 stream and the H.265 stream are extracted from the NAL unit header.

[0053] The selected NAL unit types of the H.264 stream include IDR, SPS and PPS, and the value interval of the NAL unit type of the H.264 stream is 5, 7 and 8; that is, when the value of the NAL unit type is 5, it is represented as the IDR type, when the value of the NAL unit type is 7, it is represented as the SPS type, and when the value of the NAL unit type is 8, it is represented as the PPS type.

[0054] The selected NAL unit types of the H.265 stream include VPS, SPS and PPS, and the value interval of the NAL unit type of the H.265 stream is 32, 33 and 34; that is, when the value of the NAL unit type is 32, it is represented as the VPS type, when the value of the NAL unit type is 33, it is represented as the SPS type, and when the value of the NAL unit type is 34, it is represented as the PPS type.

[0055] The NAL unit types and values belonging to the H.264 stream extracted from the NAL unit header are stored in the H.264 queue, and the NAL unit types and values belonging to the H.265 stream extracted from the NAL unit header are stored in the H.265 queue.

[0056] It is judged whether the NAL unit types of SPS / PPS / IDR of the H.264 stream exist in the H.264 queue.

[0057] It is judged whether the NAL unit types of VPS / SPS / PPS of the H.265 stream exist in the H.265 queue.

[0058] If the NAL unit types of SPS / PPS / IDR of the H.264 stream all exist in the H.264 queue, and the NAL unit types of VPS / SPS / PPS of the H.265 stream do not completely exist in the H.265 queue, it is judged that the format of the video stream is the H.264 stream.

[0059] If the NAL unit types of VPS / SPS / PPS of the H.265 stream all exist in the H.265 queue, and the NAL unit types of SPS / PPS / IDR of the H.264 stream do not completely exist in the H.264 queue, it is judged that the format of the video stream is the H.265 stream.

[0060] If other conditions occur, it is judged as a failure.

[0061] The embodiment can fully consider the value interval of the NAL unit type of the H.264 stream and the H.265 stream by identifying the key NAL unit types VPS, SPS, PPS and IDR, without multiple traversals of the video stream, and the accurate stream format can be identified at one time, and the accuracy of identification is ensured by the secondary judgment of the queue, so that the subsequent video stream coding processing can be performed according to the identified stream format.

[0062] Embodiment 2

[0063] The embodiment provides a video stream analysis system, which comprises:

[0064] An acquisition module configured to acquire a video stream;

[0065] A first identification module configured to determine whether the video stream is a program stream or an audio stream, and if so, obtain a corresponding stream format;

[0066] A second identification module configured to, if not, extract NAL unit types belonging to H.264 stream and H.265 stream respectively from a NAL unit header of the video stream, and store the NAL unit types in an H.264 queue and an H.265 queue respectively;

[0067] A determination module configured to determine that the format of the video stream is the stream format to which the NAL unit types of the queue in which all the NAL unit types exist belong, only when all the NAL unit types of one of the queues exist and the NAL unit types of the other queue do not completely exist.

[0068] It should be noted that the modules correspond to the steps described in Embodiment 1, and the modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the modules as part of the system can be executed in a computer system such as a group of computer executable instructions.

[0069] In more embodiments, the following are also provided:

[0070] An electronic device comprising a memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, the method described in Embodiment 1 is completed. For brevity, it will not be repeated here.

[0071] It should be understood that in the embodiment, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSPs, application-specific integrated circuits ASICs, ready-to-program gate arrays FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0072] The memory can include read-only memory and random access memory, and provide instructions and data to the processor, and a part of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0073] A computer readable storage medium for storing computer instructions, when the computer instructions are executed by the processor, the method described in Embodiment 1 is completed.

[0074] The method in the embodiment 1 can be directly embodied by a hardware processor or by a combination of hardware and software modules in the processor. The software modules can be located in a storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, or the like. The storage medium is located in a memory, and the processor reads information in the memory and combines the hardware to complete the steps of the above method. To avoid repetition, no further detailed description is given here.

[0075] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner depends on the specific application and design constraints of the technical solutions. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0076] Although the specific embodiments of the present application are described above in combination with the drawings, the description is not a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications or changes made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the scope of protection of the present application.

Claims

1. A method for video bitstream analysis, the method comprising: The method comprises the following steps: acquiring a video bitstream; extracting the first two bytes of the video bitstream, and judging whether the video bitstream is an AAC stream if the first two bytes match FFF*; if yes, judging whether the video bitstream is an AAC stream again by analyzing the AAC ADTS header, and judging the video bitstream as an AAC stream if yes, or judging the video bitstream as an H.264 stream or an H.265 stream if no; judging whether the video bitstream is a program stream or an audio stream, and obtaining the corresponding bitstream format if yes; or extracting the NAL unit types belonging to the H.264 stream and the H.265 stream from the NAL unit header of the video bitstream, and storing the NAL unit types in an H.264 queue and an H.265 queue respectively if no; selecting the NAL unit types of the H.264 stream, including IDR, SPS and PPS; selecting the NAL unit types of the H.265 stream, including VPS, SPS and PPS; judging the format of the video bitstream as the bitstream format of the queue with all the NAL unit types if all the NAL unit types of one queue exist and all the NAL unit types of the other queue do not exist.

2. The method of claim 1, wherein, The method judges whether the video bitstream is a program stream or an audio stream by byte matching.

3. The method of claim 2, wherein, The byte matching method comprises the following steps: extracting the first five bytes of the video bitstream, and judging the video bitstream as a PS stream if the first five bytes match 00 00 00 01 BA.

4. The method of claim 1, wherein, storing the NAL unit types belonging to the H.264 stream and their values in the H.264 queue.

5. The method of Claim 1, wherein, storing the NAL unit types belonging to the H.265 stream and their values in the H.265 queue.

6. The method of claim 4 or 5, wherein, judging the format of the video bitstream as the H.264 stream if the SPS, PPS and IDR NAL unit types of the H.264 stream in the H.264 queue all exist, and the VPS, SPS and PPS NAL unit types of the H.265 stream in the H.265 queue do not exist.

7. The method of claim 4 or 5, wherein, judging the format of the video bitstream as the H.265 stream if the VPS, SPS and PPS NAL unit types of the H.265 stream in the H.265 queue all exist, and the SPS, PPS and IDR NAL unit types of the H.264 stream in the H.264 queue do not exist.

8. A video bitstream analysis system, employing the video bitstream analysis method of any one of claims 1-7, characterized in that, The method comprises the following steps: an acquisition module configured to acquire a video bitstream; a first identification module configured to judge whether the video bitstream is a program stream or an audio stream; if yes, obtaining the corresponding bitstream format; a second identification module configured to extract the NAL unit types belonging to the H.264 stream and the H.265 stream from the NAL unit header of the video bitstream, and store the NAL unit types in an H.264 queue and an H.265 queue respectively if no; a judging module configured to judge the format of the video bitstream as the bitstream format of the queue with all the NAL unit types if all the NAL unit types of one queue exist and all the NAL unit types of the other queue do not exist.

9. An electronic device, comprising: The computer device comprises a memory and a processor, and computer instructions stored in the memory and run on the processor, and the computer instructions are run by the processor to complete the method in any one of claims 1-7. The computer device comprises a memory and a processor, and computer instructions stored in the memory and run on the processor, and the computer instructions are run by the processor to complete the method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Computer program product for storing computer instructions which, when executed by a processor, perform the method of any one of claims 1-7.

Images