Video hardware decoder circuit, bitstream parsing error detection and control method and system

Abstract

The present invention discloses a video hardware decoder circuit, a bitstream parsing error detection and control method, and a system. The method includes: performing bitstream safety range detection on the acquired video bitstream to obtain a first detection result; performing bitstream extreme value detection on the acquired video bitstream to obtain a second detection result; according to the first detection result and the second detection result, performing anomaly detection on the process of bitstream parsing of the video bitstream, and performing anomaly processing on the video bitstream or frame reset according to the anomaly detection result. The system includes a bitstream safety range detection module, an extreme value detection module, and an anomaly identification and reset module. The technical solution of the present invention can enable the video hardware decoder to have a wider error detection range for the video bitstream, more flexible detection methods, and higher detection sensitivity. After error processing of the video bitstream, the decoded video image quality is better, and at the same time, the impact on the decoding performance of the hardware decoder is smaller.

Description

Technical Field

[0001] The present invention relates to the technical field of video decoding, and particularly to a video hardware decoder circuit, a bitstream parsing error detection and control method, and a system. Background Art

[0002] With the continuous update of encoding technologies in video codec standards, the efficiency of video encoding has been improved. Therefore, the encoded video stream has better network affinity, better image quality, and stronger error resilience. Currently, video communication has become the main service of communication. However, due to various reasons such as high-efficiency compression encoding and channel transmission, the video stream is prone to bit errors and data loss during transmission. The problems that may occur during the hardware decoding of an incorrect video stream are as follows:

[0003] 1) The hardware decoder fails to recover the correct syntax elements, resulting in decoding errors. Moreover, based on existing decoding technologies, forward bit errors will spread backward, leading to a mosaic image in the decoded image of the entire video stream.

[0004] 2) When the hardware decoder decodes certain incorrect syntax elements, the entire decoding process ends erroneously, preventing the subsequent video stream from being decoded.

[0005] [[ID=!7]]3) The hardware decoder is unable to process incorrect syntax elements during decoding, resulting in abnormal freezing of the hardware decoder.

[0006] Therefore, how to quickly detect errors in the video stream (i.e., the error detection and processing technology of the video decoder) is a problem that needs to be solved during the design of each decoder.

[0007] When designing existing hardware video decoders, due to considerations of hardware logic complexity and decoding performance, almost no error detection and processing mechanisms for video stream decoding are added to the hardware video decoder. This makes it easy for the hardware decoder to freeze and display a mosaic image when processing an incorrect video stream, as Figure 1As shown. Currently, the main research on error detection and processing of video stream by hardware decoders mainly focuses on the following aspects: 1) Error detection and control mechanism at the MB level. For a macroblock with decoding errors, use adjacent correctly decoded macroblocks, macroblocks that have completed error processing, or the MV of uncompleted error processing macroblocks to predict the MV of the currently processed macroblock, and use the predicted MV to perform motion compensation using the reference frame pixels of the current frame; 2) Error detection and control mechanism at the frame level. Compare the display frame number of the frame to be decoded with the number of the most recently displayed frame. If the display number of the frame to be decoded is less than the most recently displayed frame, discard it directly; 3) Detection of network packet loss rate: Detect the network packet loss rate and the I-frame interval; Compare the network packet loss rate with a preset first threshold to obtain a first comparison result, and compare the I-frame interval with a preset second threshold to obtain a second comparison result; When an error frame is detected, select a preset mode to adjust the error frame according to the first comparison result and the second comparison result, where the error frame includes I-frames.

[0008] Problems existing in the above video stream error detection and processing mechanism:

[0009] 1) It is not suitable for implementation in hardware video decoders. The detection and processing logic is too complex, and its implementation will increase a lot of hardware resources and circuit area. At the same time, the complex hardware implementation will reduce the performance of hardware video stream decoding;

[0010] 2) The video stream error detection is not flexible enough: For example, it can only detect errors at the frame level, resulting in the inability to correctly process errors encountered at the slice level;

[0011] 3) The response of video stream error detection and processing is too slow: After detecting the error point, it cannot be processed in time, resulting in the problem of the hardware decoder crashing;

[0012] The image quality of video stream error processing is too poor: There are multi-frame or large-area garbled screens within the frame in the decoded image. Summary of the Invention

[0013] One of the purposes of the present invention is to overcome the deficiencies in the prior art. Aiming at the technical problems of complex logic, lack of flexibility, slow response, and poor image quality in the existing video stream error detection and processing mechanism, the present invention provides a video hardware decoder circuit, a bitstream parsing error detection and control method, and a system.

[0014] To achieve the above purposes, the present invention is realized through the following technical solutions:

[0015] In the first aspect, the present invention provides a bitstream parsing error detection and control method for a video hardware decoder, and the method includes:

[0016] Perform a bitstream security range detection on the acquired video bitstream to obtain a first detection result;

[0017] Perform a bitstream extreme value detection on the acquired video bitstream to obtain a second detection result;

[0018] According to the first detection result and the second detection result, perform an anomaly detection on the process of bitstream parsing of the video bitstream, and perform anomaly handling or frame reset on the video bitstream according to the anomaly detection result.

[0019] In a preferred embodiment of the present application, in the step of performing a bitstream security range detection on the acquired video bitstream to obtain a first detection result, it specifically includes:

[0020] Obtain the configuration information of the video bitstream from the driving end to obtain the bitstream length of the video bitstream included in the configuration information;

[0021] Set a bitstream pointer to record the current consumed length of the video bitstream;

[0022] By comparing the current consumed length with the bitstream length, determine whether the video bitstream has a bitstream out-of-bounds behavior;

[0023] If the judgment result is not occurred, return to repeat the judgment;

[0024] If the judgment result is occurred, output a bitstream out-of-bounds flag as the first detection result.

[0025] In a preferred embodiment of the present application, the bitstream pointer includes a byte-aligned bitstream pointer and a bit-level bitstream pointer;

[0026] The byte-aligned bitstream pointer determines whether the bitstream out-of-bounds behavior occurs by judging whether the record of the byte counter that counts the byte consumption of the video bitstream is zero; if the record of the byte counter is zero and the length of the currently operated byte-aligned bitstream pointer exceeds the length of the previous frame bitstream pointer, the judgment result is occurred;

[0027] The bit-level bitstream pointer determines whether the bitstream out-of-bounds behavior occurs by judging whether the current bitstream length calculated by the bitstream counter is greater than the current bitstream rated total length; if the current bitstream length is greater than the current bitstream rated total length, the judgment result is occurred.

[0028] In a preferred embodiment of the present application, in the step of performing a bitstream extreme value detection on the acquired video bitstream to obtain a second detection result, it specifically includes:

[0029] Obtain the configuration information of the video bitstream from the driving end to obtain different syntax elements included in the configuration information;

[0030] Perform bitstream parsing on each syntax element of the video bitstream to obtain the valid range of the corresponding value of the syntax element;

[0031] Based on the valid range of the corresponding value of each syntax element, determine whether there is a bitstream out-of-bounds behavior in the video bitstream by detecting whether the value of each syntax element is within the valid range;

[0032] If the value of the syntax element is within the valid range, return a repeated judgment;

[0033] If the value of the syntax element is outside the valid range, output an out-of-range flag as the second detection result.

[0034] In a preferred embodiment of the present application, in performing bitstream extreme value detection on the obtained video bitstream to obtain the second detection result, it specifically includes:

[0035] Obtain the configuration information of the video bitstream from the driving end to obtain different syntax elements included in the configuration information;

[0036] Perform bitstream parsing on each syntax element of the video bitstream to obtain the arithmetic coding corresponding to the syntax element;

[0037] Based on the arithmetic coding corresponding to each syntax element, determine whether there is a bitstream out-of-bounds behavior in the video bitstream by judging whether the number of leading zeros in the video bitstream is greater than the corresponding standard value;

[0038] If the number of leading zeros in the bitstream is not greater than the corresponding standard value, return a repeated judgment;

[0039] If the number of leading zeros in the bitstream is greater than the corresponding standard value, output a leading zero exception flag as the second detection result.

[0040] In a preferred embodiment of the present application, in the process of performing bitstream parsing on the video bitstream according to the first detection result and the second detection result, perform anomaly detection, and perform anomaly processing on the video bitstream or perform frame reset according to the anomaly detection result, specifically including:

[0041] According to the first detection result and the second detection result, record the error information generated during the process of performing bitstream parsing on the video bitstream;

[0042] Judge the error level recorded in the error information and perform anomaly processing on the video bitstream;

[0043] When the error level is the zero level, ignore the error information and continue normal decoding;

[0044] When the error level is the first level, the current frame is a single stripe, the decoding of the current stripe is ended, according to the coordinate information of the current frame recorded in the error information, picture information is moved from adjacent frames of the current frame to fill the current frame, and the decoding of the next stripe is started;

[0045] When the error level is the second level, the current frame is multiple stripes, the decoding of the current frame is ended, according to the coordinate information of the current frame recorded in the error information, picture information is moved from adjacent frames of the current frame to fill the current frame, and the decoding of the next frame is started;

[0046] The information about the error found at the current parsing position in the video bitstream recorded in the error information is broadcast to subsequent modules, and after the subsequent modules parse the video bitstream one by one until the error point where the error is found, a reset is performed.

[0047] In a second aspect, the present invention provides a bitstream parsing error detection and control system for a video hardware decoder. The system includes: a bitstream safety range detection module, an extreme value detection module, and an anomaly recognition and reset module. The bitstream safety range detection module and the extreme value detection module are respectively electrically connected to the anomaly recognition and reset module;

[0048] The bitstream safety range detection module is used to perform bitstream safety range detection on the obtained video bitstream to obtain a first detection result;

[0049] The extreme value detection module is used to perform bitstream extreme value detection on the obtained video bitstream to obtain a second detection result;

[0050] The anomaly recognition and reset module is used to perform anomaly detection on the process of bitstream parsing of the video bitstream according to the first detection result and the second detection result, and perform anomaly processing or frame reset on the video bitstream according to the anomaly detection result.

[0051] In a third aspect, the present invention provides a video hardware decoder circuit, including:

[0052] A bitstream preprocessing module for preprocessing the video bitstream input by the network abstraction layer;

[0053] An entropy decoding module for performing entropy decoding on the video bitstream preprocessed by the bitstream preprocessing module;

[0054] A video decoding module for decoding and filtering the video bitstream processed by the entropy decoding module;

[0055] A reconstructed frame module for performing reconstructed frame processing on the video bitstream processed by the video decoding module;

[0056] A driving end and the bitstream parsing error detection and control system as described in the second aspect;

[0057] The driving end is electrically connected to the bitstream security range detection module, the extreme value detection module, and the anomaly recognition and reset module respectively, and is used to drive each module to work;

[0058] The bitstream security range detection module is electrically connected to the bitstream preprocessing module, and is used to perform bitstream security range detection on the acquired video bitstream to obtain a first detection result;

[0059] The extreme value detection module is electrically connected to the entropy decoding module, and is used to perform bitstream extreme value detection on the acquired video bitstream according to different syntax elements in the video bitstream parsed by the entropy decoding module to obtain a second detection result.

[0060] Fourthly, the present invention provides a computer-readable storage medium, in which a computer program is stored. When it runs on a computer, it causes the computer to execute the bitstream parsing error detection control method of the video hardware decoder as described in the first aspect.

[0061] Fifthly, the present invention provides a computer program product, which includes a computer program. When it runs on a computer, it causes the computer to execute the bitstream parsing error detection control method of the video hardware decoder as described in the first aspect.

[0062] The video hardware decoder circuit, bitstream parsing error detection control method and system disclosed by the present invention enable the video hardware decoder to have a wider error detection range, more flexible detection methods and higher detection sensitivity for video bitstreams. After error processing of the video bitstream, the decoded video image quality is better, and at the same time, the decoding performance of the hardware decoder is less affected. Description of the Drawings

[0063] The present invention is described with the aid of the following drawings:

[0064] Figure 1 is the phenomenon of a garbled image that appears after decoding by a video hardware decoder in the prior art;

[0065] Figure 2 is the flowchart of the bitstream parsing error detection control method of the video hardware decoder in Embodiment 1 of the present invention;

[0066] Figure 3 is the specific flowchart of Step 100 in the bitstream parsing error detection control method of the video hardware decoder in Embodiment 1 of the present invention;

[0067] Figure 4It is a schematic diagram of a bitstream out-of-bounds behavior when performing bitstream safety range detection in the bitstream parsing error detection control method of the video hardware decoder in Embodiment 1 of the present invention;

[0068] Figure 5 It is a specific flowchart of Step 200 in the bitstream parsing error detection control method of the video hardware decoder in Embodiment 1 of the present invention;

[0069] Figure 6 It is another specific flowchart of Step 200 in the bitstream parsing error detection control method of the video hardware decoder in Embodiment 1 of the present invention;

[0070] Figure 7 It is a specific flowchart of Step 300 in the bitstream parsing error detection control method of the video hardware decoder in Embodiment 1 of the present invention;

[0071] Figure 8 It is a schematic diagram of the modules of the bitstream parsing error detection control system of the video hardware decoder in Embodiment 2 of the present invention;

[0072] Figure 9 It is a schematic diagram of the operation of the bitstream safety range detection module in the bitstream parsing error detection control system of the video hardware decoder in Embodiment 2 of the present invention;

[0073] Figure 10 It is a schematic diagram of the operation of the extreme value detection module in the bitstream parsing error detection control system of the video hardware decoder in Embodiment 2 of the present invention;

[0074] Figure 11 It is another schematic diagram of the operation of the extreme value detection module in the bitstream parsing error detection control system of the video hardware decoder in Embodiment 2 of the present invention;

[0075] Figure 12 It is a schematic diagram of the structure of the video hardware decoder circuit in Embodiment 3 of the present invention. Detailed implementation manners

[0076] To better understand the technical solutions of the present application, the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0077] It should be clear that the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0078] Embodiment 1

[0079] Embodiment 1 of the present invention discloses a method for detecting and controlling bitstream parsing errors of a video hardware decoder. During different stages of the operation of the video hardware decoder, a process of extreme value detection is inserted, and a small number of registers are used to store the results of error detection and exception handling instructions, enabling the video hardware decoder to have the ability of error detection and self-repair at the forefront of bitstream parsing at the slice layer. This can effectively reduce the interaction between the video hardware decoder and the video processing system when the video hardware decoder operates abnormally, and actively recover the error frames according to the error types, thereby effectively enhancing the robustness of the video hardware decoder.

[0080] See Figure 2 , the method of this Embodiment 1 includes:

[0081] Step \(100\): Perform bitstream safety range detection on the obtained video bitstream to obtain a first detection result;

[0082] Step \(200\): Perform bitstream extreme value detection on the obtained video bitstream to obtain a second detection result;

[0083] Step \(300\): According to the first detection result and the second detection result, perform exception detection on the process of bitstream parsing of the video bitstream, and perform exception handling or frame reset on the video bitstream according to the exception detection result.

[0084] Specifically, to solve various decoding error situations that may occur during the decoding process of the video hardware decoder (these situations have been described in the background technology above), for the video bitstream parsing process under the two widely used video coding standards of H.264 / H.265, there are corresponding interval extreme values for different syntax elements, different extreme value detections are proposed according to the internal attributes of different arithmetic coding methods for entropy decoding, and three detection mechanisms for enhancing the robustness of the hardware decoder such as bitstream length safety. In the method of this Embodiment 1, through the step flow of Step \(100\) to Step \(300\), corresponding extreme value detections are inserted into the video decoding circuit of the hardware decoder according to different syntax elements, and through multiple registers and error handling instructions connected to the video decoding circuit, it is ensured that the video hardware decoder can correctly identify an exception in the first time when an exception occurs. At the same time, because entropy decoding is in the upstream position in the pipeline of the entire video hardware decoder operation, when an exception occurs, the subsequent hardware modules are notified through the corresponding error handling instructions not to perform normal decoding after decoding to the exception point. After the video hardware decoder discovers an exception and broadcasts to all modules to complete the reset operation, data will be moved according to the coordinate position of the current exception point in the picture, and the picture of the adjacent frame corresponding picture will be used to fill the picture of the current frame error coordinate point, and all decoding work of the current frame will be ended.

[0085] See Figure 3 , in the method of this Embodiment 1, Step \(100\) specifically includes:

[0086] Step101: Obtain the configuration information of the video bitstream from the driving end to get the bitstream length of the video bitstream included in the configuration information;

[0087] Step102: Set the bitstream pointer to record the current consumed length of the video bitstream;

[0088] Step103: Determine whether a bitstream out-of-bounds behavior occurs in the video bitstream by comparing the current consumed length with the bitstream length; if the judgment result is that it does not occur, return to Step101 to repeat the judgment; if the judgment result is that it occurs, execute Step104;

[0089] Step104: Output the bitstream out-of-bounds flag as the first detection result.

[0090] Specifically, the bitstream length of the video bitstream is used as the header information for frame-level parsing. When software decoding, obtaining this header information requires configuring it to the video hardware decoder through the driving end. It is judged whether the current video hardware decoder exceeds the safe parsing range of the bitstream by whether the total length of the parsed bitstream length is out of bounds. Therefore, the video decoding circuit can judge whether a bitstream out-of-bounds phenomenon occurs by adding a corresponding bitstream pointer to record the current consumed bitstream length of the video bitstream compared with the total length. In Step102, the bitstream is generally divided into byte align and non-byte align, and the bitstream pointer (cur_bsb_cmptr) includes two types: byte-aligned bitstream pointer and bit-level bitstream pointer. See Figure 4, the byte-aligned bitstream pointer determines whether a bitstream out-of-bounds behavior occurs by checking whether the record of the byte counter that counts the byte consumption of the video bitstream is zero. If the bitstream pointer is a byte-aligned bitstream pointer, it is necessary to check whether the current byte counter (byte counter) is equal to zero. If the record of the byte counter is zero, it means that the current bitstream length of the video bitstream has been consumed to zero. However, when the length of the currently operated byte-aligned bitstream pointer exceeds the length of the previous frame bitstream pointer (Prv_bsb_cmptr), it means that a valid read operation has also occurred. Then, it is considered that a read bitstream out-of-bounds occurs, which is a decoding exception behavior, and the judgment result is that it occurs. The bit-level bitstream pointer determines whether a bitstream out-of-bounds behavior occurs by checking whether the current bitstream length calculated by the bitstream counter is greater than the current rated total bitstream length. When the bitstream pointer is a bit-level bitstream pointer, if the current bitstream length (Full_bsb_cmptr) calculated by the byte counter is greater than the current rated total bitstream length (bsb_cmptr_keep) of the video bitstream, it indicates that the bitstream decoding is out-of-bounds, and the judgment result is that it occurs. In the method of Embodiment 1 of the present invention, the video hardware decoder uses the above two detection mechanisms simultaneously, which plays a dual insurance role to ensure that the current bitstream parsing is not out-of-bounds.

[0091] See Figure 5 , in the method of Embodiment 1 of the present invention, Step200 specifically includes:

[0092] Step201: Obtain the configuration information of the video bitstream from the driver end to obtain different syntax elements included in the configuration information;

[0093] Step202: Perform bitstream parsing on each syntax element of the video bitstream to obtain the valid range of the corresponding value of the syntax element;

[0094] Step203: According to the valid range of the corresponding value of each syntax element, determine whether there is a bitstream out-of-bounds behavior in the video bitstream by detecting whether the value of each syntax element is within the valid range; if the value of the syntax element is within the valid range, return to Step201 to repeat the judgment; if the value of the syntax element is outside the valid range, execute Step204;

[0095] Step204: Output the value out-of-bounds flag as the second detection result.

[0096] Specifically, take the syntax element mb_qp_delta in bitstream parsing as an example. The position of the syntax element mb_qp_delta is within macroblock layer decoding and before residual parsing. In bitstream parsing, when the CBF (codedblock flag, which indicates whether the current block contains non-zero transform coefficients. If codedblock flag = 0, the current block does not contain non-zero transform coefficients. If codedblock flag = 1, the current block contains at least one non-zero transform coefficient) of the current block in the video bitstream is non-zero, or when the video bitstream divides the macroblock into the intra 16x16 mode, mb_qp_delta parsing is performed. The meaning of this mb_qp_delta parsing is the increment of the quantization coefficient when the macroblock performs transform quantization, as shown in Table 1 below.

[0097]

[0098] Table 1: Decoding position of mb_qt_delta

[0099] Specifically, when a slice decoding starts, slice_qp_delta is used as the quantization starting value of the current slice, but when decoding reaches a certain macroblock, the qp value of the current macroblock can be rewritten through mb_qp_delta. The value range of mb_qt_delta of the current block is [-26 + QpBdOffset / 2, 25 + QpBdOffset / 2] according to the SPEC (Advanced video coding for generic audiovisual services. Since H.264 is one of the video coding technologies named by ITU-T in the H.26x series, it is generally called the H.264 standard), and the final QP value of the current macroblock is calculated through the following formula.

[0100] QP = (QP + mb_qp_delta + 52 + 2*QpBdOffset) % (52 + QpBdOffset)) – QpBdoffset

[0101] Through analysis, it is found that the valid range of mb_qp_delta in the video bitstream is [-26, 25]. Since unsigned numbers are used in the process of video decoding by the video hardware decoder, the valid range of mb_qp_delta is shifted to the right to [0, 52]. Therefore, mb_qt_detla is only valid when it falls within this interval, otherwise it is invalid decoding. By adding real-time decoding comparison in the original video hardware decoder, when the value of mb_qp_delta parsed from the bitstream exceeds this value range, an error prompt will be given, so that the video hardware decoder can obtain effective decoding error information in the first place. Similar syntax elements to mb_qp_delta include ref_idx_l0, mvd_l0, coeff_abs_level_minus1, etc. The extreme value ranges of different syntax elements can be analyzed to effectively determine whether the video hardware decoder is within the safe decoding range.

[0102] See Figure 6 , in the method of Embodiment 1, Step200 can also be carried out in the following way, specifically including:

[0103] Step201’: Obtain the configuration information of the video bitstream from the driving end to get different syntax elements included in the configuration information;

[0104] Step202’: Parse the bitstream of each syntax element of the video bitstream to obtain the arithmetic coding corresponding to the syntax element;

[0105] Step203’: According to the arithmetic coding corresponding to each syntax element, determine whether there is a bitstream out-of-bounds behavior in the video bitstream by judging whether the number of leading zeros in the bitstream is greater than the corresponding standard value; if the number of leading zeros in the bitstream is not greater than the corresponding standard value, return to execute Step201’ to repeat the judgment; if the number of leading zeros in the bitstream is greater than the corresponding standard value, execute Step204’;

[0106] Step204’: Output the leading zero exception flag as the second detection result.

[0107] Specifically, the video bitstream has various syntax elements. During the parsing process of the video bitstream, it is possible to determine whether the video hardware decoder is in a normal decoding state based on the inherent attributes of different encoding methods used for different syntax elements. For example, Exp-Golomb is a widely adopted encoding method in the H.264 standard. Exp-Golomb coding mainly includes four types: ue(v), se(v), me(v), and te(v), which are unsigned Exp-Golomb coding, signed Exp-Golomb coding, coded Exp-Golomb, and truncated Exp-Golomb coding respectively. This application text does not expand on each Exp-Golomb coding, but only takes the unsigned Exp-Golomb coding as an example for illustration, and the other Exp-Golomb codings can be understood by referring to the unsigned Exp-Golomb coding.

[0108] The input value of the unsigned Exp-Golomb is the video bitstream being parsed, and the output value is the value obtained according to the calculation rules of Exp-Golomb for the current video bitstream. The calculation rule of the unsigned Exp-Golomb is to find the value that is not equal to zero starting from the current bit, and record the number of existing zeros as the leading zero count, and this value is used as the prefix value (prefix). The value that is symmetric to the non-zero value and the leading zeros is called the suffix value (suffix), as shown in Table 2 below.

[0109] Bitstring form Range of codeNum 1 0 <![CDATA[01x0]]> 1-2 <![CDATA[001x1x0]]> 3-6 <![CDATA[0001x2x1x0]]> 7-14 <![CDATA[00001x3x2x1x0]]> 15-30 <![CDATA[000001x4x3x2x1x0]]> 31-62 …… ……

[0110] Table 2: Exp-Golomb data format

[0111] As can be seen from Table 2 above, the number of leading zeros is the same length as the number of valid values. The suffix value is the valid value. The bits read from the video bitstream are in the order from front to back, from high to low. The calculation formula for the final codeword (codeNum) is as follows.

[0112] codeNum = 2 leadingZeroBits -1 + read_bits(leadingZeroBits)

[0113] As can be seen from the above process, the first step of decoding exponential Golomb is to continuously read zeros from the video bitstream until a non-zero value appears. First, calculate the number of leading zeros in the bitstream. Secondly, select the corresponding standard value according to the type of Golomb exponent. Then, compare the maximum value with the number of leading zeros in the video bitstream. If the number of leading zeros in the video bitstream is greater than the corresponding standard value, mark the decoding as abnormal and output an arithmetic coding exception flag to the video hardware decoder. Among them, the number of leading zeros is calculated as leadingZeroBits. By analyzing the SPEC regulations, it is found that the definition of the exponential Golomb codeword is a 32-bit unsigned integer data (unsigned interger). Therefore, the effective length of this syntax element in the video bitstream must be less than or equal to 32. So, if the number of leading zeros of the unsigned exponential Golomb is greater than 32, it is abnormal. The judgment method is to first calculate the number of leading zeros, and then judge whether the number of leading zeros is greater than 32. If the number of leading zeros is less than or equal to 32, it is within the safe range, otherwise the exponential Golomb decoding is abnormal.

[0114] See Figure 7 , in the method of Embodiment 1, Step300 specifically includes:

[0115] Step301: According to the first detection result and the second detection result, record the error information generated during the parsing of the video bitstream;

[0116] Step302: Judge the error level recorded in the error information and perform abnormal processing on the video bitstream; when the error level is the zero level, execute Step303; when the error level is the first level, execute Step304 and Step306; when the error level is the second level, execute Step305 and Step306;

[0117] Step303: Ignore the error information and continue normal decoding;

[0118] Step304: If the current frame is a single strip, end the decoding of the current strip, and according to the coordinate information of the current frame recorded in the error information, move the picture information from the adjacent frames of the current frame to fill the current frame, and start decoding the next strip;

[0119] Step305: If the current frame is multiple strips, end the decoding of the current frame, and according to the coordinate information of the current frame recorded in the error information, move the picture information from the adjacent frames of the current frame to fill the current frame, and start decoding the next frame.

[0120] Step306: Broadcast the information about the error found at the current parsing position in the video bitstream recorded in the error information to the subsequent modules, and reset after the subsequent modules parse the video bitstream one by one until the error point is found.

[0121] Specifically, in the method of Embodiment 1, in addition to constructing the logic for real-time extreme value detection and judgment during the process of parsing the video bitstream, it is also necessary to add exception handling and reset steps during the error detection process of bitstream parsing to handle error situations during the decoding process. In Step301, when the first detection result obtained from the bitstream safety range detection detects a bitstream out-of-bounds behavior, it will output a bitstream out-of-bounds flag. When the second detection result obtained from the extreme value detection detects a bitstream out-of-bounds behavior, it will output a value out-of-bounds flag or a leading zero exception flag. As long as one of the flags exists in the first detection result or the second detection result, error information such as whether an error occurs, the error level, and the type during the current decoding process will be recorded. In Embodiment 1, two registers and instructions are used to handle error situations during the decoding process, as shown in Table 3 and Table 4 below. The error type register shown in Table 3 is used to record whether an error occurs, the error level, and type information during the current decoding. The exception handling program register shown in Table 4 is used to save an entry address, and when an exception occurs, it will jump to a preset exception handling program. In Table 3 and Table 4, bits represent the positions of the stored values in the register, and the values stored in different positions have different functions. In Table 3, the position of bit 0 is used to determine whether a decoding exception occurs during the process of parsing the video bitstream. The positions of bits 1 to 3 are used to determine the error type and the way of exception handling. The positions of bits 4 to 10 are used to record the error type. The positions of bits 11 to 31 are reserved bits, which are used to store other information for future use. In Table 4, the positions of bits 0 to 31 are used for the decoding jump destination address preset for the occurrence of a decoding exception.

[0122]

[0123] Table 3: Error Type Register

[0124] Bit Name of exception handling Value of exception handling in register [31:0] BranchPC value Preset decoded jump destination address

[0125] Table 4: Exception Handling Program Register

[0126] In Step 302, when the bitstream out-of-bounds flag, value out-of-bounds flag, or leading zero anomaly flag is detected, it indicates that there is an anomaly in bitstream parsing, and it is necessary to judge the error type level. The error level can be classified into at least two categories. In this Embodiment 1, the error level includes the zero level, the first level, and the second level. For different error levels, the anomaly handling methods are different. Specifically, after an anomaly occurs during the bitstream parsing process, different handling is performed according to the impacts brought by different errors. Currently, the adopted error levels are divided into three cases. The first category is the zero level. When the error level is the zero level, the current error can be ignored, only the current error type is printed, the error information is ignored, and no other actions are taken, and the result does not affect the continuation of normal decoding. According to whether the current frame is a single slice or multiple slices, it is divided into the second category and the third category. When the current frame is a single slice, it is the second category, that is, the first level. At this time, it jumps to the anomaly handling program to end the decoding of the current slice; when the current frame is multiple slices, it is the third category, that is, the second level. At this time, it jumps to the anomaly handling program to end the decoding of the current frame. This anomaly handling can be to move the remaining picture from the neighboring frames that have been parsed in each frame of the current video bitstream according to the coordinate information of the error that has occurred in the current video bitstream to fill the current frame. If the bitstream out-of-bounds flag, value out-of-bounds flag, or leading zero anomaly flag is not detected, normal decoding continues.

[0127] In addition to performing anomaly handling when an error occurs in bitstream parsing, when the error levels triggered during the bitstream parsing process are the first level and the second level, a reset operation is performed on the video bitstream. The role of the reset is to broadcast the information of the detected error from the current bitstream parsing position to the subsequent modules. When receiving this signal, due to the execution of the pipeline, subsequent modules may still be parsing the content before the error point. Then, when a subsequent block successively parses to the error point, it is reset. Because the error handling program will next move and fill the existing data content, the subsequent modules remain idle until the end of the current frame.

[0128] Embodiment 2

[0129] Embodiment 2 of the present invention discloses a bitstream parsing error detection and control system for a video hardware decoder. By inserting an extreme value detection process at different stages of the video hardware decoder's operation and cooperating with a small number of registers to store the results of error detection and anomaly handling instructions, the video hardware decoder has the ability of error detection and self-repair at the very front end of bitstream parsing at the slice layer, which can effectively reduce the interaction with the video processing system when the video hardware decoder operates abnormally and actively recover the error frames according to the error type, thereby effectively enhancing the robustness of the video hardware decoder.

[0130] See Figure 8, the bitstream parsing error detection and control system 10 of the video hardware decoder in this Embodiment 2 includes a bitstream security range detection module 11, an extreme value detection module 12, and an anomaly recognition and reset module 13. The bitstream security range detection module 11 and the extreme value detection module 12 are respectively electrically connected to the anomaly recognition and reset module 13. The bitstream security range detection module 11 is used to perform bitstream security range detection on the acquired video bitstream to obtain a first detection result. The extreme value detection module 12 is used to perform bitstream extreme value detection on the acquired video bitstream to obtain a second detection result. The anomaly recognition and reset module 13 is used to perform anomaly detection on the process of bitstream parsing of the video bitstream according to the first detection result and the second detection result, and perform anomaly processing or frame reset on the video bitstream according to the anomaly detection result.

[0131] Specifically, to solve various decoding error situations that may occur during the decoding process of the video hardware decoder, for the video bitstream parsing process under the two widely used video coding and decoding standards of H.264 / H.265, there are corresponding interval extreme values for different syntax elements, different extreme value detections are proposed according to the internal attributes of different arithmetic coding methods for entropy decoding, and three detection mechanisms for enhancing the robustness of the hardware decoder such as bitstream length security are proposed. Through the bitstream security range detection module 11, the extreme value detection module 12, and the anomaly recognition and reset module 13, corresponding extreme value detections are inserted into the video decoding circuit of the hardware decoder according to different syntax elements, and through multiple registers and error handling instructions connected to the video decoding circuit, it is ensured that the video hardware decoder can correctly identify anomalies in the first time when an anomaly occurs. At the same time, because entropy decoding is in the upstream position in the entire pipeline of the video hardware decoder's operation, when an anomaly occurs, the subsequent hardware modules are notified through corresponding error handling instructions not to perform normal decoding after decoding to the anomaly point. After the video hardware decoder discovers an anomaly and broadcasts to all modules to complete the reset operation, data will be moved according to the coordinate position of the current anomaly point in the picture, and the picture of the adjacent frame corresponding picture will be used to fill the picture of the current frame error coordinate point, and all decoding work of the current frame will be ended.

[0132] See Figure 9, the bitstream security range detection module 11 includes a byte-aligned bitstream pointer calculation unit, a bit-level bitstream pointer calculation unit, and a bitstream out-of-bounds judgment unit. The byte-aligned bitstream pointer calculation unit and the bit-level bitstream pointer calculation unit are connected in parallel at the front end of the bitstream out-of-bounds judgment unit. When the video bitstream enters the bitstream security range detection module, the byte-aligned bitstream pointer calculation unit and the bit-level bitstream calculation unit respectively calculate the length of the current decoding corresponding bitstream pointer. The bitstream out-of-bounds judgment unit judges whether the current bitstream pointer length is out of bounds. If it is out of bounds, it outputs a bitstream out-of-bounds flag to the video hardware decoder. For the above two judgment types, the current video hardware decoder uses them simultaneously to play a dual-insurance role, ensuring that the current bitstream parsing is not out of bounds through two detection mechanisms.

[0133] See Figure 10 , the extreme value detection module 12 includes a syntax element judgment unit and a syntax element value out-of-bounds detection unit. When the video bitstream enters the extreme value detection module, after the syntax element judgment unit parses the bitstream into different syntax elements, the syntax element value out-of-bounds detection unit detects whether its value is out of bounds. If there is an out-of-bounds exception, it outputs an out-of-bounds flag to the video hardware decoder.

[0134] See Figure 11 , the extreme value detection module 12 may further include a leading zero count detection unit and, in parallel at its backend, an unsigned exponential Golomb leading zero exception detection unit (glomb_ud), a signed exponential Golomb leading zero exception detection unit (glomb_sd), a truncated exponential Golomb leading zero exception detection unit (glomb_td), and a mapped exponential Golomb leading zero exception detection unit (glomb_md). When the video bitstream enters the extreme value detection module, first, the leading zero count detection unit calculates the leading zero count of the video bitstream. Secondly, according to the Golomb exponent type, the corresponding standard value is selected through the corresponding exponential Golomb leading zero exception detection unit. Then, the maximum value is compared with the leading zero count of the video bitstream. If the leading zero count is greater than the corresponding standard value, it flags a decoding exception and outputs an arithmetic coding exception flag to the video hardware decoder.

[0135] Embodiment 3

[0136] Embodiment 3 of the present invention discloses a video hardware decoder circuit, which uses the bitstream parsing error detection and control system of the video hardware decoder disclosed in Embodiment 2 as an additional module in the circuit and executes the bitstream parsing error detection and control method of the video hardware decoder disclosed in Embodiment 1.

[0137] See Figure 12, the video hardware decoder circuit of this Embodiment 3 includes a bitstream parsing error detection and control system 10, a bitstream preprocessing module 20, an entropy decoding module 30, a video decoding module 40, a reconstructed frame module 50, and a driving end 60. Among them, the bitstream preprocessing module 20 is used to preprocess the video bitstream input by the network abstraction layer; the entropy decoding module 30 is used to perform entropy decoding on the video bitstream preprocessed by the bitstream preprocessing module; the video decoding module 40 is used to decode and filter the video bitstream processed by the entropy decoding module; the reconstructed frame module 50 is used to perform reconstructed frame processing on the video bitstream processed by the video decoding module; the driving end 60 is electrically connected to the bitstream security range detection module 11, the extreme value detection module 12, and the anomaly recognition and reset module 13 in the bitstream parsing error detection and control system 10 respectively, and is used to drive each module to work; the bitstream security range detection module 11 is electrically connected to the bitstream preprocessing module 20, and is used to perform bitstream security range detection on the obtained video bitstream to obtain a first detection result; the extreme value detection module 12 is electrically connected to the entropy decoding module 30, and is used to perform bitstream extreme value detection on the obtained video bitstream according to different syntax elements in the video bitstream parsed by the entropy decoding module 30 to obtain a second detection result.

[0138] Specifically, after the video hardware decoder circuit is powered on, configuration information is obtained from the driving end, that is, the bitstream length of the current frame decoded by software is obtained and saved to the error type register as the detection basis for the bitstream length by the bitstream security range detection module 11; the extreme values of different syntax elements are obtained and used as the range detection for different syntax elements by the extreme value detection module 12. There are three detection behaviors during the bitstream parsing process. First, the entropy decoded bitstream lengths are accumulated one by one. If the consumed bitstream length is greater than the total length, it is considered that the decoding is abnormal, otherwise the decoding is normal. Second, for the entropy decoding with an extreme value range, the decoded values will be judged according to the initial extreme value range. If it exceeds the range, it is abnormal, otherwise it is normal. Third, for the entropy decoding without an extreme value range, it will be judged according to the adopted coding attributes. If it exceeds the range, it is abnormal, otherwise it is normal. When the bitstream security range detection module 11 and the extreme value detection module 12 detect an anomaly in the video bitstream, the anomaly is processed and reset through the anomaly handling and reset module 13. If the system enables the error correction mechanism, the error type is identified and repaired according to the error type. When the error level is the zero level, the current error is ignored; when the error level is the first level, the current strip needs to be skipped; when the error level is the second level, the current frame needs to be skipped. If the system is not enabled, the currently detected anomaly is ignored.

[0139] Embodiment 4

[0140] Embodiment 4 of the present invention discloses a computer-readable storage medium, in which a computer program is stored. When it runs on a computer, it causes the computer to execute the bitstream parsing error detection and control method of the video hardware decoder disclosed in Embodiment 1.

[0141] Embodiment 5

[0142] Embodiment 5 of the present invention discloses a computer program product, which includes a computer program. When it runs on a computer, it causes the computer to execute the bitstream parsing error detection and control method of the video hardware decoder disclosed in Embodiment 1.

[0143] The video hardware decoder circuit, the bitstream parsing error detection and control method, and the system disclosed by the present invention enable the hardware decoder to have a wider error detection range for the video bitstream, a more flexible detection method, and a higher detection sensitivity. After error processing of the video bitstream, the decoded video image quality is better, and at the same time, the impact on the decoding performance of the hardware decoder is smaller.

[0144] It should be understood that the above description of the specific embodiments of the present invention is only for explaining the technical route and features of the present invention, and its purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. However, the present invention is not limited to the above specific embodiments. Any changes or modifications made within the scope of the claims of the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for detecting and controlling bitstream parsing errors in a video hardware decoder, characterized in that, The method includes: Performing bitstream security range detection on the acquired video bitstream to obtain a first detection result, which specifically includes: Obtaining the configuration information of the video bitstream from the driving end to obtain the bitstream length of the video bitstream included in the configuration information; Setting a bitstream pointer to record the current consumed length of the video bitstream; Judging whether the video bitstream has a bitstream out-of-bounds behavior by comparing the current consumed length with the bitstream length; If the judgment result is that it has not occurred, return to repeat the judgment; If the judgment result is that it has occurred, output a bitstream out-of-bounds flag as the first detection result; Performing bitstream extreme value detection on the acquired video bitstream to obtain a second detection result; According to the first detection result and the second detection result, perform anomaly detection on the process of bitstream parsing of the video bitstream, and perform anomaly processing or frame reset on the video bitstream according to the anomaly detection result.

2. The method according to claim 1, characterized in that The bitstream pointer includes a byte-aligned bitstream pointer and a bit-level bitstream pointer; The byte-aligned bitstream pointer determines whether the bitstream out-of-bounds behavior has occurred by judging whether the record of the byte counter that counts the byte consumption of the video bitstream is zero; if the record of the byte counter is zero and the length of the currently operated byte-aligned bitstream pointer exceeds the length of the previous frame bitstream pointer, the judgment result is that it has occurred; The bit-level bitstream pointer determines whether the bitstream out-of-bounds behavior has occurred by judging whether the current bitstream length calculated by the bitstream counter is greater than the current rated total bitstream length; if the current bitstream length is greater than the current rated total bitstream length, the judgment result is that it has occurred.

3. The method according to claim 1, wherein In performing bitstream extreme value detection on the acquired video bitstream to obtain a second detection result, it specifically includes: Obtaining the configuration information of the video bitstream from the driving end to obtain different syntax elements included in the configuration information; Performing bitstream parsing on each syntax element of the video bitstream to obtain the valid range of the corresponding value of the syntax element; According to the valid range of the corresponding value of each syntax element, judge whether the video bitstream has a bitstream out-of-bounds behavior by detecting whether the value of each syntax element is within the valid range; If the value of the syntax element is within the valid range, return to repeat the judgment; If the value of the syntax element is outside the valid range, output a value out-of-bounds flag as the second detection result.

4. The method according to claim 1, characterized in that, In performing bitstream extreme value detection on the acquired video bitstream to obtain a second detection result, it specifically includes: Obtaining the configuration information of the video bitstream from the driving end to obtain different syntax elements included in the configuration information; Performing bitstream parsing on each syntax element of the video bitstream to obtain the arithmetic coding corresponding to the syntax element; According to the arithmetic coding corresponding to each syntax element, judge whether the video bitstream has a bitstream out-of-bounds behavior by judging whether the number of leading zeros in the video bitstream is greater than the corresponding standard value; If the number of leading zeros in the bitstream is not greater than the corresponding standard value, return to repeat the judgment; If the number of leading zeros in the bitstream is greater than the corresponding standard value, output a leading zero anomaly flag as the second detection result.

5. The method according to claim 1, wherein During the process of performing bitstream parsing on the video bitstream according to the first detection result and the second detection result, and performing anomaly detection on the process and performing anomaly handling or frame reset on the video bitstream according to the anomaly detection result, it specifically includes: According to the first detection result and the second detection result, record the error information generated during the process of bitstream parsing of the video bitstream; Judge the error level recorded in the error information and perform anomaly handling on the video bitstream; When the error level is the zero level, ignore the error information and continue normal decoding; When the error level is the first level, the current frame is a single slice. End the decoding of the current slice, and according to the coordinate information of the current frame recorded in the error information, move the picture information from the neighboring frames of the current frame to fill the current frame, and start decoding the next slice; When the error level is the second level, the current frame is multiple slices. End the decoding of the current frame, and according to the coordinate information of the current frame recorded in the error information, move the picture information from the neighboring frames of the current frame to fill the current frame, and start decoding the next frame; Broadcast the information about the error found at the current parsing position in the video bitstream recorded in the error information to the subsequent modules, and perform reset after the subsequent modules parse the video bitstream one by one until the error point is found.

6. A bitstream parsing error detection and control system for a video hardware decoder, characterized in that, The system includes: a bitstream safety range detection module, an extreme value detection module, and an anomaly recognition and reset module. The bitstream safety range detection module and the extreme value detection module are respectively electrically connected to the anomaly recognition and reset module; The bitstream safety range detection module is used to perform bitstream safety range detection on the acquired video bitstream to obtain a first detection result, specifically including: obtaining the configuration information of the video bitstream from the driving end to obtain the bitstream length of the video bitstream included in the configuration information; setting a bitstream pointer to record the current consumed length of the video bitstream; by comparing the current consumed length with the bitstream length, judge whether the video bitstream has a bitstream out-of-bounds behavior; if the judgment result is not occurred, return for repeated judgment; if the judgment result is occurred, output a bitstream out-of-bounds flag as the first detection result; The extreme value detection module is used to perform bitstream extreme value detection on the acquired video bitstream to obtain a second detection result; The anomaly recognition and reset module is used to perform anomaly detection on the process of bitstream parsing of the video bitstream according to the first detection result and the second detection result, and perform anomaly handling or frame reset on the video bitstream according to the anomaly detection result.

7. A video hardware decoder circuit, characterized in that, It includes: A bitstream preprocessing module, which is used to preprocess the video bitstream input by the network abstraction layer; An entropy decoding module, which is used to perform entropy decoding on the video bitstream preprocessed by the bitstream preprocessing module; A video decoding module, which is used to decode and filter the video bitstream processed by the entropy decoding module; A reconstructed frame module, which is used to perform reconstructed frame processing on the video bitstream processed by the video decoding module; A driving end and the bitstream parsing error detection and control system as claimed in claim 6; The driving end is respectively electrically connected to the bitstream safety range detection module, the extreme value detection module, and the anomaly recognition and reset module, and is used to drive each module to work; The bitstream security range detection module is electrically connected to the bitstream preprocessing module, and is configured to perform bitstream security range detection on the obtained video bitstream to obtain a first detection result; The extreme value detection module is electrically connected to the entropy decoding module, and is configured to perform bitstream extreme value detection on the obtained video bitstream according to different syntax elements in the video bitstream parsed by the entropy decoding module to obtain a second detection result.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which when running on a computer, causes the computer to execute the bitstream parsing error detection control method of the video hardware decoder according to any one of claims 1 to 5.

9. A computer program product, characterized in that, The computer program product includes a computer program, which when running on a computer, causes the computer to execute the bitstream parsing error detection control method of the video hardware decoder according to any one of claims 1 to 5.

Images