Method and system for video bitstream error handling

Abstract

An error handling method and apparatus for a video bitstream for determining an error type and a frame type to select a recovery technique. The method receives a video bitstream, detects errors during decoding, and classifies the errors as syntax parsing errors, reference picture data missing or corruption, and decoding errors. The frame type is divided into a reference frame, a non-reference frame or a key frame. Based on these classifications, the method applies appropriate error handling: for partial corruption, generating replacement reference data; for complete reference loss, discarding the frame or the frame group; performing row-level hiding on the reference frame to limit error propagation; performing stripe-level hiding on the non-reference frame to improve visual quality; analyzing key frame errors to ignore slight edge errors; for serious damage, discarding the frame group; for unrecoverable syntax errors, frames are dropped.

Description

[0001] Cross-referencing

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 693,269, filed September 11, 2024, the contents of which are incorporated herein by reference. [Technical Field]

[0003] This application relates to error handling when decoding video bitstreams. [Background Technology]

[0004] Digital video compression standards such as H.264 / AVC and H.265 / HEVC achieve high compression efficiency through inter-frame prediction, where frames are reconstructed by referencing previously decoded frames. This time-dependent structure makes compressed video bitstreams highly sensitive to transmission errors, as corruption in the reference frame can propagate spatially and temporally, causing visual artifacts to persist across multiple subsequent frames. When video data is transmitted over error-prone channels such as wireless networks or broadcast systems, portions of the bitstream may be corrupted due to bit errors and packet loss, thus requiring robust error handling mechanisms in the decoder.

[0005] Traditional video decoders employ two main error handling strategies: error concealment and frame dropping. Error concealment attempts to reconstruct lost or corrupted data using spatial or temporal interpolation, preserving visual continuity but potentially introducing artifacts that propagate to dependent frames. Frame dropping completely discards corrupted frames to prevent error propagation, but this results in temporal discontinuities and a degraded user experience. Traditional decoders typically implement a single, fixed error handling strategy without considering specific error characteristics or frame types, failing to account for the different impacts of different error types and the varying roles of keyframes, reference frames, and non-reference frames within the video sequence. This inflexible approach leads to suboptimal video quality and user experience in error-prone transmission environments. [Summary of the Invention]

[0006] One embodiment provides a method for bitstream error handling in a video decoder, comprising: receiving a bitstream including data associated with a frame sequence; detecting an error while decoding a frame in the sequence from the bitstream; determining an error type associated with the error; determining a frame type of the frame; and applying an error handling method based on the error type and the frame type.

[0007] In some respects, when the error type includes lost or corrupted reference image data, the method generates alternative reference image data for partially corrupted data, or discards frames based on frame type when the reference data is completely unavailable.

[0008] In some respects, when the error type includes decoding errors, the method performs line-level hiding on reference frames to limit error propagation, performs slice-level hiding on non-reference frames to maintain visual quality, or analyzes keyframe errors to ignore small edge errors or discard frames for larger corrupted frames.

[0009] In some respects, when the error type includes a parsing error, the method discards the frame if the error is unrecoverable.

[0010] In some respects, the method outputs a decoded frame when the applied error handling method does not cause the frame to be discarded.

[0011] One embodiment provides an apparatus for bitstream error handling in a video decoder, including a memory configured to store executable instructions and a processor coupled to the memory, configured to execute the instructions to perform the error handling method described above.

[0012] To achieve the above and related objectives, certain embodiments include the following features, which are described in detail and specifically pointed out in the claims. The following description and drawings illustrate certain illustrative aspects of the embodiments. However, these aspects only show a few of several ways in which the principles of the embodiments may be adopted, and this disclosure is intended to include all such aspects and their equivalents. These and other objectives of the invention will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments, which are illustrated in various figures. [Attached Image Description]

[0013] Figure 1A and Figure 1B A block diagram of an exemplary video encoding system is shown.

[0014] Figure 2 A flowchart of a hybrid error handling method according to one embodiment is shown, illustrating the decision-making process.

[0015] Figure 3 A method for handling mixed errors in video bitstream processing according to one embodiment is shown.

Detailed Implementation Methods

[0016] Digital video compression standards such as H.264 / AVC (Advanced Video Coding) and H.265 / HEVC (High-Efficiency Video Coding) achieve high compression efficiency by employing sophisticated prediction techniques. To maximize compression efficiency, modern video codecs utilize inter-frame prediction, where frames are encoded with reference to previously decoded frames. This temporal prediction creates dependencies between frames, with the current frame potentially referencing data from one or more previously decoded reference frames. Furthermore, video frames are typically organized into Groups of Pictures (GOPs), where keyframes (I-frames) provide periodic refresh points, while predicted frames (P-frames and B-frames) are reconstructed based on reference frames.

[0017] This inter-frame dependency structure makes compressed video bitstreams highly sensitive to transmission errors and data corruption. When video data is transmitted over error-prone channels such as wireless networks, the Internet, or broadcast systems, bit errors, packet loss, and transmission delays can corrupt parts of the video bitstream. Due to the predictive dependencies inherent in video compression, a single bit error in a reference frame can propagate spatially and temporally, causing visual artifacts to persist across multiple subsequent frames.

[0018] Traditional video decoders employ various error handling strategies to mitigate the impact of bitstream corruption. These methods are generally categorized into two types: error concealment and frame dropping.

[0019] Error concealment techniques attempt to reconstruct lost or corrupted video data by estimating the missing information from the surrounding spatial or temporal context. Spatial error concealment uses pixel data from adjacent regions within the same frame, while temporal error concealment utilizes data from previously decoded frames. When a corrupted region is detected, the decoder uses interpolation, duplication, or prediction techniques to fill in the missing data. While error concealment can maintain visual continuity and provide a complete video sequence, it can introduce visual artifacts, which may propagate and accumulate over time if future frames reference the hidden region.

[0020] In contrast, frame dropping techniques completely discard corrupted frames to prevent error propagation. When corruption is detected in a frame, the decoder simply omits that frame from the decoding sequence, effectively creating a time gap. While this method prevents visual artifacts from propagating to subsequent frames, it can lead to motion discontinuities, time freezes, and a degraded user experience, especially when the interval between discarding keyframes or clean reference frames is large.

[0021] Traditional video decoders typically implement a single, fixed error handling strategy, without considering the specific characteristics of the detected errors or the type of frame being processed. For example, a decoder might be configured to always perform error hiding, always discard corrupted frames, or use simple threshold-based decisions. This inflexible approach fails to consider the different impacts of different types of errors on video quality and the different roles that various frame types play in a video sequence.

[0022] More specifically, the prior art does not adequately consider that the optimal error handling strategy depends on several factors, including: (1) the type of error detected (e.g., parsing error, missing reference data, or decoding error), (2) the type of frame affected (e.g., key frame, reference frame, or non-reference frame), (3) the extent and location of intra-frame corruption, and (4) the potential impact on subsequent frames that may rely on the current frame for prediction.

[0023] For reference frames, aggressive error hiding might be visually acceptable for the current frame, but could cause severe artifacts in subsequent frames that reference the hidden region. Conversely, for non-reference frames, discarding the frame creates unnecessary time gaps because no future frames rely on the corrupted data. For keyframes, the decision may become more critical, as keyframe corruption can affect the entire GOP, but small errors at frame edges may be visually imperceptible and can be safely ignored.

[0024] Traditional methods fail to adaptively select error handling strategies based on these contextual factors, resulting in poor video quality, inefficient bandwidth utilization, and a poor user experience in error-prone transmission environments.

[0025] This description provides an adaptive video decoder error handling method that selects a recovery strategy based on the detected error type and the characteristics of the affected frame. Unlike conventional methods that apply a single error handling method without considering the context, the disclosed hybrid method involves analyzing multiple factors to determine the appropriate response to bitstream corruption.

[0026] Different types of errors require different handling methods, and the optimal strategy depends on the role of the affected frame in the video sequence. For example, corruption in a reference frame used to predict future frames requires a different approach than corruption in a non-reference frame that only affects the currently displayed frame. Similarly, parsing errors that prevent correct bitstream interpretation require different handling than decoding errors that only affect a specific region within a frame.

[0027] The hybrid error handling method disclosed in this paper classifies detected errors into different types, including parsing errors, missing or corrupted reference image data, and decoding errors. The method also includes determining the frame type of the affected frames, distinguishing between reference frames, non-reference frames, and keyframes. Each type of frame plays a different role in the video decoding process and has a different impact on error propagation. Based on the analysis of error types and frame types, one of several error handling strategies can be selectively applied, including: generating alternative reference image data for partially corrupted reference information; discarding a single frame or an entire group of pictures (GOP) when the reference data is completely unavailable; performing line-level error hiding on reference frames to limit propagation to dependent frames; performing slice-level error hiding on non-reference frames to maintain visual quality; analyzing the error location and size within keyframes to determine whether to ignore small edge errors or discard the entire GOP to address larger corruptions; and discarding frames when parsing errors are unrecoverable.

[0028] This adaptive approach strikes a balance between visual quality, temporal continuity, and error propagation prevention, delivering improved video playback quality in error-prone transmission environments. It is applicable to wireless video streaming, digital television broadcasting, and real-time video communications, where transmission errors occur and robust error handling is required to maintain an acceptable user experience.

[0029] The following detailed description illustrates embodiments and implementations of the hybrid error handling method, including flowcharts illustrating the decision-making process, examples of different error scenarios, and how the method adjusts its response based on a combination of error types and frame types encountered during video decoding.

[0030] Figure 1A and Figure 1B An exemplary adaptive inter-frame / intra-frame video coding system for performing the video coding techniques described herein is demonstrated. The architecture of encoder 100A is as follows: Figure 1AAs shown. For the intra-frame prediction module 110, the prediction data is derived from previously encoded video data in the current image. For the inter-frame prediction module 112, motion estimation (ME) is performed at the encoder, and motion compensation (MC) is performed based on the result of the motion estimation to provide prediction data derived from other images and motion data. A selection switch 114 selects between the intra-frame prediction module 110 and the inter-frame prediction module 112. The selected prediction data is provided to the adder 116 to form the prediction error, also known as the residual. These residuals are then processed by the transform module (T) 118, followed by the quantization module (Q) 120. The transformed and quantized residuals are then encoded by the entropy encoder 122 to be included in the video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with additional information, such as motion and coding mode information associated with intra-frame and inter-frame prediction, and other information such as parameters associated with the loop filters applied to the base image region. Figure 1A As shown, additional information associated with the intra-frame prediction module 110, the inter-frame prediction module 112, and the in-loop filter (ILPF) 130 is provided to the entropy encoder 122.

[0031] When using inter-frame prediction mode, the reference image or multiple reference images must also be reconstructed at the encoder. Therefore, the transformed and quantized residuals are processed by the inverse quantization module (IQ) 124 and the inverse transform module (IT) 126 to recover the residuals. These residuals are then added back to the prediction data 136 in the reconstruction module (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in the reference picture buffer 134 and used to predict other frames.

[0032] like Figure 1AAs shown, the input video data undergoes a series of encoding operations in the encoding system. Due to these encoding operations, the reconstructed video data generated from REC 128 may suffer various impairments. To improve video quality, a loop filter 130 is applied before storing the reconstructed video data in the reference image buffer 134. The loop filter 130 may include various filtering operations, such as a deblocking filter (DF), a sample adaptive offset (SAO), and an adaptive loop filter (ALF). Since the decoder needs to apply the same filtering operations, the loop filter information can be incorporated into the bitstream. Therefore, the loop filter information is provided to the entropy encoder 122 to be incorporated into the encoded bitstream. Figure 1A As shown, the loop filter 130 processes the reconstructed video data before the filtered samples are stored in the reference image buffer 134. Figure 1A The encoding system architecture shown represents an exemplary structure of a typical video encoder that can be implemented in various video coding standards, such as High Efficiency Video Coding (HEVC), VP8, VP9, ​​Advanced Video Coding (H.264), or Versatile Video Coding (VVC).

[0033] like Figure 1B As shown, decoder 100B shares several functional similarities with encoder, but operates in a complementary manner to reconstruct the original video data. Unlike encoder, which requires transform module 118 and quantization module 120 for compression, decoder can implement inverse quantization module 124 and inverse transform module 126 to reverse the compression process. In decoder, entropy decoder 140 replaces encoder's entropy encoder 122. Entropy decoder 140 is responsible for interpreting the received video bitstream, extracting the quantized transform coefficients and necessary coding information, including ILPF information, intra-frame prediction information, and inter-frame prediction information.

[0034] The decoder's intra-prediction module 150 can be more efficient than its encoder counterpart because it does not need to perform a computationally intensive mode search process. Instead, it directly generates the intra-prediction signal by applying intra-prediction information received from the entropy decoder 140. This information precisely specifies which prediction mode to use, thereby reducing the extensive mode evaluation process required at the encoder end.

[0035] Similarly, the inter-frame prediction process of the decoder can be simpler than that of the encoder. The motion compensation module (MC) 152 needs to perform motion compensation operations based on the motion vectors and reference image information received by the entropy decoder 140. This is simpler than the inter-frame prediction process of the encoder, which needs to perform motion estimation simultaneously to find the optimal motion vectors and motion compensation to generate the prediction signal. The decoder can apply the received motion information to reconstruct the inter-frame prediction block and access the necessary reference image data from its reference image buffer 134.

[0036] Figure 2 A flowchart of method 200 is shown, illustrating the decision-making process for hybrid error handling in a decoder (e.g., decoder 100B) for video bitstream processing according to one embodiment. The flowchart illustrates how the method determines the error type associated with a detected error, determines the frame type of the frame being processed, and applies error handling methods based on the characteristics of the error type and frame type. Error types include parsing errors, reference frame errors, and decoding errors. Parsing errors may occur during bitstream header interpretation and structure verification. Reference image data errors may involve missing or corrupted images used for inter-frame prediction, while decoding errors cover problems occurring during coding tree unit (CTU) processing and pixel data reconstruction. Errors occurring during CTU processing and pixel data reconstruction in the slice data portion are classified as decoding errors. Furthermore, if the slice data contains invalid, unexpected, or out-of-range values, or its structure does not conform to the expected syntax, these are also counted as decoding errors. Frame types include the type of the current frame and the type of the reference frame for the current frame. Keyframes provide independent decoding points. Reference frames can be used to predict future frames, while non-reference frames can be used for the current display without affecting the decoding of subsequent frames.

[0037] The process begins in step S202 (Start) and immediately proceeds to step S204, in which syntax parsing error detection is performed while decoding a frame from the sequence in the bitstream. In step S204, the method determines whether a syntax parsing error has occurred while parsing NAL (Network Abstraction Layer, or NAL) unit headers, slice headers, or other syntax elements.

[0038] When a syntax parsing error is detected in step S204, the method proceeds to step S206 to assess the severity of the syntax parsing error. In step S206, the method determines whether the syntax parsing error is irrecoverable, meaning that the bitstream structure has been severely damaged to the point that continued parsing would be unreliable or impossible.

[0039] If the syntax parsing error in step S206 is unrecoverable, the method proceeds to step S208 to discard the frame. This represents an error handling method in which corrupted frames are completely discarded to prevent the propagation of syntax-level corruption.

[0040] If the syntax parsing error in step S206 is recoverable, the method proceeds to step S210 to bypass the syntax error. This error handling method allows the decoder to continue processing despite minor syntax irregularities without fundamentally disrupting the bitstream structure.

[0041] After the syntax error assessment, or if no syntax parsing error is detected in step S204, the method proceeds to step S212 to assess the availability of the Reference Picture List (RPL) data. In step S212, it is determined whether the reference picture data required for the current frame is fully available in the decoded picture buffer.

[0042] In this step, the method determines the error type, including lost or corrupted RPL data. This step examines the RPL data carried in the bitstream of the current frame. The RPL specifies which previously decoded images are needed from the Decoded Picture Buffer (DPB) to predict the current frame. The decoder verifies that all reference images listed in the RPL actually exist and are accessible in the DPB.

[0043] When it is determined in step S212 that the RPL data is not fully available, the method proceeds to step S214 to assess partial availability. In step S214, it is determined whether the RPL data is partially corrupted or completely lost. This assessment distinguishes between a situation where some reference images in the RPL are lost or corrupted while others remain accessible, and a situation where all required reference images in the DPB are lost.

[0044] If the RPL data is partially damaged or lost in step S214, the method proceeds to step S216 to generate alternative reference image data for the damaged or lost portion.

[0045] When it is determined in step S214 that the reference image data is completely unavailable, the method proceeds to step S218 to determine the frame type. In step S218, the frame type is determined, specifically, it is evaluated whether the current frame is a reference frame or a non-reference frame.

[0046] If the frame type in step S218 is a reference frame, the method proceeds to step S220 to discard a group of frames associated with that reference frame. This error handling method removes the entire group of pictures (GOP) to prevent the propagation of temporal errors due to a lack of reference dependencies.

[0047] If the frame type in step S218 is a non-reference frame, the method proceeds to step S222 to discard the single frame. Since non-reference frames are not used as prediction sources for subsequent frames, the removal of a single frame does not cause dependency chain failure.

[0048] When the reference image data is fully available in step S212, the method proceeds to step S224 for decoding error detection. In step S224, it is determined whether a decoding error occurred during macroblock processing, transform coefficient decoding, or motion vector reconstruction.

[0049] If no decoding error is detected in step S224, the method proceeds directly to step S236, and the error handling process ends after the frame is successfully decoded.

[0050] When a decoding error is detected in step S224, the method proceeds to step S225 to classify the frame type. In step S225, it is determined whether the frame type is a reference frame, which can be used for prediction of subsequent frames.

[0051] If the frame type in step S225 is a non-reference frame, the method proceeds to step S228 to perform piece-level hiding on one or more erroneous regions in the non-reference frame. This error handling method applies comprehensive spatial hiding to the entire affected region to maintain visual integrity without considering temporal error propagation.

[0052] If the frame type in step S225 is a reference frame, the method proceeds to step S226 for keyframe classification. In step S226, it is determined whether the frame type of the current frame is a keyframe, which serves as an independent decoding point and typically affects the entire GOP structure.

[0053] If the frame type in step S226 is a keyframe, the method proceeds to step S230 to perform error region analysis. In step S230, the error location and error region size within the keyframe are analyzed, specifically assessing whether the size of the affected region is below a predetermined threshold and whether the error location is at the edge of the frame.

[0054] If the size of the affected area of ​​the error within the keyframe in step S230 is less than a predetermined threshold and the error is located at the edge of the keyframe, the method proceeds to step S234 to ignore the error. This error handling method recognizes that small errors at the frame edges have minimal visual impact and can be safely ignored.

[0055] When the error criteria of step S230 are not met (the affected area exceeds the threshold or is not at the frame edge), the method proceeds to step S232 to discard a set of frames associated with the keyframe. This error handling method can prevent significant visual artifacts from propagating throughout the GOP structure.

[0056] If the frame type is not a keyframe, the method proceeds to step S227 to perform row-level hiding of one or more error regions in the reference frame. This error handling method applies minimal spatial hiding to limit error propagation to dependent frames that will use this frame for prediction.

[0057] It should be noted that the order of steps S225 and S226 can be interchanged for flexible implementation.

[0058] All error handling paths converge in step S236, at which point the hybrid error handling method ends. During various error handling processes, when the applied error handling method does not result in a dropped frame, the decoded frame is output for display or further processing. This output occurs after the successful application of error handling techniques, such as alternative reference image data generation, line-level hiding, slice-level hiding, or error bypass operations. Conditional outputs provide information that successfully processed or sufficiently hidden frames are passed to subsequent stages of the video decoding pipeline. Damaged frames identified as discardable can be excluded from the output stream to maintain overall video quality and prevent error propagation.

[0059] This flowchart illustrates the selection of an adaptive error handling method based on a system evaluation of error type characteristics (syntax parsing errors, missing or corrupted reference image data, or decoding errors) and frame type attributes (keyframe, reference frame, or non-reference frame). The exemplary technical implementation shown in Method 200 provides optimized processing for each combination of error type and frame type, balancing visual quality preservation, temporal continuity maintenance, and error propagation prevention, depending on the specific characteristics of the detected corruption and the role of the affected frame in the video sequence structure.

[0060] Figure 3 A hybrid error handling method 300 for video bitstream processing is illustrated, applicable to one embodiment of a decoder (e.g., decoder 100B). Method 300 represents a method that includes... Figure 2 This paper presents a high-level procedural flow detailing the decision-making process, providing a system-adaptive error recovery method based on contextual analysis of error characteristics and frame attributes. Method 300 includes the following steps:

[0061] S302: Receive a bit stream including data associated with a frame sequence;

[0062] S304: An error was detected while decoding frames in this sequence from the bitstream;

[0063] S306: Determine the error type associated with this error;

[0064] S308: Determine the frame type of this frame; and

[0065] S310: Apply the error handling method based on the error type and the frame type.

[0066] In step S302, a bitstream is received, which includes data associated with a frame sequence. This step involves acquiring and initially buffering compressed video data encoded according to a video compression standard (e.g., H.264 / AVC or H.265 / HEVC). The bitstream contains Network Abstraction Layer (NAL) units that encapsulate various types of video data, including parameter sets, title sequences, and compressed frame data. The frame sequence represents video content arranged chronologically and may include different frame types, such as intra-coded frames (I-frames), predictive frames (P-frames), and bidirectional predictive frames (B-frames). The received bitstream may originate from various sources, including network transmissions, storage media, or broadcast channels, and may contain data segments that have been transmitted, corrupted, or lost during encoding, transmission, or storage.

[0067] In step S304, error detection is performed while decoding a frame from the sequence from the bitstream. This step covers multiple levels of error detection occurring throughout the decoding pipeline. Initial error detection involves parsing and verifying syntax elements in the NAL unit header, slice header, and parameter set structure to identify malformed or invalid syntax structures. Subsequent error detection occurs within the decoding process itself, where errors may be identified in transform coefficient decoding, motion vector reconstruction, reference image management, or macroblock processing. Error detection mechanisms may use cyclic redundancy checks (CRC), parity bits, syntax verification rules, range checks of decoding parameters, and consistency verification between interdependent syntax elements. Errors detected in this step may range from minor syntax irregularities that can be bypassed to severe corruption that prevents meaningful frame reconstruction.

[0068] In step S306, the error type associated with the detected error is determined. This classification process categorizes errors into different types based on their characteristics, severity, and impact on the decoding process. Error types include syntax parsing errors, missing or corrupted Reference Picture List (RPL) data, and decoding errors occurring during pixel data reconstruction. Syntax parsing errors may include invalid parameter values, malformed headers, missing start codes, or violations of syntax constraints. RPL data errors may include previously decoded frames (required in inter-frame prediction) being unavailable, corrupted, or partially corrupted. Decoding errors may include failures in transform coefficient processing, motion compensation, intra-frame prediction, or other reconstruction operations.

[0069] In step S308, the frame type of the currently processed frame is determined. This step involves analyzing the characteristics of the frame to classify it based on its role in the video sequence structure and its relationship to other frames. Frame type determination includes identifying whether the current frame is a keyframe (I-frame), i.e., a frame that provides an independent decoding reference point; a reference frame, i.e., a frame used for inter-frame prediction of subsequent frames; or a non-reference frame, i.e., a frame used only for the current display and not affecting the decoding of future frames. Furthermore, frame type analysis may distinguish different prediction structures, such as P-frames that reference previous frames and B-frames that may reference both previous and future frames. Different frame types have different effects on temporal error propagation; reference frames may affect multiple subsequent frames, while the effect of non-reference frames is limited to a local range.

[0070] In step S310, an error handling method is applied based on the error type determined in step S306 and the frame type determined in step S308. An error recovery strategy is adaptively selected by comprehensively considering error characteristics and frame attributes. The application of error handling methods includes frame dropping, error hiding, reference image data generation, and error bypass operations. For parsing errors, frame dropping can be applied to unrecoverable errors, while syntax bypass can be applied to recoverable errors. For RPL data problems, alternative reference image data can be generated for partially corrupted cases, or frames or frame groups can be dropped for cases where the reference is completely lost. For decoding errors, different hiding granularities can be applied according to the frame type; for example, row-level hiding can be used for reference frames to minimize propagation effects, while slice-level hiding can be used for non-reference frames to maximize visual quality.

[0071] After applying the error handling method in step S310, the method may include an additional step of outputting a decoded frame if the applied error handling method does not result in a dropped frame. The conditional output step ensures that frames successfully handled, adequately hidden, or effectively recovered through the hybrid error handling method are forwarded to the display pipeline or subsequent processing stages. Frames that have been handled through alternative reference image data generation, line-level hiding, slice-level hiding, or successful error bypass operations are considered suitable for output, while frames marked as dropped due to unrecoverable syntax errors, complete loss of reference image data, or severe keyframe corruption can be excluded from the output stream. The adaptability of method 300 provides context-based error recovery, which outperforms traditional single-strategy methods.

[0072] The terminology used in the various embodiments described herein is intended to describe particular embodiments and should not be construed as limiting. In this specification and the appended claims, the singular forms “a,” “an,” and “a” also include the plural forms unless the context clearly indicates otherwise. It should be understood that the term “and / or” as used herein is intended to cover any and all possible combinations of one or more of the associated listed items. Furthermore, the terms “comprising,” “including,” and / or “comprises” as used herein indicate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0073] Unless otherwise specified, the term "some" means one or more. Various combinations of "at least one" or "one or more" followed by a list should be interpreted as including any combination of the listed items, including single items and multiple items. In the context of this specification, the terms "coupled," "connected," "in connection," "electrically connected," and similar expressions are used interchangeably to broadly indicate the state of an electrical or electronic connection. Furthermore, when one entity electrically transmits and / or receives information signals to another entity, regardless of the signal type or transmission medium, it is considered to be "communicating" with the other entity.

[0074] The ordinal designations (such as "first", "second", etc.) used in the specification and claims are used to distinguish multiple instances of elements with similar names. These designations do not imply any inherent order, priority, or chronological order, but are only used to uniquely identify and distinguish different instances of elements. The directional terms used in the embodiments (such as up, down, left, right, upper side, lower side, front, or rear) refer only to the directions in the drawings and are for illustrative purposes only.

[0075] The terms used in this specification and the appended claims, such as “substantially,” “about,” “usually,” “essentially,” “almost,” “about,” and similar expressions, are used to account for variations in precision, manufacturing tolerances, measurement accuracy, environmental conditions, and inherent material properties. These variations may range from ±20% in a wider range of applications to ±10%, ±5%, ±3%, ±2%, ±1%, or ±0.5% in more precise implementations. The specific degree of variation covered by these terms is determined by the nature of the described components, relationships, or parameters and by the understanding of one of ordinary skill in the art.

[0076] Various exemplary components, logic, logic blocks, modules, circuits, operations, and algorithmic processes related to the embodiments disclosed herein can be implemented as electronic hardware, firmware, software, or a combination thereof. The interchangeability of hardware, firmware, and software depends on the specific application and design constraints of the overall system. Hardware and data processing devices used to implement the various components described herein may include one or more of the following: general-purpose single-chip or multi-chip processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any suitable combination thereof.

[0077] A general-purpose processor may include a microprocessor, or alternatively, any conventional processor, controller, microcontroller, or state machine. In some implementations, the processor may be implemented through a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other suitable configuration. In some embodiments, a particular process, operation, or method may be performed by circuitry designed specifically for a particular function, optimized to improve performance, efficiency, or other relevant metrics.

[0078] In some respects, the subject matter described herein can be implemented as software. The various functions of the disclosed components, or the steps of the methods, operations, processes, or algorithms described herein, can be implemented by one or more modules in one or more computer programs. These computer programs may include non-transitory processor-executable or computer-executable instructions encoded on one or more tangible processor-readable or computer-readable storage media. These storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any medium capable of storing program code.

[0079] Various modifications to the embodiments described herein will likely be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit or scope of this disclosure. Therefore, the claims are not intended to limit the embodiments shown herein, but are to be given the broadest scope consistent with this disclosure. Various features described herein in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0080] The operations depicted in a specific order in the figures should not be construed as requiring strict adherence to that order in practice, nor should they imply that all shown operations must be performed. Additional, undepicted operations may be added at different points in the depicted sequence; these operations may occur before, after, simultaneously with, or between the shown operations. The various figures and component diagrams provided are for illustrative purposes only and are not drawn to scale, intended to facilitate understanding of the described embodiments, and not to limit the scope of the invention to the specific arrangements depicted.

[0081] Although the invention has been described in conjunction with certain embodiments, those skilled in the art will understand that various modifications and adaptations can be made without departing from the scope of the invention. The specific embodiments presented are intended to illustrate the invention and not to limit its application or construction. Those skilled in the art will readily observe that many modifications and alterations can be made to the apparatus and methods while retaining the teachings of the invention. Therefore, the foregoing disclosure should be interpreted only by the limits of the appended claims.

Claims

1. A method for bitstream error handling in a video decoder, comprising: Receive a bitstream that includes data associated with a frame sequence; An error was detected while decoding frames in the sequence from the bitstream; Determine the error type associated with this error; Determine the frame type of this frame; as well as Apply the error handling method based on the error type and the frame type.

2. The method as described in claim 1, characterized in that, This error type includes missing or corrupted reference image data, and the applicable error handling methods include: In response to the determination that the reference image data is partially damaged, alternative reference image data is generated for the damaged portion.

3. The method as described in claim 1, characterized in that, This error type includes missing or corrupted reference image data, and the applicable error handling methods include: In response to determining that the reference image data is completely unavailable and that the frame type includes non-reference frames, the frame is discarded; or In response to determining that the reference image data is completely unavailable and that the frame type includes a reference frame, a set of frames associated with the reference frame is discarded.

4. The method as described in claim 1, characterized in that, The error type includes decoding errors, the frame type includes reference frames, and the error handling method includes: Perform line-level hiding on one or more error regions in the reference frame.

5. The method as described in claim 1, characterized in that, The error type includes decoding errors, the frame type includes non-reference frames, and the error handling method includes: Perform slice-level hiding on one or more erroneous regions in the non-reference frame.

6. The method as described in claim 1, characterized in that, The error type includes decoding errors, the frame type includes keyframes, and the error handling method applied includes: In response to determining that the affected area of ​​the error in the keyframe is larger than a predetermined threshold or that the location of the error in the keyframe is not at the edge of the frame, a set of frames associated with the keyframe is discarded.

7. The method as described in claim 1, characterized in that, The error type includes decoding errors, the frame type includes keyframes, and the error handling method applied includes: In response to determining that the affected area of ​​the error in the keyframe is smaller than a predetermined threshold and the error is located at the edge of the keyframe, the error is ignored.

8. The method as described in claim 1, characterized in that, This error type includes syntax parsing errors, and the methods for handling this error include: In response to the determination that the syntax parsing error is unrecoverable, the frame is discarded.

9. The method as described in claim 1, characterized in that, Further includes: In response to the application of the error handling method and the determination that the frame will not be discarded, the decoded frame is output.

10. An apparatus for bitstream error handling in a video decoder, comprising: Memory configured to store executable instructions; as well as The processor coupled to the memory is configured to execute the instruction to: Receive a bitstream that includes data associated with a frame sequence; An error was detected while decoding frames in the sequence from the bitstream; Determine the error type associated with this error; Determine the frame type of this frame; as well as Apply the error handling method based on the error type and the frame type.

11. The apparatus as claimed in claim 10, characterized in that, The processor is further configured as follows: In response to determining that the error type includes missing or corrupted reference image data and that the reference image data is partially corrupted, alternative reference image data is generated for the corrupted portion.

12. The apparatus as claimed in claim 10, characterized in that, The processor is further configured as follows: In response to determining that the error type includes missing or corrupted reference image data and that the reference image data is completely unavailable: If the frame type includes non-reference frames, discard the frame; or If the frame type includes a reference frame, discard the set of frames associated with that reference frame.

13. The apparatus as claimed in claim 10, characterized in that, The processor is further configured as follows: In response to determining that the error type includes a decoding error and the frame type includes a reference frame, line-level hiding is performed on one or more error regions.

14. The apparatus as claimed in claim 10, characterized in that, The processor is further configured as follows: In response to determining that the error type includes a decoding error and the frame type includes a non-reference frame, slice-level hiding is performed on one or more error regions.

15. The apparatus as claimed in claim 10, characterized in that, The processor is also configured to: when determining that the error type includes decoding errors and the frame type includes keyframes: if the size of the affected area of ​​the error within the keyframe is greater than a predetermined threshold or the location of the error within the keyframe is not at the edge of the frame, then discard a set of frames associated with the keyframe.

16. The apparatus as claimed in claim 10, characterized in that, The processor is also configured to ignore an error when it is determined that the error type includes a decoding error and the frame type includes a keyframe: if the size of the affected area of ​​the error within the keyframe is less than a predetermined threshold and the error is located at the edge of the keyframe.

17. The apparatus as claimed in claim 10, characterized in that, The processor is also configured to discard the frame when it is determined that the error type includes a parsing error and that the parsing error is unrecoverable.

18. The apparatus as claimed in claim 10, characterized in that, The processor is also configured to output the decoded frame if the frame is not dropped during error handling.

Images