Video decoding method and device, electronic equipment and computer readable storage medium

Abstract

The application discloses a video decoding method, device, electronic equipment and computer readable storage medium, comprising: receiving a video code stream to be played, calling a hardware decoder to decode, and when detecting a hardware decoding exception, determining an abnormal timestamp of an instant decoding refresh frame by reversely searching a video code stream sequence, reacquiring a corresponding video code stream based on the abnormal timestamp and executing hardware decoding again; when recovery fails, jumping to a next instant decoding refresh frame position to continue hardware decoding, and when hardware decoding recovery fails again, switching to a software decoder to decode. By constructing a decoding strategy combining multi-level hardware decoding recovery and software decoding, the video decoding recovery capability in an abnormal code stream scene can be improved while the low power consumption and high efficiency advantages of hardware decoding are maintained, thereby effectively reducing the freezing or black screen phenomenon in the video playing process.

Description

Technical Field

[0001] This application relates to the field of video playback technology, and in particular to a video decoding method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] In current video playback, when a hardware decoder encounters a decoding error, it is usually handled by simply reporting the error and terminating playback. This can easily lead to problems such as video playback stuttering, black screens, or direct exit from playback. It is difficult to improve playback stability in abnormal bitstream scenarios while maintaining the low power consumption advantage of hardware decoding. Therefore, there is an urgent need for a video decoding processing method that can effectively recover from decoding errors.

[0003] With the development of network technology and the continuous enrichment of video content formats, various video sources are constantly emerging, including live video, short videos, online movies and TV shows, and video resources in home network storage (NAS). This has led to increasingly complex video data transmission methods and playback scenarios. In practical applications, users typically receive the video stream through a player and call a decoder to decode it, thereby displaying the video image.

[0004] In existing video playback systems, decoders generally include hardware decoders and software decoders. Hardware decoders have advantages such as high decoding efficiency and low power consumption, and are therefore usually used preferentially during video playback. However, in cases of network transmission anomalies or incomplete bitstream data, hardware decoders may fail to decode. Current technologies typically only report the error information to the user and terminate playback, making it difficult to guarantee the continuity and stability of video playback when decoding anomalies occur. Summary of the Invention

[0005] This application provides a video decoding method, apparatus, electronic device, and computer-readable storage medium, which can improve the video decoding recovery capability in abnormal bitstream scenarios while maintaining the advantages of low power consumption and high efficiency of hardware decoding.

[0006] One technical solution adopted in this application is to provide a video decoding method, the video decoding method comprising: Receive the video stream to be played and call the hardware decoder to perform first-level hardware decoding on the video stream; When the first-level hardware decoding fails, the video stream is searched in reverse according to the video stream sequence to determine the abnormal timestamp of the instantaneous decoding refresh frame; The video stream is reacquired based on the abnormal timestamp, and the hardware decoder is invoked to perform secondary hardware decoding on the reacquired video stream. When the secondary hardware decoding fails, the process jumps to the next real-time decoding refresh frame and calls the hardware decoder to perform tertiary hardware decoding based on the next real-time decoding refresh frame. When the third-level hardware decoding fails, the software decoder is switched to perform software decoding on the subsequently received video stream.

[0007] Another technical solution adopted in this application is: providing a video decoding device, the video decoding device comprising: The first decoding module is used to receive the video stream to be played and call the hardware decoder to perform first-level hardware decoding on the video stream. An anomaly determination module is used to perform a reverse search on the video bitstream according to the video bitstream sequence when the first-level hardware decoding is abnormal, and determine the abnormal timestamp of the instantaneous decoding refresh frame. The second decoding module is used to reacquire the video stream based on the abnormal timestamp and call the hardware decoder to perform secondary hardware decoding on the reacquired video stream. The third decoding module is used to jump to the next real-time decoding refresh frame when the second-level hardware decoding fails, and to call the hardware decoder to perform third-level hardware decoding based on the next real-time decoding refresh frame; The software decoding module is used to switch to a software decoder to perform software decoding on the subsequently received video stream when the third-level hardware decoding fails.

[0008] Another technical solution adopted in this application is: providing an electronic device, the electronic device comprising: Memory, used to store executable program code; A processor is configured to call and run the executable program code from the memory, causing the electronic device to perform the video decoding method as described in any of the preceding descriptions.

[0009] Another technical solution adopted in this application is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the video decoding method described in any of the above claims.

[0010] This application provides a video decoding method, which includes: receiving a video stream to be played, calling a hardware decoder for decoding, and when a hardware decoding anomaly is detected, determining the abnormal timestamp of the real-time decoding refresh frame by reverse searching the video stream sequence, re-acquiring the corresponding video stream based on the abnormal timestamp, and performing hardware decoding again; if recovery fails, jumping to the next real-time decoding refresh frame position to continue hardware decoding; and if hardware decoding recovery fails again, switching to a software decoder for decoding. By constructing a decoding strategy that combines multi-level hardware decoding recovery with software decoding, the video decoding recovery capability under abnormal stream scenarios can be improved while maintaining the advantages of low power consumption and high efficiency of hardware decoding, thereby effectively reducing stuttering or black screen phenomena during video playback. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating the first embodiment of the video decoding method of this application; Figure 2 This is a flowchart illustrating the second embodiment of the video decoding method of this application; Figure 3 This is a schematic diagram of the hardware decoder switching and software decoder switching strategy of the video decoding method of this application; Figure 4 This is a schematic diagram of an exemplary implementation process of the video decoding method of this application, which switches between a hardware decoder and a software decoder. Figure 5 This is a decoding diagram illustrating the video decoding method of this application that jumps to the next real-time decoding refresh frame; Figure 6 This is an exemplary structural block diagram of an electronic device using the video decoding method of this application; Figure 7 This is an exemplary structural block diagram of a computer-readable storage medium for the video decoding method of this application. Detailed Implementation

[0012] The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0013] This embodiment can be implemented based on the Android system platform. For example, during the operation of the Android system, video data can be obtained from an online video server or live streaming server through network protocols (such as WebRTC, HTTP, etc.), or video files stored on the terminal device or network attached storage (NAS) can be read through the local file system. Regarding the player implementation, the MediaPlayer or NuPlayer components natively provided by the Android system can be used, or third-party open-source player frameworks (such as ExoPlayer, FFmpeg, or VLC) can be used to parse and play the video stream.

[0014] During video decoding, a hardware decoder can be used to decode the video stream, leveraging hardware acceleration to reduce CPU usage and power consumption. For example, when decoding H.264 or H.265 encoded video streams, the platform's hardware decoding capabilities are prioritized. However, in practical applications, due to potential irregularities in the video stream or data loss during network transmission, key parameter information may be missing, such as the Sequence Parameter Set (SPS) or Picture Parameter Set (PPS), causing anomalies in the hardware decoder during initialization or decoding.

[0015] Specifically, when the hardware decoder fails to obtain complete SPS or PPS information during the decoding process, it will lead to incorrect decoding parameter configuration, resulting in decoding failure. For example, if the hardware decoder cannot restore a valid decoding state after attempting data padding, it may return an error code (e.g., cb decret = -5), thus failing to decode and output a complete video frame, or having no valid data output in the decoding buffer, causing the decoding process to be interrupted. These situations will directly cause abnormal phenomena such as video playback stuttering and black screens.

[0016] In existing technologies, the aforementioned decoding anomaly issues are typically addressed by passively waiting for new Instantaneous Decoding Refresh (IDR) frames to appear in subsequent video streams to resume decoding. Since IDR frames contain complete key parameter information (such as SPS, PPS, etc.), the hardware decoder can re-establish the decoding state at these frames, thereby restoring normal playback. However, this recovery method relies on the stream's inherent structure and lacks an active recovery mechanism. When IDR frame intervals are long or the anomaly persists, it will lead to prolonged playback interruptions.

[0017] Based on this, a multi-level decoding recovery strategy is proposed to address the aforementioned hardware decoding anomaly scenarios. This strategy introduces an active recovery mechanism when a decoding anomaly occurs to improve the continuity and stability of video playback. For some embodiments, please refer to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the video decoding method of this application. It should be noted that if substantially the same result is achieved, the method of this application is not necessarily identical. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, the video decoding method includes: S101: Receive the video stream to be played and call the hardware decoder to perform first-level hardware decoding on the video stream.

[0018] In this context, a video stream refers to a continuous data stream formed after compression and encoding according to video coding standards. It consists of multiple video frames arranged in the encoded order and is provided to a player for parsing and decoding through online, local, or live streaming methods. Hardware decoders execute video decoding algorithms using hardware circuitry. Software decoders execute video decoding algorithms using software algorithms.

[0019] As an example, a user watches online videos through a video playback application on their terminal device. The video server continuously transmits video stream data compressed and encoded according to video encoding standards, such as H.264 or H.265, to the terminal device. After receiving the video stream to be played, the terminal device sends the video stream to the video decoding component through the player module, and first calls the hardware decoder in the terminal device for decoding processing, thereby decoding the compressed and encoded video stream into displayable video frame data and outputting it to the display module for playback.

[0020] S102, when the first-level hardware decoding is abnormal, the video bitstream is searched in reverse according to the video bitstream sequence to determine the abnormal timestamp of the instantaneous decoding refresh frame.

[0021] In this process, reverse search can be performed by following the encoding order of the video bitstream, starting from the bitstream position where the decoding anomaly occurred and searching backward along the time sequence until the most recent first instantaneous decoding refresh frame is located.

[0022] An Instantaneous Decoding Refresh Frame (IDR frame) is a keyframe type in video coding standards. When packet loss or data anomalies occur during transmission, an IDR frame can serve as a valid reference frame to restart the decoding process. An anomaly timestamp can be a video playback time identifier corresponding to when the hardware decoder detects a decoding anomaly, used to indicate the location of the decoding anomaly in the bitstream.

[0023] As an example, during video playback, if the hardware decoder encounters an error during decoding due to network packet loss, abnormal bitstream data, or missing video encoding information—for example, missing key parameter information or reference frame information—and thus cannot correctly decode the current video frame, the system detects a Level 1 hardware decoding anomaly. In this case, the player module discards the cached video bitstream data and, based on the current decoding position's location within the video bitstream sequence, performs a reverse search of the video bitstream in chronological order to find the nearest real-time decoding refresh frame. For instance, if an anomaly occurs in the bitstream data at the current playback position, the system can discard the cached video bitstream data, search backwards for the position of the previous real-time decoding refresh frame, and obtain the corresponding timestamp information to determine the abnormal timestamp of that real-time decoding refresh frame.

[0024] S103, based on the abnormal timestamp, reacquire the video stream and call the hardware decoder to perform secondary hardware decoding on the reacquired video stream.

[0025] As an example, after determining an abnormal timestamp, the player module can re-request the video stream data at the corresponding time position from the video data source based on the timestamp, or reread the video data segment at the corresponding time position from the local cache and input the re-acquired video stream into the hardware decoder for decoding processing again. For example, the player can rewind the playback position to the corresponding real-time decoding refresh frame position based on the abnormal timestamp and reload the real-time decoding refresh frame and its subsequent stream data to attempt to restore the normal decoding process.

[0026] S104, when the secondary hardware decoding fails, jump to the next real-time decoding refresh frame and call the hardware decoder to perform tertiary hardware decoding based on the next real-time decoding refresh frame.

[0027] Among them, Level 1 hardware decoding, Level 2 hardware decoding, and Level 3 hardware decoding represent a tiered recovery decoding process performed on the same hardware decoder according to a preset recovery strategy after a decoding anomaly is detected. Level 1 hardware decoding is the normal decoding stage, Level 2 hardware decoding is the hardware decoding process after reacquiring the bitstream based on the abnormal timestamp, and Level 3 hardware decoding is the process of jumping to the corresponding position of the next instant decoding refresh frame and performing hardware decoding again after Level 2 hardware decoding fails.

[0028] As an example, if the video stream structure still has anomalies or the keyframe information is incomplete, then the secondary hardware decoding will still fail to recover the decoding. In this case, the player module discards the cached video stream data, locates the video stream position corresponding to the next real-time decoding refresh frame based on the frame structure of the video stream, and uses this position as the new decoding starting point.

[0029] S105, when the third-level hardware decoding fails, switch to software decoder to perform software decoding on the subsequently received video stream.

[0030] As an example, if three levels of hardware decoding still cannot restore normal decoding, for example due to severe non-standard encoding or data corruption in the video stream, the player module can discard the cached video stream data and perform a decoding strategy switching operation, switching the current video decoding method from a hardware decoder to a software decoder. At this time, the player will input the subsequently received video streams to the software decoder for decoding processing.

[0031] This embodiment receives the video stream to be played, calls a hardware decoder for decoding, and when a hardware decoding anomaly is detected, it determines the abnormal timestamp of the real-time decoding refresh frame by reverse searching the video stream sequence. Based on the abnormal timestamp, it reacquires the corresponding video stream and performs hardware decoding again. If recovery fails, it jumps to the next real-time decoding refresh frame to continue hardware decoding. When hardware decoding recovery fails again, it switches to a software decoder for decoding. By constructing a decoding strategy that combines multi-level hardware decoding recovery and software decoding, it can improve the video decoding recovery capability in abnormal bitstream scenarios while maintaining the advantages of low power consumption and high efficiency of hardware decoding, thereby effectively reducing stuttering or black screen phenomena during video playback.

[0032] Please see Figure 2 , Figure 2 This is a flowchart illustrating a second embodiment of the video decoding method of this application. The method includes the following steps: S201: Receive the video stream to be played and call the hardware decoder to perform first-level hardware decoding on the video stream.

[0033] S202, when the first-level hardware decoding fails, the video frame position corresponding to the abnormal timestamp is used as the starting position to read the video frames in the video stream data backward.

[0034] As an example, when the player detects an abnormal decoding status during the execution of the first-level hardware decoding, it can perform backtracking analysis on the current video bitstream. For example, it can obtain the corresponding abnormal timestamp based on the abnormal information returned by the decoder, and use the video frame position corresponding to the abnormal timestamp as the starting position of the backtracking search. Then, it can read the video frame data in the video bitstream backward in the time order of the video bitstream, and backtrack the bitstream content before the current abnormal position frame by frame.

[0035] S203, within the preset backtracking window, the frame type of the read video frames is identified, and when the first instant decoding refresh frame is found, the corresponding abnormal timestamp is determined.

[0036] As an example, the player module can identify the frame type of the read video frames within a preset backtracking window. For instance, it can identify the keyframe type based on the frame type identifier information in the video encoding standard and determine whether the current frame is an instant-on-demand refresh frame during the backtracking search. When the first instant-on-demand refresh frame is found within the backtracking window, it can be determined that this instant-on-demand refresh frame can be used as a reference frame to restart decoding, and the timestamp corresponding to this instant-on-demand refresh frame can be determined as an abnormal timestamp for decoding recovery.

[0037] S204, Determine the target time position of the video stream based on the abnormal timestamp.

[0038] As an example, the player can determine the corresponding target time position of the video stream in the video timeline based on the abnormal timestamp. This target time position can indicate the point in time when the player needs to reacquire video data, enabling the decoder to re-establish the correct decoding reference relationship based on real-time decoding refresh frames.

[0039] S205, Generate video stream request parameters based on the target time position.

[0040] As an example, when the video data source is a network data source, the request parameters may include, but are not limited to, start timestamp parameters, segment index parameters, byte range parameters, and corresponding playlist positioning information. For instance, in a scenario based on the HTTP segmented transmission protocol, the corresponding segment number can be calculated based on the target time position, and a URL request address containing that segment number can be generated, or a request parameter with a time offset can be generated, so that the server returns video stream data containing the target time position.

[0041] When the video data source is a local storage data source, the request parameter can be file read offset information. For example, the corresponding file byte offset position can be obtained by mapping the timestamp with the real-time decoding refresh frame index table (such as the time index in the index file or container encapsulation), and a file read instruction can be generated so that the player can start reading the video stream from the data segment corresponding to the target time position.

[0042] Furthermore, to ensure decoding continuity, the request parameters can also include preload range information, that is, extending the preset time window forward or backward based on the target time position, which can obtain complete keyframe data, thereby ensuring that the decoder can resume decoding from the complete keyframe.

[0043] S206, obtain the video stream corresponding to the target time position through a network data source or a local storage data source, and call the hardware decoder to perform secondary hardware decoding on the re-acquired video stream.

[0044] As an example, the player obtains the video stream data corresponding to the target time position through a network data source or a local storage data source based on the generated video stream request parameters, and inputs the newly obtained video stream to the hardware decoder to perform decoding processing again, thereby performing secondary hardware decoding in an attempt to restore the normal playback process of the video. This can quickly restore the decoding state when the hardware decoding is abnormal, thus improving the stability of video playback.

[0045] S207, when the secondary hardware decoding fails, obtain the source duration of the video stream, and parse the image group parameters and video frame rate parameters from the video stream.

[0046] As an example, when a secondary hardware decoding anomaly is detected, the player module parses the current video stream and obtains the Group of Pictures (GOP) parameters and video frame rate parameters from the video stream's header information or sequence parameter information. The GOP parameters characterize the frame structure relationship between adjacent real-time decoding refresh frames, while the video frame rate parameters represent the number of video frames per unit time.

[0047] S208, calculate the time interval between two adjacent instantaneous decoding refresh frames based on the image group parameters and the video frame rate parameters.

[0048] As an example, the player estimates the possible time position of the next real-time decoding refresh frame based on the parsed image group parameters and video frame rate parameters.

[0049] Among them, the image group parameter is used to characterize the number of frames contained between two adjacent instant decoding refresh frames, while the video frame rate parameter is used to characterize the number of video frames per unit time. By dividing the number of frames by the video frame rate parameter, the corresponding time span, i.e. the time interval, can be obtained, which can be used to characterize the playback time offset caused by performing an instant decoding refresh frame jump.

[0050] In some embodiments, S208 may include: determining the number of frames between adjacent instantaneous decoding refresh frames based on the length of the image group parameter; dividing the number of frames by the video frame rate parameter to obtain the time interval between adjacent instantaneous decoding refresh frames.

[0051] As an example, to determine whether to perform a jump recovery operation based on instantaneous decoded refresh frames, it is necessary to calculate the time span between two adjacent instantaneous decoded refresh frames. Specifically, the number of frames between adjacent instantaneous decoded refresh frames is first determined based on the group of pictures (GOP) parameters parsed from the video stream. Since the group of pictures parameter describes the number of video frames contained in a complete group of pictures, and each group of pictures typically starts with an instantaneous decoded refresh frame, the length of the group of pictures can characterize the number of frames contained between two adjacent instantaneous decoded refresh frames.

[0052] After obtaining the frame count, the system further performs time conversion by combining the video frame rate parameter from the video stream. The video frame rate parameter represents the number of video frames displayed per unit time. Therefore, by dividing the frame count by the video frame rate parameter, the time interval between adjacent real-time decoding refresh frames can be obtained. This transforms the structural information, originally represented by frame count, into time-dimensional information, enabling the system to judge and control the decoding recovery strategy based on a time scale. For example, it can be used to determine whether a preset time threshold condition is met, thereby deciding whether to perform an IDR frame jump recovery operation.

[0053] S209, when the time interval is less than a preset time threshold and the timestamp of the next instant decoding refresh frame estimated based on the time interval is less than the duration of the source video, jump to the next instant decoding refresh frame and call the hardware decoder to perform three-level hardware decoding based on the next instant decoding refresh frame.

[0054] As an example, the time interval is compared with a preset time threshold. If the time interval is less than the preset time threshold, a decoding recovery operation is performed to jump to the next instant decoding refresh frame. If the time interval is greater than the preset time threshold, the IDR frame jump recovery strategy is abandoned, and the current decoding strategy is maintained or the software decoding process is directly entered.

[0055] Therefore, by introducing a time-span-based constraint mechanism, the image group parameters are transformed into temporal semantics, which enables effective control over the decoding and recovery strategy. This ensures that the decoding and recovery operation is only triggered when the jump cost is acceptable, thereby avoiding playback jumps or stuttering caused by long-distance jumps. At the same time, it reduces invalid recovery attempts and improves overall decoding stability and user experience.

[0056] S210, when the third-level hardware decoding fails, stop the current hardware decoder's decoding process, release the corresponding decoding resources, and report video buffer prompt information to the upper-layer application.

[0057] As an example, if a decoding error still occurs during the three-level hardware decoding process, the player module can stop the current hardware decoder's decoding process and release the relevant decoding resources. At the same time, it can report video buffering prompt information to the upper-layer application, so that the player can enter the buffering state and prompt the user interface that the video is resuming playback.

[0058] S211, initialize the software decoder, and after receiving a new real-time decoding refresh frame, input the subsequently received video bitstream into the software decoder for decoding processing.

[0059] As an example, when the player continues to receive new real-time decoding refresh frames, it inputs the subsequently received video stream into the software decoder for decoding processing and outputs the corresponding video frame data for video playback. Therefore, in the event of continuous hardware decoding failures, video decoding can continue through software decoding, thereby reducing prolonged stuttering or black screen phenomena during video playback and improving video playback stability.

[0060] In some embodiments, please refer to Figure 3 The player receives the video stream data to be played and determines whether a hardware decoder is currently used for video decoding. If a hardware decoder is not currently used, the player proceeds with the normal playback process.

[0061] When determining whether a hardware decoder is currently in use, it checks whether any playback errors have occurred during the current playback process. If no playback errors have occurred, the decoding error handling process is initiated. If playback errors have occurred, it indicates that the current hardware decoding environment may have persistent abnormalities or incompatibility issues. In this case, switching to a software decoder can avoid multiple buffering issues caused by repeated hardware decoder errors.

[0062] After entering the decoding exception handling process, please refer to Figure 4 When a hardware decoder malfunction is detected, the player performs a cache management operation, discarding the cached video stream data and reporting a buffering warning to the upper-layer application, thus putting the player into a buffering state. Then, the system determines the current video playback scenario.

[0063] Please see Figure 5 In the current live streaming scenario, since video data is real-time and cannot be traced back, a strategy of switching software decoders is implemented to perform software decoding on the subsequently received video streams.

[0064] When the current scenario is not live, please refer to Figure 4 and Figure 5The system then enters a multi-level hardware decoding recovery mechanism based on the timeline. It records the timestamp of the decoding anomaly and executes a first-level recovery strategy based on this timestamp: re-requesting the video stream data corresponding to that timestamp and calling the hardware decoder for decoding. If the first-level recovery fails, a second-level recovery strategy is initiated: calculating the time interval between two adjacent instantaneous decoding refresh frames based on the image group parameters and video frame rate parameters; if the time interval is less than a preset time threshold (e.g., a time threshold of 2), and the timestamp of the next instantaneous decoding refresh frame estimated based on the time interval is less than the source video duration, the system jumps to the next IDR frame and calls the hardware decoder again for decoding.

[0065] If decoding is successful at the next IDR frame, the hardware decoder will continue to play the video; if decoding still fails, the system will jump back to the abnormal timestamp position of the initially decoded IDR frame to avoid timeline offset caused by invalid jumps.

[0066] After jumping back to the abnormal timestamp position of the initially decoded IDR frame, a three-level recovery strategy is entered: if the hardware decoder is determined to be unable to resume normal operation, hardware decoding is stopped, decoding resources are released, and the system switches to the software decoder. Upon receiving a new IDR frame, the video stream is input into the software decoder for decoding, thus ensuring the continuity of video playback.

[0067] Please see Figure 6 , Figure 6 This is an exemplary structural block diagram of an electronic device using the video decoding method of this application. Figure 6 As shown, the electronic device 600 of this application may include a processor 601 and a memory 602, wherein the processor 601 and the memory 602 communicate via a bus. The memory 602 stores program instructions for video decoding. When the program instructions are executed by the processor 601, the processor performs the aforementioned method steps to implement a video decoding method in the above embodiments.

[0068] Please see Figure 7 , Figure 7 This is an exemplary structural block diagram of a computer-readable storage medium for the video decoding method of this application. Figure 7 As shown, the computer-readable storage medium 700 stores a computer program 701. When the computer program 701 is run by a processor on a computer, it causes the computer to perform the aforementioned method steps to implement a video decoding method in the above embodiments.

[0069] The above solution receives the video stream to be played, calls a hardware decoder for decoding, and when a hardware decoding anomaly is detected, it determines the abnormal timestamp of the real-time decoding refresh frame by reverse searching the video stream sequence. Based on the abnormal timestamp, it reacquires the corresponding video stream and performs hardware decoding again. If recovery fails, it jumps to the next real-time decoding refresh frame to continue hardware decoding. When hardware decoding recovery fails again, it switches to a software decoder. By constructing a decoding strategy that combines multi-level hardware decoding recovery and software decoding, it can improve the video decoding recovery capability in abnormal bitstream scenarios while maintaining the advantages of low power consumption and high efficiency of hardware decoding, thereby effectively reducing stuttering or black screen phenomena during video playback.

[0070] In the several embodiments provided in this application, it should be understood that the disclosed methods, electronic devices, and storage media can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0071] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0072] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0073] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the video decoding method described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A video decoding method, characterized in that, The video decoding method includes: Receive the video stream to be played and call the hardware decoder to perform first-level hardware decoding on the video stream; When the first-level hardware decoding fails, the video stream is searched in reverse according to the video stream sequence to determine the abnormal timestamp of the instantaneous decoding refresh frame; The video stream is reacquired based on the abnormal timestamp, and the hardware decoder is invoked to perform secondary hardware decoding on the reacquired video stream. When the secondary hardware decoding fails, the process jumps to the next real-time decoding refresh frame and calls the hardware decoder to perform tertiary hardware decoding based on the next real-time decoding refresh frame. When the third-level hardware decoding fails, the software decoder is switched to perform software decoding on the subsequently received video stream.

2. The method according to claim 1, characterized in that, The step of performing a reverse search on the video bitstream according to the video bitstream sequence to determine the abnormal timestamp of the real-time decoding refresh frame includes: Using the video frame position corresponding to the abnormal timestamp as the starting position, read the video frames in the video stream data backward. Within a preset backtracking window, the read video frames are identified by their frame type, and when the first instant decoding refresh frame is found, the corresponding abnormal timestamp is determined.

3. The method according to claim 1, characterized in that, The step of re-acquiring the video stream based on the abnormal timestamp includes: Based on the abnormal timestamp, determine the target time position of the video stream; Based on the target time location, generate video stream request parameters; Obtain the video stream corresponding to the target time location through a network data source or a local storage data source.

4. The method according to claim 1, characterized in that, The jump to the next real-time decoding refresh frame includes: The video stream source duration is obtained, and image group parameters and video frame rate parameters are parsed from the video stream. Calculate the time interval between two adjacent instantaneous decoding refresh frames based on the image group parameters and the video frame rate parameters; When the time interval is less than a preset time threshold and the timestamp of the next instant decoding refresh frame estimated based on the time interval is less than the duration of the source video, the process jumps to the next instant decoding refresh frame.

5. The method according to claim 4, characterized in that, The step of calculating the time interval between two adjacent instantaneous decoding refresh frames based on the image group parameters and the video frame rate parameters includes: The number of frames between adjacent instantaneous decoding refresh frames is determined based on the length of the image group parameters. Divide the number of frames by the video frame rate parameter to obtain the time interval between adjacent instantaneous decoding refresh frames.

6. The method according to claim 1, characterized in that, After the step of identifying a hardware decoding anomaly, and before the step of performing a reverse search of the video bitstream according to the video bitstream sequence to determine the anomaly timestamp of the real-time decoding refresh frame, the following steps are included: When a decoding error is detected, the video stream data in the player's buffer is cleared, and the current player state is recorded. Generate a video buffering prompt event based on the player's status; The video buffering notification event is sent to the application layer via a preset player event callback interface to trigger the video buffering display.

7. The method according to claim 1, characterized in that, The switching software decoder performs software decoding on subsequently received video streams, including: Stop the current hardware decoder's decoding process and release the corresponding decoding resources, and report video buffering prompt information to the upper-layer application; The software decoder is initialized, and upon receiving a new real-time decoding refresh frame, the subsequently received video stream is input into the software decoder for decoding processing.

8. A video decoding device, characterized in that, The video decoding device includes: The first decoding module is used to receive the video stream to be played and call the hardware decoder to perform first-level hardware decoding on the video stream. An anomaly determination module is used to perform a reverse search on the video bitstream according to the video bitstream sequence when the first-level hardware decoding is abnormal, and determine the abnormal timestamp of the instantaneous decoding refresh frame. The second decoding module is used to reacquire the video stream based on the abnormal timestamp and call the hardware decoder to perform secondary hardware decoding on the reacquired video stream. The third decoding module is used to jump to the next real-time decoding refresh frame when the second-level hardware decoding fails, and to call the hardware decoder to perform third-level hardware decoding based on the next real-time decoding refresh frame; The software decoding module is used to switch to a software decoder to perform software decoding on the subsequently received video stream when the third-level hardware decoding fails.

9. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable program code; A processor is configured to call and run the executable program code from the memory, causing the electronic device to perform the video decoding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the video decoding method as described in any one of claims 1 to 7.

Images