A video decoding method, device, apparatus and storage medium

Abstract

The application provides a video decoding method, device and equipment and a storage medium. The video decoding method can intelligently identify different error types in the process of video streaming decoding, and execute error-tolerant processing instructions matched with the error types according to the actual error types encountered, without immediately interrupting the decoding process, thereby forming a gradual error recovery mechanism, which is beneficial to improving the error tolerance of the entire video streaming decoding process.

Description

Technical Field

[0001] This application relates to the field of video transmission technology, and more specifically, to a video decoding method, apparatus, device, and storage medium. Background Technology

[0002] In existing video decoding systems, video streams are transmitted from the video sender to the video receiver via the RTSP (Real Time Streaming Protocol, a network protocol for real-time streaming media transmission). At the video receiver, the received video stream can be decoded and displayed using a decoder.

[0003] Currently, when network errors (such as temporary or permanent network errors) are encountered during video stream transmission, existing video decoding methods do not differentiate between different types of network errors. Instead, they directly interrupt the video decoding process as soon as a network error is detected. This results in poor fault tolerance in the video transmission decoding process, which can easily cause video playback interruptions and a poor user experience. Summary of the Invention

[0004] In view of this, this application provides a video decoding method, apparatus, device, and storage medium that can intelligently identify different error types during the video stream transmission decoding process, and execute fault-tolerant processing instructions matching the actual error type encountered, without immediately interrupting the decoding process, thereby forming a progressive error recovery mechanism that is beneficial to improving the fault tolerance of the entire video stream transmission decoding process.

[0005] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings.

[0006] In a first aspect, embodiments of this application provide a video decoding method, the video decoding method comprising: A data transmission channel is established with the video sending end based on the network parameters pre-configured for the target network protocol. The system receives a video stream sent by the video transmitter through the data transmission channel and sequentially reads data packets from the received video stream; wherein, the data packets represent video data after video encoding. In response to the successful reading of the current data packet, the successfully read current data packet is sent to the job queue associated with the decoder, and the decoder sequentially performs video decoding on the data packets in the job queue to obtain video frame data; In response to a failure to read the current data packet, the system receives an error type return value after the failure and executes a fault-tolerance processing instruction matching the error type indicated by the error type return value. The fault-tolerance processing instruction is either a retry read instruction or a skip read instruction. The retry read instruction indicates a processing instruction to reread the current data packet, while the skip read instruction indicates a processing instruction to set a target status flag for the current data packet and skip the current data packet to continue reading the next data packet from the video stream.

[0007] Secondly, embodiments of this application provide a video decoding device, the video decoding device comprising: The parameter configuration module is used to establish a data transmission channel with the video sender based on the network parameters pre-configured for the target network protocol. The data reading module is used to receive the video stream sent by the video sending end through the data transmission channel, and to read data packets sequentially from the received video stream; wherein, the data packets represent video data after video encoding; The data sending module is used to send the successfully read current data packet to the job queue associated with the decoder in response to the successful reading of the current data packet, and to perform video decoding on the data packets in the job queue in sequence by the decoder to obtain video frame data; The error recovery module is used to respond to the failure to read the current data packet, receive the error type return value obtained after the read failure, and execute a fault-tolerant processing instruction matching the read error type according to the read error type indicated by the error type return value; wherein, the fault-tolerant processing instruction is a retry read instruction or a skip read instruction, the retry read instruction is a processing instruction to reread the current data packet, and the skip read instruction is a processing instruction to set the target status flag of the current data packet and skip the current data packet to continue reading the next data packet from the video stream.

[0008] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the video decoding method described above.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the video decoding method described above.

[0010] The technical solutions provided by the embodiments of this application may include the following beneficial effects: This application provides a video decoding method, apparatus, device, and storage medium. Based on pre-configured network parameters for the target network protocol, a data transmission channel is established between the device and the video transmitter. The device receives a video stream from the video transmitter through the data transmission channel and sequentially reads data packets from the received video stream. In response to a successful read of a current data packet, the successfully read current data packet is sent to a job queue associated with the decoder. The decoder then sequentially decodes the data packets in the job queue to obtain video frame data. In response to a failed read of a current data packet, an error type return value is received, and a fault-tolerant processing instruction matching the read error type is executed according to the error type indicated by the error type return value. Thus, this application can intelligently identify different error types during video stream transmission decoding and execute fault-tolerant processing instructions matching the actual error type encountered, without immediately interrupting the decoding process. This forms a progressive error recovery mechanism, which is beneficial for improving the fault tolerance of the entire video stream transmission decoding process. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This paper shows a schematic diagram of the structure of a video transmission decoding system provided in an embodiment of this application; Figure 2 A flowchart illustrating a video decoding method provided in an embodiment of this application is shown; Figure 3 This illustration shows a schematic diagram of the structure of a video decoding device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0014] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0015] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0016] In one embodiment of this application, a video decoding method can be run in a video decoding device, which can be installed on the video receiving end side; wherein, Figure 1 This application provides a schematic diagram of the structure of a video transmission decoding system according to an embodiment of the present application. Figure 1 As shown, the video transmission decoding system may include the aforementioned video receiver and at least one video transmitter. On the video receiver side, a video decoding device can receive the video stream sent by the video transmitter and perform video decoding on the received video stream to obtain a specific video that can be played on the video receiver.

[0017] To facilitate understanding of the embodiments of this application, a video decoding method, apparatus, device, and storage medium provided in the embodiments of this application will be described in detail below.

[0018] Reference Figure 2 As shown, Figure 2 The diagram illustrates a flowchart of a video decoding method provided in an embodiment of this application, wherein the video decoding method includes steps S101-S104; specifically: S101 establishes a data transmission channel with the video sender based on network parameters pre-configured for the target network protocol.

[0019] Here, the target network protocol can be the RTSP protocol used to establish video streaming connections (i.e., the aforementioned data transmission channel).

[0020] Specifically, when performing step S101, the video decoding device can configure the specific parameter value corresponding to each network parameter in the form of key-value pairs, according to each network parameter name and its corresponding preset parameter value, thereby establishing a data transmission channel (equivalent to a video stream connection) between the video receiver and the video sender based on the configured network parameters.

[0021] For example, network parameters can be configured as follows using the preset parameter values ​​corresponding to each network parameter stored in AVDictionary (a data structure in the FFmpeg toolkit used to pass configuration parameters): Setting the network parameter buffer_size to 4194304 (4MB) can increase the receiving buffer of the video stream, which is suitable for network transmission application scenarios with network jitter and high bitrate streams. Set the value of the network parameter udp_buf_size to 4194304 (4MB) to set the UDP receive buffer size; Set the value of the network parameter rtp_buf_size to 4194304 (4MB) to set the RTP buffer size; Setting the value of the network parameter rtsp_transport to tcp prioritizes TCP transmission, which helps improve the stability of video stream transmission. Setting the network parameter stimeout to 5000000 (5 seconds) helps prevent video stream disconnections due to network jitter. Setting the network parameter max_delay to 500000 (500ms) allows for a larger buffer delay on the video receiving end.

[0022] Here, considering that the range of values ​​for the same network parameter may vary depending on the type of network transmission application scenario, as an optional embodiment, in the subsequent video transmission decoding process, when a change in the network condition corresponding to the data transmission channel is detected, the video decoding device can determine the target network transmission application scenario that matches the changed network condition from multiple preset network transmission application scenarios; then, the video decoding device can dynamically adjust the pre-configured network parameters according to the target network parameters associated with the target network transmission application scenario.

[0023] It should be noted that the above-mentioned preset network transmission application scenarios may include, but are not limited to: the first network transmission application scenario where network jitter occurs, the second network transmission application scenario where high bit rate streams occur, the third network transmission application scenario where network instability occurs, and the fourth network transmission application scenario where network latency is sensitive.

[0024] For example, when network jitter is detected, the video decoding device can determine the target network transmission application scenario as the first network transmission application scenario from the multiple preset network transmission application scenarios. If the value of the network parameter buffer_size is not 4194304 (4MB) applicable to the first network transmission application scenario, the value of the network parameter buffer_size can be dynamically adjusted to 4194304 (4MB) based on the fact that the target network parameter associated with the network parameter buffer_size in the first network transmission application scenario is 4194304 (4MB).

[0025] It should be noted that if the above data transmission channel is successfully established, the subsequent step S102 can be executed. If the above data transmission channel fails to be established, the relevant network resources can be released and the specific error type of the failure to establish the above data transmission channel can be returned so that the relevant control personnel on the video receiving end can make timely corrections.

[0026] S102, receive the video stream sent by the video sending end through the data transmission channel, and read data packets sequentially from the received video stream.

[0027] Here, on the video sending end, the video data to be sent to the video receiving end can be encoded to obtain the data packet corresponding to the video data (that is, the data packet represents the video data after video encoding). The aforementioned video stream is equivalent to a video data sequence composed of multiple continuously sent data packets. In other words, the data packet is equivalent to the basic unit of video transmission.

[0028] Specifically, as the video transmitter continuously sends data packets to the video receiver through the aforementioned data transmission channel, the video decoding device on the video receiver side can also continuously receive data packets. The queue of received data packets is the received video stream. Therefore, the video decoding device can read data packets sequentially from the received video stream (e.g., read data packets sequentially from the video stream in order of reception time from early to late).

[0029] For example, on the video receiving end, the video decoding device can use the av_read_frame function (a core function for reading data packets from media files or video streams) to read data packets sequentially from the received video stream in order of reception time from earliest to latest.

[0030] S103, in response to the successful reading of the current data packet, the successfully read current data packet is sent to the job queue associated with the decoder, and the data packets in the job queue are sequentially video decoded by the decoder to obtain video frame data.

[0031] Here, taking the reading of data packets from the received video stream using the av_read_frame function as an example, the av_read_frame function will send the reading result of the current data packet back to the video decoding device after the reading is completed, so that the video decoding device can determine whether the current data packet has been read successfully based on the reading result sent back by the av_read_frame function.

[0032] Specifically, if the current data packet is successfully read, the video decoding device can send the successfully read current data packet to the decoder. On the decoder side, the received data packets (i.e., the successfully read data packets) will form a job queue in the decoding buffer according to the order of their reception time, so that the decoder can perform video decoding on the data packets in the job queue in sequence to obtain the video decoding result (i.e., the aforementioned video frame data) corresponding to each data packet in the job queue.

[0033] It should be noted that, considering the possibility that the decoder may encounter corrupted video frames in the data packets during video decoding, in order to improve the decoder's tolerance to corrupted video frames, as an optional embodiment, the video decoding device can also configure fault tolerance parameters for the decoder before starting the decoder (i.e., before sequentially decoding the data packets in the job queue through the decoder); wherein, the configuration operation of the fault tolerance parameters includes at least one of the following: 1. Configure the error detection parameter of the decoder as the first parameter, which indicates that the decoder is allowed to enable the aggressive error detection mode (that is, configure err_detect as aggressive).

[0034] 2. Configure the error recovery parameter of the decoder as the second parameter, which indicates that the decoder is allowed to enable error recovery mode (that is, configure the error_resilience parameter to 1).

[0035] 3. Configure the frame skipping parameter of the decoder as the third parameter, which means that the decoder is allowed to skip the lost video frames during video decoding (that is, configure the skip_frame parameter as default).

[0036] 4. Configure the number of threads parameter of the decoder as the fourth parameter. The fourth parameter indicates that the decoder is allowed to automatically set the number of threads (that is, configure the threads parameter to auto, so that the decoder can set the number of threads according to the actual video decoding situation, which is beneficial to improving the decoding performance).

[0037] 5. Configure the number of thread type parameters of the decoder as the fifth parameter. The fifth parameter indicates that the decoder is allowed to enable the parallel decoding process at both the frame level and the slice level (that is, configure the thread_type parameter to frame+slice).

[0038] S104, in response to the failure to read the current data packet, receive the error type return value obtained after the read failure, and execute the fault tolerance processing instruction matching the read error type according to the read error type indicated by the error type return value.

[0039] Here, taking the reading of data packets from the received video stream using the av_read_frame function as an example, for the current data packet being read, the av_read_frame function will send the reading result corresponding to the current data packet back to the video decoding device after the reading is finished. If the reading result is a reading failure, the reading result sent back by the av_read_frame function will be an error type return value that can indicate the specific error type of this reading failure (i.e., the reading error type mentioned above).

[0040] Specifically, the aforementioned fault-tolerant processing instructions are either retry read instructions or skip read instructions. The retry read instruction indicates a processing instruction to reread the current data packet, while the skip read instruction indicates a processing instruction to set a target status flag (i.e., the read_frame_eof flag, used to indicate that the av_read_frame function encounters the end of the data file and cannot continue reading) for the current data packet and skip the current data packet to continue reading the next data packet from the video stream.

[0041] It should be noted that, based on the fact that the aforementioned fault-tolerance instructions are either retry read instructions or skip read instructions, in the video transmission decoding process, regardless of the type of read error detected by the av_read_frame function, the video decoding device will only reread the current data packet that failed to be read or continue reading the next data packet in the video stream, without affecting the video decoding process being executed on the decoder side. That is, there are multiple data packets in the job queue on the decoder side whose read time is earlier than the current data packet. Therefore, even if the current data packet fails to be read, it will not affect the decoder from continuing to decode the data packets already in the job queue, thus forming a progressive error recovery mechanism, which is beneficial to improving the fault tolerance of the entire video stream transmission decoding process.

[0042] Specifically, as an optional embodiment, step S104 can be performed according to the method shown in steps a1-a2 below: Step a1: If the error type returned by the error type is the first read error type, then determine to execute the skip read instruction.

[0043] Here, the first read error type indicates that the end of the video stream has been reached and data cannot be read further.

[0044] Specifically, when the error type return value of the av_read_frame function is AVERROR_EOF (end of file), it can be determined that the read error type indicated by the error type return value is the first read error type. At this time, the video decoding device can set the target status flag (i.e., the read_frame_eof flag) for the current data packet that failed to be read, and skip the current data packet to continue reading the next data packet from the video stream (i.e., execute the skip read instruction).

[0045] Step a2: If the error type indicated by the error type return value is a read error type other than the first read error type, then determine to execute the retry read instruction.

[0046] Here, in addition to the first read error type indicated by the error type return value AVERROR_EOF, the possible read error types are: the second read error type indicated by the error type return value AVERROR(EAGAIN) (resource temporarily unavailable), and the third read error type indicated by other error type return values ​​(i.e., other error type return values ​​besides AVERROR_EOF and AVERROR(EAGAIN)).

[0047] Specifically, if the error type return value of the av_read_frame function is AVERROR(EAGAIN), it can be determined that the read error type indicated by the error type return value is the second read error type. At this time, the video decoding device can execute continue to retry reading (that is, execute the retry reading instruction) on the above-mentioned current data packet that failed to be read.

[0048] Specifically, if the error type return value returned by the av_read_frame function is one of the other error types mentioned above, it can be determined that the read error type indicated by the error type return value is the third read error type. At this time, the video decoding device can record the relevant logs and continue to try to read the current data packet that failed to be read (that is, execute the retry read instruction) so as to provide the video transmission decoding system with a chance to recover the network, instead of immediately returning a network error and interrupting the video decoding process.

[0049] Here, after sending a data packet to the decoder (e.g., via avcodec_send_packet), if the data packet fails to be sent and the error type return value is AVERROR(EAGAIN), the video decoding device can set the again flag for the failed data packet to indicate that the video decoding device can retry sending the data packet to the decoder later (e.g., resend the data packet to the decoder after waiting for a preset time). If the data packet is sent successfully, the send_cnt count can be incremented to determine the number of data packets successfully sent to the decoder.

[0050] Specifically, after successfully sending the data packet to the decoder, the video decoding device can detect whether there are video frame processing errors in the video frame data (i.e., the video decoding result of the data packet) returned by the decoder by performing the following steps b1-b4: Step b1: Receive the current video frame data obtained after video decoding from the decoder.

[0051] Specifically, the video decoding device can use the avcodec_receive_frame function to receive the current video frame data obtained by the decoder after decoding the data packet. After the reception is completed, the avcodec_receive_frame function will send the reception result corresponding to the current video frame data back to the video decoding device, so that the video decoding device can determine whether the current video frame data has been successfully received based on the reception result fed back by the avcodec_receive_frame function.

[0052] Step b2: In response to the successful reception of the current video frame data, verify the validity of the current video frame data.

[0053] Specifically, if the avcodec_receive_frame function returns a result indicating successful reception of the current video frame data, then during step b2, the video decoding device can verify the validity of the current video frame data using at least one of the following verification methods: 1. Check if the frame pointer in the current video frame data is NULL (the purpose is to prevent null pointer access). If the frame pointer is NULL, it can be determined that the data of the current video frame is invalid.

[0054] 2. Check if the frame->data[0] pointer in the current video frame data is NULL (the purpose is to ensure that the luminance component data is accessible). If the frame->data[0] pointer is NULL, it can be determined that the data of the current video frame is invalid.

[0055] 3. Check if frame->width in the current video frame data is greater than 0 (the purpose is to ensure that the image width is a valid value). If frame->width ≤ 0, it can be determined that the data of the current video frame is invalid.

[0056] 4. Check if frame->height in the current video frame data is greater than 0 (the purpose is to ensure that the image height is a valid value). If frame->height ≤ 0, it can be determined that the data of the current video frame is invalid.

[0057] Step b3: If it is determined that the current video frame data is invalid, skip the current video frame data, continue to receive the next video frame data from the decoder, and verify the validity of the next video frame data.

[0058] Specifically, regarding when the current video frame data is considered invalid, please refer to the specific verification method in step b2 above. Repeated points will not be repeated here.

[0059] Step b4: If it is determined that the current video frame data is valid, then the current video frame data is converted to a different color space to obtain video frame data in the target format.

[0060] Here, regarding the specific circumstances of when the current video frame data is considered valid, please refer to the specific verification method in step b2 above. Repeated points will not be repeated here.

[0061] Specifically, when performing color space conversion on the current video frame data, the sws_scale function can be used to convert the current video frame data into a target format (such as NV21 format) to obtain video frame data in the target format; wherein, the target format refers to a format that meets the video display requirements of the video receiver.

[0062] It should be noted that when the sws_scale function performs color space conversion on the current video frame data (that is, converts the current video frame data to the target format), it also feeds back the conversion result corresponding to the current video frame data to the video decoding device. In other words, after performing color space conversion on the current video frame data, the video decoding device can receive the conversion return value obtained after color space conversion on the current video frame data (that is, the above conversion result).

[0063] Specifically, if the conversion return value indicates that the conversion failed (e.g., the conversion return value scaleRet<= 0), then the current video frame data can be skipped, and the color space conversion can continue for the next video frame data that is valid.

[0064] Specifically, if the conversion return value indicates that the conversion was successful (e.g., the conversion return value scaleRet > 0), the YUV data (video frame data in the target format) can be encapsulated into a jbyteArray. The encapsulated jbyteArray can then be passed to the Java layer through a JNI callback function to complete the processing of the current video frame data.

[0065] Here, after performing step b1 above, the video decoding device processes the current video frame data that failed to be received according to the method shown in steps c1-c3 below, specifically: Step c1: In response to the failure to receive the current video frame data, determine that the decoder has encountered a decoding error, and receive the decoding error return value returned by the decoder.

[0066] Here, the avcodec_receive_frame function can receive the current video frame data obtained by the decoder after decoding the data packet. After the reception is completed, the avcodec_receive_frame function will send the reception result corresponding to the current video frame data back to the video decoding device. If the reception result is a reception failure, the reception result sent back by the avcodec_receive_frame function will be a decoding error return value that can indicate the specific error type of this reception failure (i.e., the decoding error type mentioned above).

[0067] Step c2: When the decoding error type indicated by the decoding error return value is the first decoding error type, a retry flag is set for the current video frame data, and after waiting for a preset time, the decoder is controlled to resend the current video frame data carrying the retry flag.

[0068] Here, the first decoding error type indicates that the resource is temporarily unavailable. That is, if the decoding error return value fed back by the avcodec_receive_frame function is AVERROR(EAGAIN) (resource is temporarily unavailable), then it can be determined that the decoding error type indicated by the decoding error return value is the first decoding error type.

[0069] Specifically, if the decoding error return value fed back by the avcodec_receive_frame function is AVERROR(EAGAIN), it can be determined that the decoding error type indicated by the decoding error return value is the first decoding error type. At this time, the video decoding device can set a retry flag (i.e., the again flag) for the aforementioned current video frame data that failed to be received, and after waiting for a preset time, control the decoder to resend the current video frame data carrying the retry flag.

[0070] Step c3: When the decoding error type indicated by the decoding error return value is a decoding error type other than the first decoding error type, continue to receive the next video frame data from the decoder and verify the data validity of the next video frame data.

[0071] Here, in addition to the first decoding error type indicated by the decoding error return value AVERROR(EAGAIN), the possible decoding error types are: the second decoding error type indicated by the decoding error return value AVERROR_EOF, and the third decoding error type (equivalent to a permanent network error) indicated by other decoding error return values ​​(i.e., other decoding error return values ​​besides AVERROR_EOF and AVERROR(EAGAIN)).

[0072] Specifically, if the decoding error return value reported by the avcodec_receive_frame function is AVERROR_EOF, it can be determined that the decoding error type indicated by the decoding error return value is the second decoding error type. At this time, the video decoding device can set the read end flag for the above-mentioned current video frame data that failed to be received, continue to receive the next video frame data from the decoder, and verify the data validity of the next video frame data.

[0073] Specifically, if the decoding error return value returned by the avcodec_receive_frame function is one of the other decoding error return values ​​mentioned above, it can be determined that the decoding error type indicated by the decoding error return value is the third decoding error type. At this time, the video decoding device can skip the current video frame data that failed to be received and continue to receive the next video frame data from the decoder, and verify the validity of the data of the next video frame data.

[0074] Based on the video decoding method provided in the embodiments of this application, different error types can be intelligently identified during the video stream transmission decoding process, and fault-tolerant processing instructions matching the actual error type can be executed without immediately interrupting the decoding process, thereby forming a progressive error recovery mechanism, which is beneficial to improving the fault tolerance of the entire video stream transmission decoding process.

[0075] Based on the same inventive concept, this application also provides a video decoding device corresponding to the above-described video decoding method. Since the principle of the video decoding device in the embodiments of this application is similar to that of the above-described video decoding method in the embodiments of this application, the implementation of the video decoding device can refer to the implementation of the above-described video decoding method, and the repeated parts will not be described again.

[0076] Reference Figure 3 As shown, Figure 3 This illustration shows a structural schematic diagram of a video decoding device provided in an embodiment of this application. The video decoding device includes: The parameter configuration module 301 is used to establish a data transmission channel with the video sending end based on the network parameters pre-configured for the target network protocol. The data reading module 302 is used to receive the video stream sent by the video sending end through the data transmission channel, and to read data packets sequentially from the received video stream; wherein, the data packets represent video data after video encoding; The data sending module 303 is used to send the successfully read current data packet to the job queue associated with the decoder in response to the successful reading of the current data packet, and to perform video decoding on the data packets in the job queue in sequence by the decoder to obtain video frame data; Error recovery module 304 is used to respond to the failure to read the current data packet, receive the error type return value obtained after the read failure, and execute a fault-tolerant processing instruction matching the read error type according to the read error type indicated by the error type return value; wherein, the fault-tolerant processing instruction is a retry read instruction or a skip read instruction, the retry read instruction is a processing instruction to reread the current data packet, and the skip read instruction is a processing instruction to set the target status flag of the current data packet and skip the current data packet to continue reading the next data packet from the video stream.

[0077] In an optional implementation, before sequentially performing video decoding on the data packets in the job queue via the decoder, the data sending module 303 is further configured to: Configure fault tolerance parameters for the decoder; wherein the configuration of the fault tolerance parameters includes at least one of the following: Configure the error detection parameter of the decoder as a first parameter, wherein the first parameter indicates that the decoder is allowed to enable an aggressive error detection mode; Configure the error recovery parameter of the decoder to a second parameter, wherein the second parameter indicates that the decoder is allowed to enable error recovery mode; Configure the frame skipping parameter of the decoder as a third parameter, which indicates that the decoder is allowed to skip lost video frames during video decoding; Configure the number of threads parameter of the decoder as the fourth parameter, which indicates that the decoder is allowed to automatically set the number of threads; Configure the number of thread type parameters of the decoder as the fifth parameter, which indicates that the decoder is allowed to enable parallel decoding processes at both the frame level and the slice level.

[0078] In an optional implementation, when executing a fault-tolerant processing instruction matching the read error type based on the read error type indicated by the error type return value, the error recovery module 304 is configured to: If the error type return value indicates that the read error type is the first read error type, then the skip read instruction is executed; wherein, the first read error type indicates that the end of the video stream data has been reached and data cannot be read further; If the error type returned by the error type is a read error type other than the first read error type, then the retry read instruction is executed.

[0079] In an optional embodiment, the video decoding device further includes a decoder fault-tolerant configuration module, wherein the decoder fault-tolerant configuration module is used for: Receive the current video frame data obtained after video decoding from the decoder; In response to the successful reception of the current video frame data, the validity of the current video frame data is verified; If the current video frame data is determined to be invalid, the current video frame data is skipped, and the next video frame data is received from the decoder, and the validity of the next video frame data is verified. If the current video frame data is determined to be valid, then the current video frame data is converted to a different color space to obtain video frame data in the target format; wherein, the target format represents a format that meets the video display requirements of the video receiving end.

[0080] In an optional implementation, after receiving the video frame data obtained after video decoding from the decoder, the decoder fault-tolerant configuration module is further configured to: In response to the failure to receive the current video frame data, it is determined that the decoder has encountered a decoding error, and the decoding error return value returned by the decoder is received; When the decoding error return value indicates a first decoding error type, a retry flag is set for the current video frame data, and after waiting for a preset time, the decoder is controlled to resend the current video frame data carrying the retry flag; wherein, the first decoding error type indicates that the resource is temporarily unavailable; When the decoding error return value indicates a decoding error type other than the first decoding error type, the next video frame data continues to be received from the decoder, and the validity of the next video frame data is verified.

[0081] In an optional implementation, after performing color space conversion on the current video frame data, the decoder fault-tolerant configuration module is further configured to: Receive the conversion return value obtained after color space conversion of the current video frame data; If the conversion return value indicates that the conversion failed, the current video frame data is skipped, and the color space conversion is performed on the next video frame data that is valid.

[0082] In an optional embodiment, the video decoding device further includes a parameter adjustment module, wherein the parameter adjustment module is used for: When a change in the network condition corresponding to the data transmission channel is detected, a target network transmission application scenario that matches the changed network condition is determined from multiple preset network transmission application scenarios. The pre-configured network parameters are dynamically adjusted based on the target network parameters associated with the target network transmission application scenario.

[0083] like Figure 4 As shown, this application embodiment also provides an electronic device 400 for executing the video decoding method in this application. The electronic device includes a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor 402. The memory 401 and the processor 402 are connected via a bus for communication. When the processor 402 executes the computer program, it implements the steps of the video decoding method described above.

[0084] Specifically, the memory 401 and processor 402 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 402 runs the computer program stored in the memory 401, it can execute the video decoding method described above.

[0085] Corresponding to the video decoding method in this application, this application embodiment also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the steps of the video decoding method described above.

[0086] Specifically, the storage medium can be a general-purpose storage medium, such as a removable disk or hard disk. When the computer program on the storage medium is run, it can execute the video decoding method described above.

[0087] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0088] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0090] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0091] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0092] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A video decoding method, characterized in that, The video decoding method includes: A data transmission channel is established with the video sending end based on the network parameters pre-configured for the target network protocol. The system receives a video stream sent by the video transmitter through the data transmission channel and sequentially reads data packets from the received video stream; wherein, the data packets represent video data after video encoding. In response to the successful reading of the current data packet, the successfully read current data packet is sent to the job queue associated with the decoder, and the decoder sequentially performs video decoding on the data packets in the job queue to obtain video frame data; In response to a failure to read the current data packet, the system receives an error type return value after the failure and executes a fault-tolerance processing instruction matching the error type indicated by the error type return value. The fault-tolerance processing instruction is either a retry read instruction or a skip read instruction. The retry read instruction indicates a processing instruction to reread the current data packet, while the skip read instruction indicates a processing instruction to set a target status flag for the current data packet and skip the current data packet to continue reading the next data packet from the video stream.

2. The video decoding method according to claim 1, characterized in that, Before sequentially performing video decoding on the data packets in the job queue via the decoder, the video decoding method further includes: Configure fault tolerance parameters for the decoder; wherein the configuration of the fault tolerance parameters includes at least one of the following: Configure the error detection parameter of the decoder as a first parameter, wherein the first parameter indicates that the decoder is allowed to enable an aggressive error detection mode; Configure the error recovery parameter of the decoder to a second parameter, wherein the second parameter indicates that the decoder is allowed to enable error recovery mode; Configure the frame skipping parameter of the decoder as a third parameter, which indicates that the decoder is allowed to skip lost video frames during video decoding; Configure the number of threads parameter of the decoder as the fourth parameter, which indicates that the decoder is allowed to automatically set the number of threads; Configure the number of thread type parameters of the decoder as the fifth parameter, which indicates that the decoder is allowed to enable parallel decoding processes at both the frame level and the slice level.

3. The video decoding method according to claim 1, characterized in that, The step of executing a fault-tolerant processing instruction matching the read error type based on the read error type indicated by the error type return value includes: If the error type return value indicates that the read error type is the first read error type, then the skip read instruction is executed; wherein, the first read error type indicates that the end of the video stream data has been reached and data cannot be read further; If the error type returned by the error type is a read error type other than the first read error type, then the retry read instruction is executed.

4. The video decoding method according to claim 1, characterized in that, The video decoding method further includes: Receive the current video frame data obtained after video decoding from the decoder; In response to the successful reception of the current video frame data, the validity of the current video frame data is verified; If the current video frame data is determined to be invalid, the current video frame data is skipped, and the next video frame data is received from the decoder, and the validity of the next video frame data is verified. If the current video frame data is determined to be valid, then the current video frame data is converted to a different color space to obtain video frame data in the target format; wherein, the target format represents a format that meets the video display requirements of the video receiving end.

5. The video decoding method according to claim 4, characterized in that, After receiving the video frame data obtained after video decoding from the decoder, the video decoding method further includes: In response to the failure to receive the current video frame data, it is determined that the decoder has encountered a decoding error, and the decoding error return value returned by the decoder is received; When the decoding error return value indicates a first decoding error type, a retry flag is set for the current video frame data, and after waiting for a preset time, the decoder is controlled to resend the current video frame data carrying the retry flag; wherein, the first decoding error type indicates that the resource is temporarily unavailable; When the decoding error return value indicates a decoding error type other than the first decoding error type, the next video frame data continues to be received from the decoder, and the validity of the next video frame data is verified.

6. The video decoding method according to claim 4, characterized in that, After performing color space conversion on the current video frame data, the video decoding method further includes: Receive the conversion return value obtained after color space conversion of the current video frame data; If the conversion return value indicates that the conversion failed, the current video frame data is skipped, and the color space conversion is performed on the next video frame data that is valid.

7. The video decoding method according to claim 1, characterized in that, The video decoding method further includes: When a change in the network condition corresponding to the data transmission channel is detected, a target network transmission application scenario that matches the changed network condition is determined from multiple preset network transmission application scenarios. The pre-configured network parameters are dynamically adjusted based on the target network parameters associated with the target network transmission application scenario.

8. A video decoding device, characterized in that, The video decoding device includes: The parameter configuration module is used to establish a data transmission channel with the video sender based on the network parameters pre-configured for the target network protocol. The data reading module is used to receive the video stream sent by the video sending end through the data transmission channel, and to sequentially read data packets from the received video stream; wherein, the data packets represent video data after video encoding; The data sending module is used to send the successfully read current data packet to the job queue associated with the decoder in response to the successful reading of the current data packet, and to perform video decoding on the data packets in the job queue in sequence by the decoder to obtain video frame data; The error recovery module is used to respond to the failure to read the current data packet, receive the error type return value obtained after the read failure, and execute a fault-tolerant processing instruction matching the read error type according to the read error type indicated by the error type return value; wherein, the fault-tolerant processing instruction is a retry read instruction or a skip read instruction, the retry read instruction is a processing instruction to reread the current data packet, and the skip read instruction is a processing instruction to set the target status flag of the current data packet and skip the current data packet to continue reading the next data packet from the video stream.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the video decoding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the video decoding method as described in any one of claims 1 to 7.

Images