Error recovery method, device and equipment of AVS3-oriented video coding and decoding system and storage medium

Abstract

The invention discloses an error recovery method, device and equipment of an AVS3-oriented video coding and decoding system and a storage medium, and relates to the technical field of computers, and the method comprises the steps: separating a video frame code stream into a picture header packet and a plurality of Patch packets based on a start code through a channel simulator, packaging identification information to generate a to-be-processed network packet, and transmitting the to-be-processed network packet to a server; judging whether the picture header packet and each Patch packet are lost or not, and if the picture header packet is lost or all the Patch packets are lost, judging that the current code stream is unavailable; if the picture header packet is not lost and at least one Patch packet is not lost, the picture header packet and the unlost Patch packet are written into an output code stream, and the AVS3 decoder judges the frame loss condition according to the display sequence interval of the current frame and the previous frame in the output code stream and performs data recovery by adopting a corresponding error recovery method. Error recovery efficiency of a video coding and decoding system is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an error recovery method, apparatus, device, and storage medium for a video codec system for AVS3. Background Technology

[0002] Currently, in the Ultra Low Latency Coding (ULL) mode of the AVS3 standard, video frame data passes through the channel during transmission, which may result in packet loss. Traditional packet loss handling schemes mainly focus on the loss of entire frame data, that is, they determine frame loss by detecting the POC (Picture Order Count) interval in the frame sequence and cover up the error by copying the previous frame.

[0003] However, in actual transmission scenarios, frame data consists of multiple data packets at different levels: Picture Header (PH): Contains header information for the frame, such as frame type, POC, Decoding Order Index (DOI), Reference Frame List (RPL), and other key information.

[0004] Image Patch / Slice Data: Contains the actual image encoding data; a frame can contain multiple patches.

[0005] When PH is complete but some patches are lost, or when PH is complete but all patch data is lost, existing technologies cannot effectively handle these finer-grained packet loss scenarios.

[0006] Deficiencies of existing technology: 1. Inability to distinguish between PH loss and Patch loss: Existing technology processes frame data as a whole and cannot identify the loss of different components within the frame; 2. Loss of multiple consecutive PH frames leads to missing reference frames: When multiple consecutive PH frames are lost, the reference frames in the Decoding Image Buffer (DPB) will be incorrectly cleared, causing subsequent frames to be unable to be decoded due to the lack of reference frames. 3. Lack of fine-grained channel simulation capabilities: Existing channel simulation systems only support packet loss simulation at the whole frame level and cannot simulate packet loss at the PH level and Patch level.

[0007] As can be seen from the above, improving the efficiency of error recovery for AVS3-oriented video codec systems is an urgent problem to be solved. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide an error recovery method, apparatus, device, and storage medium for AVS3-based video codec systems, which can improve the efficiency of error recovery for AVS3-based video codec systems. The specific solution is as follows: Firstly, this application provides an error recovery method for a video codec system based on AVS3, including: Using a channel simulator and based on the start code, the bitstream data of each image frame in the video data is separated into image header packets and several patch packets. The image header packets and their corresponding identification information, as well as each patch packet and its corresponding identification information, are encapsulated into independent network packets to be processed. The receiving end receives each of the network packets to be processed, and uses a preset packet loss probability model to determine whether the image header packet corresponding to the network packet to be processed is lost and whether each of the patch packets is lost, so as to obtain the network packet reception status corresponding to each of the image frames. If the network packet reception status indicates that the image header packet is lost, or all the patch packets are lost, then the current bitstream data is determined to be unusable and the current bitstream data is skipped. If the image header packet is not lost and at least one patch packet is not lost, then the current bitstream data corresponding to the image header packet and each unlost patch packet is written into the output bitstream. The display order interval between the current frame and the previous frame in the output bitstream is determined using the decoder corresponding to AVS3. Based on the display order interval, it is determined whether the current bitstream data has frame loss due to image header packet loss, image header packet being complete but some patch packets being lost, or image header packet being complete but all patch packets being lost. If so, the current bitstream data is recovered based on the corresponding error recovery method to obtain the data recovery result.

[0009] Optionally, the step of using a channel simulator and based on a start code to separate the bitstream data of each image frame in the video data into image header packets and several patch packets, and encapsulating the image header packets with their corresponding identification information and each patch packet with its corresponding identification information into independent network packets to be processed, includes: Video data is acquired using a channel simulator, and the bitstream data of each image frame is segmented based on the start code to obtain the corresponding image header packet and several patch packets corresponding to the image header packet. Configure first identification information for the image header packet, and configure second identification information corresponding to the patch index of each patch packet for each patch packet; Based on a preset transmission unit size, the image header packet and the corresponding first identification information, and each patch packet and the corresponding second identification information are encapsulated to obtain several network packets to be processed; wherein, the network packets to be processed carrying the image header packet and the network packets to be processed carrying the patch packet are independent of each other at the encapsulation level.

[0010] Optionally, the step of receiving each of the network packets to be processed at the receiving end, and determining whether the image header packet corresponding to the network packet to be processed is lost and whether each of the patch packets is lost using a preset packet loss probability model, to obtain the network packet reception status corresponding to each of the image frames, includes: At the receiving end, each of the network packets to be processed is acquired, and the network packets to be processed are identified based on the identification information to obtain the image header packet and each of the patch packets; Using a preset packet loss probability model and based on the image header packet, it is determined whether the image header packet of the current image frame is lost. If the receiving end does not receive the image header packet, it is determined that the image header packet of the current image frame is lost, and the corresponding loss determination result is obtained. The preset packet loss probability model is used to determine whether each of the patch packets in the current image frame is lost based on each patch packet. If the receiving end does not receive the patch packet, the corresponding patch packet is determined to be lost, and the corresponding loss determination result is obtained. Based on the loss determination results of each Patch packet, determine whether all Patch packets in the current image frame are lost. If all Patch packets are determined to be lost, generate a full Patch loss indication for the current image frame. Based on the loss determination results of the image header packet, the loss determination results of each patch packet, and the full patch loss indication, network packet reception status information corresponding to the current image frame is generated; the network packet reception status information is used to characterize the transmission quality of the current image frame in channel simulation.

[0011] Optionally, if the network packet reception status indicates that the image header packet is lost, or all the patch packets are lost, then the current bitstream data is determined to be unusable and the current bitstream data is skipped. If the image header packet is not lost, and at least one patch packet is not lost, then the current bitstream data corresponding to the image header packet and each unlost patch packet is written into the output bitstream, including: Based on the network packet reception status information, determine whether the image header packet of the current image frame is lost, and determine whether all patch packets of the current image frame are lost. If it is determined that the image header packet of the current image frame is lost, or if it is determined that all the patch packets are lost, then the bitstream data of the current image frame is determined to be unusable, the current image frame is skipped, and then writing the bitstream data of the current image frame into the output bitstream is prohibited. If it is determined that the image header packet of the current image frame is not lost, and at least one patch packet is not lost, then the bitstream data portion of the current image frame is determined to be available, and the bitstream data corresponding to the image header packet is written to the output bitstream. Then, the bitstream data corresponding to the patch packet that is determined to be not lost is written to the output bitstream, and the patch packet that is determined to be lost is skipped to prevent the corresponding bitstream data from being written to the output bitstream, thus obtaining the output bitstream.

[0012] Optionally, based on the display order interval, it is determined whether the current bitstream data has frame drops due to image header packet loss. If so, data recovery is performed on the current bitstream data using the corresponding error recovery method to obtain the data recovery result, including: The corresponding image frame data is read from the output bitstream, and when the image header packet of the current image frame is read, the bitstream data is subjected to start code detection based on a preset start code set to obtain the start code detection result; the start code set includes an image header start code for identifying the image header packet and a patch start code for identifying the patch packet; If the start code detection result indicates that after reading the image header packet of the current image frame, the next start code read within a preset time range is the image header start code, it is determined that all the patch packets of the current image frame have been lost, and a full patch loss indication corresponding to the current image frame is generated. If the start code detection result indicates that the image header packet of each image frame has been successfully read, the current display order number of the current image frame in the image frame sequence is determined based on the image sequence parameters carried by the image header packet of the current image frame, and the previous display order number of the last successfully decoded image frame is obtained. Then, the display order interval between the current display order number and the previous display order number is determined. If the display sequence interval is greater than a preset consecutive number unit, it is determined that there is at least one intermediate image frame with a completely lost image header packet between the previous successfully decoded image frame and the current image frame, and an image header loss indication corresponding to the intermediate image frame is generated. The lost image frames are determined based on the full patch loss indication and the image header loss indication; If the lost image frame is the first image frame, and the image header packet of the first image frame is lost or all patch packets are lost, then decoding failure is determined, and a return message indicating decoding failure is generated. If the lost image frame is not the first image frame, then the image data of the previous successfully decoded image frame is obtained, and the image data of the previous successfully decoded image frame is used as the recovered image data of the lost image frame to obtain the corresponding data recovery result.

[0013] Optionally, it is determined whether the current bitstream data has frame drops due to either a complete image header packet with partial patch packet loss or a complete image header packet with all patch packets loss. If so, data recovery is performed on the current bitstream data based on the corresponding error recovery method to obtain the data recovery result, including: If the display order interval is equal to the preset consecutive numbering unit, then the current image frame is determined to be a consecutive image frame, and the first receiving status of the image header packet of the current image frame and the second receiving status of each Patch packet are obtained; The data integrity status of the current image frame is determined based on the first reception status and the reception status. If the data integrity status indicates that the image header packet of the current image frame is not lost, and all patch packets in the current image frame are lost, then the current image frame is determined to be in the first state where the image header is intact but all patches are lost. If the data integrity status indicates that the image header packet of the current image frame is not lost, and there is at least one unlost patch packet and at least one lost patch packet in the current image frame, then the current image frame is determined to be in the second state where the image header is complete and some patches are lost. Based on the first state, copy the image data of the previously successfully decoded image frame, and set the copy result as the recovered image data of the current image frame; Based on the second state, a reference patch packet with the same patch index as the lost patch packet is obtained from the previously successfully decoded image frame, and the reference patch packet is set as the recovery patch packet of the lost patch packet. Then, all unlost patch packets in the current image frame are retained. The data recovery result of the current image frame is generated based on the image header packet of the current image frame, the recovered image data, each of the unlost patch packets, and the recovery patch packet.

[0014] Optionally, after determining whether the current bitstream data has frame drops due to image header packet loss, complete image header packet with partial Patch packet loss, or complete image header packet with all Patch packets loss based on the display order interval, the method further includes: The successfully decoded image frame corresponding to the output bitstream is set as a reference frame and stored in the decoded image buffer; Based on the current image frame to be decoded, the image header packet corresponding to the current image frame to be decoded is parsed from the output bitstream, and the reference frame list information of the current image frame to be decoded is extracted from the image header packet; the reference frame list information includes the display order number of the expected reference frame; Based on the reference frame list information, determine whether the display order number of each reference frame stored in the decoded image buffer exists in the reference frame list information. If it does not exist, mark the reference frame as an unusable reference frame and remove the unusable reference frame from the decoded image buffer. The number of available reference frames in the decoded image buffer is detected. If the number of reference frames is not greater than a preset threshold, the most recently successfully decoded image frame is forcibly retained in the decoded image buffer. Check whether the reference frame corresponding to the display order number of each expected reference frame in the reference frame list information exists in the current decoded image buffer. If it does not exist, select an available reference frame from the decoded image buffer as a replacement reference frame, and set the display order number of the replacement reference frame to the display order number of the expected reference frame to obtain a new reference frame list. Based on the new reference frame list, the corresponding reference frame image data is obtained from the decoded image buffer, and the reference frame image data is used to decode the current image frame to obtain the data recovery result corresponding to the current image frame.

[0015] Secondly, this application provides an error recovery device for an AVS3 video codec system, comprising: The bitstream data separation module is used to separate the bitstream data of each image frame in the video data into image header packets and several patch packets based on the start code using a channel simulator. The image header packets and their corresponding identification information, as well as each patch packet and its corresponding identification information, are encapsulated into independent network packets to be processed. The network packet judgment module is used to receive each network packet to be processed by the receiving end, and use a preset packet loss probability model to determine whether the image header packet corresponding to the network packet to be processed is lost and whether each Patch packet is lost, so as to obtain the network packet reception status corresponding to each image frame. The output bitstream generation module is used to determine that the current bitstream data is unusable and skip the current bitstream data if the network packet reception status indicates that the image header packet is lost, or if all the patch packets are lost; if the image header packet is not lost and at least one of the patch packets is not lost, the current bitstream data corresponding to the image header packet and each unlost patch packet is written into the output bitstream. The data recovery result determination module is used to determine the display order interval between the current frame and the previous frame in the output bitstream using the decoder corresponding to AVS3, and to determine whether the current bitstream data has frame loss due to image header packet loss, image header packet being complete and part of the patch packet being lost, or image header packet being complete and all patch packets being lost, based on the display order interval. If so, the current bitstream data is recovered based on the corresponding error recovery method to obtain the data recovery result.

[0016] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned error recovery method for a video codec system for AVS3.

[0017] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned error recovery method for a video codec system for AVS3.

[0018] As can be seen from the above, before performing error recovery for the AVS3-oriented video codec system, this application needs to use a channel simulator and based on the start code to separate the bitstream data of each image frame in the video data into image header packets and several patch packets. The image header packets and their corresponding identification information, as well as each patch packet and its corresponding identification information, are then encapsulated into independent network packets to be processed. The receiving end receives each network packet to be processed and uses a preset packet loss probability model to determine whether the image header packet and each patch packet corresponding to the network packet to be processed are lost, thus obtaining the network packet reception status corresponding to each image frame. If the network packet reception status indicates that the image header packet is lost, or all patch packets are lost... If the current bitstream data is deemed unusable, it is skipped. If the image header packet is not lost and at least one patch packet is not lost, the current bitstream data corresponding to the image header packet and each unlost patch packet is written into the output bitstream. The display order interval between the current frame and the previous frame in the output bitstream is determined using the decoder corresponding to AVS3. Based on the display order interval, it is determined whether the current bitstream data has frame loss due to image header packet loss, image header packet being complete and some patch packets being lost, or image header packet being complete and all patch packets being lost. If so, the current bitstream data is recovered based on the corresponding error recovery method to obtain the data recovery result.

[0019] Therefore, this application first requires using a channel simulator and based on a start code to separate the bitstream data of each image frame in the video data into image header packets and several patch packets. The image header packets and their corresponding identifiers, as well as each patch packet and its corresponding identifier, are then encapsulated into independent network packets to be processed. Secondly, the receiving end receives each network packet to be processed and uses a preset packet loss probability model to determine whether the image header packet and each patch packet corresponding to the network packet to be processed are lost, thus obtaining the network packet reception status corresponding to each image frame. If the network packet reception status indicates that the image header packet is lost, or all patch packets are lost, then the current bitstream data is determined to be unusable. The system skips the current bitstream data. If the image header packet is not lost, and at least one patch packet is not lost, the current bitstream data corresponding to the image header packet and each surviving patch packet is written to the output bitstream. Then, the display order interval between the current frame and the previous frame in the output bitstream is determined using the decoder corresponding to AVS3. Based on the display order interval, it is determined whether the current bitstream data has frame drops due to image header packet loss, image header packet integrity with partial patch packet loss, or image header packet integrity with all patch packets loss. Finally, if any such cases exist, the current bitstream data is recovered using the corresponding error recovery method to obtain the data recovery result. In this way, the efficiency of error recovery for AVS3-oriented video codec systems is improved, thereby enhancing the user experience. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 This application discloses a flowchart of an error recovery method for a video codec system based on AVS3. Figure 2 This is a schematic diagram illustrating a specific process for processing image packages disclosed in this application; Figure 3 This is a schematic diagram of the error recovery device for an AVS3 video encoding and decoding system disclosed in this application; Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Currently, in the ultra-low latency codec mode of the AVS3 standard, video frame data passes through the channel during transmission, which may result in packet loss. Traditional packet loss handling schemes mainly focus on the loss of entire frame data, that is, detecting frame loss by detecting the POC interval in the frame sequence and covering the error by copying the previous frame. However, in actual transmission scenarios, existing technologies cannot effectively handle these finer-grained packet loss scenarios. To this end, this application provides an error recovery method for AVS3 video codec systems, which can improve the efficiency of error recovery for AVS3 video codec systems.

[0024] See Figure 1 As shown, this embodiment of the invention discloses an error recovery method for a video codec system for AVS3, including: Step S11: Using a channel simulator and based on the start code, separate the bitstream data of each image frame in the video data into image header packets and several patch packets, and encapsulate the image header packets and their corresponding identification information, as well as each patch packet and its corresponding identification information, into independent network packets to be processed.

[0025] In this embodiment, the error recovery process for the AVS3-oriented video codec system is as follows: The channel simulator supports independent packet loss simulation at both the PH and Patch levels. The decoder can detect and distinguish three scenarios: missing PH, missing partial PH patch, and missing all PH patch. When a reference frame becomes unavailable due to consecutive lost PH frames, the most recently successfully decoded frame is used to replace the missing reference frame and decoding continues. Ensure that all frames can be decoded normally in all frame loss scenarios except for the first frame loss.

[0026] It is worth mentioning that the technical solution of this application embodiment includes the following three core modules: Module 1: Fine-grained Channel Simulation: Separate frame data into PH packets and Patch packets, and perform packet loss simulation together. When the PH is lost or all Patches are lost, skip the entire frame data and do not write it into the output bitstream. When some Patches are lost, skip the data of the Patch containing the lost packet and write it into the bitstream.

[0027] Module 2: Decoder Error Detection & Concealment: PH (Profile Key) loss is detected via POC (Profile of Concept) intervals, and complete patch loss is detected through bitstream parsing. Lost frames are replaced with copies of the most recently successfully decoded frames. If the PH is not lost, and only some patches are lost, patch error mitigation techniques are used for recovery.

[0028] Module 3: Reference Frame Recovery in Scenarios of Missing Reference Frames: The most recently successfully decoded frame is retained during the DPB reference frame marking process. When the required reference frame is unavailable, the nearest available frame in the decoding buffer is used as a replacement.

[0029] In this embodiment, the channel simulation processing logic is as follows: During the packetization stage of the channel simulator (packetize function), the bitstream data of each frame is separated into independent PH packets and Patch packets according to the start code. The PH and each Patch are then divided into independent network packets according to the MTU size. PH packet: patchID = -1, indicating it is an image header packet; Patch packet: patchID = i (i is the Patch index, starting from 0), identifies the data packet of the i-th Patch; Subsequently, channel packet loss simulation was performed on the PH packet and each Patch packet together, and each packet was independently determined to be lost based on the packet loss probability model.

[0030] Specifically, using a channel simulator and based on a start code, the bitstream data of each image frame in the video data is separated into image header packets and several patch packets. The image header packets and their corresponding identifiers, as well as each patch packet and its corresponding identifier, are then encapsulated into independent network packets to be processed. This process includes: acquiring video data using a channel simulator and segmenting the bitstream data of each image frame based on a start code to obtain corresponding image header packets and several patch packets corresponding to the image header packets; configuring first identifier information for the image header packets and configuring second identifier information corresponding to the patch index of each patch packet; encapsulating the image header packets and their corresponding first identifier information, and each patch packet and its corresponding second identifier information, based on a preset transmission unit size, to obtain several network packets to be processed; wherein the network packets to be processed carrying image header packets and the network packets to be processed carrying patch packets are independent of each other at the encapsulation level.

[0031] Step S12: Receive each of the network packets to be processed using the receiving end, and use a preset packet loss probability model to determine whether the image header packet corresponding to the network packet to be processed is lost and whether each of the patch packets is lost, so as to obtain the network packet reception status corresponding to each of the image frames.

[0032] In this embodiment, the receiving end needs to analyze and process the packets after channel simulation: First, count the packet loss: traverse all received packets in each frame and record phLost (whether the PH packet is lost) and lossPatches[i] (whether the i-th patch is lost). Second, check whether all patches are lost: traverse the loss flags of all patches; if all are 1, then allPatchesLost = 1.

[0033] Specifically, the receiving end receives each network packet to be processed and uses a preset packet loss probability model to determine whether the image header packet and each patch packet corresponding to the network packet to be processed are lost, thereby obtaining the network packet reception status corresponding to each image frame. This can include: acquiring each network packet to be processed at the receiving end and identifying the network packet to be processed based on the identification information to obtain the image header packet and each patch packet; using the preset packet loss probability model and based on the image header packet to determine whether the image header packet of the current image frame is lost; if the receiving end does not receive the image header packet, it is determined that the image header packet of the current image frame is lost, and the corresponding loss determination result is obtained; using the preset packet loss probability model and based on each patch packet to determine whether the image header packet of the current image frame is lost. The system determines whether each patch packet of the current image frame is lost. If the receiver does not receive a patch packet, the corresponding patch packet is considered lost, and the corresponding loss determination result is obtained. Based on the loss determination results of each patch packet, the system determines whether all patch packets in the current image frame are lost. If all patch packets are considered lost, a full patch loss indication for the current image frame is generated. Based on the loss determination results of the image header packet, the loss determination results of each patch packet, and the full patch loss indication, network packet reception status information corresponding to the current image frame is generated. The network packet reception status information is used to characterize the transmission quality of the current image frame in channel simulation.

[0034] Step S13: If the network packet reception status indicates that the image header packet is lost, or all the patch packets are lost, then the current bitstream data is determined to be unusable and the current bitstream data is skipped. If the image header packet is not lost and at least one patch packet is not lost, then the current bitstream data corresponding to the image header packet and each unlost patch packet is written into the output bitstream.

[0035] In this embodiment, the availability of frame data needs to be determined. If phLost or allPatchesLost is true, the entire frame data is unavailable, and the frame is skipped and not written to the output bitstream. Otherwise, a PH packet is written. Then, the PatchID of the missing Patch data packet is determined. All packets of the PatchID containing the missing packet are skipped and written to the output bitstream. If a PatchID does not contain a missing packet, it is written to the bitstream normally.

[0036] The above judgment logic is as follows: PH loss: The entire frame is unusable (the decoder cannot parse the frame type, POC, RPL and other key information of the frame). All patches are missing: the entire frame is unusable (PH is present but there is no image data); The PH is complete and at least one Patch is not lost: Write the PH and the live Patch, and the decoder can decode the live Patch normally.

[0037] Specifically, if the network packet reception status indicates that the image header packet is lost, or all patch packets are lost, then the current bitstream data is determined to be unusable and skipped. If the image header packet is not lost, and at least one patch packet is not lost, then the image header packet and the current bitstream data corresponding to each unlost patch packet are written into the output bitstream. This can include: determining whether the image header packet of the current image frame is lost based on the network packet reception status information, and determining whether all patch packets of the current image frame are lost; if the image header packet of the current image frame is determined to be lost, or if all patch packets are lost, then the current bitstream data is skipped. If all h packets are lost, the current image frame's bitstream data is determined to be unusable, and the current image frame is skipped. Writing the current image frame's bitstream data to the output bitstream is then prohibited. If the current image frame's header packet is not lost, and at least one patch packet is not lost, the current image frame's bitstream data is determined to be partially usable. The bitstream data corresponding to the header packet is written to the output bitstream. Then, the bitstream data corresponding to the patch packet determined to be not lost is written to the output bitstream. The patch packets determined to be lost are skipped to prevent the corresponding bitstream data from being written to the output bitstream, thus obtaining the output bitstream.

[0038] Step S14: Use the decoder corresponding to AVS3 to determine the display order interval between the current frame and the previous frame in the output bitstream, and based on the display order interval, determine whether the current bitstream data has frame loss caused by image header packet loss, image header packet complete and part of the patch packet lost, and image header packet complete and all patch packets lost. If so, perform data recovery on the current bitstream data based on the corresponding error recovery method to obtain the data recovery result.

[0039] In this embodiment, in one specific implementation, the present application requires frame loss detection and error masking at the decoding end: After reading the PH of the current frame (start code 0xB3 or 0xB6), the Patch data (start code range 0x00~0x8E) should normally be read immediately afterwards. If the PH start code of the next frame (0xB3 or 0xB6) is encountered directly after reading the PH, it means that all Patch data of the current frame has been lost.

[0040] It's worth noting that in the decoder's main loop, after successfully decoding the PH of a frame, PH loss is detected at the POC interval. If the difference between the current frame's POC and the POC of the previously successfully decoded frame is greater than 1, then a frame's PH has been lost (the entire frame is skipped by channel emulation). When PH loss, partial PH patch loss, or complete PH patch loss is detected, different actions are taken depending on the situation: When the first frame PH is lost or the first frame Patch is completely lost, there is no available previous frame to copy, decoding fails, and error code -1 is returned.

[0041] Specifically, based on the display order interval, it is determined whether the current bitstream data has frame loss due to the loss of image header packets. If so, the current bitstream data is recovered using the corresponding error recovery method to obtain the data recovery result. This may include: reading the corresponding image frame data from the output bitstream, and performing start code detection on the bitstream data based on a preset start code set when reading the image header packet of the current image frame, to obtain the start code detection result; the start code set includes an image header start code for identifying image header packets and a patch start code for identifying patch packets; if the start code detection result indicates that the next start code read within a preset time range after reading the image header packet of the current image frame is an image header start code, it is determined that all patch packets of the current image frame have been lost, and a full patch loss indication corresponding to the current image frame is generated; if the start code detection result indicates that the image header packets of each image frame have been successfully read, the current display order number of the current image frame in the image frame sequence is determined based on the image sequence parameters carried by the image header packet of the current image frame.

[0042] Furthermore, in this embodiment, it is necessary to obtain the previous display sequence number of the previously successfully decoded image frame, and then determine the display sequence interval between the current display sequence number and the previous display sequence number. If the display sequence interval is greater than a preset consecutive number unit, it is determined that there is at least one intermediate image frame with a completely lost image header packet between the previously successfully decoded image frame and the current image frame, and an image header loss indication corresponding to the intermediate image frame is generated. The lost image frame is determined based on the full patch loss indication and the image header loss indication. If the lost image frame is the first image frame, and the image header packet of the first image frame is lost or all patch packets are lost, it is determined that the decoding has failed, and a return message indicating decoding failure is generated. If the lost image frame is not the first image frame, the image data of the previous successfully decoded image frame is obtained, and the image data of the previous successfully decoded image frame is used as the recovery image data of the lost image frame to obtain the corresponding data recovery result.

[0043] Furthermore, when the first frame's key signature (PH) is complete and not all patches are lost, the first frame's data is recovered using patch spatial error recovery technology, and the lost patch data is recovered using the successfully decoded patch data from the first frame. Secondly, in non-first frame scenarios, if PH loss or complete patch loss is detected, the data from the previous frame is directly copied for the recovery of the current frame. Alternatively, in non-first frame scenarios, if only some patch data is lost, the corresponding patch data from the previous frame is directly copied for the recovery of the current frame.

[0044] Specifically, it determines whether the current bitstream data has frame drops due to either a complete image header packet with some patch packets missing or a complete image header packet with all patch packets missing. If so, it performs data recovery on the current bitstream data based on the corresponding error recovery method to obtain the data recovery result. This may include: if the display order interval is equal to a preset consecutive numbering unit, then the current image frame is determined to be a consecutive image frame, and the first reception status of the image header packet and the second reception status of each patch packet of the current image frame are obtained; the data integrity status of the current image frame is determined based on the first reception status and the reception status; if the data integrity status indicates that the image header packet of the current image frame is not lost, and all patch packets in the current image frame have been lost, then the current image frame is determined to be in the first state of complete image header and all patch packets lost.

[0045] Furthermore, if the data integrity status indicates that the image header of the current image frame is not lost, and there is at least one unlost patch packet and at least one lost patch packet in the current image frame, then the current image frame is determined to be in the second state where the image header is intact but some patches are lost. Based on the first state, the image data of the previously successfully decoded image frame is copied, and the copy result is set as the recovered image data of the current image frame. Based on the second state, a reference patch packet with the same patch index as the lost patch packet is obtained from the previously successfully decoded image frame, and the reference patch packet is set as the recovered patch packet of the lost patch packet. Then, all unlost patch packets in the current image frame are retained. Based on the image header of the current image frame, the recovered image data, each unlost patch packet, and the recovered patch packet, the data recovery result of the current image frame is generated.

[0046] In this embodiment, the present application requires decoding recovery in the scenario of missing reference frames: In the scenario of multiple consecutive frames of PH being lost, a key problem will occur: all reference frames of the current frame are not in the decoding buffer.

[0047] In the standard reference picture marking process, the processing flow for the PH corresponding to POC75 is as follows: Figure 2 As shown: 1. Traverse the reference frames in the decoding buffer (such as POC65-69); 2. Check if each reference frame is in the RPL of POC75; 3. The RPL of POC 75 includes POC70-74; 4. POCs 65-69 are not in RPL, therefore they are marked as "no longer used" and removed from DPB; 5. Clear the decoding buffer, pm->cur_num_ref_pics=0 (the number of currently available reference frames in the decoding buffer); 6. The decoder returned an error due to the lack of a reference frame, and all subsequent frames could not be decoded.

[0048] It is worth mentioning that, during the reference frame marking process, this embodiment of the application needs to retain the most recently successfully decoded frame (the frame corresponding to last_suc_poc) in the decoding buffer, even if the frame is not in the current frame's RPL. After the reference frame marking is completed, the decoding buffer must retain at least the frame corresponding to last_suc_poc (such as POC 69), ensuring that pm->cur_num_ref_pics>=1. ​​When the reference frame specified in the RPL cannot be found in the decoding buffer, the available frame in the decoding buffer that is closest to the required reference frame is selected as the replacement.

[0049] In one specific implementation, when POC75 needs to reference POC70-74, all four reference frame positions are replaced with the only available POC69 in the decoding buffer, that is, the actual RPL becomes [L0:69 69 69 69], that is, the decoder uses POC69 as all reference frames to actually decode POC 5.

[0050] Specifically, after determining whether the current bitstream data suffers from frame loss due to missing image header packets, intact image header packets with partial patch packet loss, or intact image header packets with all patch packets loss, based on the display order interval, the process may further include: setting successfully decoded image frames corresponding to the output bitstream as reference frames and storing them in the decoded image buffer; parsing the image header packets corresponding to the current image frame to be decoded from the output bitstream based on the current image frame to be decoded, and extracting the reference frame list information of the current image frame to be decoded from the image header packets; the reference frame list information includes the display order number of the expected reference frames; determining whether the display order number of each reference frame stored in the decoded image buffer exists in the reference frame list information based on the reference frame list information; if not, marking the reference frame as unusable. The system first checks the number of available reference frames in the decoded image buffer. If the number of available reference frames is not greater than a preset threshold, the most recently successfully decoded image frame is forcibly retained in the decoded image buffer. Then, it checks whether the reference frame corresponding to the display order number of each desired reference frame in the reference frame list exists in the current decoded image buffer. If it does not exist, an available reference frame is selected from the decoded image buffer as a replacement reference frame, and the display order number of the replacement reference frame is set to the display order number of the desired reference frame, resulting in a new reference frame list. Based on the new reference frame list, the corresponding reference frame image data is obtained from the decoded image buffer, and the reference frame image data is used to decode the current image frame to obtain the data recovery result corresponding to the current image frame.

[0051] As can be seen from the above, the embodiments of this application first require using a channel simulator and based on the start code to separate the bitstream data of each image frame in the video data into image header packets and several patch packets, and then encapsulate the image header packets and their corresponding identification information, as well as each patch packet and its corresponding identification information, into independent network packets to be processed; secondly, the receiving end receives each network packet to be processed, and uses a preset packet loss probability model to determine whether the image header packet and each patch packet corresponding to the network packet to be processed are lost, thereby obtaining the network packet reception status corresponding to each image frame; if the network packet reception status indicates that the image header packet is lost, or all patch packets are lost, then it is determined that the current bitstream data is unusable. If the image header packet and at least one patch packet are not lost, the current bitstream data is skipped. If the image header packet and each surviving patch packet are not lost, the current bitstream data is written to the output bitstream. Then, the display order interval between the current frame and the previous frame in the output bitstream is determined using the decoder corresponding to AVS3. Based on the display order interval, it is determined whether the current bitstream data has frame drops due to image header packet loss, image header packet integrity and partial patch packet loss, or image header packet integrity and all patch packet loss. Finally, if any such cases exist, the current bitstream data is recovered using the corresponding error recovery method to obtain the data recovery result. In this way, the error recovery process of the AVS3-oriented video codec system is improved, thereby enhancing the user experience.

[0052] Accordingly, see Figure 3 As shown, this application also provides an error recovery device for an AVS3 video codec system, comprising: The bitstream data separation module 11 is used to separate the bitstream data of each image frame in the video data into image header packets and several patch packets based on the start code using a channel simulator, and to encapsulate the image header packets and their corresponding identification information and each patch packet and its corresponding identification information into independent network packets to be processed. The network packet judgment module 12 is used to receive each network packet to be processed by the receiving end, and use a preset packet loss probability model to determine whether the image header packet corresponding to the network packet to be processed is lost and whether each Patch packet is lost, so as to obtain the network packet receiving status corresponding to each image frame. The output bitstream generation module 13 is used to determine that the current bitstream data is unusable and skip the current bitstream data if the network packet reception status indicates that the image header packet is lost, or all the patch packets are lost; if the image header packet is not lost and at least one of the patch packets is not lost, the current bitstream data corresponding to the image header packet and each unlost patch packet is written into the output bitstream. The data recovery result determination module 14 is used to determine the display order interval between the current frame and the previous frame in the output bitstream using the decoder corresponding to AVS3, and to determine whether the current bitstream data has frame loss due to image header packet loss, image header packet being complete and part of the patch packet being lost, or image header packet being complete and all patch packets being lost, based on the display order interval. If so, the current bitstream data is recovered based on the corresponding error recovery method to obtain the data recovery result.

[0053] In some specific embodiments, the bitstream data separation module 11 may specifically include: The bitstream data separation subunit is used to acquire video data using a channel simulator and to segment the bitstream data of each image frame based on the start code to obtain the corresponding image header packet and several patch packets corresponding to the image header packet. The identification information configuration unit is used to configure first identification information for the image header packet and configure second identification information corresponding to the patch index of each patch packet for each patch packet. The network packet to be processed determination unit is used to encapsulate the image header packet with the corresponding first identification information and each patch packet with the corresponding second identification information based on a preset transmission unit size to obtain a plurality of network packets to be processed; wherein, the network packets to be processed carrying the image header packet and the network packets to be processed carrying the patch packet are independent of each other at the encapsulation level.

[0054] In some specific embodiments, the network packet determination module 12 may specifically include: A network packet identification unit is used to acquire each network packet to be processed at the receiving end, and identify the network packet to be processed based on the identification information to obtain the image header packet and each patch packet; The first loss determination result generation unit is used to determine whether the image header packet of the current image frame is lost based on the image header packet using a preset packet loss probability model. If the receiving end does not receive the image header packet, it is determined that the image header packet of the current image frame is lost, and the corresponding loss determination result is obtained. The second loss determination result generation unit is used to determine whether each of the Patch packets of the current image frame is lost based on the preset packet loss probability model and each Patch packet. If the receiving end does not receive the Patch packet, it determines that the corresponding Patch packet is lost and obtains the corresponding loss determination result. The first loss indication generation unit is used to determine whether all Patch packets in the current image frame are lost based on the loss determination results of each Patch packet. If all Patch packets are determined to be lost, a full Patch loss indication for the current image frame is generated. The network packet reception status information determination unit is used to generate network packet reception status information corresponding to the current image frame based on the loss determination result of the image header packet, the loss determination result of each patch packet, and the full patch loss indication; the network packet reception status information is used to characterize the transmission quality of the current image frame in channel simulation.

[0055] In some specific embodiments, the output stream generation module 13 may specifically include: The image header packet determination unit is used to determine whether the image header packet of the current image frame is lost based on the network packet reception status information, and to determine whether all Patch packets of the current image frame are lost. The bitstream data writing unit is used to determine that the bitstream data of the current image frame is unavailable if it is determined that the image header packet of the current image frame is lost, or if it is determined that all the patch packets are lost, and then skip the current image frame and prevent the bitstream data of the current image frame from being written to the output bitstream. The output bitstream generation unit is configured to determine that the bitstream data portion of the current image frame is available if it is determined that the image header packet of the current image frame is not lost and at least one patch packet is not lost. The unit then writes the bitstream data corresponding to the image header packet to the output bitstream, writes the bitstream data corresponding to the patch packet that is determined to be not lost to the output bitstream, and skips the patch packets that are determined to be lost, so as to prevent the corresponding bitstream data from being written to the output bitstream, thereby obtaining the output bitstream.

[0056] In some specific embodiments, the data recovery result determination module 14 may specifically include: The image frame data reading unit is used to read the corresponding image frame data from the output bitstream, and when the image header packet of the current image frame is read, to perform start code detection on the bitstream data based on a preset start code set to obtain the start code detection result; the start code set includes an image header start code for identifying the image header packet and a patch start code for identifying the patch packet; The second loss indication generation unit is used to determine that all the patch packets of the current image frame have been lost if the start code detection result indicates that the next start code read within a preset time range after reading the image header packet of the current image frame is the image header start code, and to generate a full patch loss indication corresponding to the current image frame. The display order interval determination unit is used to determine the current display order number of the current image frame in the image frame sequence based on the image sequence parameters carried by the image header packet of the current image frame when the start code detection result indicates that the image header packet of each image frame has been successfully read. It also obtains the previous display order number of the last successfully decoded image frame and then determines the display order interval between the current display order number and the previous display order number. The third loss indication generation unit is used to determine, if the display order interval is greater than a preset consecutive numbering unit, that there is at least one intermediate image frame between the previous successfully decoded image frame and the current image frame where the image header packet is completely lost, and to generate an image header loss indication corresponding to the intermediate image frame. The missing image frame determination unit is used to determine the missing image frame based on the full patch loss indication and the image header loss indication; The return information generation unit is used to determine that decoding has failed if the lost image frame is the first image frame and the image header packet or all patch packets of the first image frame are lost, and to generate return information indicating decoding failure. The data recovery result determination unit is used to obtain the image data of the previous successfully decoded image frame if the lost image frame is not the first image frame, and use the image data of the previous successfully decoded image frame as the recovered image data of the lost image frame to obtain the corresponding data recovery result.

[0057] In some specific embodiments, the data recovery result determination module 14 may specifically include: A receiving status generation unit is used to determine that the current image frame is a continuous image frame if the display order interval is equal to the preset continuous number unit, and to obtain the first receiving status of the image header packet of the current image frame and the second receiving status of each Patch packet. A data integrity status determination unit is used to determine the data integrity status of the current image frame based on the first receiving status and the receiving status. The first state generation unit is used to determine that the current image frame is in the first state of complete image header and all patches lost if the data integrity state indicates that the image header packet of the current image frame is not lost and all patch packets of the current image frame are lost. The second state generation unit is used to determine that the current image frame is in a second state where the image header is complete and some patches are missing if the data integrity state indicates that the image header packet of the current image frame is not lost, and there is at least one unlost patch packet and at least one lost patch packet in the current image frame. The image data recovery determination unit is used to copy the image data of the previously successfully decoded image frame based on the first state, and set the copy result as the recovered image data of the current image frame; The reference patch packet acquisition unit is used to acquire a reference patch packet with the same patch index as the lost patch packet from the previously successfully decoded image frame based on the second state, set the reference patch packet as the recovery patch packet of the lost patch packet, and then retain all unlost patch packets in the current image frame. The data recovery result determination subunit is used to generate the data recovery result of the current image frame based on the image header packet of the current image frame, the recovered image data, each of the unlost patch packets and the recovered patch packet.

[0058] In some specific embodiments, the error recovery device for the AVS3-oriented video codec system may further include: A reference frame storage unit is used to set successfully decoded image frames corresponding to the output bitstream as reference frames and store them in the decoded image buffer; The reference frame list information extraction unit is used to parse the image header packet corresponding to the current image frame to be decoded from the output bitstream based on the current image frame to be decoded, and extract the reference frame list information of the current image frame to be decoded from the image header packet; the reference frame list information includes the display order number of the expected reference frame; The image removal unit is used to determine whether the display order number of each reference frame stored in the decoded image buffer exists in the reference frame list information based on the reference frame list information. If it does not exist, the reference frame is marked as an unusable reference frame and the unusable reference frame is removed from the decoded image buffer. The reference frame number determination unit is used to detect the number of available reference frames in the decoded image buffer. If the number of reference frames is not greater than a preset threshold, the most recently successfully decoded image frame is forcibly retained in the decoded image buffer. The alternative reference frame selection unit is used to check whether the reference frame corresponding to the display order number of each expected reference frame in the reference frame list information exists in the current decoded image buffer. If it does not exist, an available reference frame is selected from the decoded image buffer as an alternative reference frame, and the display order number of the alternative reference frame is set to the display order number of the expected reference frame to obtain a new reference frame list. The data recovery result determination subunit is used to obtain the corresponding reference frame image data from the decoded image buffer based on the new reference frame list, and to use the reference frame image data to decode the current image frame to obtain the data recovery result corresponding to the current image frame.

[0059] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the error recovery method for the AVS3 video codec system disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be a computer.

[0060] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0061] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0062] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the error recovery method for the AVS3 video codec system executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0063] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned error recovery method for an AVS3-oriented video codec system. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0064] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0065] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0066] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0067] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0068] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An error recovery method for a video codec system based on AVS3, characterized in that, include: Using a channel simulator and based on the start code, the bitstream data of each image frame in the video data is separated into image header packets and several patch packets. The image header packets and their corresponding identification information, as well as each patch packet and its corresponding identification information, are encapsulated into independent network packets to be processed. The receiving end receives each of the network packets to be processed, and uses a preset packet loss probability model to determine whether the image header packet corresponding to the network packet to be processed is lost and whether each of the patch packets is lost, so as to obtain the network packet reception status corresponding to each of the image frames. If the network packet reception status indicates that the image header packet is lost, or all the patch packets are lost, then the current bitstream data is determined to be unusable and the current bitstream data is skipped. If the image header packet is not lost and at least one patch packet is not lost, then the current bitstream data corresponding to the image header packet and each unlost patch packet is written into the output bitstream. The display order interval between the current frame and the previous frame in the output bitstream is determined using the decoder corresponding to AVS3. Based on the display order interval, it is determined whether the current bitstream data has frame loss due to image header packet loss, image header packet being complete but some patch packets being lost, or image header packet being complete but all patch packets being lost. If so, the current bitstream data is recovered based on the corresponding error recovery method to obtain the data recovery result.

2. The error recovery method for a video codec system for AVS3 according to claim 1, characterized in that, The method involves using a channel simulator and based on a start code to separate the bitstream data of each image frame in the video data into image header packets and several patch packets. The image header packets and their corresponding identifiers, as well as each patch packet and its corresponding identifier, are then encapsulated into independent network packets to be processed, including: Video data is acquired using a channel simulator, and the bitstream data of each image frame is segmented based on the start code to obtain the corresponding image header packet and several patch packets corresponding to the image header packet. Configure first identification information for the image header packet, and configure second identification information corresponding to the patch index of each patch packet for each patch packet; Based on a preset transmission unit size, the image header packet and the corresponding first identification information, and each patch packet and the corresponding second identification information are encapsulated to obtain several network packets to be processed; wherein, the network packets to be processed carrying the image header packet and the network packets to be processed carrying the patch packet are independent of each other at the encapsulation level.

3. The error recovery method for a video codec system for AVS3 according to claim 1, characterized in that, The step of receiving each of the network packets to be processed at the receiving end and determining whether the image header packet corresponding to each network packet to be processed is lost and whether each of the patch packets is lost using a preset packet loss probability model, to obtain the network packet reception status corresponding to each image frame, includes: At the receiving end, each of the network packets to be processed is acquired, and the network packets to be processed are identified based on the identification information to obtain the image header packet and each of the patch packets; Using a preset packet loss probability model and based on the image header packet, it is determined whether the image header packet of the current image frame is lost. If the receiving end does not receive the image header packet, it is determined that the image header packet of the current image frame is lost, and the corresponding loss determination result is obtained. The preset packet loss probability model is used to determine whether each of the patch packets in the current image frame is lost based on each patch packet. If the receiving end does not receive the patch packet, the corresponding patch packet is determined to be lost, and the corresponding loss determination result is obtained. Based on the loss determination results of each Patch packet, determine whether all Patch packets in the current image frame are lost. If all Patch packets are determined to be lost, generate a full Patch loss indication for the current image frame. Based on the loss determination results of the image header packet, the loss determination results of each patch packet, and the full patch loss indication, network packet reception status information corresponding to the current image frame is generated; the network packet reception status information is used to characterize the transmission quality of the current image frame in channel simulation.

4. The error recovery method for a video codec system for AVS3 according to claim 1, characterized in that, If the network packet reception status indicates that the image header packet is lost, or all the patch packets are lost, then the current bitstream data is determined to be unusable and is skipped. If the image header packet is not lost, and at least one patch packet is not lost, then the current bitstream data corresponding to the image header packet and each unlost patch packet is written into the output bitstream, including: Based on the network packet reception status information, determine whether the image header packet of the current image frame is lost, and determine whether all patch packets of the current image frame are lost. If it is determined that the image header packet of the current image frame is lost, or if it is determined that all the patch packets are lost, then the bitstream data of the current image frame is determined to be unusable, the current image frame is skipped, and then writing the bitstream data of the current image frame into the output bitstream is prohibited. If it is determined that the image header packet of the current image frame is not lost, and at least one patch packet is not lost, then the bitstream data portion of the current image frame is determined to be available, and the bitstream data corresponding to the image header packet is written to the output bitstream. Then, the bitstream data corresponding to the patch packet that is determined to be not lost is written to the output bitstream, and the patch packet that is determined to be lost is skipped to prevent the corresponding bitstream data from being written to the output bitstream, thus obtaining the output bitstream.

5. The error recovery method for a video codec system for AVS3 according to claim 1, characterized in that, Based on the display order interval, it is determined whether the current bitstream data has frame drops due to image header packet loss. If so, data recovery is performed on the current bitstream data using the corresponding error recovery method to obtain the data recovery result, including: The corresponding image frame data is read from the output bitstream, and when the image header packet of the current image frame is read, the bitstream data is subjected to start code detection based on a preset start code set to obtain the start code detection result; the start code set includes an image header start code for identifying the image header packet and a patch start code for identifying the patch packet; If the start code detection result indicates that after reading the image header packet of the current image frame, the next start code read within a preset time range is the image header start code, it is determined that all the patch packets of the current image frame have been lost, and a full patch loss indication corresponding to the current image frame is generated. If the start code detection result indicates that the image header packet of each image frame has been successfully read, the current display order number of the current image frame in the image frame sequence is determined based on the image sequence parameters carried by the image header packet of the current image frame, and the previous display order number of the last successfully decoded image frame is obtained. Then, the display order interval between the current display order number and the previous display order number is determined. If the display sequence interval is greater than a preset consecutive number unit, it is determined that there is at least one intermediate image frame with a completely lost image header packet between the previous successfully decoded image frame and the current image frame, and an image header loss indication corresponding to the intermediate image frame is generated. The lost image frames are determined based on the full patch loss indication and the image header loss indication; If the lost image frame is the first image frame, and the image header packet of the first image frame is lost or all patch packets are lost, then decoding failure is determined, and a return message indicating decoding failure is generated. If the lost image frame is not the first image frame, then the image data of the previous successfully decoded image frame is obtained, and the image data of the previous successfully decoded image frame is used as the recovered image data of the lost image frame to obtain the corresponding data recovery result.

6. The error recovery method for a video codec system for AVS3 according to claim 5, characterized in that, Determine whether the current bitstream data suffers from frame drops due to either a complete image header packet with partial or complete patch packet loss, or a complete image header packet with all patch packets lost. If so, perform data recovery on the current bitstream data using the corresponding error recovery method to obtain the data recovery result, including: If the display order interval is equal to the preset consecutive numbering unit, then the current image frame is determined to be a consecutive image frame, and the first receiving status of the image header packet of the current image frame and the second receiving status of each Patch packet are obtained; The data integrity status of the current image frame is determined based on the first reception status and the reception status. If the data integrity status indicates that the image header packet of the current image frame is not lost, and all patch packets in the current image frame are lost, then the current image frame is determined to be in the first state where the image header is intact but all patches are lost. If the data integrity status indicates that the image header packet of the current image frame is not lost, and there is at least one unlost patch packet and at least one lost patch packet in the current image frame, then the current image frame is determined to be in the second state where the image header is complete and some patches are lost. Based on the first state, copy the image data of the previously successfully decoded image frame, and set the copy result as the recovered image data of the current image frame; Based on the second state, a reference patch packet with the same patch index as the lost patch packet is obtained from the previously successfully decoded image frame, and the reference patch packet is set as the recovery patch packet of the lost patch packet. Then, all unlost patch packets in the current image frame are retained. The data recovery result of the current image frame is generated based on the image header packet of the current image frame, the recovered image data, each of the unlost patch packets, and the recovery patch packet.

7. The error recovery method for a video codec system for AVS3 according to claim 1, characterized in that, After determining, based on the display order interval, whether the current bitstream data suffers from frame drops due to lost image header packets, intact image header packets with partial patch packet loss, or intact image header packets with all patch packets loss, the process further includes: The successfully decoded image frame corresponding to the output bitstream is set as a reference frame and stored in the decoded image buffer; Based on the current image frame to be decoded, the image header packet corresponding to the current image frame to be decoded is parsed from the output bitstream, and the reference frame list information of the current image frame to be decoded is extracted from the image header packet; the reference frame list information includes the display order number of the expected reference frame; Based on the reference frame list information, determine whether the display order number of each reference frame stored in the decoded image buffer exists in the reference frame list information. If it does not exist, mark the reference frame as an unusable reference frame and remove the unusable reference frame from the decoded image buffer. The number of available reference frames in the decoded image buffer is detected. If the number of reference frames is not greater than a preset threshold, the most recently successfully decoded image frame is forcibly retained in the decoded image buffer. Check whether the reference frame corresponding to the display order number of each expected reference frame in the reference frame list information exists in the current decoded image buffer. If it does not exist, select an available reference frame from the decoded image buffer as a replacement reference frame, and set the display order number of the replacement reference frame to the display order number of the expected reference frame to obtain a new reference frame list. Based on the new reference frame list, the corresponding reference frame image data is obtained from the decoded image buffer, and the reference frame image data is used to decode the current image frame to obtain the data recovery result corresponding to the current image frame.

8. An error recovery device for a video codec system for AVS3, characterized in that, include: The bitstream data separation module is used to separate the bitstream data of each image frame in the video data into image header packets and several patch packets based on the start code using a channel simulator. The image header packets and their corresponding identification information, as well as each patch packet and its corresponding identification information, are encapsulated into independent network packets to be processed. The network packet judgment module is used to receive each network packet to be processed by the receiving end, and use a preset packet loss probability model to determine whether the image header packet corresponding to the network packet to be processed is lost and whether each Patch packet is lost, so as to obtain the network packet reception status corresponding to each image frame. The output bitstream generation module is used to determine that the current bitstream data is unusable and skip the current bitstream data if the network packet reception status indicates that the image header packet is lost, or if all the patch packets are lost; if the image header packet is not lost and at least one of the patch packets is not lost, the current bitstream data corresponding to the image header packet and each unlost patch packet is written into the output bitstream. The data recovery result determination module is used to determine the display order interval between the current frame and the previous frame in the output bitstream using the decoder corresponding to AVS3, and to determine whether the current bitstream data has frame loss due to image header packet loss, image header packet being complete and part of the patch packet being lost, or image header packet being complete and all patch packets being lost, based on the display order interval. If so, the current bitstream data is recovered based on the corresponding error recovery method to obtain the data recovery result.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the error recovery method for a video codec system for AVS3 as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the error recovery method for a video codec system for AVS3 as described in any one of claims 1 to 7.

Images