Intra error recovery method and device for avs3 key frame, equipment and medium

Abstract

The application discloses an intra-frame error recovery method and device of an AVS3 key frame, equipment and a medium, relates to the technical field of video coding, and comprises the following steps: acquiring target key frame data sent by an encoding end; performing integrity detection on the target key frame data; if there is a missing slice in the target key frame, acquiring a luminance sketch in the image header of the target key frame; performing error concealment on the target key frame to acquire a corresponding intermediate key frame; performing up-sampling on a target sketch block in the luminance sketch to acquire a corresponding up-sampled sketch block; and fusing the up-sampled sketch block and a target key frame block in the intermediate key frame to supplement missing data of the target key frame, so that intra-frame error recovery is realized. By fusing the luminance sketch and the intermediate key frame, the problem that the overall luminance of the reconstruction result is greatly deviated from the original content is solved.

Description

Technical Field

[0001] This invention relates to the field of video encoding and decoding technology, and in particular to methods, apparatus, devices and media for intra-frame error recovery of AVS3 keyframes. Background Technology

[0002] In the AVS3 video codec standard, the ultra-low latency coding mode is designed for real-time video communication scenarios (such as video conferencing, telemedicine, cloud gaming, etc.), requiring extremely low end-to-end latency. In this mode, video frames are divided into multiple independently coded patches, each of which can be independently encapsulated into one or more network data packets, achieving subframe-level transmission and decoding. In unreliable network transmission environments, some data packets may fail to arrive at the decoding end on time due to packet loss, latency jitter, or other reasons.

[0003] Currently, pure spatial interpolation is commonly used to fill in missing data and correct errors within keyframes. However, this method can lead to a significant deviation between the overall brightness of the reconstructed result and the original content. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for intra-frame error recovery of AVS3 keyframes, which solves the problem of large deviation between the overall brightness of the reconstructed result and the original content by fusing the brightness sketch with the intermediate keyframes. The specific solution is as follows:

[0005] Firstly, this application provides an intra-frame error recovery method for AVS3 keyframes, applied at the decoding end, including:

[0006] The target keyframe data is obtained by the encoding end based on the AVS3 standard; wherein the target keyframe data is obtained by the encoding end downsampling the luminance component of the current keyframe, generating a corresponding luminance sketch, embedding the luminance sketch into the image header of the current keyframe to obtain the target keyframe, and then slicing the target keyframe; the resolution of the luminance sketch is lower than the resolution of the current keyframe.

[0007] Integrity checks are performed on the target keyframe data. If a missing slice exists in the target keyframe, the brightness sketch in the image header of the target keyframe is obtained.

[0008] The target keyframe is processed using video image error concealment technology to obtain the corresponding intermediate keyframe. The target sketch block in the brightness sketch corresponding to the missing slice is upsampled to obtain the corresponding upsampled sketch block. The upsampled sketch block and the target keyframe block in the intermediate keyframe corresponding to the missing slice are fused to supplement the missing data of the target keyframe and realize intra-frame error recovery. The resolution of the upsampled sketch is the same as the resolution of the missing slice.

[0009] Optionally, the process of generating the luminance sketch at the encoding end and embedding the luminance sketch into the image header of the current keyframe includes:

[0010] The luminance component of the current keyframe is downsampled and mean filtered according to a preset downsampling ratio to obtain the luminance sketch.

[0011] The brightness sketch is embedded into the image header of the current keyframe using horizontal differential pulse modulation and exponential Golomb coding techniques.

[0012] Optionally, obtaining the brightness sketch from the image header of the target keyframe includes:

[0013] The flag bits in the image header are parsed. If the flag bits indicate that the brightness sketch exists in the image header, the brightness sketch is obtained from the image header using horizontal differential pulse modulation and exponential Golomb coding techniques.

[0014] Optionally, the processing of the target keyframe based on video image error concealment technology to obtain the corresponding intermediate keyframes includes:

[0015] Determine the target position of the missing data corresponding to the missing slice in the target keyframe, and use the pixel values ​​in the neighborhood of the target position to perform spatial interpolation on the target keyframe to obtain the intermediate keyframe.

[0016] Optionally, the fusion of the upsampled sketch patch and the target keyframe patch in the intermediate keyframe corresponding to the missing slice includes:

[0017] The target fusion weight corresponding to the target keyframe image block is determined based on the number of neighborhoods corresponding to the target location, and the upsampled sketch and the intermediate keyframe are weighted and fused according to the target fusion weight.

[0018] Optionally, the number of neighborhoods corresponding to the target location is positively correlated with the magnitude of the target fusion weight.

[0019] Optionally, the intra-frame error recovery method for AVS3 keyframes further includes:

[0020] Acquire video frame data sent by the encoding end based on the AVS3 standard; wherein, the video frame data is the data obtained by the encoding end after slicing several video frames;

[0021] Determine whether a target video frame exists in the video frame data; wherein, the target video frame is a non-critical frame with missing slices;

[0022] If the target video frame exists in the video frame data, then determine the first position of the missing data corresponding to the missing slice in the target video frame, and obtain the reference frame corresponding to the target video frame from the video frame data;

[0023] A second position corresponding to the first position in the reference frame is determined, and the pixels of the second position are copied to the first position to complete the intra-frame error recovery of the target video frame.

[0024] Secondly, this application provides an intra-frame error recovery device for AVS3 keyframes, applied at the decoding end, comprising:

[0025] A keyframe data acquisition module is used to acquire target keyframe data sent by the encoding end based on the AVS3 standard; wherein, the target keyframe data is obtained by the encoding end downsampling the luminance component of the current keyframe to generate a corresponding luminance sketch, embedding the luminance sketch into the image header of the current keyframe to obtain the target keyframe, and then slicing the target keyframe; the resolution of the luminance sketch is lower than the resolution of the current keyframe;

[0026] The brightness sketch acquisition module is used to perform integrity detection on the target keyframe data. If there are missing slices in the target keyframe, the brightness sketch in the image header of the target keyframe is acquired.

[0027] The data fusion module is used to process the target keyframe based on video image error concealment technology to obtain the corresponding intermediate keyframe, upsample the target sketch block in the brightness sketch corresponding to the missing slice to obtain the corresponding upsampled sketch block, and fuse the upsampled sketch block and the target keyframe block in the intermediate keyframe corresponding to the missing slice to supplement the missing data of the target keyframe and realize intra-frame error recovery; wherein, the resolution of the upsampled sketch is the same as the resolution of the missing slice.

[0028] Thirdly, this application provides an electronic device, comprising:

[0029] Memory, used to store computer programs;

[0030] A processor for executing the computer program to implement the aforementioned intra-frame error recovery method for AVS3 keyframes.

[0031] Fourthly, this application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned intra-frame error recovery method for AVS3 keyframes.

[0032] This application first acquires target keyframe data sent by the encoding end based on the AVS3 standard. The target keyframe data is obtained by downsampling the luminance component of the current keyframe to generate a corresponding luminance sketch, embedding the luminance sketch into the image header of the current keyframe to obtain the target keyframe, and then slicing the target keyframe. The resolution of the luminance sketch is lower than that of the current keyframe. Then, the target keyframe data undergoes integrity detection. If a missing slice exists in the target keyframe, the luminance sketch in the image header of the target keyframe is acquired. Finally, the target keyframe is processed using video image error concealment technology to obtain corresponding intermediate keyframes. The target sketch block in the luminance sketch corresponding to the missing slice is upsampled to obtain a corresponding upsampled sketch block. The upsampled sketch block and the target keyframe block in the intermediate keyframe corresponding to the missing slice are fused to supplement the missing data in the target keyframe and achieve intra-frame error recovery. As can be seen, this application generates a low-resolution brightness sketch by downsampling the brightness components of keyframes at the encoding end and embeds it into the image header, providing the decoding end with prior information on global brightness and low-frequency structure. By obtaining this sketch and upsampling it when a slice is missing at the decoding end, and then fusing it with the error hiding result, the restoration process no longer relies solely on local neighborhood interpolation, but introduces overall brightness distribution constraints, thereby effectively correcting the brightness deviation caused by pure spatial interpolation and solving the problem of large deviation between the overall brightness of the reconstruction result and the original content. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of the intra-frame error recovery method for AVS3 keyframes disclosed in this application;

[0035] Figure 2 This is a schematic diagram of the structure of an intra-frame error recovery device for AVS3 keyframes disclosed in this application;

[0036] Figure 3 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] See Figure 1 As shown, this embodiment of the invention discloses an intra-frame error recovery method for AVS3 keyframes, applied to the decoding end, including:

[0039] Step S11: Obtain target keyframe data sent by the encoding end based on the AVS3 standard; wherein, the target keyframe data is obtained by the encoding end downsampling the luminance component of the current keyframe, generating a corresponding luminance sketch, embedding the luminance sketch into the image header of the current keyframe to obtain the target keyframe, and then slicing the target keyframe; the resolution of the luminance sketch is lower than the resolution of the current keyframe.

[0040] In this embodiment, the post-processing filter entry function ec_post_filter_lost_patches() first performs the following condition determination:

[0041] Frame type check: Post-processing is only performed on I-frames (i.e., keyframes) (slice_type == SLICE_I). P / B frame loss patches are recovered using temporal copying, eliminating over-smoothing artifacts caused by spatial interpolation and requiring no post-processing.

[0042] Missing Patch Check: Traverse all Patch indices and use the array to determine which Patches were missing in the current frame and have undergone error hiding.

[0043] Patch information acquisition: Calculate the pixel-level boundary coordinates (left_pel, up_pel, right_pel, down_pel) of the patch based on its row and column position in the grid, and perform error recovery based on these coordinates.

[0044] That is, in this embodiment, it is also possible to acquire video frame data sent by the encoding end based on the AVS3 standard; wherein, the video frame data is the data obtained by the encoding end after slicing several video frames; determine whether there is a target video frame in the video frame data; wherein, the target video frame is a non-critical frame with missing slices; if the target video frame exists in the video frame data, determine the first position of the missing data corresponding to the missing slice in the target video frame, and obtain the reference frame corresponding to the target video frame from the video frame data; determine the second position in the reference frame corresponding to the first position, and copy the pixels of the second position to the first position to complete the intra-frame error recovery of the target video frame.

[0045] That is, in this embodiment, it will first determine whether the acquired video frame data is keyframe data. If so, it will be processed using a luminance sketch. If not, it will be directly restored using a temporal copy method.

[0046] In addition, in this embodiment, the process of generating a luminance sketch at the encoding end and embedding the luminance sketch into the image header of the current key frame includes: performing downsampling and mean filtering on the luminance component of the current key frame according to a preset downsampling ratio to obtain a luminance sketch; and embedding the luminance sketch into the image header of the current key frame using horizontal differential pulse modulation technology and exponential Golomb coding technology.

[0047] In other words, this embodiment proposes a method based on low-resolution sketch guidance. The encoder embeds an extremely low-resolution luminance sketch in the picture header, providing a global structural prior for error hiding in the decoder. After spatial weighted interpolation and fusion with the sketch, the reconstructed region of the missing patch in the I-frame is generated, significantly improving the visual quality of the error-hidden region. The encoder downsamples the original I-frame image by 1 / 16 (i.e., a preset downsampling ratio) to generate a low-resolution luminance sketch, which is then embedded in the picture header and transmitted via horizontal DPCM + signed exponential Golomb coding. After spatial weighted interpolation, the decoder bilinearly upsamples the sketch to the full patch resolution and adaptively fuses the interpolation result with the sketch based on the number of effective neighbors, providing a basic reconstruction with correct DC and low-frequency structure for subsequent filtering.

[0048] It should be noted that spatial weighted interpolation only uses distance-weighted averaging of the boundary pixels of the neighboring patch. When the number of effective neighbors is insufficient (e.g., only 2-3 directional neighbors are available for a corner patch), the reconstruction quality is severely degraded. More importantly, pure spatial interpolation completely lacks prior knowledge of the overall brightness level (DC component) and low-frequency structure (large-scale brightness distribution) of the lost region, which may lead to significant deviations between the reconstruction results and the original content in terms of brightness.

[0049] Therefore, in this embodiment, when encoding an I-frame, the encoding end downsamples the luminance component of the original image to generate a low-resolution luminance sketch with a resolution of 1 / S (S=EC_SKETCH_SUBSAMPLE=16).

[0050] sw = pic_w / S (Sketch width);

[0051] sh = pic_h / S (Sketch height);

[0052] For each sketch pixel (sx, sy), calculate the arithmetic mean of its corresponding S×S original pixel block. This operation is equivalent to an S×S mean filter + S times downsampling, which preserves the DC and low-frequency information of the original image, while compressing the data volume to 1 / S² (i.e. 1 / 256) of the original brightness.

[0053] Sketch data is embedded in the picture header and compressed using horizontal DPCM (Differential Pulse Code Modulation) + signed exponential Golomb coding. For each row of sketch pixels, the prediction residual is calculated using the left neighboring pixels as the predicted value. The DPCM residual is then written to the bitstream using signed exponential Golomb coding via the `com_bsw_write_se()` function. This coding method uses shorter codewords for small residual values ​​close to zero and longer codewords for larger residuals, making it well-suited to the Laplace distribution characteristics of DPCM residuals. A 1-bit `ec_sketch_enable_flag` is written before the sketch data to indicate whether the current picture header contains sketch data. Only I-frames transmit the sketch.

[0054] Furthermore, before the intra-frame error recovery process officially begins, this embodiment also requires modifications to the AVS3 standard syntax. The key modified syntax is as follows:

[0055] picture_header() { / type /

[0056] ...(existing syntax elements)...

[0057] if ( slice_type == SLICE_I ) {

[0058] ec_sketch_enable_flag u(1)

[0059] if (ec_sketch_enable_flag) {

[0060] ec_sketch_data()

[0061] }

[0062] }

[0063] stuffing_bit f(1)

[0064] while (!byte_aligned()) {

[0065] stuffing_zero_bit f(1)

[0066] }

[0067] }

[0068] ec_sketch_data() { / Type /

[0069] SketchWidth = (horizontal_size + 15) >> 4

[0070] SketchHeight = (vertical_size + 15) >> 4

[0071] for (y = 0; y < SketchHeight; y++) {

[0072] prev = 0

[0073] for (x = 0; x < SketchWidth; x++) {

[0074] sketch_luma_residual[y][x] se(v)

[0075] }

[0076] }

[0077] }

[0078] The `ec_sketch_enable_flag` (i.e., the flag) syntax element only appears when `slice_type == SLICE_I`. When it equals 1, it indicates that the current image header carries 1 / 16 spatial resolution luminance submap data, which can be used by the decoder to assist in error masking recovery when a patch is lost. When it equals 0, the current image header does not carry submap data.

[0079] SketchWidth and SketchHeight: The width and height of the sub-image, derived from the image dimensions in the sequence header, do not need to be explicitly transferred.

[0080] `sketch_luma_residual[y][x]`: The horizontal DPCM residual of the luminance pixels in the sub-image, encoded in signed exponential Golomb code (se(v)). Each row is predicted independently, with the first row predicted as 0 and the remaining rows predicted as the reconstructed values ​​of their left neighbors. The residual values ​​are integers, and the reconstructed values ​​must be cropped to the nearest integer. .

[0081] Step S12: Perform integrity detection on the target keyframe data. If there are missing slices in the target keyframe, obtain the brightness sketch in the image header of the target keyframe.

[0082] The decoding process at the decoding end in this embodiment is as follows:

[0083] enter:

[0084] sketch_luma_residual[ y ][ x ] / Residual read from the bitstream /

[0085] SketchWidth, SketchHeight

[0086] BitDepth

[0087] MaxVal = ( 1 << BitDepth ) - 1

[0088] Output:

[0089] SketchBuf[y][x] / Reconstructed subimage brightness pixels /

[0090] process:

[0091] for y = 0 .. SketchHeight - 1

[0092] prev = 0

[0093] for x = 0 .. SketchWidth - 1

[0094] cur = prev + sketch_luma_residual[ y ][ x ]

[0095] SketchBuf[ y ][ x ] = Clip3( 0, MaxVal, cur )

[0096] prev = SketchBuf[ y ][ x ]

[0097] endfor;

[0098] endfor;

[0099] In this embodiment, obtaining the luminance sketch in the image header of the target keyframe includes: parsing the flag bits in the image header; if the flag bits indicate the presence of a luminance sketch in the image header, then the luminance sketch is obtained from the image header using horizontal differential pulse modulation and exponential Golomb coding techniques. When the flag bit is equal to 1, it indicates that the current image header carries 1 / 16 spatial resolution luminance sub-map data, which can be used by the decoding end to assist in error masking and recovery when a patch is lost.

[0100] Specifically, when parsing the image header, the decoding end performs the reverse process of the sketch data.

[0101] 1. Read ec_sketch_enable_flag (1-bit);

[0102] 2. If the flag is 1, calculate the sketch dimensions (sw, sh) based on the image width and height in the sequence header;

[0103] 3. Read the signed exponential Golomb-encoded DPCM residuals line by line to obtain a brightness sketch.

[0104] Step S13: Process the target keyframe based on video image error hiding technology to obtain the corresponding intermediate keyframe. Upsample the target sketch block in the brightness sketch corresponding to the missing slice to obtain the corresponding upsampled sketch block. Then, fuse the upsampled sketch block and the target keyframe block in the intermediate keyframe corresponding to the missing slice to supplement the missing data of the target keyframe and realize intra-frame error recovery. The resolution of the upsampled sketch is the same as the resolution of the missing slice.

[0105] In this embodiment, the target keyframe is processed based on video image error hiding technology to obtain the corresponding intermediate keyframe, including: determining the target position of the missing data corresponding to the missing slice in the target keyframe, and using the pixel values ​​in the neighborhood of the target position to perform spatial interpolation on the target keyframe to obtain the intermediate keyframe.

[0106] In other words, in the case of a missing frame patch: since there is no reference frame available for the I-frame, a spatial weighted interpolation error hiding method (i.e. video image error hiding technology) is adopted. The boundary pixels of the neighboring patches that have been correctly decoded in 8 directions (up, down, left, right, upper left, upper right, lower left, lower right) are used to fill the missing area (i.e. the target position of the missing data in the target keyframe) through distance weighted interpolation (to obtain the intermediate keyframe).

[0107] Then, the pixel values ​​of the original sketch are accumulated and restored, and cropped to the valid range [0, max_val];

[0108] When performing error hiding on a missing patch at the decoding end, the low-resolution sketch is upsampled to the full resolution size of the patch using bilinear interpolation (to obtain intermediate keyframes).

[0109] In addition, the upsampled sketch block and the target keyframe block in the intermediate keyframe corresponding to the missing slice are fused, including: determining the target fusion weight corresponding to the target keyframe image block according to the number of neighborhoods corresponding to the target position, and performing weighted fusion of the upsampled sketch and the intermediate keyframe according to the target fusion weight.

[0110] It should be noted that the number of neighborhoods corresponding to the target location is positively correlated with the target fusion weight.

[0111] Specifically, the spatially weighted interpolation result is fused with the upsampled sketch using a weighted method. The fusion weight is adaptively determined by the number of effective neighboring patches, which ranges from 0 to 8. The core idea of ​​this adaptive strategy is that the more neighbors there are, the more reliable the spatial interpolation, and therefore the higher the weight should be. When there are insufficient neighbors, the global DC and low-frequency structure provided by the sketch become more reliable priors and should be given higher weights. However, the sketch weight is always kept above 0.5, as prior knowledge reflects the actual effect better than interpolated pixels, but it still needs to be dynamically adjusted based on the correct pixels of the surrounding patches. It should be noted that sketch guidance only applies to the luma component (Y). The chroma component is not fused because the sketch does not contain chroma information.

[0112] The code that merges the intermediate keyframes and the upsampled sketch is as follows:

[0113] Subgraph-assisted missing patch brightness restoration:

[0114] enter:

[0115] P / The pixel region where the patch is missing [y_w × y_h] /

[0116] N / Number of valid neighborhood patches (0..8) /

[0117] WeightedInterp[ ][ ] / Spatial weighted interpolation results /

[0118] SketchBuf[ ][ ] / The reconstructed subgraph /

[0119] EC_SKETCH_SUBSAMPLE = 16;

[0120] Output:

[0121] RecPatch[ ][ ] / Recovery results after fusion /

[0122] process:

[0123] = N / 16.0;

[0124] / The more neighborhood the interpolation, the more reliable it is, but at least half of the subgraph needs to be used. /

[0125] for y = 0 .. y_h – 1;

[0126] for x = 0 .. y_w – 1;

[0127] / Bilinear upsampling: subgraph coordinates /

[0128] fx = (P.left + x) / EC_SKETCH_SUBSAMPLE

[0129] fy = (P.top + y) / EC_SKETCH_SUBSAMPLE

[0130] SketchUpsample[ y ][ x ] = BilinearInterp( SketchBuf, fx, fy )

[0131] / Weighted stacking /

[0132] RecPatch[ y ][ x ] = Clip3( 0, MaxVal,

[0133] Round( α × WeightedInterp[ y ][ x ]

[0134] + ( 1 - ) × SketchUpsample[ y ][ x ] ) )

[0135] endfor

[0136] endfor

[0137] / Then, the three-stage EC post-processing filter (ec_pf_process_patch) continues to be executed. /

[0138] As can be seen, this application generates a low-resolution brightness sketch by downsampling the brightness components of keyframes at the encoding end and embeds it into the image header, providing the decoding end with prior information on global brightness and low-frequency structure. By obtaining this sketch and upsampling it when a slice is missing at the decoding end, and then fusing it with the error hiding result, the restoration process no longer relies solely on local neighborhood interpolation, but introduces overall brightness distribution constraints, thereby effectively correcting the brightness deviation caused by pure spatial interpolation and solving the problem of large deviation between the overall brightness of the reconstruction result and the original content.

[0139] See Figure 2 As shown, this embodiment of the invention discloses an intra-frame error recovery device for AVS3 keyframes, applied at the decoding end, comprising:

[0140] The keyframe data acquisition module 11 is used to acquire target keyframe data sent by the encoding end based on the AVS3 standard; wherein, the target keyframe data is obtained by the encoding end downsampling the luminance component of the current keyframe, generating a corresponding luminance sketch, embedding the luminance sketch into the image header of the current keyframe to obtain the target keyframe, and then slicing the target keyframe; the resolution of the luminance sketch is lower than the resolution of the current keyframe;

[0141] The brightness sketch acquisition module 12 is used to perform integrity detection on the target keyframe data. If there are missing slices in the target keyframe, the brightness sketch in the image header of the target keyframe is acquired.

[0142] The data fusion module 13 is used to process the target keyframe based on video image error hiding technology to obtain the corresponding intermediate keyframe, upsample the target sketch block in the brightness sketch corresponding to the missing slice to obtain the corresponding upsampled sketch block, and fuse the upsampled sketch block and the target keyframe block in the intermediate keyframe corresponding to the missing slice to supplement the missing data of the target keyframe and realize intra-frame error recovery; wherein, the resolution of the upsampled sketch is the same as the resolution of the missing slice.

[0143] In some specific embodiments, the encoding end may specifically include:

[0144] The luminance component processing module is used to downsample and mean filter the luminance component of the current key frame according to a preset downsampling ratio to obtain the luminance sketch.

[0145] A luminance sketch embedding module is used to embed the luminance sketch into the image header of the current keyframe using horizontal differential pulse modulation and exponential Golomb coding techniques.

[0146] In some specific embodiments, the brightness sketch acquisition module 12 may specifically include:

[0147] A luminance sketch acquisition unit is used to parse the flag bits in the image head. If the flag bits indicate that the luminance sketch exists in the image head, the luminance sketch is acquired from the image head using horizontal differential pulse modulation and exponential Golomb coding techniques.

[0148] In some specific embodiments, the data fusion module 13 may specifically include:

[0149] The intermediate keyframe acquisition unit is used to determine the target position of the missing data corresponding to the missing slice in the target keyframe, and to perform spatial interpolation on the target keyframe using the pixel values ​​in the neighborhood of the target position to obtain the intermediate keyframe.

[0150] In some specific embodiments, the data fusion module 13 may specifically include:

[0151] The data fusion unit is used to determine the target fusion weight corresponding to the target keyframe image block according to the number of neighborhoods corresponding to the target location, and to perform weighted fusion of the upsampled sketch and the intermediate keyframe according to the target fusion weight.

[0152] In some specific embodiments, the intra-frame error recovery device for AVS3 keyframes further includes:

[0153] The video frame acquisition module is used to acquire video frame data sent by the encoding end based on the AVS3 standard; wherein, the video frame data is the data obtained by the encoding end after slicing several video frames;

[0154] The video frame determination module is used to determine whether a target video frame exists in the video frame data; wherein, the target video frame is a non-critical frame with missing slices;

[0155] The reference frame acquisition module is used to determine the first position of the missing data corresponding to the missing slice in the target video frame if the target video frame exists in the video frame data, and to acquire the reference frame corresponding to the target video frame from the video frame data.

[0156] The pixel copy module is used to determine the second position in the reference frame corresponding to the first position, and copy the pixels of the second position to the first position to complete the intra-frame error recovery of the target video frame.

[0157] Furthermore, embodiments of this application also disclose an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0158] Figure 3 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the intra-frame error recovery method for AVS3 keyframes disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0159] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0160] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0161] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the intra-frame error recovery method for AVS3 keyframes executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0162] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned intra-frame error recovery method for AVS3 keyframes. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0163] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0164] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0165] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0166] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0167] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for intra-frame error recovery of AVS3 keyframes, characterized in that, Applied to the decoding end, including: The target keyframe data is obtained by the encoding end based on the AVS3 standard; wherein the target keyframe data is obtained by the encoding end downsampling the luminance component of the current keyframe, generating a corresponding luminance sketch, embedding the luminance sketch into the image header of the current keyframe to obtain the target keyframe, and then slicing the target keyframe; the resolution of the luminance sketch is lower than the resolution of the current keyframe. Integrity checks are performed on the target keyframe data. If a missing slice exists in the target keyframe, the brightness sketch in the image header of the target keyframe is obtained. The target keyframe is processed using video image error concealment technology to obtain the corresponding intermediate keyframe. The target sketch block in the brightness sketch corresponding to the missing slice is upsampled to obtain the corresponding upsampled sketch block. The upsampled sketch block and the target keyframe block in the intermediate keyframe corresponding to the missing slice are fused to supplement the missing data of the target keyframe and realize intra-frame error recovery. The resolution of the upsampled sketch is the same as the resolution of the missing slice.

2. The intra-frame error recovery method for AVS3 keyframes according to claim 1, characterized in that, The process of generating the brightness sketch at the encoding end and embedding the brightness sketch into the image header of the current keyframe includes: The luminance component of the current keyframe is downsampled and mean filtered according to a preset downsampling ratio to obtain the luminance sketch. The brightness sketch is embedded into the image header of the current keyframe using horizontal differential pulse modulation and exponential Golomb coding techniques.

3. The intra-frame error recovery method for AVS3 keyframes according to claim 2, characterized in that, The step of obtaining the brightness sketch from the image header of the target keyframe includes: The flag bits in the image header are parsed. If the flag bits indicate that the brightness sketch exists in the image header, the brightness sketch is obtained from the image header using horizontal differential pulse modulation and exponential Golomb coding techniques.

4. The intra-frame error recovery method for AVS3 keyframes according to claim 1, characterized in that, The process of processing the target keyframes based on video image error concealment technology to obtain corresponding intermediate keyframes includes: Determine the target location of the missing data corresponding to the missing slice in the target keyframe, and use the pixel values ​​in the neighborhood of the target location to perform spatial interpolation on the target keyframe to obtain the intermediate keyframe.

5. The intra-frame error recovery method for AVS3 keyframes according to claim 4, characterized in that, The fusion of the upsampled sketch block and the target keyframe block in the intermediate keyframe corresponding to the missing slice includes: The target fusion weight corresponding to the target keyframe image block is determined based on the number of neighborhoods corresponding to the target location, and the upsampled sketch and the intermediate keyframe are weighted and fused based on the target fusion weight.

6. The intra-frame error recovery method for AVS3 keyframes according to claim 5, characterized in that, The number of neighborhoods corresponding to the target location is positively correlated with the magnitude of the target fusion weight.

7. The intra-frame error recovery method for AVS3 keyframes according to any one of claims 1 to 6, characterized in that, Also includes: Acquire video frame data sent by the encoding end based on the AVS3 standard; wherein, the video frame data is the data obtained by the encoding end after slicing several video frames; Determine whether a target video frame exists in the video frame data; wherein, the target video frame is a non-critical frame with missing slices; If the target video frame exists in the video frame data, then determine the first position of the missing data corresponding to the missing slice in the target video frame, and obtain the reference frame corresponding to the target video frame from the video frame data; A second position corresponding to the first position in the reference frame is determined, and the pixels of the second position are copied to the first position to complete the intra-frame error recovery of the target video frame.

8. An intra-frame error recovery device for AVS3 keyframes, characterized in that, Applied to the decoding end, including: A keyframe data acquisition module is used to acquire target keyframe data sent by the encoding end based on the AVS3 standard; wherein, the target keyframe data is obtained by the encoding end downsampling the luminance component of the current keyframe to generate a corresponding luminance sketch, embedding the luminance sketch into the image header of the current keyframe to obtain the target keyframe, and then slicing the target keyframe; the resolution of the luminance sketch is lower than the resolution of the current keyframe; The brightness sketch acquisition module is used to perform integrity detection on the target keyframe data. If there are missing slices in the target keyframe, the brightness sketch in the image header of the target keyframe is acquired. The data fusion module is used to process the target keyframe based on video image error concealment technology to obtain the corresponding intermediate keyframe, upsample the target sketch block in the brightness sketch corresponding to the missing slice to obtain the corresponding upsampled sketch block, and fuse the upsampled sketch block and the target keyframe block in the intermediate keyframe corresponding to the missing slice to supplement the missing data of the target keyframe and realize intra-frame error recovery; wherein, the resolution of the upsampled sketch is the same as the resolution of the missing slice.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the intra-frame error recovery method for AVS3 keyframes as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the intra-frame error recovery method for AVS3 keyframes as described in any one of claims 1 to 7.

Images