Video inter-frame decoding error image recovery method and device and computer equipment

Abstract

The invention relates to a video inter-frame decoding error image recovery method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring to-be-processed image information of a target decoding frame; wherein the target decoding frame comprises at least one decoding block; according to the to-be-processed image information, calculating area proportion data of an intra-frame prediction block in the target decoding frame to obtain frame mode attribute information of an area where a current decoding block is located; determining motion attribute information and texture attribute information of an area where the current decoding block is located according to the to-be-processed image information and the frame mode attribute information; and determining motion vector data of the current decoding block according to the frame mode attribute information, the motion attribute information and the texture attribute information, and performing image restoration on the current decoding block according to the motion vector data. By adopting the method, the motion vector calculation accuracy of the decoded error block can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of video image processing, and in particular to a method, device, computer equipment, storage medium and computer program product for recovering erroneous images during video inter-frame decoding. Background Art

[0002] Hardware video decoder decoding error detection and processing is an important implementation that needs to be considered in the design phase. The hardware video decoder decoding error detection algorithm triggers the corresponding error handling function of the hardware decoder based on the type of decoding error detected.

[0003] The accuracy of the motion vector of the inter-frame decoding error block determines the matching degree between the prediction block data used for filling and the current decoding block data. A high matching degree means that the image quality filled with the current decoding error block is optimal. However, in the related art, the filling method of the inter-frame decoding error block of the hardware video decoder still has many shortcomings.

[0004] In related technologies, the motion vector of the decoded error block is uniformly set to (0,0), that is, the predicted data block at the same position of the reference frame is used to fill the current decoded error block. However, in video scenes with intense motion, the image data blocks filled in this way have serious block effects. If the number of decoded error blocks is large, the entire frame image is likely to have a screen distortion problem, and the accuracy is low. The calculation of the motion vector of the decoded error block supported by the hardware decoder has very low accuracy, which also leads to inaccurate predicted block data obtained by the decoded error block. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for video inter-frame decoding error image recovery that can improve the accuracy of motion vector calculation of decoding error blocks in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for recovering an erroneous image after decoding an inter-frame video, comprising:

[0007] Acquire the image information to be processed of the target decoding frame; wherein the target decoding frame includes at least one decoding block;

[0008] Calculating area ratio data of the intra-frame prediction block in the target decoded frame according to the image information to be processed, so as to obtain frame mode attribute information of the area where the current decoded block is located;

[0009] Determine motion attribute information and texture attribute information of the area where the current decoding block is located according to the image information to be processed and the frame mode attribute information;

[0010] The motion vector data of the current decoding block is determined according to the frame mode attribute information, the motion attribute information and the texture attribute information, and image restoration is performed on the current decoding block according to the motion vector data.

[0011] In one of the embodiments, calculating the area ratio data of the intra-frame prediction block in the target decoded frame according to the image information to be processed to obtain the frame mode attribute information of the area where the current decoded block is located includes:

[0012] Calculating area ratio data of the decoded intra-frame prediction blocks in the target decoded frame according to the image information to be processed;

[0013] Update the comparison code stream data according to the area ratio data;

[0014] The frame mode attribute information of the area where the current decoding block is located is determined according to the compared code stream data.

[0015] In one embodiment, updating the comparison stream data according to the area ratio data includes:

[0016] When the area ratio data is greater than a preset intra-frame prediction value, calculating current actual code stream data;

[0017] The current actual code stream data is used as the new comparison code stream data.

[0018] In one of the embodiments, determining the frame mode attribute information of the area where the current decoding block is located according to the comparison bitstream data includes:

[0019] Calculating the bitstream data volume ratio of the target decoding frame according to the comparison bitstream data, and determining the frame mode attribute information of the area where the current decoding block is located according to the bitstream data volume ratio;

[0020] Accordingly, determining the motion vector data of the current decoding block according to the frame mode attribute information, the motion attribute information and the texture attribute information includes:

[0021] When the proportion of the bitstream data amount of the current decoding frame is less than a preset bitstream threshold, selecting the motion vector data of the co-located block of the current decoding block as the position vector data of the current decoding block;

[0022] When the proportion of the bitstream data amount of the current decoding frame is greater than or equal to the preset bitstream threshold, the motion vector data of the current decoding block is determined according to the motion attribute information and the texture attribute information.

[0023] In one of the embodiments, determining the motion attribute information of the area where the current decoding block is located according to the to-be-processed image information and the frame mode attribute information includes:

[0024] Determine an inter-frame prediction block in the target decoding frame according to the frame mode attribute information;

[0025] Extracting, from the image information to be processed, first motion vector data of the inter-frame prediction block in the upper coding tree unit row of the current decoding block, and calculating the area ratio of the inter-frame prediction block whose first motion vector data is greater than a first preset vector threshold, to obtain first ratio data;

[0026] Extracting second motion vector data of the inter-frame prediction block in the left-neighboring coding tree unit of the current decoding block from the image information to be processed, and calculating the area ratio of the inter-frame prediction block whose second motion vector data is greater than a second preset vector threshold, to obtain second ratio data;

[0027] The motion attribute information of the area where the current decoding block is located is determined according to the first scale data and the second scale data.

[0028] In one of the embodiments, determining the texture attribute information of the area where the current decoding block is located according to the image information to be processed and the frame mode attribute information includes:

[0029] Extracting upper neighbor area data of the minimum coding unit in the upper neighbor coding tree unit row of the current decoding block from the image information to be processed, and calculating upper neighbor area ratio data according to the upper neighbor area data;

[0030] Extracting the left neighboring area data of the minimum coding unit in the left neighboring coding tree unit of the current decoding block from the image information to be processed, and calculating the left neighboring area ratio data according to the left neighboring area data;

[0031] The texture attribute information of the area where the current decoding block is located is determined according to the upper neighbor area ratio data and the left neighbor area ratio data.

[0032] In a second aspect, the present application further provides a video inter-frame decoding error image recovery device, comprising:

[0033] An information acquisition module, used to acquire the to-be-processed image information of a target decoding frame; wherein the target decoding frame includes at least one decoding block;

[0034] A frame mode determination module, used to calculate the area ratio data of the intra-frame prediction block in the target decoded frame according to the image information to be processed, so as to obtain the frame mode attribute information of the area where the current decoded block is located;

[0035] An attribute analysis module, used to determine motion attribute information and texture attribute information of the area where the current decoding block is located according to the image information to be processed and the frame mode attribute information;

[0036] The image restoration module is used to determine the motion vector data of the current decoding block according to the frame mode attribute information, the motion attribute information and the texture attribute information, and perform image restoration on the current decoding block according to the motion vector data.

[0037] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0038] Acquire the image information to be processed of the target decoding frame; wherein the target decoding frame includes at least one decoding block;

[0039] Calculating area ratio data of the intra-frame prediction block in the target decoded frame according to the image information to be processed, so as to obtain frame mode attribute information of the area where the current decoded block is located;

[0040] Determine motion attribute information and texture attribute information of the area where the current decoding block is located according to the image information to be processed and the frame mode attribute information;

[0041] The motion vector data of the current decoding block is determined according to the frame mode attribute information, the motion attribute information and the texture attribute information, and image restoration is performed on the current decoding block according to the motion vector data.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0043] Acquire the image information to be processed of the target decoding frame; wherein the target decoding frame includes at least one decoding block;

[0044] Calculating area ratio data of the intra-frame prediction block in the target decoded frame according to the image information to be processed, so as to obtain frame mode attribute information of the area where the current decoded block is located;

[0045] Determine motion attribute information and texture attribute information of the area where the current decoding block is located according to the image information to be processed and the frame mode attribute information;

[0046] The motion vector data of the current decoding block is determined according to the frame mode attribute information, the motion attribute information and the texture attribute information, and image restoration is performed on the current decoding block according to the motion vector data.

[0047] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0048] Acquire the image information to be processed of the target decoding frame; wherein the target decoding frame includes at least one decoding block;

[0049] Calculating area ratio data of the intra-frame prediction block in the target decoded frame according to the image information to be processed, so as to obtain frame mode attribute information of the area where the current decoded block is located;

[0050] Determine motion attribute information and texture attribute information of the area where the current decoding block is located according to the image information to be processed and the frame mode attribute information;

[0051] The motion vector data of the current decoding block is determined according to the frame mode attribute information, the motion attribute information and the texture attribute information, and image restoration is performed on the current decoding block according to the motion vector data.

[0052] The above-mentioned video inter-frame decoding error image recovery method, device, computer equipment, storage medium and computer program product obtain the image information of at least one decoding block by obtaining the to-be-processed image information of the target decoding frame, thereby calculating the area ratio data of the intra-frame prediction block obtained by predicting according to the intra-frame data of the target decoding frame in the target decoding frame, and can obtain the frame mode attribute information of the area where the current decoding block is located, so as to characterize the probability of skipping the prediction mode frame in the current decoding frame, and then determine the motion attribute information and texture attribute information of the area where the current decoding block is located according to the to-be-processed image information and the frame mode attribute information, and then determine the motion vector data of the current decoding block according to the frame mode attribute information, the motion attribute information and the texture attribute information, so as to select a better motion vector, and restore the image of the current decoding block according to the motion vector data, so as to ensure that the decoding error block obtains a prediction data block with a small error, and the inter-frame decoding block image is also referred to by the subsequent decoding frame, so as to improve the accuracy of motion vector calculation in the whole link, and improve the image quality of the video decoding sequence as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1A diagram of an application environment of a method for restoring an erroneous image during video inter-frame decoding in an embodiment;

[0055] Figure 2 A schematic diagram of a flow chart of a method for restoring an erroneous image during inter-frame decoding of a video in one embodiment;

[0056] Figure 3 This is a flow chart of step S204 of a method for restoring an erroneous image during inter-frame decoding of a video in an embodiment;

[0057] Figure 4 A schematic diagram of an architecture for implementing a method for restoring an erroneous image during inter-frame decoding of a video in one embodiment;

[0058] Figure 5 It is a structural block diagram of a device for recovering an erroneous image from video inter-frame decoding in one embodiment;

[0059] Figure 6 is an internal structure diagram of a computer device in one embodiment;

[0060] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] The video inter-frame decoding error image recovery method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network to collect the image information to be processed and send it to the server 104. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The data storage system can be used to store the image information to be processed of the target decoded frame, as well as the frame mode attribute information, motion attribute information and texture attribute information of the decoded block obtained by analysis. Among them, the terminal 102 can be but not limited to various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be applied to a hardware video decoder, which can be implemented with an independent server or a server cluster consisting of multiple servers.

[0063] In an exemplary embodiment, Figure 2 As shown, a method for recovering an image with error during decoding of a video frame is provided. Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps S202 to S208. Among them:

[0064] Step S202, obtaining the to-be-processed image information of the target decoding frame.

[0065] The target decoding frame includes at least one decoding block; the image information to be processed includes attribute information of at least one padding block to be processed and adjacent padding blocks, and the attribute information includes at least area information and motion vector information. Exemplarily, the server 104 can select a target decoding frame according to a decoded error image during inter-frame decoding, thereby correspondingly obtaining its image information to be processed.

[0066] Step S204, calculating the area ratio data of the intra-frame prediction block in the target decoding frame according to the image information to be processed, so as to obtain the frame mode attribute information of the area where the current decoding block is located.

[0067] For example, the server 104 may calculate the area ratio data of the intra-frame prediction block in the target decoded frame according to the image information to be processed, and obtain the frame mode attribute information intra_pred_block_area_ratio of the area where the current decoded block is located according to the area ratio data. f (abbreviated as ratio f ).

[0068] Among them, the frame mode attribute information is used to characterize the probability of skipping the prediction mode frame in the area where the current decoding block is located. Intra-prediction is a technology used to predict a single frame in video coding. In intra-prediction, each block (or area) of the video only depends on other parts that have been encoded or decoded in the same frame, and does not depend on the information of the previous and next frames. It is different from inter-prediction, which is based on other frames for prediction.

[0069] Among them, Skip Mode Frame is a technology in video coding that is used to speed up processing and reduce data transmission. When the encoder believes that a video frame or some areas (blocks) in it are very similar to the previous frame, it can use "skip prediction mode", that is, no longer perform complex prediction or encoding processing on these areas, but directly reuse the data in the previous frame.

[0070] Step S206: determining the motion attribute information and texture attribute information of the area where the current decoding block is located according to the image information to be processed and the frame mode attribute information.

[0071] Among them, the motion attribute information is used to characterize the motion attribute of the area where the current decoding error block is located. The motion attribute refers to the motion of a certain area in the video between different frames, whether a certain block or area of ​​the video is stationary or moving between adjacent frames, or the speed and direction of the motion. The motion attribute includes motion speed, motion direction and motion state (i.e. whether it is moving): the motion speed refers to how fast a certain area moves between video frames; the motion direction refers to the direction in which the target area moves, such as left, right, up or down; whether it is moving refers to whether a certain area remains stationary or moves between different frames.

[0072] Among them, texture attribute information is used to characterize the texture attributes of the area where the current decoding error block is located. Texture attributes refer to the visual details and structural complexity of a certain area in a video or image. Simply put, it describes the smoothness or complexity of the surface of an area, such as whether it contains many details, lines, edges, color changes, etc. When the decoder processes the video, the texture attributes can be used to determine whether a certain area of ​​the video is complex or simple. If the decoding error occurs in a complex area, more complex calculations are required to repair the area to ensure the restoration of details. If the error occurs in a smooth area, it may be much simpler to repair.

[0073] Exemplarily, the server 104 can calculate the area ratio data of the inter-frame prediction block whose motion vector data is greater than the preset vector threshold in the target decoded frame according to the image information to be processed and the frame mode attribute information, so as to obtain the motion attribute information and texture attribute information of the area where the current decoded block is located. In this process, in order to count the area ratio of the motion vector (MV) of the adjacent block of the decoded error block exceeding the threshold, the server 104 can determine the inter-frame prediction block in the target decoded frame according to the frame mode attribute information; extract the first motion vector data of the inter-frame prediction block in the upper adjacent coding tree unit row of the current decoded block from the image information to be processed, and calculate the area ratio of the inter-frame prediction block whose first motion vector data is greater than the first preset vector threshold to obtain the first ratio data; extract the second motion vector data of the inter-frame prediction block in the left adjacent coding tree unit of the current decoded block from the image information to be processed, and calculate the area ratio of the inter-frame prediction block whose second motion vector data is greater than the second preset vector threshold to obtain the second ratio data; determine the motion attribute information of the area where the current decoded block is located according to the first ratio data and the second ratio data.

[0074] Furthermore, the server 104 may calculate and store the area ratio moving_area_ratio of the inter-frame prediction block motion vector MV of the upper adjacent coding tree unit (CTU) row greater than the set threshold top-ctu :

[0075]

[0076] Among them, moving_area_ratio top-ctu The calculation formula is shown above, where ctu stands for top-ctu;

[0077] Next, the server 104 may calculate and store the area ratio moving_area_ratio of the inter-frame prediction block motion vector MV of the left adjacent coding tree unit (CTU) greater than the set threshold left-ctu :

[0078] Among them, moving_area_ratio left-ctu The calculation formula is shown above, where ctu stands for left-ctu; determine the motion attribute of the area where the current decoding block is located:

[0079]

[0080] Exemplarily, the server 104 may calculate the area ratio data of the minimum coding unit in the adjacent decoding blocks of the current decoding block according to the image information to be processed and the frame mode attribute information, so as to obtain the texture attribute information of the area where the current decoding block is located. In this process, in order to count the area ratio of the minimum coding unit (CU) of the adjacent blocks of the decoding error block, the server 104 may extract the upper neighboring area data of the minimum coding unit in the upper neighboring coding tree unit row of the current decoding block from the image information to be processed, and calculate the upper neighboring area ratio data according to the upper neighboring area data; extract the left neighboring area data of the minimum coding unit in the left neighboring coding tree unit of the current decoding block from the image information to be processed, and calculate the left neighboring area ratio data according to the left neighboring area data; determine the texture attribute information of the area where the current decoding block is located according to the upper neighboring area ratio data and the left neighboring area ratio data.

[0081] Furthermore, the server 104 may calculate and store the minimum coding unit (CU) area ratio of the upper adjacent coding tree unit (CTU) row min_cu_area_ratio top-ctu :

[0082]

[0083] Among them, min_cu_area_ratio top-ctu The calculation formula is as above, where ctu stands for top-ctu;

[0084] Next, the server 104 may calculate and store the min_cu_area_ratio of the minimum coding unit (CU) area of ​​the left-neighbor coding tree unit left-ctu :

[0085] Among them, min_cu_area_ratio left-ctu The calculation formula is as above, where ctu stands for left-ctu; the texture attribute of the area where the current decoding block is located is then determined:

[0086]

[0087] Step S208, determining the motion vector data of the current decoding block according to the frame mode attribute information, the motion attribute information and the texture attribute information, and performing image restoration on the current decoding block according to the motion vector data.

[0088] Exemplarily, the server 104 can analyze the characteristics of the current decoded block through frame mode attribute information, motion attribute information and texture attribute information. The frame mode attribute information is used to determine whether the current decoded block is predicted based on intra-frame or inter-frame, so as to determine whether it is necessary to refer to the data of other frames; the motion attribute information is used to provide the motion of the current block between different frames. The server 104 can infer the moving direction and speed of the current block based on this information, so as to calculate a suitable motion vector. This motion vector represents the position of the current decoded block in the reference frame, and the server 104 can find the corresponding data in the reference frame for matching. Finally, the texture attribute information is used to characterize the texture complexity of the current decoded block to help optimize the decoding process. If the texture of the block is more complex, the server 104 needs to process the recovery of the area more finely to retain the details of the image. Finally, the server 104 can locate the corresponding image data in the reference frame by the motion vector determined according to the above information, and perform accurate image restoration on the current decoded block.

[0089] In the above-mentioned video inter-frame decoding error image recovery method, by obtaining the to-be-processed image information of the target decoding frame, the image information of at least one decoding block is obtained, so as to calculate the area ratio data of the intra-frame prediction block obtained by predicting according to the intra-frame data of the target decoding frame in the target decoding frame, so as to obtain the frame mode attribute information of the area where the current decoding block is located, so as to characterize the probability of skipping the prediction mode frame in the current decoding frame, and then determine the motion attribute information and texture attribute information of the area where the current decoding block is located according to the to-be-processed image information and the frame mode attribute information, and then determine the motion vector data of the current decoding block according to the frame mode attribute information, the motion attribute information and the texture attribute information, so as to select a better motion vector, and perform image restoration on the current decoding block according to the motion vector data, so as to ensure that the decoding error block obtains a prediction data block with a small error, and the inter-frame decoding block image is also referred to by the subsequent decoding frame, so as to improve the accuracy of motion vector calculation in the whole link, and improve the image quality of the video decoding sequence as a whole.

[0090] In an exemplary embodiment, Figure 3 As shown, step S204 includes steps S302 to S306. Among them:

[0091] Step S302: Calculate area ratio data of decoded intra-frame prediction blocks in a target decoded frame according to the image information to be processed.

[0092] For example, the server 104 may calculate the intra-prediction block area ratio of the most recently decoded frame to obtain the area ratio data intra_pred_block_area_ratio f (abbreviated as ratio f ):

[0093] intra_pred_block_area f

[0094] =sum(width intra_pred_block_i *height intra_pred_block_i )

[0095] decoded_frame_area f =width decoded_frame *height decoded_frame

[0096]

[0097] Step S304: update the comparison code stream data according to the area ratio data.

[0098] For example, the server 104 may calculate the current actual bitstream data when the area ratio data is greater than the preset intra-frame prediction value; and use the current actual bitstream data as the new comparison bitstream data, and the calculation formula is:

[0099]

[0100] Among them, intra_pred_area thr can be set to 0.9.

[0101] Step S306: determining the frame mode attribute information of the area where the current decoding block is located according to the compared code stream data.

[0102] Exemplarily, the server 104 can calculate the bitstream data volume ratio of the target decoded frame based on the comparison bitstream data, and determine the frame mode attribute information of the area where the current decoded block is located based on the bitstream data volume ratio. According to the above steps, the server 104 can calculate the bitstream size ratio of the current decoded frame using the following formula:

[0103]

[0104] Accordingly, the server 104 can select the motion vector data of the co-located block of the current decoding block as the position vector data of the current decoding block when the proportion of the bitstream data volume of the current decoding frame is less than the preset bitstream threshold; and determine the motion vector data of the current decoding block according to the motion attribute information and the texture attribute information when the proportion of the bitstream data volume of the current decoding frame is greater than or equal to the preset bitstream threshold.

[0105] In another embodiment, Figure 4 As shown, the server 104 can be applied to a hardware video decoder (VideoDecoder), and the server 104 can obtain the to-be-processed image information of the target decoded frame, thereby calculating the intra-frame prediction block area in the decoded frame according to the to-be-processed image information, thereby performing skip prediction mode frame detection, and obtaining frame mode attribute information; calculating the area ratio of the motion vector of the adjacent blocks of the decoded error block exceeding the threshold, thereby performing motion area detection, and obtaining motion attribute information; calculating the minimum coding unit area ratio of the adjacent blocks of the decoded error block, thereby performing texture smoothing area detection, and obtaining texture attribute information, thereby analyzing the characteristics of the current decoded block to determine whether the current decoded block is predicted based on intra-frame or inter-frame, determining the motion of the current block between different frames and the texture complexity of the current decoded block, and then calculating the inter-frame decoded error block motion vector MV according to the attributes of the area where the current decoded block is located. The calculation process of the decoded error block motion vector MV is as follows:

[0106]

[0107] Among them, in bitstream_ratio f <threshold bitstream_size If the conditions are met, col_mv poc_nearest It is the motion vector (MV) of the co-located block in the decoded frame closest to the current decoded frame image display order (POC). If the co-located block is not inter-frame predicted, the motion vector of the adjacent block of the co-located block is taken. If the MV of the adjacent block is not available, the MV is set to (0,0).

[0108] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0109] Based on the same inventive concept, the embodiment of the present application also provides a video inter-frame decoding error image recovery device for implementing the video inter-frame decoding error image recovery method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more video inter-frame decoding error image recovery device embodiments provided below can refer to the limitations of the video inter-frame decoding error image recovery method above, and will not be repeated here.

[0110] In an exemplary embodiment, Figure 5 As shown, a video inter-frame decoding error image recovery device is provided, comprising: an information acquisition module 502, a frame mode determination module 504, a property analysis module 506 and an image recovery module 508, wherein:

[0111] The information acquisition module 502 is used to acquire the to-be-processed image information of the target decoding frame; wherein the target decoding frame includes at least one decoding block;

[0112] The frame mode determination module 504 is used to calculate the area ratio data of the intra-frame prediction block in the target decoding frame according to the image information to be processed, so as to obtain the frame mode attribute information of the area where the current decoding block is located;

[0113] The attribute analysis module 506 is used to determine the motion attribute information and texture attribute information of the area where the current decoding block is located according to the image information to be processed and the frame mode attribute information;

[0114] The image restoration module 508 is used to determine the motion vector data of the current decoding block according to the frame mode attribute information, the motion attribute information and the texture attribute information, and perform image restoration on the current decoding block according to the motion vector data.

[0115] In one embodiment, the frame mode determination module 504 includes:

[0116] An area calculation unit, used for calculating area ratio data of a decoded intra-frame prediction block in a target decoding frame according to the image information to be processed;

[0117] A data updating unit, used for updating the comparison code stream data according to the area ratio data;

[0118] The attribute determination unit is used to determine the frame mode attribute information of the area where the current decoding block is located according to the comparison code stream data.

[0119] In one of the embodiments, the data updating unit is specifically used to: calculate the current actual code stream data when the area ratio data is greater than the preset intra-frame prediction value; and use the current actual code stream data as the new comparison code stream data.

[0120] In one of the embodiments, the attribute determination unit is specifically used to: calculate the bitstream data volume ratio of the target decoding frame according to the comparison bitstream data, and determine the frame mode attribute information of the area where the current decoding block is located according to the bitstream data volume ratio;

[0121] Accordingly, the image recovery module 508 is specifically used to: when the proportion of the bitstream data volume of the current decoded frame is less than the preset bitstream threshold, select the motion vector data of the co-located block of the current decoded block as the position vector data of the current decoded block; when the proportion of the bitstream data volume of the current decoded frame is greater than or equal to the preset bitstream threshold, determine the motion vector data of the current decoded block according to the motion attribute information and the texture attribute information.

[0122] In one of the embodiments, the attribute analysis module 506 is specifically used to: determine the inter-frame prediction block in the target decoding frame according to the frame mode attribute information; extract the first motion vector data of the inter-frame prediction block in the upper adjacent coding tree unit row of the current decoding block from the image information to be processed, and calculate the area ratio of the inter-frame prediction block whose first motion vector data is greater than the first preset vector threshold value to obtain the first proportional data; extract the second motion vector data of the inter-frame prediction block in the left adjacent coding tree unit of the current decoding block from the image information to be processed, and calculate the area ratio of the inter-frame prediction block whose second motion vector data is greater than the second preset vector threshold value to obtain the second proportional data; determine the motion attribute information of the area where the current decoding block is located according to the first proportional data and the second proportional data.

[0123] In one of the embodiments, the attribute analysis module 506 is also used to: extract the upper neighboring area data of the minimum coding unit in the upper neighboring coding tree unit row of the current decoding block from the image information to be processed, and calculate the upper neighboring area ratio data based on the upper neighboring area data; extract the left neighboring area data of the minimum coding unit in the left neighboring coding tree unit of the current decoding block from the image information to be processed, and calculate the left neighboring area ratio data based on the left neighboring area data; determine the texture attribute information of the area where the current decoding block is located based on the upper neighboring area ratio data and the left neighboring area ratio data.

[0124] Each module in the above-mentioned video inter-frame decoding error image recovery device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0125] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the image information to be processed of the target decoding frame, as well as the frame mode attribute information, motion attribute information and texture attribute information of the decoded block obtained by analysis. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for recovering an erroneous image from decoding between video frames is implemented.

[0126] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for recovering an erroneous image from an inter-frame decoding of a video is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0127] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0128] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: obtaining image information to be processed of a target decoded frame; wherein the target decoded frame includes at least one decoded block; calculating area ratio data of an intra-frame prediction block in the target decoded frame according to the image information to be processed to obtain frame mode attribute information of a region where a current decoded block is located; determining motion attribute information and texture attribute information of a region where a current decoded block is located according to the image information to be processed and the frame mode attribute information; determining motion vector data of the current decoded block according to the frame mode attribute information, the motion attribute information and the texture attribute information, and performing image restoration on the current decoded block according to the motion vector data.

[0129] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on the image information to be processed, the area ratio data of the decoded intra-frame prediction block in the target decoding frame is calculated; based on the area ratio data, the comparison code stream data is updated; and based on the comparison code stream data, the frame mode attribute information of the area where the current decoding block is located is determined.

[0130] In one embodiment, when the processor executes the computer program, the following steps are further implemented: when the area ratio data is greater than a preset intra-frame prediction value, current actual code stream data is calculated; and the current actual code stream data is used as new comparison code stream data.

[0131] In one embodiment, when the processor executes the computer program, the following steps are further implemented: calculating the proportion of the code stream data volume of the target decoding frame according to the comparison code stream data, and determining the frame mode attribute information of the area where the current decoding block is located according to the proportion of the code stream data volume; when the proportion of the code stream data volume of the current decoding frame is less than a preset code stream threshold, selecting the motion vector data of the same-position block of the current decoding block as the position vector data of the current decoding block; when the proportion of the code stream data volume of the current decoding frame is greater than or equal to the preset code stream threshold, determining the motion vector data of the current decoding block according to the motion attribute information and the texture attribute information.

[0132] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining the inter-frame prediction block in the target decoding frame according to the frame mode attribute information; extracting the first motion vector data of the inter-frame prediction block in the upper adjacent coding tree unit row of the current decoding block from the image information to be processed, and calculating the area ratio of the inter-frame prediction block whose first motion vector data is greater than the first preset vector threshold, to obtain first proportional data; extracting the second motion vector data of the inter-frame prediction block in the left adjacent coding tree unit of the current decoding block from the image information to be processed, and calculating the area ratio of the inter-frame prediction block whose second motion vector data is greater than the second preset vector threshold, to obtain second proportional data; determining the motion attribute information of the area where the current decoding block is located according to the first proportional data and the second proportional data.

[0133] In one embodiment, when the processor executes the computer program, the following steps are also implemented: extracting the upper neighboring area data of the minimum coding unit in the upper neighboring coding tree unit row of the current decoding block from the image information to be processed, and calculating the upper neighboring area ratio data based on the upper neighboring area data; extracting the left neighboring area data of the minimum coding unit in the left neighboring coding tree unit of the current decoding block from the image information to be processed, and calculating the left neighboring area ratio data based on the left neighboring area data; determining the texture attribute information of the area where the current decoding block is located based on the upper neighboring area ratio data and the left neighboring area ratio data.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: obtaining image information to be processed of a target decoded frame; wherein the target decoded frame includes at least one decoded block; calculating area ratio data of an intra-frame prediction block in the target decoded frame according to the image information to be processed, so as to obtain frame mode attribute information of a region where a current decoded block is located; determining motion attribute information and texture attribute information of a region where a current decoded block is located according to the image information to be processed and the frame mode attribute information; determining motion vector data of the current decoded block according to the frame mode attribute information, the motion attribute information and the texture attribute information, and performing image restoration on the current decoded block according to the motion vector data.

[0135] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the image information to be processed, the area ratio data of the decoded intra-frame prediction block in the target decoding frame is calculated; based on the area ratio data, the comparison code stream data is updated; and based on the comparison code stream data, the frame mode attribute information of the area where the current decoding block is located is determined.

[0136] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: when the area ratio data is greater than a preset intra-frame prediction value, current actual code stream data is calculated; and the current actual code stream data is used as new comparison code stream data.

[0137] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: calculating the proportion of the bitstream data volume of the target decoding frame according to the comparison bitstream data, and determining the frame mode attribute information of the area where the current decoding block is located according to the bitstream data volume proportion; when the bitstream data volume proportion of the current decoding frame is less than a preset bitstream threshold, selecting the motion vector data of the same-position block of the current decoding block as the position vector data of the current decoding block; when the bitstream data volume proportion of the current decoding frame is greater than or equal to the preset bitstream threshold, determining the motion vector data of the current decoding block according to the motion attribute information and the texture attribute information.

[0138] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining the inter-frame prediction block in the target decoding frame according to the frame mode attribute information; extracting the first motion vector data of the inter-frame prediction block in the upper adjacent coding tree unit row of the current decoding block from the image information to be processed, and calculating the area ratio of the inter-frame prediction block whose first motion vector data is greater than the first preset vector threshold, to obtain first proportional data; extracting the second motion vector data of the inter-frame prediction block in the left adjacent coding tree unit of the current decoding block from the image information to be processed, and calculating the area ratio of the inter-frame prediction block whose second motion vector data is greater than the second preset vector threshold, to obtain second proportional data; determining the motion attribute information of the area where the current decoding block is located according to the first proportional data and the second proportional data.

[0139] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: extracting the upper neighboring area data of the minimum coding unit in the upper neighboring coding tree unit row of the current decoding block from the image information to be processed, and calculating the upper neighboring area ratio data based on the upper neighboring area data; extracting the left neighboring area data of the minimum coding unit in the left neighboring coding tree unit of the current decoding block from the image information to be processed, and calculating the left neighboring area ratio data based on the left neighboring area data; determining the texture attribute information of the area where the current decoding block is located based on the upper neighboring area ratio data and the left neighboring area ratio data.

[0140] In one embodiment, a computer program product is provided, comprising a computer program, which implements the following steps when executed by a processor: obtaining image information to be processed of a target decoded frame; wherein the target decoded frame comprises at least one decoded block; calculating area ratio data of an intra-frame prediction block in the target decoded frame according to the image information to be processed, so as to obtain frame mode attribute information of a region where a current decoded block is located; determining motion attribute information and texture attribute information of a region where a current decoded block is located according to the image information to be processed and the frame mode attribute information; determining motion vector data of the current decoded block according to the frame mode attribute information, the motion attribute information and the texture attribute information, and performing image restoration on the current decoded block according to the motion vector data.

[0141] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the image information to be processed, the area ratio data of the decoded intra-frame prediction block in the target decoding frame is calculated; based on the area ratio data, the comparison code stream data is updated; and based on the comparison code stream data, the frame mode attribute information of the area where the current decoding block is located is determined.

[0142] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: when the area ratio data is greater than a preset intra-frame prediction value, current actual code stream data is calculated; and the current actual code stream data is used as new comparison code stream data.

[0143] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: calculating the proportion of the bitstream data volume of the target decoding frame according to the comparison bitstream data, and determining the frame mode attribute information of the area where the current decoding block is located according to the bitstream data volume proportion; when the bitstream data volume proportion of the current decoding frame is less than a preset bitstream threshold, selecting the motion vector data of the same-position block of the current decoding block as the position vector data of the current decoding block; when the bitstream data volume proportion of the current decoding frame is greater than or equal to the preset bitstream threshold, determining the motion vector data of the current decoding block according to the motion attribute information and the texture attribute information.

[0144] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining the inter-frame prediction block in the target decoding frame according to the frame mode attribute information; extracting the first motion vector data of the inter-frame prediction block in the upper adjacent coding tree unit row of the current decoding block from the image information to be processed, and calculating the area ratio of the inter-frame prediction block whose first motion vector data is greater than the first preset vector threshold, to obtain first proportional data; extracting the second motion vector data of the inter-frame prediction block in the left adjacent coding tree unit of the current decoding block from the image information to be processed, and calculating the area ratio of the inter-frame prediction block whose second motion vector data is greater than the second preset vector threshold, to obtain second proportional data; determining the motion attribute information of the area where the current decoding block is located according to the first proportional data and the second proportional data.

[0145] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: extracting the upper neighboring area data of the minimum coding unit in the upper neighboring coding tree unit row of the current decoding block from the image information to be processed, and calculating the upper neighboring area ratio data based on the upper neighboring area data; extracting the left neighboring area data of the minimum coding unit in the left neighboring coding tree unit of the current decoding block from the image information to be processed, and calculating the left neighboring area ratio data based on the left neighboring area data; determining the texture attribute information of the area where the current decoding block is located based on the upper neighboring area ratio data and the left neighboring area ratio data.

[0146] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0147] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for restoring an image with errors during inter-frame decoding of a video, characterized in that: The method comprises: Acquire the image information to be processed of the target decoding frame; wherein the target decoding frame includes at least one decoding block; Calculating area ratio data of the intra-frame prediction block in the target decoded frame according to the image information to be processed, so as to obtain frame mode attribute information of the area where the current decoded block is located; Determine motion attribute information and texture attribute information of the area where the current decoding block is located according to the image information to be processed and the frame mode attribute information; The motion vector data of the current decoding block is determined according to the frame mode attribute information, the motion attribute information and the texture attribute information, and image restoration is performed on the current decoding block according to the motion vector data.

2. The method according to claim 1, characterized in that The step of calculating the area ratio data of the intra-frame prediction block in the target decoded frame according to the image information to be processed to obtain the frame mode attribute information of the area where the current decoded block is located includes: Calculating area ratio data of the decoded intra-frame prediction blocks in the target decoded frame according to the image information to be processed; Update the comparison code stream data according to the area ratio data; The frame mode attribute information of the area where the current decoding block is located is determined according to the comparison code stream data.

3. The method according to claim 2, characterized in that The updating of the comparison code stream data according to the area ratio data includes: When the area ratio data is greater than a preset intra-frame prediction value, calculating current actual code stream data; The current actual code stream data is used as the new comparison code stream data.

4. The method according to claim 2, characterized in that: The determining, according to the comparison bitstream data, the frame mode attribute information of the area where the current decoding block is located comprises: Calculating the bitstream data volume ratio of the target decoding frame according to the comparison bitstream data, and determining the frame mode attribute information of the area where the current decoding block is located according to the bitstream data volume ratio; Accordingly, determining the motion vector data of the current decoding block according to the frame mode attribute information, the motion attribute information and the texture attribute information includes: When the proportion of the bitstream data amount of the current decoding frame is less than a preset bitstream threshold, selecting the motion vector data of the co-located block of the current decoding block as the position vector data of the current decoding block; When the proportion of the bitstream data amount of the current decoding frame is greater than or equal to the preset bitstream threshold, the motion vector data of the current decoding block is determined according to the motion attribute information and the texture attribute information.

5. The method according to claim 1, characterized in that The determining, according to the to-be-processed image information and the frame mode attribute information, the motion attribute information of the area where the current decoding block is located comprises: Determine an inter-frame prediction block in the target decoding frame according to the frame mode attribute information; Extracting, from the image information to be processed, first motion vector data of the inter-frame prediction block in the upper coding tree unit row of the current decoding block, and calculating the area ratio of the inter-frame prediction block whose first motion vector data is greater than a first preset vector threshold, to obtain first ratio data; Extracting second motion vector data of the inter-frame prediction block in the left-neighboring coding tree unit of the current decoding block from the image information to be processed, and calculating the area ratio of the inter-frame prediction block whose second motion vector data is greater than a second preset vector threshold, to obtain second ratio data; The motion attribute information of the area where the current decoding block is located is determined according to the first scale data and the second scale data.

6. The method according to claim 1, characterized in that The determining, according to the to-be-processed image information and the frame mode attribute information, the texture attribute information of the area where the current decoding block is located comprises: Extracting upper neighbor area data of the minimum coding unit in the upper neighbor coding tree unit row of the current decoding block from the image information to be processed, and calculating upper neighbor area ratio data according to the upper neighbor area data; Extracting, from the image information to be processed, the left neighboring area data of the minimum coding unit in the left neighboring coding tree unit of the current decoding block, and calculating the left neighboring area ratio data according to the left neighboring area data; The texture attribute information of the area where the current decoding block is located is determined according to the upper neighbor area ratio data and the left neighbor area ratio data.

7. A device for restoring an image with errors during inter-frame decoding of a video, characterized in that: The device comprises: An information acquisition module, used to acquire the to-be-processed image information of a target decoding frame; wherein the target decoding frame includes at least one decoding block; A frame mode determination module, used to calculate the area ratio data of the intra-frame prediction block in the target decoded frame according to the image information to be processed, so as to obtain the frame mode attribute information of the area where the current decoded block is located; An attribute analysis module, used to determine motion attribute information and texture attribute information of the area where the current decoding block is located according to the image information to be processed and the frame mode attribute information; The image restoration module is used to determine the motion vector data of the current decoding block according to the frame mode attribute information, the motion attribute information and the texture attribute information, and perform image restoration on the current decoding block according to the motion vector data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images