Image decoding device and method

Abstract

The invention relates to an image decoding device and method. An image decoding method includes: determining an occurrence of an error block for an Nth image picture generated according to an inter-image coding technique; enabling the image coding device to receive error return information when the image decoding device receives the corresponding block of the (N + P-1) th image picture; capturing motion vector information and residual value information of a block before the error block occurs in the Nth image picture, and decoding according to an inter-image coding technology; capturing a translation vector from the (N-1) th image picture identified as the translation image picture to set the translation vector as motion vector information, setting residual value information as zero, and decoding a block from the error block to the last block of the (N + P-1) th image picture according to an inter-image coding technology; and capturing motion vector information and residual value information corresponding to blocks of the (N + P) th image picture, and decoding the blocks according to an inter-image coding technology.

Description

Technical Field

[0001] This invention relates to image decoding technology, and more particularly to an image decoding apparatus and method. Background Technology

[0002] As consumers demand more multimedia entertainment and higher display quality, the requirements for resources related to video content processing, transmission, and storage, such as memory size and bandwidth, are also increasing. Many standards have been established to ensure the display quality of video content, while significant progress has also been made in compression and decompression efficiency.

[0003] However, data loss is inevitable during video streaming. When data is lost, the video decoding device will be unable to decode based on the information transmitted by the video encoding device, such as motion vector information and residual information. Without a corresponding mechanism, the video decoding device will be unable to decode normally in such a situation, let alone generate a decoded video image. Summary of the Invention

[0004] In view of the problems of the prior art, one object of the present invention is to provide an image decoding apparatus and method to improve the prior art.

[0005] This invention includes an image decoding method applied in an image decoding device, comprising: determining the occurrence of an error block in an Nth image frame generated by inter-image coding technology from an image stream from an image encoding device, where N is an integer greater than 1; transmitting error reporting information to the image encoding device, so that the image encoding device receives the error reporting information when the image decoding device receives the corresponding block of the N+P-1th image frame in the image stream, and the image encoding device encodes the N+Pth image frame in the image stream according to the inter-image coding technology with reference to a reference image frame before the Nth image frame where no error occurred, where P is an integer greater than or equal to 1; and extracting motion vectors from the image stream of multiple first blocks corresponding to the Nth image frame before the occurrence of the error block. The image stream extracts motion vector information and residual information to perform a decoding procedure on the first block according to the image coding technique to generate a first decoding result; it extracts a translation vector from the (N-1)th image frame identified as a panning image frame, sets the motion vector information as the translation vector and sets the residual information to zero, and performs a decoding procedure on multiple second blocks from the erroneous block of the Nth image frame to the last block of the N+P-1th image frame according to the image coding technique to generate a second decoding result; and it extracts motion vector information and residual information of multiple inter-image reference blocks corresponding to the N+Pth image frame from the image stream, and performs a decoding procedure on the inter-image reference blocks according to the image coding technique to generate an inter-image reference decoding result.

[0006] The present invention further includes an image decoding apparatus, comprising: a front-end decoding circuit and a back-end decoding circuit. The front-end decoding circuit is configured to: determine the occurrence of an error block in the Nth image frame generated by inter-image coding technology from the image stream from the image encoding device, where N is an integer greater than 1; and transmit error reporting information to the image encoding device, so that the image encoding device receives the error reporting information when the image decoding device receives the corresponding block of the N+P-1th image frame in the image stream, and enables the image encoding device to encode the N+Pth image frame in the image stream according to the inter-image coding technology with reference to a reference image frame before the Nth image frame where no error occurred, where P is an integer greater than or equal to 1. The back-end decoding circuit is configured to: when the front-end decoding circuit extracts motion vector information and residual value information of multiple first blocks before the occurrence of the error block in the corresponding Nth image frame from the image stream, perform a decoding procedure on the first block according to the inter-image coding technique to generate a first decoding result; when the front-end decoding circuit extracts the translation vector from the N-1th image frame identified as a translation image frame, sets the motion vector information as the translation vector and sets the residual value information to zero, perform a decoding procedure on multiple second blocks from the error block of the Nth image frame to the last block of the N+P-1th image frame sequentially with reference to the N-1th to N+P-2th image frames according to the inter-image coding technique to generate a second decoding result; and when the front-end decoding circuit extracts motion vector information and residual value information of multiple inter-image reference blocks corresponding to the N+Pth image frame from the image stream, perform a decoding procedure on the inter-image reference blocks according to the inter-image coding technique to generate an inter-image reference decoding result.

[0007] Regarding the features, implementation, and effects of this case, the preferred embodiments are described in detail below with reference to the drawings. Attached Figure Description

[0008] Figure 1 This shows a block diagram of an image encoding device and an image decoding device according to one embodiment of the present invention;

[0009] Figure 2A This diagram illustrates a plurality of image frames transmitted by an image stream in one embodiment of the present invention.

[0010] Figure 2B This diagram illustrates a plurality of image frames transmitted by an image stream in one embodiment of the present invention.

[0011] Figure 3 This diagram illustrates an image processing block in an image frame according to one embodiment of the present invention; and

[0012] Figure 4 This diagram shows a flowchart of an image decoding method according to one embodiment of the present invention. Detailed Implementation

[0013] One objective of this invention is to provide an image decoding apparatus and method. When an error block is determined to occur in the Nth image frame, a translation vector is extracted from the (N-1)th image frame identified as a translation image frame. The motion vector information is set as the translation vector, and the residual value information is set to zero. Decoding is then performed on these image blocks from the error block up to the last block of the (N+P-1)th image frame containing the corresponding block where the image encoding device receives the error report information. This approach allows the decoding process to continue while maintaining the smoothness of the decoded image frame, thus improving the accuracy of the decoding.

[0014] Please refer to Figure 1 . Figure 1 This diagram shows a block diagram of an image encoding device 100 and an image decoding device 110 according to one embodiment of the present invention.

[0015] The image encoding device 100 is configured to perform FM on multiple image frames according to an inter-image encoding technique, for example... Figure 1 The exemplary N-1th image frame FM N-1 The Nth video frame FM N ..., the N+Pth video frame FM N+P Encode the video stream IS. Here, N is an integer greater than 1, and P is an integer greater than or equal to 1.

[0016] To be more specific, FM using the Nth video frame N Taking the image frame to be encoded as an example, the image encoding device 100 encodes it according to the corresponding reference image data. The image encoding device 100 can use inter-frame encoding technology to enable FM encoding of the Nth image frame. N At least one previous image frame is used as reference image data. For example, when the reference image data is the previous image frame, the image encoding device 100 will make the (N-1)th image frame FM. N-1 FM for the Nth image frame as reference image data N Encoding is performed. However, in other embodiments, the image encoding device 100 may also FM the Nth image frame. N The first K images (not shown in the figure) are used as reference image data, or the Nth image is FM. N Multiple preceding images are used as reference image data. This invention is not limited thereto.

[0017] On the other hand, the image encoding device 100 can also use intra-frame encoding technology to enable the Nth image frame to be encoded to FM.N Use its own internal image content as reference image data.

[0018] In the context of inter-image coding techniques, the image coding apparatus 100 performs motion compensation on the motion vector information of the current image frame relative to the reference image data to obtain residual information, and then performs processing such as, but not limited to, discrete cosine transform (DCT), quantization, and entropy coding to generate an image stream IS. Conversely, in the context of intra-image coding techniques, the image coding apparatus 100 performs processing such as, but not limited to, discrete cosine transform, quantization, and entropy coding on the current image frame to generate an image stream IS.

[0019] It should be noted that for the same image frame, the image encoding device 100 can use either inter-image encoding technology or intra-image encoding technology to encode different image blocks.

[0020] The image decoding device 110 is configured to receive and decode the image stream IS to generate multiple decoded image frames FD, for example... Figure 1 The (N-1)th decoded image frame FD shown N-1 The Nth decoded image frame FD N ..., the N+Pth decoded image frame FD N+P The image decoding device 110 includes a front-end decoding circuit 120 and a back-end decoding circuit 130.

[0021] The front-end decoding circuit 120 is configured to perform processing on the video stream IS, such as, but not limited to, entropy decoding, inverse quantization (IQ), and inverse discrete cosine transform (IT), to extract motion vector information MV and residual information RES corresponding to different video frames.

[0022] The back-end decoding circuit 130 is configured to perform a decoding process. In one embodiment, the back-end decoding circuit 130 includes an estimation circuit 140, an image reconstruction circuit 150, and a deblocking operation circuit 160.

[0023] Corresponding to the inter-image coding technique, the estimation circuit 140 calculates and generates multiple estimated pixels EP based on the reference image data RD and motion vector information MV during the decoding process. The image reconstruction circuit 150 calculates and generates multiple reconstructed pixels RP based on the estimated pixels EP and residual information RES. The deblocking circuit 160 performs deblocking operations based on the reconstructed pixels RP to generate a decoding result, and then outputs the decoded image frame FD based on the decoding result. The decoded image frame FD can be fed back to the estimation circuit 140 as the reference image data RD.

[0024] In the corresponding intra-image coding technique, since motion compensation is not required, the front-end decoding circuit 120 does not need to extract the corresponding motion vector information MV, but only the corresponding residual information RES. The estimation circuit 140 can calculate and generate estimated pixels EP based on the decoded portion of the same image frame as reference image data RD. The image reconstruction circuit 150 calculates and generates reconstructed pixels RP based on the estimated pixels EP and the residual information RES. The reconstructed pixels RP can be fed back to the estimation circuit 140 as the reference image data RD as the decoded portion of the same image frame. The deblocking circuit 160 performs deblocking operations based on the reconstructed pixels RP to generate a decoding result, and then outputs the decoded image frame FD based on the decoding result.

[0025] In some usage scenarios, data loss may occur in the video stream IS, causing the video decoding device 110 to detect an error. The following will use the (N-1)th video frame FM of the video stream IS as an example. N-1 The Nth video frame FM N ..., the N+Pth video frame FM N+P The Nth video frame in FM N Using the detection of errors as an example, the processing mechanism of the image decoding device 110 will be explained.

[0026] Please refer to Figure 2A . Figure 2A This diagram illustrates multiple image frames transmitted by an image stream IS in one embodiment of the present invention. More specifically, in Figure 2A The image in the middle is the (N-1)th image frame (FM) transmitted by the video stream IS. N-1 The Nth video frame FM N And the N+1th video frame FM N+1 .

[0027] In one embodiment, each image frame in the image stream IS is like the Nth image frame FM. N The image shown contains multiple image processing blocks. These image processing blocks have a size that can be read and processed by the front-end decoding circuit 120, for example, a size of 64×64. Figure 2A In the image, only one image processing block 200 is represented. Each image processing block contains multiple coding unit blocks. These coding unit blocks are the block size for one encoding operation by the image coding device 100, and can have different sizes such as 32×32, 16×16, 8×8, etc. Figure 2A In this example, only one coding unit block 210 is shown in the image processing block 200. In reality, the image processing block 200 may contain multiple coding unit blocks of the same or different sizes. It should be noted that the sizes of the image processing blocks and coding unit blocks described above are merely examples. The invention is not limited thereto.

[0028] The front-end decoding circuit 120 generates the Nth image frame FM from the image stream IS from the image encoding device 100 according to the inter-image coding technique. N Determine the occurrence of erroneous block EB.

[0029] Once the front-end decoding circuit 120 determines that an error block EB has occurred, the front-end decoding circuit 120 transmits error report information ER to the video encoding device 100, so that the video encoding device 100 can receive the N+P-1th video frame FM in the video stream from the video decoding device 110. N+P-1 When the corresponding block CB is reached, an error report message ER is received. In this embodiment, P is 1. In other words, in Figure 2A In the image, the corresponding block CB is located in the Nth image frame FM. N middle.

[0030] The image encoding device 100 will view the N+Pth image frame in FM mode. N+P (In this embodiment, the N+1th image frame is FM) N+1 This is a new image slice, referencing the N+P-1th image frame FM according to inter-image coding techniques. N+P-1 (In this embodiment, the Nth image frame is FM) N The reference image frame that did not previously cause an error is compared to the N+Pth image frame FM in the video stream IS. N+P Encoding (in this embodiment, the N+1th image frame FM) N+1 In one embodiment, this reference image frame is the image frame preceding the (N+P-1)th image frame, i.e., the (N+P-2)th image frame FM. N+P-2 (In this embodiment, the N-1th image frame is FM) N-1 ).

[0031] The front-end decoding circuit 120 captures the corresponding Nth video frame (FM) from the video stream IS. NThe movement vector information MV and residual information RES of multiple first blocks 220 prior to the occurrence of the erroneous block EB. Figure 2A In the diagram, the first block 220 before the occurrence of error block EB is depicted as a block with a forward slash.

[0032] The back-end decoding circuit 130 performs a decoding procedure on the first block 200 according to the image inter-coding technology to generate the first decoding result DR1.

[0033] More specifically, for the Nth image frame FM N The back-end decoding circuit 130 captures previously decoded image frames according to inter-image coding techniques (e.g., the estimation circuit 140 receives the decoded image frame FD fed back from the deblocking operation circuit 160). N-1 The first block 220 is decoded using the reference image data RD. In the decoding process, the estimation circuit 140 calculates and generates the estimated pixel EP based on the reference image data RD and the motion vector information MV. The image reconstruction circuit 150 calculates and generates the reconstructed pixel RP based on the estimated pixel EP and the residual information RES. The deblocking operation circuit 160 performs deblocking operation based on the reconstructed pixel RP to generate the first decoding result DR1.

[0034] On the other hand, the front-end decoding circuit 120 FMs the (N-1)th image frame identified as a panning image frame. N-1 The translation vector is extracted, and the movement vector information MV is set as the translation vector, while the residual information RES is set to zero. The method for extracting the translation vector and the method for identifying the image frame as a translated image frame will be discussed later.

[0035] The back-end decoding circuit 130 uses inter-image coding technology to decode the Nth image frame using FM. N Error block EB to the N+P-1th image frame FM N+P-1 (In this embodiment, the Nth image frame is FM) N The last block of the FM video feed contains multiple second blocks 230 that sequentially refer to the (N-1)th video frame. N-1 Up to the N+P-2th video frame FM N+P-2 The decoding process generates a second decoding result, DR2. Figure 2A In the middle, the error block EB to the Nth image frame FM N The second block 230 of the last block is drawn as a dotted block.

[0036] More specifically, the back-end decoding circuit 130 corresponds to the (N-1)th video frame in FM mode. N-1 Previous decoded video footage FD N-1As reference image data RD, the second block 230 is decoded. The estimation circuit 140 calculates and generates an estimated pixel EP based on the reference image data RD and the translation vector information MV. The image reconstruction circuit 150 calculates and generates a reconstructed pixel RP based on the estimated pixel EP and the residual value information RES (which is zero). The deblocking circuit 160 performs a deblocking operation based on the reconstructed pixel RP to generate a second decoding result DR2.

[0037] The back-end decoding circuit 130 outputs the first decoding result DR1 and the second decoding result DR2 as the Nth decoded image frame FD. N More specifically, after generating the first decoding result DR1 and the second decoding result DR2, the deblocking circuit 160 of the back-end decoding circuit 130 merges the first decoding result DR1 and the second decoding result DR2 to output the Nth decoded image frame FD. N .

[0038] Therefore, in this embodiment, the Nth image frame FM N The first block 220 in the reference is the (N-1)th image frame FM. N-1 Multiple blocks 240 in the image are generated based on the motion vector information MV and residual information RES transmitted from the image encoding device 100. Figure 2A In the middle, block 240 is drawn as a blank block. On the other hand, the Nth image frame FM N The second block 230 in the image references the (N-1)th image frame FM based on the translation vector. N-1 Multiple blocks 250 within. Figure 2A In the diagram, block 250 is depicted as a gray block.

[0039] In this embodiment, the N+Pth image frame FM N+P (In this embodiment, the N+1th image frame is FM) N+1 It contains only multiple inter-image reference blocks 260. Figure 2A In the image reference block 260, the area between images is drawn as a reverse diagonal block.

[0040] The front-end decoding circuit 120 captures the corresponding (N+1)th video frame (FM) from the video stream IS. N+1 The image reference block 260 contains the motion vector information MV and residual information RES.

[0041] The back-end decoding circuit 130 performs a decoding procedure on the inter-image reference block 260 according to the inter-image coding technology to generate the inter-image reference decoding result ITE.

[0042] More specifically, the back-end decoding circuit 130 uses inter-image coding technology to extract previously decoded image frames as reference image data RD to perform a decoding process on the inter-image reference block 260. The previously decoded image frame is the Nth image frame FM. N The decoded video frame corresponding to the previous video frame where no error occurred, such as the previous video frame, i.e., the (N-1)th video frame FM. N-1 The corresponding decoded image screen. The estimation circuit 140, the image reconstruction circuit 150, and the deblocking operation circuit 160 sequentially perform the decoding process to generate the inter-image reference decoding result ITE.

[0043] The back-end decoding circuit 130 further outputs the inter-image reference decoding result ITE as the (N+1)th decoded image frame FD. N+1 More specifically, after generating the inter-image reference decoding result ITE, the deblocking operation circuit 160 of the back-end decoding circuit 130 will output the inter-image reference decoding result ITE as the (N+1)th decoded image frame FD. N+1 .

[0044] Figure 2B This diagram illustrates multiple image frames transmitted by an image stream IS in one embodiment of the present invention. More specifically, in Figure 2B The image in the middle is the (N-1)th image frame (FM) transmitted by the video stream IS. N-1 The Nth video frame FM N The N+1th video frame FM N+1 And the N+2nd video frame FM N+2 .

[0045] The aforementioned image frame includes content such as image processing blocks and, more specifically, encoding unit blocks within those image processing blocks. Figure 2A The same applies, so I won't repeat it here.

[0046] Similar to correspondence Figure 2A The operation mode of the front-end decoding circuit is 120 pairs of the Nth image frame FM. N The system determines the occurrence of an erroneous block EB and transmits an error report message ER to the image encoding device 100, so that the image encoding device 100 can receive the N+P-1th image frame FM from the image stream in the image decoding device 110. N+P-1 When the corresponding block CB is reached, an error report message ER is received. In this embodiment, the value of P is greater than 1, and in Figure 2B This is illustrated using a value of P equal to 2. In other words, in Figure 2B In the image, the corresponding block CB is located in the (N+1)th image frame FM. N+1 middle.

[0047] The image encoding device 100 will view the N+Pth image frame in FM mode. N+P (In this example, the N+2th video frame is FM) N+2 () is a new image slice, which is FMed according to the inter-image coding technique for the N+Pth image frame in the image stream IS. N+P Encoding. Similarly, in one embodiment, this reference image frame is the image frame preceding the Nth image frame, i.e., the (N-1)th image frame FM. N-1 .

[0048] The front-end decoding circuit 120 and the back-end decoding circuit 130 FM the Nth image frame. N The processing method performed on the first block 220 before the occurrence of the erroneous block EB is the same as Figure 2A The same applies, and the first decoding result DR1 is generated by the back-end decoding circuit 130, which will not be described in detail here.

[0049] On the other hand, the front-end decoding circuit 120 FMs the (N-1)th image frame identified as a panning image frame. N-1 Extract the translation vector, set the movement vector information MV as the translation vector, and set the residual information RES to zero.

[0050] The back-end decoding circuit 130 uses inter-image coding technology to decode the Nth image frame using FM. N Error block EB to the N+P-1th image frame FM N+P-1 (In this example, the N+1th video frame is FM) N+1 The last block of the FM video feed contains multiple second blocks 230 that sequentially refer to the (N-1)th video frame. N-1 Up to the N+P-2th video frame FM N+P-2 The decoding process generates a second decoding result, DR2. Figure 2B In the middle, the error block EB to the N+1th image frame FM N+1 The second block 230 of the last block is drawn as a dotted block.

[0051] More specifically, the back-end decoding circuit 130 corresponds to the (N-1)th video frame in FM mode. N-1 Previous decoded video footage FD N-1 As reference image data RD, for the Nth image frame FM N The second block 230 references the previously decoded image frame FD based on the translation vector. N-1 A decoding process is performed, in which the estimation circuit 140, the image reconstruction circuit 150, and the deblocking operation circuit 160 are sequentially processed to produce the first part of the second decoding result DR2.

[0052] The back-end decoding circuit 130 outputs the first decoding result DR1 and the first part of the second decoding result DR2 as the Nth decoded image frame FD. N More specifically, after generating the first decoding result DR1 and the first part of the second decoding result DR2, the deblocking circuit 160 of the back-end decoding circuit 130 merges the first decoding result DR1 and the first part of the second decoding result DR2 to output the Nth decoded image frame FD. N .

[0053] Furthermore, the back-end decoding circuit 130 corresponds to the Nth image frame in FM mode. N Previous decoded video footage FD N As reference image data RD, for the N+1th image frame FM N+1 The second block 230 references the previously decoded image frame FD based on the translation vector. N A decoding process is performed, in which the estimation circuit 140, the image reconstruction circuit 150, and the deblocking operation circuit 160 are processed sequentially to produce the second part of the second decoding result DR2.

[0054] The back-end decoding circuit 130 outputs the second part of the second decoding result DR2 as the (N+1)th decoded image frame FD. N+1 More specifically, after generating the second part of the second decoding result DR2, the deblocking operation circuit 160 of the back-end decoding circuit 130 will output the second part of the second decoding result DR2 as the (N+1)th decoded image frame FD. N+1 .

[0055] Since the motion vector information MV is a translation vector and the residual information RES is zero, the first part of the second decoding result DR2 (i.e., the Nth decoded image frame FD) N The portion corresponding to the second block 230 is based on the translation vector reference image data RD, that is, the (N-1)th decoded image frame FD. N-1 This corresponds to the decoding of block 250. The second part of the second decoding result DR2 (i.e., the N+1th decoded image frame FD) N+1 The content of the reference image data RD is based on the translation vector reference, which is the Nth decoded image frame FD. N It is obtained through decoding.

[0056] It should be noted that when P is any integer greater than 2, the back-end decoding circuit 130 can output the second part of the second decoding result DR2 as the (N+1)th to (N+P-1)th decoded image frames in the manner described above. Furthermore, when P is any integer greater than 2, the (N+1)th to (N+P-1)th decoded image frames will be generated sequentially based on a reference to the previous decoded image frame. That is, the (N+1)th decoded image frame is generated by referencing the Nth decoded image frame, the (N+2)th decoded image frame is generated by referencing the (N+1)th decoded image frame, and so on, until the (N+P-1)th decoded image frame is generated by referencing the (N+P-2)th decoded image frame.

[0057] In this embodiment, the N+Pth image frame FM N+P (In this embodiment, the N+2th image frame is FM) N+2 It contains only multiple inter-image reference blocks 260. Figure 2A In the image reference block 260, the area between images is drawn as a reverse diagonal block.

[0058] The front-end decoding circuit 120 captures the corresponding (N+2)th video frame (FM) from the video stream IS. N+2 The image reference block 260 contains the motion vector information MV and residual information RES.

[0059] The back-end decoding circuit 130 performs a decoding procedure on the inter-image reference block 260 according to the inter-image coding technique to generate an inter-image reference decoding result ITE, and the decoding procedure performed by the back-end decoding circuit 130 is consistent with the corresponding... Figure 2A The decoding procedure is the same and will not be repeated here.

[0060] The back-end decoding circuit 130 further outputs the inter-image reference decoding result ITE as the N+Pth decoded image frame FD. N+P (In this example, the N+2nd decoded image frame is FD) N+2 More specifically, after generating the inter-image reference decoding result ITE, the deblocking operation circuit 160 of the back-end decoding circuit 130 will output the inter-image reference decoding result ITE as the N+2th decoded image frame FD. N+2 .

[0061] In one embodiment, in both of the above embodiments, the image encoding device 100 can, without receiving the image decoding device 110's FM image related to the (N+P)th image frame... N+2 When checking the error report, determine the N+Pth video frame FM. N+2No error block occurred, and the next (N+P+1) video frame is referenced to the previously preset video frame using inter-video coding technology (e.g., the previous video frame, i.e., the N+Pth video frame FM). N+2 The image decoding device 110 uses the corresponding decoded image frame instead of the decoded image frame corresponding to the reference image frame that did not have an error before the Nth image frame. Therefore, the image decoding device 110 can extract the corresponding motion vector information and residual information based on the inter-image coding technique to perform the decoding process. However, the present invention is not limited thereto.

[0062] Based on the above methods Figure 1 The back-end decoding circuit 130 can FM the Nth image frame error block to the N+P-1th image frame based on the N-1th image frame identified as a panning image frame. N-1 The translation vector is used for decoding to maintain the smoothness of the decoded image.

[0063] The following will first explain how the translation vector is obtained.

[0064] Please refer to Figure 3 . Figure 3 This diagram illustrates an image processing block IB in an image frame according to one embodiment of the present invention. In one embodiment, the image processing block IB is 64×64 in size and includes encoding unit blocks 300 to 312.

[0065] For each image frame, the front-end decoding circuit 120 extracts multiple unit block movement vectors corresponding to the encoding unit block in each image processing block.

[0066] by Figure 3 Taking coding unit blocks 300-312 as examples, coding unit block 300 has a size of 32×32 and a unit block movement vector MV1 of (5,0). Coding unit block 301 has a size of 16×16 and a unit block movement vector MV2 of (-2,0). Coding unit block 302 has a size of 16×16 and a unit block movement vector MV3 of (4,1). Coding unit block 303 has a size of 16×16 and is a block coded according to the intra-image coding technique, but does not have a unit block movement vector, and is represented by N / A.

[0067] Coding unit block 304 has a size of 8×8 and is a block coded according to intra-image coding techniques without a unit block movement vector, denoted as N / A. Coding unit block 305 has a size of 8×8 and has a unit block movement vector MV1 of (5,0). Coding unit block 306 has a size of 8×8 and has a unit block movement vector MV1 of (5,0). Coding unit block 307 has a size of 8×8 and has a unit block movement vector MV4 of (6,0).

[0068] Coding unit block 308 has a size of 16×16 and a unit block movement vector MV1 of (5,0). Coding unit block 309 has a size of 16×16 and is a block coded according to intra-image coding technology without a unit block movement vector, denoted as N / A. Coding unit block 310 has a size of 16×16 and is a block coded according to intra-image coding technology without a unit block movement vector, denoted as N / A. Coding unit block 311 has a size of 16×16 and a unit block movement vector MV5 of (12,7). Coding unit block 312 has a size of 32×32 and a unit block movement vector MV6 of (4,0).

[0069] The front-end decoding circuit 120 performs a first statistical procedure on the unit block movement vector corresponding to each image processing block of each image frame. In one embodiment, the front-end decoding circuit 120 performs the following steps for each image processing block in the first statistical procedure.

[0070] First, the front-end decoding circuit 120 sets the first weight of each coding unit block according to the size of the coding unit block. Figure 3 In one embodiment, when 8×8 is a unit weight value, the coding unit blocks 300 to 312 have first weights of 16, 4, 4, 4, 1, 1, 1, 1, 4, 4, 4, 4, 12 unit weight values, respectively.

[0071] Next, the front-end decoding circuit 120 sums the first weights of the coding unit blocks corresponding to the movement vector of each unit block to generate a cumulative first weight value for the movement vector of each unit block. Figure 3 In one embodiment, the front-end decoding circuit 120 superimposes the first weights of coding unit blocks 300, 305, 306, and 308 to obtain the first weight cumulative value 22 (=16+1+1+4) of the unit block movement vector MV1. Since each of the unit block movement vectors MV2 to MV6 corresponds to only one coding unit block, the first weight cumulative values ​​of the unit block movement vectors MV2 to MV6 are 4, 4, 1, 4, and 16, respectively.

[0072] The front-end decoding circuit 120 determines the largest unit block movement vector with the largest first weight cumulative value and the second largest unit block movement vector with the second largest first weight cumulative value among the unit block movement vectors. Figure 3 In one embodiment, the front-end decoding circuit 120 determines that the unit block movement vector MV1 with the largest first weight cumulative value 22 is the largest unit block movement vector, and determines that the unit block movement vector MV6 with the second largest first weight cumulative value 16 is the second largest unit block movement vector.

[0073] The front-end decoding circuit 120 superimposes the first weighted cumulative value corresponding to at least one unit block movement vector that has a vector value differing from the largest unit block movement vector by a preset range, and the largest first weighted cumulative value, to generate a first unit block cumulative value corresponding to the largest unit block movement vector. In one embodiment, the vector value can be defined as the result of adding the X and Y values ​​of each movement vector. The preset range can be defined as the vector value of the largest unit block movement vector plus or minus, for example, but not limited to, a range of 6. Figure 3 In this embodiment, since the largest unit block movement vector is (5,0), its vector value is 5+0=5. Therefore, the first weighted cumulative values ​​4, 1, and 16 of the unit block movement vectors MV3, MV4, and MV6, whose vector values ​​are between -1 and 11, are all superimposed with the largest first weighted cumulative value 22 to generate the first unit block cumulative value 43 (=22+4+1+16) of the corresponding largest unit block movement vector MV1. It should be noted that the above definition of vector values ​​and preset ranges is only an example. The present invention is not limited thereto.

[0074] The front-end decoding circuit 120 superimposes the first weighted cumulative value and the second largest first weighted cumulative value corresponding to at least one unit block movement vector that has a vector value differing from the second largest unit block movement vector within a preset range, to generate a second unit block cumulative value corresponding to the second largest unit block movement vector. Figure 3 In an embodiment, under the same numerical example, the second largest unit block movement vector MV6 is (4,0), and its vector value is 4+0=4. Therefore, the first weighted cumulative values ​​22, 4, and 1 of the unit block movement vectors MV1, MV3, and MV4, whose vector values ​​are in the preset range of -2 to 10, are all superimposed with the second largest first weighted cumulative value 16 to generate the second unit block cumulative value 43 (=16+22+4+1) corresponding to the second largest unit block movement vector MV6.

[0075] The front-end decoding circuit 120 determines the largest cumulative value of the first and second unit blocks, and selects the unit block movement vector corresponding to the largest cumulative value from the largest and second largest unit block movement vectors as the most frequently occurring unit block movement vector. Figure 3 In this embodiment, both the first unit block cumulative value and the second unit block cumulative value are 43. The front-end decoding circuit 120 can select either one, for example, the first unit block cumulative value is the maximum unit block cumulative value, and select the corresponding maximum unit block movement vector MV1 as the unit block movement vector with the highest frequency of occurrence.

[0076] Furthermore, the front-end decoding circuit 120 sets multiple processing block movement vectors corresponding to each image processing block, wherein the processing block movement vectors are the unit block movement vectors that appear most frequently in each image processing block. Figure 3 In one embodiment, the front-end decoding circuit 120 sets the most frequently occurring unit block movement vector MV1 in the image processing block 300 as the processing block movement vector of the image processing block 300.

[0077] After setting the processing block movement vectors corresponding to all image processing blocks of an image frame in the above manner, the front-end decoding circuit 120 performs a second statistical procedure on the processing block movement vectors corresponding to the image processing blocks of each image frame.

[0078] The second statistical procedure is essentially the same as the first statistical procedure, except that the role of the coding unit block is replaced by the image processing block, and the role of the unit block movement vector is replaced by the processing block movement vector.

[0079] More specifically, in the second statistical procedure, the front-end decoding circuit 120 sets a second weight for each image processing block according to the size of the image processing block, and superimposes the second weights of the image processing blocks corresponding to the movement vector of each processing block to generate a cumulative value of the second weight of the movement vector of each processing block.

[0080] Furthermore, the front-end decoding circuit 120 determines the largest processing block movement vector with the largest second weight cumulative value and the second largest processing block movement vector with the second largest second weight cumulative value among the processing block movement vectors. It then superimposes the second weight cumulative value and the largest second weight cumulative value corresponding to at least one processing block movement vector that has a vector value that differs from the largest processing block movement vector within a preset range to generate a first processing block cumulative value corresponding to the largest processing block movement vector. It also superimposes the second weight cumulative value and the second largest second weight cumulative value corresponding to at least one processing block movement vector that has a vector value that differs from the second largest processing block movement vector within a preset range to generate a second processing block cumulative value corresponding to the second largest processing block movement vector.

[0081] Finally, the front-end decoding circuit 120 determines the maximum cumulative value of the processing block among the cumulative values ​​of the first and second processing blocks, and selects the processing block movement vector that corresponds to the maximum cumulative value from the maximum and second largest processing block movement vectors as the processing block movement vector with the highest frequency of occurrence.

[0082] Furthermore, the front-end decoding circuit 120 sets multiple frame movement vectors corresponding to the image frames, where each frame movement vector is the movement vector of the processing block that appears most frequently in each image frame. The front-end decoding circuit 120 can capture the (N-1)th image frame FM. N-1 The corresponding screen movement vector is used as the translation vector.

[0083] The following will explain how to identify a panning image.

[0084] For each image frame, the front-end decoding circuit 120 calculates the sum of the second weighted cumulative values ​​of the motion vectors of all processed blocks, and further calculates the maximum motion vector ratio between the maximum processed block cumulative value and the sum. In a numerical example, the maximum processed block cumulative value of an image frame is 40, the sum of the second weighted cumulative values ​​of the motion vectors of all processed blocks is 50, and the maximum motion vector ratio will therefore be 0.8.

[0085] For each image frame, the front-end decoding circuit 120 sequentially determines that the most frequently occurring processing block motion vector (i.e., the image motion vector of an image frame) and the image motion vectors of the K consecutive previous image frames all have vector values ​​that differ within a specific range, so that the threshold value decreases by K preset values ​​without being less than the limit value. Furthermore, the front-end decoding circuit 120 identifies any image frame whose maximum motion vector ratio is greater than the threshold value as a panning image frame.

[0086] In a numerical example, with a preset threshold of 0.9, a specific range of 6, a limit of 0.6, and a preset value of 0.05, if the motion vector of an image frame differs from the motion vectors of zero previous images by less than 6, the threshold will not decrease and will remain at 0.9. In this case, the aforementioned image frame will not be identified as a panning image frame because the maximum motion vector ratio (0.8) is less than the threshold value (0.9). However, if the motion vector of an image frame differs from the motion vectors of three previous images by less than 6, the threshold will decrease by 3 × 0.05 = 0.15 while remaining at a value not less than 0.6, resulting in a threshold value of 0.75. In this case, the aforementioned image frame will be identified as a panning image frame because the maximum motion vector ratio (0.8) is greater than the threshold value (0.75).

[0087] In one embodiment, the front-end decoding circuit 120 may dynamically lower the threshold value by storing the motion vectors of multiple image frames through, for example, but not limited to, a first-in-first-out circuit (not shown) further included in the image decoding device 110, so that when the motion vector values ​​of the image frame are close to those of the previous consecutive image frames, it is easier to be judged as a translational image frame.

[0088] It should be noted that when a scene has a static background and only a moving foreground object, the vector value of the motion vector in the most frequently processed block may be zero. Therefore, the front-end decoding circuit 120 may not adjust the threshold value when the corresponding motion vector value is zero, in order to avoid misjudging the scene as a panning image.

[0089] In some technologies, when data loss occurs in the received video stream, the image decoding device will be unable to extract the motion vector information and residual information of the image, and thus will be unable to correctly generate the decoded image.

[0090] The image decoding device of the present invention, when determining that an erroneous block occurs in the Nth image frame, extracts a translation vector from the (N-1)th image frame identified as a translational image frame, sets the movement vector information to the translation vector and the residual value information to zero, and decodes these image blocks from the erroneous block up to the last block of the (N+P-1)th image frame where the corresponding block for which the image encoding device receives the error report information. This method allows the decoding process to continue and maintains the smoothness of the decoded image frame while improving the accuracy of decoding.

[0091] Please refer to Figure 4 . Figure 4 This diagram shows a flowchart of an image decoding method 400 in one embodiment of the present invention.

[0092] In addition to the aforementioned apparatus, the present invention also discloses an image decoding method 400, applicable to, for example, but not limited to, [other applications]. Figure 1 In the image decoding apparatus 110. One of the image decoding methods 400 is implemented, for example. Figure 4 As shown, it includes the following steps.

[0093] In step S410, the Nth image frame FM generated from the image stream IS from the image encoding device 100 according to the inter-image coding technique is... N Determine if an erroneous block EB has occurred, where N is an integer greater than 1.

[0094] In step S420, error reporting information ER is transmitted to the image encoding device 100, so that the image encoding device 100 receives the N+P-1th image frame FM in the image stream from the image decoding device 110. N+P-1 When the corresponding block CB receives the error report information ER, the image encoding device 100 encodes the (N+P)th image in the image stream IS using inter-image encoding techniques, referencing a reference image that did not experience an error before the Nth image. N+P , where P is an integer greater than or equal to 1.

[0095] In step S430, the corresponding Nth image frame FM is captured from the image stream IS. N The motion vector information MV and residual information RES of the first block 220 before the occurrence of the error block are used to generate the first decoding result DR1 by decoding the first block 220 according to the inter-image coding technique.

[0096] In step S440, from the (N-1)th image frame identified as a panning image frame FM N-1 Extract the translation vector, set the motion vector information MV as the translation vector, and set the residual information RES to zero, so as to process the image from the Nth image frame FM according to the inter-image coding technique. N Error block up to the N+P-1th video frame FM N+P-1 The second block 230 of the last block sequentially refers to the N-1th to N+P-2th image frames to perform the decoding process and generate the second decoding result DR2.

[0097] In step S450, the corresponding N+P-th image frame FM is captured from the image stream IS. N+P The motion vector information MV and residual information RES of the inter-image reference block 260 are used to generate the inter-image reference decoding result ITE by decoding the inter-image reference block 260 according to the inter-image coding technique.

[0098] It should be noted that the above-described implementation is merely an example. In other embodiments, those skilled in the art can make modifications without departing from the spirit of the invention.

[0099] In summary, the image decoding apparatus and method of the present invention, when determining that an erroneous block occurs in the Nth image frame, extracts a translation vector from the (N-1)th image frame identified as a translational image frame, sets the movement vector information as a translation vector and sets the residual value information to zero, and decodes these image blocks from the erroneous block up to the last block of the (N+P-1)th image frame where the corresponding block for which the image encoding device receives the error report information. This approach allows the decoding process to continue while maintaining the smoothness of the decoded image frame, thus improving the accuracy of the decoding.

[0100] Although the embodiments of this case are described above, these embodiments are not intended to limit this case. Those skilled in the art can make changes to the technical features of this case based on the express or implied content of this case. All such changes may fall within the scope of patent protection sought in this case. In other words, the scope of patent protection in this case shall be determined by the scope of the patent application in this specification.

[0101] [Symbol Explanation]

[0102] 100: Image encoding device

[0103] 110: Image decoding device

[0104] 120: Front-end decoding circuit

[0105] 130: Back-end decoding circuit

[0106] 140: Estimation Circuit

[0107] 150: Image Reconstruction Circuit

[0108] 160: Deblocking circuit

[0109] 200: Image Processing Block

[0110] 210: Coding unit block

[0111] 220: Block 1

[0112] 230: Second Block

[0113] Blocks 240 and 250

[0114] 260: Image Inter-reference Block

[0115] 300~312: Coding unit blocks

[0116] 400: Image Decoding Method

[0117] S410~S450: Steps

[0118] CB: Corresponding block

[0119] DR1: First Decoding Result

[0120] DR2: Second Decoding Result

[0121] EB: Error Block

[0122] EP: Estimated Pixels

[0123] ER: Error Reporting Information

[0124] FD: Decoded video footage

[0125] FD N-1 The (N-1)th decoded image frame

[0126] FD N The Nth decoded image frame

[0127] FD N+1 The (N+1)th decoded image frame

[0128] FD N+2 The (N+2)th decoded image frame

[0129] FD N+P The N+Pth decoded image frame

[0130] FM: Video feed

[0131] FM N-1 The (N-1)th image frame

[0132] FM N The Nth image frame

[0133] FM N+1 The (N+1)th image frame

[0134] FM N+2 The (N+2)th image frame

[0135] FM N+P The N+Pth image frame

[0136] IB: Image Processing Block

[0137] IS: Video Streaming

[0138] ITE: Inter-image reference decoding results

[0139] MV: Movement Vector Information

[0140] MV1~MV6: Movement Vectors

[0141] RD: Reference Image Data

[0142] RES: Residual Value Information

[0143] RP: Reconstructed Pixels

Claims

1. An image decoding method, applied in an image decoding device, comprising: For the video stream from the video encoding device, the occurrence of an error block is determined based on the Nth video frame generated by the inter-video encoding technique, where N is an integer greater than 1; Error reporting information is transmitted to the image encoding device so that the image encoding device receives the error reporting information when the image decoding device receives the corresponding block of the N+P-1th image frame in the image stream, and the image encoding device encodes the N+Pth image frame in the image stream with reference to the reference image frame before the Nth image frame that did not have an error, according to the inter-image coding technique, where P is an integer greater than or equal to 1; The motion vector information and residual information of multiple first blocks before the occurrence of the error block in the Nth image frame are extracted from the image stream, so as to generate a first decoding result by decoding the first blocks according to the inter-image coding technology. A translation vector is extracted from the (N-1)th image frame identified as a translation image frame. This translation vector information is set as the translation vector, and the residual value information is set to zero. Then, according to the inter-image coding technique, a second decoding result is generated by sequentially referencing the (N-1)th to (N+P-2)th image frames to perform the decoding process on multiple second blocks from the erroneous block of the Nth image frame to the last block of the (N+P-1)th image frame. The motion vector information and residual information of multiple inter-image reference blocks corresponding to the N+Pth image frame are extracted from the image stream, and the inter-image reference blocks are decoded according to the inter-image coding technology to generate inter-image reference decoding results.

2. An image decoding device, comprising: The front-end decoding circuit is configured as follows: In the video stream from the video encoding device, the occurrence of an error block is determined based on the Nth video frame generated by inter-video coding technology, where N is an integer greater than 1; and Error reporting information is transmitted to the image encoding device so that the image encoding device receives the error reporting information when the image decoding device receives the corresponding block of the N+P-1th image frame in the image stream, and the image encoding device encodes the N+Pth image frame in the image stream with reference to the reference image frame before the Nth image frame that did not have an error, according to the inter-image coding technique, where P is an integer greater than or equal to 1; The back-end decoding circuit is configured as follows: When the front-end decoding circuit extracts the motion vector information and residual information of multiple first blocks corresponding to the Nth image frame before the occurrence of the error block from the image stream, it performs a decoding procedure on the first blocks according to the inter-image coding technology to generate a first decoding result. When the front-end decoding circuit extracts a translation vector from the (N-1)th image frame identified as a translation image frame, sets the motion vector information as the translation vector, and sets the residual value information to zero, it performs the decoding process on multiple second blocks from the erroneous block of the Nth image frame to the last block of the (N+P-1)th image frame in sequence, referencing the (N-1)th to (N+P-2)th image frames, according to the inter-image coding technique, to generate a second decoding result; and When the front-end decoding circuit extracts the motion vector information and residual information of multiple inter-image reference blocks corresponding to the N+Pth image frame from the image stream, it performs the decoding procedure on these inter-image reference blocks according to the inter-image coding technology to generate inter-image reference decoding results.

3. The image decoding apparatus according to claim 2, wherein the back-end decoding circuit is configured to: When P is 1, the first decoding result and the second decoding result are output as the Nth decoded image frame, and the inter-image reference decoding result is output as the N+Pth decoded image frame; and When P is greater than 1, the first decoding result and the first part of the second decoding result are output as the Nth decoded image frame, the second part of the second decoding result is output as the N+1th to N+P-1th decoded image frames, and the inter-image reference decoding result is output as the N+Pth decoded image frame.

4. The image decoding apparatus according to claim 2, wherein the back-end decoding circuit is configured to: The decoding process uses previously decoded image frames as reference image data, based on the inter-image coding technique; and The decoded portion of the N+Pth image frame is set as the reference image data for the decoding process based on the image coding technology.

5. The image decoding apparatus according to claim 4, wherein the reference image frame is the (N-1)th image frame, and the back-end decoding circuit is configured to: For the N+Pth image frame, the previously decoded image frame corresponding to the N-1th image frame is extracted as the reference image data for the decoding process according to the inter-image coding technique.

6. The image decoding apparatus according to claim 5, wherein the back-end decoding circuit comprises: The estimation circuit is configured to calculate and generate multiple estimated pixels in the decoding process based on the reference image data and the motion vector information, according to the inter-image coding technique. An image reconstruction circuit is configured to calculate and generate a plurality of reconstructed pixels in the decoding process based on the estimated pixels and the residual information. as well as The deblocking circuit is configured to perform deblocking operations on the reconstructed pixels in the decoding process to generate the first decoding result for the first block and the second decoding result for the second block.

7. The image decoding apparatus according to claim 5, wherein the back-end decoding circuit comprises: An estimation circuit is configured to calculate and generate multiple estimated pixels based on the reference image data in the decoding process, according to the inter-image coding technique. Image reconstruction circuitry, configured to calculate and generate a plurality of reconstructed pixels in the decoding process based on the estimated pixels and the residual information; and The deblocking circuit is configured to perform deblocking operations on the reconstructed pixels in the decoding process to generate an inter-image reference decoding result corresponding to the inter-image reference blocks.

8. The image decoding apparatus according to claim 2, wherein each of the plurality of image frames in the image stream comprises a plurality of image processing blocks, and the image processing blocks respectively comprise a plurality of encoding unit blocks, and the front-end decoding circuit is further configured to: For each of the image frames, extract multiple unit block movement vectors corresponding to the coding unit blocks in each of the image processing blocks; A first statistical procedure is performed on the unit block movement vector corresponding to each of the image processing blocks of each of the image frames; For each of the image frames, set multiple processing block movement vectors corresponding to the image processing blocks, wherein each of the processing block movement vectors is one of the most frequently occurring unit block movement vectors in each of the image processing blocks. A second statistical procedure is performed on the movement vector of the processing block corresponding to the processing block of each of the image frames; Multiple frame movement vectors are set for the corresponding image frames, wherein each frame movement vector is a movement vector of one of the most frequently occurring processing blocks in each of the image frames; and Extract one of the image movement vectors corresponding to the (N-1)th image frame as the translation vector.

9. The image decoding apparatus according to claim 8, wherein the front-end decoding circuit is further configured to: In this first statistical procedure, for each of these image processing blocks: Set the first weight of each of the coding unit blocks according to the size of the coding unit blocks; The first weights of the corresponding coding unit blocks for each of the such unit block movement vectors are superimposed to generate the first weight cumulative value of each of the such unit block movement vectors; Determine the largest unit block movement vector with the largest first weight cumulative value and the second largest unit block movement vector with the second largest first weight cumulative value among these unit block movement vectors; The first weight cumulative value corresponding to at least one of the unit block movement vectors that has a vector value that differs from the maximum unit block movement vector within a preset range is superimposed with the maximum first weight cumulative value to generate a first unit block cumulative value corresponding to the maximum unit block movement vector. The first weight cumulative value corresponding to at least one of the cell block movement vectors that has a vector value that differs from the second largest cell block movement vector within the preset range, and the second largest first weight cumulative value are superimposed to generate a second cell block cumulative value corresponding to the second largest cell block movement vector. as well as Determine the largest cumulative value among the cumulative values ​​of the first and second unit blocks, and select the unit block movement vector corresponding to the largest cumulative value from the largest and second largest unit block movement vectors as one of the most frequently occurring unit block movement vectors; and In this second statistical procedure, for each of these image frames: A second weight is set for each of the image processing blocks according to the size of the image processing blocks; The second weights of the image processing blocks corresponding to each of the processing block movement vectors are superimposed to generate a second weight cumulative value for each of the processing block movement vectors. Determine the largest processing block movement vector with the largest second weight cumulative value and the second largest processing block movement vector with the second largest second weight cumulative value among these processing block movement vectors; The second weight cumulative value corresponding to at least one of the processing block movement vectors that has a difference between the vector values ​​of the maximum processing block movement vector and the maximum second weight cumulative value are superimposed to generate a first processing block cumulative value corresponding to the maximum processing block movement vector. as well as The second weight cumulative value corresponding to at least one of the processing block movement vectors that has a difference between the vector values ​​of the second largest processing block movement vector and the second largest second weight cumulative value are superimposed to generate a second processing block cumulative value corresponding to the second largest processing block movement vector. as well as Determine the largest cumulative value of the processing block among the first and second cumulative values ​​of the processing block, and select the processing block movement vector that corresponds to the largest cumulative value of the processing block from the largest and second largest processing block movement vectors as one of the most frequently occurring processing block movement vectors.

10. The image decoding apparatus of claim 9, wherein the front-end decoding circuit is further configured for each of the image frames to: Calculate the sum of the second weighted cumulative values ​​of all such processing block movement vectors, and if the maximum processing block cumulative value is not zero, further calculate the maximum movement vector ratio between the maximum processing block cumulative value and the sum; Sequentially determine that the most frequently occurring movement vector of the processing block and the movement vectors of the K consecutive previous image frames all have a difference of a vector value within a specific range, so that the threshold value decreases by K preset values ​​without being less than the limit value; and Identify any of the image frames whose maximum motion vector ratio is greater than the threshold value as the translation image frame.

Images