Symbolizing device and program

Abstract

To improve coding efficiency.SOLUTION: A coding device 1 comprises: a frame memory 21 which stores a frame obtained by locally decoding an original image; an image processing unit 10 which performs image processing on a reference frame acquired from the frame memory 21 and generates one or more filter processing frames with varied frequency components; and an inter-screen prediction unit 20 which generates an inter-screen prediction image for a frame to be encoded, by referring to the one or more filter processing frames.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an encoding device and a program.

Background Art

[0002] In encoding methods such as H.265 / HEVC (High Efficiency Video Coding) and H.266 / VVC (Versatile Video Coding), it is known to perform moving image encoding using pixel correlation (see, for example, Non-Patent Document 1). In inter-picture prediction, prediction is performed by referring to reference frames before and after an encoding target frame (prediction target frame). Generally, when a prediction target unit and a decoded unit of its reference frame are the same object, it is considered that the pixel correlation between the same objects is high, so that inter-picture prediction can be performed with high accuracy.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in conventional encoding techniques, when the sharpness differs between frames, inter-picture prediction is difficult and there is a problem that the encoding efficiency tends to decrease.

[0005] In view of such circumstances, an object of the present invention is to provide an encoding device and a program capable of improving the encoding efficiency of moving images with different sharpness between frames.

Means for Solving the Problems

[0006] An encoding device according to one embodiment is an encoding device for encoding a source image, comprising: a frame memory for storing frames obtained by locally decoding the source image; an image processing unit for performing image processing on a reference frame obtained from the frame memory to generate one or more filtered frames with altered frequency components; and an inter-frame prediction unit for generating inter-frame prediction images for a frame to be encoded by referring to the one or more filtered frames.

[0007] Furthermore, in one embodiment, the image processing unit may perform a process to blur or sharpen the high-frequency components of the reference frame to generate the filtered frame.

[0008] Furthermore, in one embodiment, the image processing unit may include: a multi-resolution decomposition unit that performs multi-resolution decomposition on the reference frame to generate an n1-level decomposition image with a decomposition order of n1 and an n2-level decomposition image with a decomposition order of n2 greater than n1; a registration unit that determines the most similar block in the lowest frequency component image of the n2-level decomposition image for each divided block of the lowest frequency component image of the n1-level decomposition image and generates registration information indicating the position of the similar block; a high-frequency component allocation unit that identifies the position of the similar block according to the registration information, identifies a block at the same spatial phase position as the similar block in each frequency component image of the n2-level decomposition image as an allocation block, and generates a high-frequency component allocation image in which the allocation block is allocated to the same spatial phase position as the divided block in the high-frequency component of the n1-level decomposition image; and a sharpening reconstruction unit that performs multi-resolution reconstruction on the high-frequency component allocation image to generate the filtered frame in which the high-frequency components of the reference frame are sharpened.

[0009] Furthermore, in one embodiment, the registration unit may generate the registration information only when the difference between the pixel value of the divided block and the pixel value of the similar block exceeds the registration threshold.

[0010] Furthermore, in one embodiment, the high-frequency component allocation unit may classify the n2-layer decomposed image into high-frequency component images and low-frequency component images within each frequency component image of the n3 layer, where the decomposition level is one less than n2; identify a block with the same spatial phase position as the similar block within the low-frequency component image as a comparison block; and allocate the allocation block if the difference between the pixel value of the allocation block and the pixel value of the comparison block exceeds the first allocation threshold within each frequency component image of the n3 layer.

[0011] Furthermore, in one embodiment, the high-frequency component allocation unit may classify the n2-layer decomposed image into high-frequency component images and low-frequency component images within each frequency component image of the n3 layer, where the decomposition level is one less than n2. Within the low-frequency component image, it may identify a block at the same spatial phase position as the similar block as a comparison block. If the difference between the pixel value of the similar block and the pixel value of the comparison block exceeds the second allocation threshold, the allocation block in the same frequency component image as the comparison block may be allocated in the n3 layer.

[0012] Furthermore, in one embodiment, the system may further include a geometric transformation unit that performs a geometric transformation on the reference frame and generates one or more geometrically transformed reference frames, and the image processing unit may perform image processing on the geometrically transformed reference frames.

[0013] Furthermore, in one embodiment, the reference frame may further include one or more frames that are temporally prior to the frame to be encoded, or one or more frames that are temporally later to the frame to be encoded.

[0014] Furthermore, the program according to one embodiment causes the computer to function as the encoding device. [Effects of the Invention]

[0015] According to the present invention, it is possible to improve the encoding efficiency of images. [Brief explanation of the drawing]

[0016] [Figure 1] It is a block diagram showing a configuration example of an encoding device according to an embodiment. [Figure 2] It is a block diagram showing a configuration example of an image processing unit in an encoding device according to an embodiment. [Figure 3] It is a block diagram showing a configuration example of an edge enhancement image generation unit in an encoding device according to an embodiment. [Figure 4] It is a diagram for explaining an example of registration processing in an encoding device according to an embodiment. [Figure 5] It is a diagram for explaining an example of assignment processing in an encoding device according to an embodiment. [Figure 6] It is a diagram showing a first example of accuracy determination of assignment in an encoding device according to an embodiment. [Figure 7] It is a diagram showing a second example of accuracy determination of assignment in an encoding device according to an embodiment. [Figure 8] It is a block diagram showing a configuration example of a blurring image generation unit in an encoding device according to an embodiment. [Figure 9] It is a diagram for explaining an example of high-frequency component suppression processing in an encoding device according to an embodiment. [Figure 10] It is a block diagram showing a modified example of an image processing unit in an encoding device according to an embodiment.

Embodiments of the Invention

[0017] Hereinafter, an embodiment will be described in detail with reference to the drawings.

[0018] FIG. 1 is a block diagram showing a configuration example of an encoding device according to an embodiment. The encoding device 1 shown in FIG. 1 includes an image processing unit 10, a block division unit 11, a subtraction unit 12, a conversion unit 13, a visual activity calculation unit 14, a QP determination unit 15, a quantization unit 16, an inverse quantization unit 17, an inverse conversion unit 18, an addition unit 19, an intra-frame prediction unit 20, a frame memory 21, an inter-frame prediction unit 22, a switching unit 23, and an entropy encoding unit 24.

[0019] The block division unit 11 generates divided blocks obtained by dividing an input image (original image) as coding units (CUs: Coding Units), and outputs them to the subtraction unit 12, the visual activity calculation unit 14, and the inter-picture prediction unit 22.

[0020] The subtraction unit 12 subtracts the pixel values of a prediction unit input from the intra-picture prediction unit 20 or the inter-picture prediction unit 22, which will be described later, from the pixel values of each coding unit input from the block division unit 11, generates a residual signal indicating the difference in pixel values between the coding unit and the prediction unit, and outputs it to the conversion unit 13.

[0021] The conversion unit 13 further divides the residual signal input from the subtraction unit 12 into transform units (TUs: Transform Units), performs a conversion process such as an orthogonal transformation for each TU to calculate conversion coefficients, and outputs them to the quantization unit 16.

[0022] The visual activity calculation unit 14 calculates the visual activity of the coding unit input from the block division unit 11, and outputs it to the QP determination unit 15. When the coding unit contains many high-frequency components, the visual activity becomes large, and when the coding unit does not contain many high-frequency components, the visual activity becomes small.

[0023] The QP determination unit 15 determines a quantization parameter (QP value) based on the visual activity input from the visual activity calculation unit 14, and outputs it to the quantization unit 16.

[0024] The quantization unit 16 divides the conversion coefficients input from the conversion unit 13 by a quantization step corresponding to the QP value input from the QP determination unit 15 to perform quantization, thereby generating quantization coefficients, and outputs them to the inverse quantization unit 17 and the entropy coding unit 24.

[0025] The inverse quantization unit 17 restores the conversion coefficients by multiplying the quantization coefficients input from the quantization unit 16 by the quantization step, and outputs them to the inverse conversion unit 18.

[0026] The inverse transform unit 18 performs an inverse transform process (a process that reverses the transformation performed by the transform unit 13) on the transformation coefficients input from the inverse quantization unit 17 to restore the residual signal and outputs it to the adder unit 19. For example, if the transform unit 13 performs a discrete cosine transform, the inverse transform unit 18 performs an inverse discrete cosine transform.

[0027] The adder 19 adds the residual signal input from the inverse transformer 18 and the prediction unit input from the switching unit 23 to generate a decoded unit, which is then output to the frame memory 21 and the in-screen prediction unit 20.

[0028] The inverse quantization unit 17, the inverse transform unit 18, and the adder unit 19 constitute a local decoding unit, which performs local decoding. Specifically, the local decoding unit restores the transformation coefficients by multiplying the quantization coefficients by the quantization step, performs an inverse transform on the transformation coefficients to restore the residual signal, and adds the residual signal and the prediction unit to generate a decoded unit.

[0029] The in-screen prediction unit 20 refers to the decoded units input from the addition unit 19, performs in-screen prediction (intra prediction) for each prediction unit (PU), generates an in-screen predicted image, and outputs it to the switching unit 23.

[0030] The frame memory 21 stores the decoded units input from the adder 19 on a frame-by-frame basis. In other words, the frame memory 21 stores the frames that have been locally decoded after the original image has been quantized.

[0031] The image processing unit 10 acquires a reference frame from the frame memory 21, performs image processing on the reference frame to generate one or more filtered frames with altered frequency components, and outputs them to the inter-frame prediction unit 22. The image processing unit 10 may also perform blurring or sharpening of the high-frequency components of the reference frame to generate filtered frames. Details of the image processing unit 10 will be described later.

[0032] The inter-frame prediction unit 22 divides the encoding units input from the block division unit 11 into prediction units. Then, the inter-frame prediction unit 22 refers to one or more filtered frames input from the image processing unit 10 in addition to the reference frame, and generates motion vectors for each prediction unit using methods such as block matching. The inter-frame prediction unit 22 may adopt the motion vector that provides the best encoding efficiency through RD optimization or the like. The inter-frame prediction unit 22 performs motion compensation prediction based on the motion vector for each prediction unit, generates an inter-frame prediction image for the frame to be encoded, and outputs it to the switching unit 23. Conventional encoding devices perform inter-frame prediction by referring to a reference frame, whereas the encoding device 1 differs in that it performs inter-frame prediction by referring to a filtered frame in addition to the reference frame.

[0033] The switching unit 23 switches between the in-screen prediction image input from the in-screen prediction unit 20 and the inter-screen prediction image input from the inter-screen prediction unit 22 to obtain a prediction image, which is then output to the subtraction unit 12 and the addition unit 19.

[0034] The entropy coding unit 24 performs entropy coding on the quantization coefficients input from the quantization unit 16, the prediction mode information input from the in-screen prediction unit 20, and the motion vector information input from the inter-screen prediction unit 22, compresses the data, generates a bitstream, and outputs it to the outside of the coding device 1. Any entropy coding scheme can be used for entropy coding, such as zero-order exponential Golomb coding or context-based adaptive binary arithmetic coding (CABAC).

[0035] <Image Processing> Next, the processing of the image processing unit 10 will be described. Figure 2 is a block diagram showing an example of the configuration of the image processing unit 10. The image processing unit 10 shown in Figure 2 comprises a multi-resolution decomposition unit 30, a sharpened image generation unit 40, and a blurred image generation unit 50.

[0036] The image processing unit 10 inputs (acquires) a reference frame locally decoded by the local decoding unit (inverse quantization unit 17, inverse transformation unit 18, and addition unit 19) from the frame memory 21. A reference frame is a frame referenced when performing inter-frame prediction processing in the inter-frame prediction unit 22. The reference frame is defined by an encoding scheme such as H.265 / HEVC or H.266 / VVC. If the frame to be encoded is a P-picture (Predictive Picture), there is one reference frame; if the frame to be encoded is a B-picture (Bidirectionally Predictive Picture), there are two reference frames. The image processing unit 10 may also input (acquire) the frame before and / or the frame after the frame to be encoded from the frame memory 21. The frame before means one or more frames that are temporally before the frame to be encoded, and the frame after means one or more frames that are temporally after the frame to be encoded. In the embodiments described below, the image processing unit 10 will be described as inputting only the reference frame.

[0037] The multi-resolution decomposition unit 30 performs multi-resolution decomposition on a reference frame input from the frame memory 21 to generate an n1-level decomposition image with a decomposition level of n1, and an n2-level decomposition image with a decomposition level greater than n1, n2. The multi-resolution decomposition unit 30 then outputs the n1-level decomposition image and the n2-level decomposition image to the sharpened image generation unit 40, and outputs the n1-level decomposition image to the blurred image generation unit 50.

[0038] The multi-resolution decomposition unit 30 may perform multi-resolution decomposition by wavelet packet decomposition. By performing wavelet packet decomposition, the phase information of each frequency component image can be treated as having the same size. In this embodiment, the multi-resolution decomposition unit 30 performs wavelet packet decomposition with decomposition levels n1=1 and n2=2. The multi-resolution decomposition unit 30 may obtain the information of decomposition levels n1 and n2 from an external source.

[0039] The sharpening image generation unit 40 receives the n1-layer decomposed image and the n2-layer decomposed image from the multi-resolution decomposition unit 30. The sharpening image generation unit 40 then assigns the high-frequency bandwidth of the n2-layer decomposed image to the high-frequency bandwidth of the n1-layer decomposed image, and then performs multi-resolution reconstruction to generate a filtered frame (sharpened reference frame) in which the reference frame has been sharpened. The sharpening image generation unit 40 outputs the generated sharpened reference frame to the inter-screen prediction unit 22.

[0040] The blurred image generation unit 50 receives an n1-layer decomposed image from the multi-resolution decomposition unit 30. The blurred image generation unit 50 then suppresses the high-frequency bandwidth of the n1-layer decomposed image and performs multi-resolution reconstruction to generate a filtered frame (blurred reference frame) with a blurred reference frame. The blurred image generation unit 50 outputs the generated blurred reference frame to the inter-screen prediction unit 22.

[0041] <<Image Sharpening Generation Unit>> Next, the details of the sharpening image generation unit 40 will be described. Figure 3 is a block diagram showing an example configuration of the sharpening image generation unit 40. The sharpening image generation unit 40 shown in Figure 3 comprises a registration unit 41, a high-frequency component allocation unit 42, and a sharpening reconstruction unit 43.

[0042] The registration unit 41 divides the lowest frequency component image of the n1-level decomposed image into divided blocks of a predetermined size (for example, 8 x 8 pixels). Then, for each divided block in the lowest frequency component image of the n1-level decomposed image, the registration unit 41 determines the similar block with the highest similarity (correlation) to the divided block in the lowest frequency component image of the n2-level decomposed image by block matching. The registration unit 41 then generates registration information indicating the location of the similar block and outputs it to the high-frequency component assignment unit 42.

[0043] Referring to Figure 4, a specific example of the registration process by the registration unit 41 will be explained. In this example, the multi-resolution decomposition unit 30 performs wavelet packet decomposition with a decomposition order of 1 on the first reference frame to generate a 1-layer decomposed image F1, and performs wavelet packet decomposition with a decomposition order of 2 on the second reference frame to generate a 2-layer decomposed image F2. The first and second reference frames may be the same frame. By performing multi-resolution decomposition of the second reference frame with a decomposition order of n2=2, four frequency component images (LL,HL,LH,HH) are generated as a 1-layer decomposed image. In addition, sixteen frequency component images (LLLL,LLLH,LLHL,LLHH,HLLL,HLLH,HLHL,HLHH,LHLL,LHLH,LHHL,LHHH,HHLL,HHLH,HHHL,HHHH) are generated as a 2-layer decomposed image F2.

[0044] The registration unit 41 performs registration (alignment) of each divided block B of the lowest frequency component image LL of the first-layer decomposed image F1 with the lowest frequency component image LLLL of the second-layer decomposed image F2, and determines the similar block S within the lowest frequency component image LLLL that has the highest similarity (correlation) with divided block B.

[0045] The registration unit 41 may perform a registration accuracy check and generate registration information only if the accuracy is high.

[0046] The process when the registration unit 41 performs accuracy determination will be described below. For example, the registration unit 41 determines that the accuracy of registration is high and generates registration information only when the difference between the pixel value of the divided block B and the pixel value of the similar block S exceeds the registration threshold. That is, if the difference is less than or equal to the registration threshold, it determines that the accuracy of registration is low and does not generate registration information. The registration unit 41 may calculate SAD (Sum of Absolute Difference) or SSD (Sum of Squared Difference) as the difference in pixel values.

[0047] The high-frequency component allocation unit 42 identifies the position of the similar block S according to the registration information input from the registration unit 41. The high-frequency component allocation unit 42 then identifies the block at the same spatial phase position as the similar block S within each frequency component image of the n2-layer decomposed image F2 as the allocated block A. The high-frequency component allocation unit 42 then generates a high-frequency component allocation image in which the allocated block A is allocated to the same spatial phase position as the divided block B within the high-frequency components of the n1-layer decomposed image F1. The high-frequency component allocation unit 42 then outputs the high-frequency component allocation image to the sharpening reconstruction unit 43.

[0048] Referring to Figure 5, the allocation process by the high-frequency component allocation unit 42 will be explained in detail. Here, as in Figure 4, n1=1 and n2=2. The high-frequency component allocation unit 42 identifies the position of similar blocks S within the lowest frequency component image LLLL of the two-layer decomposed image F2 according to the registration information. In Figure 5, this alignment is indicated by a dashed arrow.

[0049] The high-frequency component allocation unit 42 identifies the block with the same spatial phase position as the similar block S within the high-frequency component images (LLHL, LLLH, LLHH, HLHL, HLLH, HLHH, LHHL, LHLH, LHHH, HHHL, HHLHH) in each frequency component image with resolution level n2-1=1 of the two-layer decomposed image F2 as allocation block A. For the sake of explanation, in Figure 5, allocation block A is shown only for the high-frequency component images LLHL, HLHL, LHHL, and HHHL. The high-frequency component allocation unit 42 then allocates the allocation block A within the horizontal low-frequency and vertical high-frequency component images (LLHL, HLHL, LHHL, HHHL) in each frequency component image with resolution level 1 of the two-layer decomposed image F2 to the same spatial phase position as the divided block B within the horizontal low-frequency and vertical high-frequency component image (HL) of the one-layer decomposed image F1. In Figure 5, this allocation is shown by four solid arrows.

[0050] Similarly, the high-frequency component allocation unit 42 allocates allocation block A within the horizontal high-frequency / vertical low-frequency component images (LLLH, HLLH, LHLH, HHLH) in each frequency component image of the first decomposition level of the two-layer decomposed image F2 to the same spatial phase position as division block B within the horizontal high-frequency / vertical low-frequency component image (LH) of the one-layer decomposed image F1. Similarly, the high-frequency component allocation unit 42 allocates allocation block A within the horizontal high-frequency / vertical high-frequency component images (LLHH, HLHH, LHHH, HHHH) in each frequency component image of the first decomposition level of the two-layer decomposed image F2 to the same spatial phase position as division block B within the horizontal high-frequency / vertical high-frequency component image (HH) of the one-layer decomposed image F1.

[0051] The high-frequency component allocation unit 42 may perform an allocation accuracy determination and only perform the allocation if the accuracy is high.

[0052] The process when the high-frequency component allocation unit 42 performs accuracy determination will be described below. The high-frequency component allocation unit 42 classifies the n2-layer decomposed image into high-frequency component images and low-frequency component images at a layer with a decomposition level one less than n2. Within the low-frequency component image, it identifies blocks with the same spatial phase position as similar blocks as comparison blocks. Only if the difference between the pixel value of the allocation block in the high-frequency component image and the comparison block in the low-frequency component image exceeds the first allocation threshold, it determines that the allocation accuracy is high and allocates the allocation block. That is, if the difference exceeds the first allocation threshold, it determines that the allocation accuracy is low and does not allocate the allocation block. This comparison may be performed for each decomposed image at a layer with a decomposition level one less than n2. The high-frequency component allocation unit 42 may calculate SAD or SSD as the difference in pixel values.

[0053] Referring to Figure 6, a first example of assignment accuracy determination is explained. Here, as in Figures 4 and 5, n1=1 and n2=2. The high-frequency component assignment unit 42 classifies the two-layer decomposed image F2 into high-frequency component images (shown with a darker background in Figure 6) and low-frequency component images (shown with a lighter background in Figure 6) for each frequency component image of the n3 layers, where the decomposition level is one less than n2. Within each low-frequency component image, it identifies a block with the same spatial phase position as the similar block S as a comparison block C. The high-frequency component assignment unit 42 then calculates the difference between the pixel value of assignment block A and the pixel value of comparison block C within each frequency component image (HL, LH, HH) of the n3 layers, and assigns assignment block A if the difference exceeds the first assignment threshold T1. Note that this accuracy determination is not performed for the frequency component image LL because there is no comparison block C.

[0054] Referring to Figure 7, a second example of assignment accuracy determination will be explained. Here, as in Figures 4 to 6, n1=1 and n2=2. The high-frequency component assignment unit 42 identifies the comparison block C, as in the first example. The high-frequency component assignment unit 42 then calculates the difference between the pixel value of the similar block S and the pixel value of the comparison block C, and if this difference exceeds the second assignment threshold T2, it assigns an assignment block A in the same frequency component image as the comparison block C in the n3 layer. For example, in the 1-layer decomposed image HL, if the difference exceeds the second assignment threshold T2, it assigns three assignment blocks A in the 1-layer decomposed image HL. Note that since there is no comparison block C in the frequency component image LL, this accuracy determination is not performed. Alternatively, the high-frequency component assignment unit 42 may combine the first and second examples of accuracy determination and assign an assignment block A if both conditions are met.

[0055] The sharpening reconstruction unit 43 performs multi-resolution reconstruction on the high-frequency component-assigned image input from the high-frequency component-assigned unit 42 to generate a filtered frame (sharpened reference frame) in which the high-frequency components of the reference frame have been sharpened. For example, if the n1-level decomposed image is an image generated by wavelet packet decomposition with a decomposition level of n1 on the reference frame, the sharpening reconstruction unit 43 performs wavelet packet reconstruction with a decomposition level of n1 on the high-frequency component-assigned image.

[0056] <<Blurred Image Generation Unit>> Next, the details of the blurred image generation unit 50 will be described. Figure 8 is a block diagram showing an example configuration of the blurred image generation unit 50. The blurred image generation unit 50 shown in Figure 8 comprises a high-frequency component suppression unit 51 and a blurred reconstruction unit 52.

[0057] The high-frequency component suppression unit 51 generates a high-frequency component suppressed image by suppressing the high-frequency components of the n1-layer decomposed image input from the multi-resolution decomposition unit 30, and outputs it to the blur reconstruction unit 52.

[0058] Figure 9 shows an example of high-frequency component suppression processing by the high-frequency component suppression unit 51. The high-frequency component suppression unit 51 generates a high-frequency component suppressed image by multiplying the pixel values ​​of the high-frequency component images (HL, LH, HH) of the n1-level decomposed image by a blur coefficient γ. The blur coefficient γ is a value less than 1, for example, γ = 0.9. The high-frequency component suppression unit 51 may also obtain information on the blur coefficient γ from an external source.

[0059] The blur reconstruction unit 52 performs multi-resolution reconstruction on the high-frequency component suppression image input from the high-frequency component suppression unit 51 to generate a filtered frame (blurred reference frame) in which the high-frequency components of the reference frame are blurred. For example, if the n1-level decomposed image is an image generated by wavelet packet decomposition with n1 levels of decomposition on the reference frame, the blur reconstruction unit 52 performs wavelet packet reconstruction with n1 levels of decomposition on the high-frequency component suppression image.

[0060] <<Modified image sharpening unit>> Next, a modified version of the sharpened image generation unit 40 will be described. The sharpened image generation unit 40 may increase the high-frequency components by multiplying by a coefficient, similar to the blurred image generation unit 50. In other words, in the modified version, the sharpened image generation unit 40 generates an image with increased high-frequency components by multiplying the pixel values ​​of the high-frequency component images (HL, LH, HH) of the n1-level decomposed image by a sharpening coefficient γ'. The sharpening coefficient γ' is a value greater than 1, for example, γ' = 1.1. The sharpened image generation unit 40 may obtain information on the sharpening coefficient γ' from an external source. Then, the sharpened image generation unit 40 performs multi-resolution reconstruction on the image with increased high-frequency components to generate a filtered frame (sharpened reference frame) in which the high-frequency components of the reference frame have been sharpened.

[0061] <Variations of the image processing unit> Next, a modified version of the image processing unit 10 will be described. Figure 10 is a block diagram showing an example configuration of the image processing unit 10a, which is a modified version of the image processing unit 10. The image processing unit 10a includes a geometric transformation unit 60, a multi-resolution decomposition unit 30, a sharpened image generation unit 40, and a blurred image generation unit 50. The image processing unit 10a differs from the image processing unit 10 in that it further includes a geometric transformation unit 60.

[0062] The geometric transformation unit 60 receives a reference frame from the frame memory 21, performs a geometric transformation on the reference frame, and generates one or more geometrically transformed reference frames. The geometric transformation unit 60 then outputs one or more geometrically transformed reference frames (a group of geometrically transformed images) to the multi-resolution decomposition unit 30. For example, the geometric transformation unit 60 performs an affine transformation as the geometric transformation. The geometric transformation parameters are scaling a={0.9,1.0,1.1}, rotation θ={1 / 16π,0,-1 / 16π}, and no skew. When this affine transformation is performed, a group of nine geometrically transformed images is generated for each reference frame.

[0063] The multi-resolution decomposition unit 30 performs multi-resolution decomposition on the reference frame after geometric transformation. Other processing is the same as that of the image processing unit 10, so the explanation is omitted.

[0064] <Program> To function as the encoding device 1 described above, a computer capable of executing program instructions can also be used. Here, the computer may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, etc. The program instructions may be program code, code segments, etc., for executing the required task.

[0065] A computer comprises a processor, a memory unit, an input unit, an output unit, and a communication interface. The processor is specifically a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), SoC (System on a Chip), etc., and may be composed of multiple processors of the same or different types. The processor controls each of the above components and performs various arithmetic operations by reading and executing programs from the memory unit.

[0066] The program may be recorded on a computer-readable recording medium. Using such a medium, the program can be installed on the computer. The recording medium on which the program is recorded may be a non-transitory recording medium. Non-transitory recording media are not particularly limited, but may include, for example, CD-ROMs, DVD-ROMs, or USB (Universal Serial Bus) memory. Alternatively, the program may be downloaded from an external device via a network.

[0067] As described above, in the present invention, image processing is performed on a reference frame acquired from the frame memory 21 to generate one or more filtered frames with altered frequency components, and an inter-frame prediction image is generated for the frame to be encoded by referring to the filtered frame. Therefore, since the number of frames referenced in inter-frame prediction is increased compared to conventional encoding methods, the accuracy of encoding is improved and the encoding efficiency can be improved.

[0068] The filtered frame should preferably be a sharpened reference frame with the high-frequency components of the reference frame blurred, and / or a blurred reference frame with the high-frequency components of the reference frame blurred. Even when encoding images with low correlation between screens, using a sharpened or blurred reference frame is expected to improve the correlation with the frame to be encoded.

[0069] Furthermore, the number of filtered frames can be increased by performing a geometric transformation on the reference frame before filtering, or by filtering the frame before or after the frame to be encoded. In this case, the number of frames referenced in inter-screen prediction increases even further, making it possible to further improve encoding efficiency.

[0070] Although the embodiments described above are representative examples, it will be apparent to those skilled in the art that many modifications and substitutions are possible within the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited by the embodiments described above, and various modifications or changes are possible without departing from the scope of the claims. For example, it is possible to integrate multiple component blocks shown in the configuration diagram of the embodiments, or to divide a single component block. [Explanation of symbols]

[0071] 1 Encoding device 10,10a Image Processing Unit 11 Block division section 12 Subtraction Unit 13 Conversion section 14 Visual Activity Calculation Unit 15 QP decision section 16 Quantization section 17 Inverse quantization section 18 Inverse Transform Section 19 Addition section 20 In-screen prediction section 21 frame memory 22 Inter-screen prediction unit 23 Switching section 24 Entropy coding unit 30 Multi-resolution decomposition part 40 Image Sharpening Generation Unit 41 Registration Department 42 High-frequency component allocation section 43 Sharpening reconstruction part 50 Blurred Image Generation Unit 51 High-frequency component suppression section 52 Blurred Reconstruction Section 60 Geometric Transformation Section

Claims

1. 1. A coding device for coding an original image, comprising: a frame memory for storing frames obtained by locally decoding the original image; an image processing unit that performs image processing on the reference frame acquired from the frame memory to generate one or more filter processed frames in which frequency components are changed; an inter prediction unit that generates an inter prediction image for a frame to be coded by referring to the one or more filtered frames; An encoding device comprising:

2. The encoding device according to claim 1 , wherein the image processing unit performs a process of blurring or sharpening high frequency components of the reference frame to generate the filtered frame.

3. The image processing unit a multi-resolution decomposition unit that performs multi-resolution decomposition on the reference frame to generate an n1-layer decomposed image having a decomposition order of n1 and an n2-layer decomposed image having a decomposition order of n2 greater than n1; a registration unit that determines a similar block with the highest similarity in the lowest frequency component image of the n2-layer decomposition image for each divided block of the lowest frequency component image of the n1-layer decomposition image, and generates registration information indicating the position of the similar block; a high frequency component allocation unit that identifies the position of the similar block according to the registration information, identifies a block at the same spatial phase position as the similar block in each frequency component image of the n2 layer decomposition image as an allocated block, and generates a high frequency component allocated image in which the allocated block is allocated to the same spatial phase position as the divided block in the high frequency component of the n1 layer decomposition image; a sharpening reconstruction unit that performs multi-resolution reconstruction on the high-frequency component allocated image to generate the filtered frame in which the high-frequency components of the reference frame are sharpened; The encoding device according to claim 1 or 2, comprising:

4. The encoding device according to claim 3 , wherein the registration unit generates the registration information when a difference between pixel values ​​of the divided block and pixel values ​​of the similar block exceeds a registration threshold.

5. 5. The encoding device according to claim 3, wherein the high frequency component allocation unit classifies the n2-layer decomposition image into a high frequency component image and a low frequency component image within each frequency component image of n3 layers, the decomposition rank of which is one less than n2, identifies a block within the low frequency component image that has the same spatial phase position as the similar block as a comparison block, and allocates the allocation block when a difference between pixel values ​​of the allocation block and pixel values ​​of the comparison block within each frequency component image of the n3 layers exceeds a first allocation threshold.

6. 6. The encoding device according to claim 3, wherein the high frequency component allocation unit classifies the n2-layer decomposition image into high frequency component images and low frequency component images within each frequency component image of n3 layers, where the decomposition rank is one less than n2, identifies a block within the low frequency component image that has the same spatial phase position as the similar block as the comparison block, and allocates the allocation block within the frequency component image that is the same as the comparison block in the n3 layers when a difference between pixel values ​​of the similar block and pixel values ​​of the comparison block exceeds a second allocation threshold.

7. a geometric transformation unit that performs a geometric transformation on the reference frame to generate one or more reference frames after the geometric transformation; the image processing unit performs image processing on the reference frame after the geometric transformation. Encoding device according to any one of claims 1 to 6.

8. The encoding device according to claim 1 , wherein the reference frames further include one or more frames located temporally before the current frame to be encoded or one or more frames located temporally after the current frame to be encoded.

9. A program for causing a computer to function as the encoding device according to any one of claims 1 to 8.

Images