Image decoding device and image decoding method

Abstract

To provide an image encoding device, an image decoding device, and a program for improving encoding efficiency.SOLUTION: An image decoding device that decodes signals that have been coded by dividing frames that constitute a moving image into blocks includes a prediction unit that generates block-based predicted images including predicted signals by performing signal prediction on each pixel signal on a block-by-block basis, an acquisition unit that acquires a control identification signal that indicates the type of selection control selected by the coding side, a filtering unit that generates a new block of the predicted image by filtering the predicted signal using decoded adjacent signals adjacent to the block of the predicted image on the basis of the control identification signal, an inverse quantization unit that performs corresponding inverse quantization processing on quantized transform coefficients to restore transform coefficient signals, and an inverse orthogonal transform unit that reconstructs block-by-block predicted residual signals by performing inverse transform processing determined on the basis of the control identification signal on the transform coefficient signals.SELECTED DRAWING: Figure 17

Description

[Technical field]

[0001] The present invention relates to an image encoding device, an image decoding device, and programs thereof that are applicable to video encoding formats such as MPEG-2, AVC / H.264, MPEG-H, and HEVC / H.265. [Background technology]

[0002] It is known that image signals, whether still images or moving images, generally have high signal correlation between adjacent pixels. Taking advantage of this property, video coding methods such as MPEG-2, AVC / H.264, MPEG-H, and HEVC / H.265 provide two prediction modes, intra prediction and inter prediction. Intra prediction is a technique for performing signal prediction only with signals within a coding frame, and performs DC prediction, planar prediction, or directional prediction on pixel signals (original signals) in a coding target block of an original image using pixel signals of coded and decoded blocks adjacent to the left or upper side of the coding target block, and generates a block of a predicted image made of pixel signals (prediction signals) predicted by extrapolation. In this way, pixel signals in the coding target block are efficiently predicted by using pixel signals of decoded blocks adjacent to the coding target block.

[0003] For example, referring to Fig. 29, the nature of the prediction residual signal by intra prediction will be described by taking prediction in the horizontal direction (horizontal prediction) of intra prediction as an example. In the example shown in Fig. 29, for an original signal in the encoding target block Bko of an original image consisting of pixel signals p of a block size of horizontal 4 pixels x vertical 4 pixels (hereinafter, simply indicated as "4 x 4", the same applies to other block sizes) (see Fig. 29(a)), prediction is performed using the pixel signal of the decoded block adjacent to the left as a reference signal Sr for horizontal prediction, and a block Bkp of a predicted image consisting of a prediction signal corresponding to the pixel signal of the encoding target block is generated (see Fig. 29(b)).

[0004] Then, a block Bkd of a prediction residual signal can be obtained from the difference between the original signal in the block Bko to be coded in the original image and the prediction signal of the block Bkp of the prediction image (see FIG. 29(c)). The prediction residual signal by this intra prediction has the property that the closer the pixel position is to the reference signal, the smaller the residual component (the smaller the signal strength) because the prediction efficiency is higher; in other words, the farther away from the reference signal, the lower the prediction efficiency, and therefore the larger the residual component (the larger the signal strength) (see FIG. 29(d)).

[0005] Here, in the example shown in Fig. 29, prediction is performed using inter-pixel correlation in the horizontal direction, but the signal of the block that has been coded and decoded is also present above the block to be coded. In the example shown in Fig. 29, efficient prediction is performed using horizontal correlation by horizontal intra prediction to generate a prediction residual signal, but vertical correlation is not used.

[0006] Therefore, in the currently defined H.265, a high correlation between the pixel signals in the blocks above the block to be coded is utilized, and a filter process that reflects the signal fluctuations for the blocks above is applied only to the topmost predicted signal in the block Bkp of the predicted image in order to improve the accuracy of the predicted signal. This makes it possible to utilize not only the horizontal correlation due to the horizontal prediction, but also the vertical correlation, and the average residual component in the block Bkd of the predicted residual signal is further reduced, improving the coding efficiency. Similarly, in the case of vertical intra-prediction, a filter process is applied using the horizontally adjacent pixels not used in the prediction, and in the case of DC prediction that does not use the horizontal and vertical adjacent pixels, a filter process using the horizontal and vertical adjacent pixels is applied, thereby achieving a significant improvement in coding efficiency compared to previous video coding methods such as MPEG-2 and H.264.

[0007] In addition, in inter prediction, motion prediction is performed from a temporally adjacent reference frame that has been coded and decoded, a motion vector is calculated, and a predicted image is generated using this. The difference between the predicted image and the original image is generated as a prediction residual signal. However, while in conventional techniques, the high correlation of signals with adjacent blocks is actively used in intra prediction, such properties that indicate the high correlation of signals with adjacent blocks are not used in inter prediction.

[0008] Then, a block of a prediction residual signal generated through intra prediction or inter prediction is subjected to an orthogonal transform process and a quantization process, and is coded.

[0009] In the currently specified H.265, the orthogonal transform process for intra prediction mainly uses DCT (Discrete Cosine Transform), but a mode is provided that applies orthogonal transform process using DST (Discrete Sine Transform) to the prediction residual signal of intra prediction for some block sizes. However, when the block size of the block to be coded is large (larger than 4x4), the DST is applied depending on the signal characteristics of the blocks to the left and above. The application of DST is not permitted by the standard because it reduces the benefits of its application and is costly to implement. In addition, in the orthogonal transform process for inter prediction in the currently specified H.265, DCT is applied regardless of the block size.

[0010] It has been shown that a feature of motion compensation processing in inter prediction is that the amount of prediction error is statistically large at the end points of the prediction region related to the coding (i.e., the prediction residual signal for pixel signals located on the periphery of the block of the predicted image) (see, for example, Non-Patent Document 1). Therefore, when this prediction region is further divided into blocks to form a transform region, there exists a transform region whose features match the basic features of the DST used when coding the prediction residual signal. A technique for applying DST utilizing this has been disclosed (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0011] [Patent Document 1] JP 2014-36278 A [Non-patent literature]

[0012] [Non-Patent Document 1] Cheng, Wentao, et al., "Characteristic Analysis of Motion-Compensated Interframe Difference Signal Based on Statistical Motion Distribution Model", D-II, Vol. J84-D-II, No. 9, pp. 2001-2010, September 1, 2001 Summary of the Invention [Problem to be solved by the invention]

[0013] As mentioned above, Non-Patent Document 1 indicates that a characteristic of motion compensation processing in inter prediction is that the amount of prediction error is statistically large at the ends of the prediction area related to the encoding (i.e., the prediction residual signal for pixel signals located on the periphery of the block of the predicted image), and based on this, Patent Document 1 discloses a technique for applying DST.

[0014] However, because this is a statistically-based indication, there is a problem that applying DST to images that deviate from the statistics (for example, when there are features such as strong edges near the periphery of the predicted image block) may actually reduce the coding efficiency.

[0015] An object of the present invention is to provide an image encoding device, an image decoding device, and programs therefor that improve encoding efficiency. [Means for solving the problem]

[0016] The image decoding device of the present invention is an image decoding device that decodes a signal that has been coded by dividing a frame that constitutes a moving image into blocks, and is characterized in that it comprises: a prediction unit that generates a block-based predicted image including a predicted signal by performing signal prediction on each pixel signal in block units; an acquisition unit that acquires a control identification signal that indicates the type of selection control selected by the coding side; a filtering processing unit that generates a new block of the predicted image by filtering the predicted signal using decoded adjacent signals adjacent to the block of the predicted image based on the control identification signal; an inverse quantization unit that performs a corresponding inverse quantization process on quantized transform coefficients to restore a transform coefficient signal; and an inverse transform unit that reconstructs a block-based predicted residual signal by performing an inverse transform process determined based on the control identification signal on the transform coefficient signal. In an image encoding process in one embodiment, a low-pass filter process is applied to the predicted image used in inter prediction using the encoded and decoded adjacent signals in the same manner as in intra prediction, thereby reducing the prediction residual signal for the leftmost and topmost regions of the predicted image of inter prediction, and the blocks of the prediction residual signal are divided into blocks, and it is determined without a flag whether to apply DST or DCT according to the direction of the low-pass filter process, and the corresponding orthogonal transform process is executed. In an image decoding process in one embodiment, it is also determined without a flag whether to apply DST or DCT according to the direction of the low-pass filter process, and the original image is restored.

[0017] That is, an image coding device of one embodiment is an image coding device that divides an original image of each frame constituting a moving image into blocks and codes the divided original image, and is characterized by comprising: adjacent pixel non-reference prediction means for generating a block-by-block predicted image made of a predicted signal by a predetermined adjacent pixel non-reference prediction that performs signal prediction for each pixel signal of the original image of each block, without using a decoded adjacent signal; filter processing means for generating a new predicted image block by applying low-pass filter processing to a predicted signal located at a boundary of the predicted image block, using the decoded adjacent signals adjacent to the block of the predicted image; prediction residual signal generation means for calculating an error of each prediction signal of the block of the predicted image after the low-pass filter processing for each pixel signal of a block to be coded of the original image, and generating a prediction residual signal of each block; block division means for dividing the prediction residual signal of each block into a predetermined block shape, and orthogonal transform means having an orthogonal transform selection and application means for selecting and applying a plurality of types of orthogonal transform processing to each divided block of the prediction residual signal divided into blocks, depending on the position to which the low-pass filter processing has been applied.

[0018] In one embodiment, the predetermined neighboring pixel non-reference prediction includes any one of inter prediction, intra block copy prediction, and cross component signal prediction.

[0019] In one embodiment of the image encoding device, the filter processing means performs the low-pass filter processing so as to smooth out prediction signals adjacent to the decoded adjacent signals among prediction signals within a block of the predicted image generated by the adjacent pixel non-reference prediction means.

[0020] In one embodiment of the image encoding device, the block division means in the orthogonal transform means is characterized in that, when dividing a block of a prediction residual signal corresponding to a block of a prediction image having a block size larger than a size obtained by doubling a predetermined block size vertically and horizontally, the divided block at a position adjacent to the decoded adjacent signal is divided to have the predetermined block size.

[0021] In one embodiment of the image encoding device, the block division means in the orthogonal transform means sets a predetermined block size for a divided block including a pixel position to which the low-pass filter processing has been applied.

[0022] Moreover, in the image encoding device of one embodiment, the orthogonal transform selection application means in the orthogonal transform means applies a first orthogonal transform process of vertical DST and horizontal DCT to the divided block of the prediction residual signal located at the top when the pixel to which the low-pass filter process has been applied is at the top and the application direction of the low-pass filter process is vertical, applies a second orthogonal transform process of horizontal DST and vertical DCT to the divided block of the prediction residual signal located at the leftmost when the pixel to which the low-pass filter process has been applied is at the leftmost and the application direction of the low-pass filter process is horizontal, applies a third orthogonal transform process of vertical and horizontal DST when the pixel to which the low-pass filter process has been applied is at the top and at the leftmost and the application direction of the low-pass filter process is a direction of application of a corner filter, and applies a fourth orthogonal transform process of DCT in the vertical and horizontal directions to the divided block of the prediction residual signal in which no pixel to which the low-pass filter process has been applied exists.

[0023] In addition, in one embodiment of the image encoding device, the filter processing means is characterized in that it has a means for obtaining a local decoded image for each divided block based on the block division means, and replacing a prediction signal in a block of a predicted image generated by the adjacent pixel non-reference prediction each time a local decoded image is obtained, and performing low-pass filter processing according to each block size, in block size units of the divided block, using the decoded adjacent signals adjacent to the predicted image among the prediction signals in the replaced predicted image block, and the local decoded image for each divided block.

[0024] Furthermore, an image decoding device of one embodiment is an image decoding device that divides a frame constituting a moving image into blocks and decodes an encoded signal, and is characterized in that it comprises: an adjacent pixel non-reference prediction means for generating a block-by-block predicted image consisting of a predicted signal by a predetermined adjacent pixel non-reference prediction that performs signal prediction for each pixel signal in a block unit without using a decoded adjacent signal; a filter processing means for generating a new predicted image block by applying low-pass filter processing to a predicted signal located at a boundary of the block of the predicted image using the decoded adjacent signals adjacent to the block of the predicted image; an orthogonal transform selection and application means for selecting and applying multiple types of inverse orthogonal transform processing according to the position to which the low-pass filter processing was applied to the transform coefficients of the restored block unit that were divided into blocks by the image encoding side, thereby generating divided blocks of a prediction residual signal; and an inverse orthogonal transform means having a block reconstruction means for reconstructing a block corresponding to the block size of the prediction image based on the divided blocks of the prediction residual signal.

[0025] In addition, in one embodiment of the image decoding device, the filter processing means is characterized in that it has a means for obtaining a local decoded image for each divided block based on the block division means, and replacing a prediction signal in a block of a predicted image generated by the adjacent pixel non-reference prediction each time the local decoded image is obtained, and performing low-pass filter processing according to each block size, in block size units of the divided block, using the decoded adjacent signals adjacent to the predicted image among the prediction signals in the replaced predicted image block, and the local decoded image for each divided block.

[0026] Furthermore, the program of an embodiment causes a computer to function as the image encoding device or the image decoding device of an embodiment. Effect of the Invention

[0027] According to one embodiment, a signal error occurring between a prediction signal of a prediction image and an adjacent decoded block signal is reduced, improving the coding efficiency, and thus an image coding device and an image decoding device of a video coding method with high coding efficiency can be realized. That is, on the coding side, it is possible to reduce the residual component in the prediction residual signal, and the coding efficiency can be improved by reducing the amount of information to be coded and transmitted. Also, on the decoding side, it is possible to decode even with such a reduced amount of information. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1A is a block diagram of a filter processing unit for a predicted image in an image encoding device according to a first embodiment of the present invention, and FIG. 1B is an explanatory diagram showing an example of filtering of the predicted image. [Diagram 2] 1A, 1B, and 1C are explanatory diagrams illustrating adjacent pixel non-reference prediction according to the present invention. [Diagram 3] 1 is a block diagram showing an example of an image encoding device according to a first embodiment of the present invention. [Figure 4]4 is a flowchart relating to a filtering process of a predicted image in the image encoding device according to the first embodiment of the present invention. [Diagram 5] FIG. 2 is an explanatory diagram relating to a filtering process of a predicted image in the image encoding device according to the first embodiment of the present invention. [Figure 6] FIG. 2 is a block diagram of a filter processing unit related to a predicted image and its surroundings in the image decoding device according to the first embodiment of the present invention. [Figure 7] FIG. 1 is a block diagram showing an example of an image decoding device according to a first embodiment of the present invention. [Figure 8] FIG. 11 is a block diagram of a filter processing unit in an image encoding device or an image decoding device according to a second embodiment of the present invention. [Figure 9] 10 is a flowchart relating to a filtering process of a predicted image in an image encoding device or an image decoding device according to a second embodiment of the present invention. [Figure 10] 13(a) and 13(b) are explanatory diagrams relating to a filtering process of a predicted image in an image encoding device or an image decoding device according to a second embodiment of the present invention, respectively. [Figure 11] FIG. 13A is a block diagram of the filter processing unit and the orthogonal transform selection control unit in an image encoding device according to a third embodiment of the present invention, and FIGS. 13B and 13C are diagrams showing an example of a base waveform for orthogonal transform. [Figure 12] FIG. 11 is a block diagram showing a first example of an image encoding device according to a third embodiment of the present invention. [Figure 13] 13 is a flowchart relating to an orthogonal transform selection control process of a first example in an image encoding device according to a third embodiment of the present invention. [Figure 14] FIG. 11 is a block diagram showing a second embodiment of the image encoding device according to the third embodiment of the present invention. [Figure 15] 13 is a flowchart relating to an orthogonal transform selection control process of a second example in an image encoding device according to a third embodiment of the present invention. [Figure 16] FIG. 11 is a block diagram of the periphery of a filtering processing unit and an inverse orthogonal transform selection control unit in an image decoding device according to a third embodiment of the present invention. [Figure 17] FIG. 11 is a block diagram showing an example of an image decoding device according to a third embodiment of the present invention. [Figure 18] 13 is a flowchart relating to an inverse orthogonal transform selection control process of an example of an image decoding device according to a third embodiment of the present invention. [Figure 19] FIG. 13A is a block diagram of a filter processing unit for a predicted image in an image encoding device according to a fourth embodiment of the present invention, and FIG. 13B is an explanatory diagram showing an example of filtering of the predicted image. [Figure 20] 1A, 1B, and 1C are explanatory diagrams illustrating orthogonal transform application blocks after block division of a predicted image according to the present invention. [Figure 21] FIG. 13 is a block diagram showing an example of an image encoding device according to a fourth embodiment of the present invention. [Figure 22] 13 is a flowchart relating to a filtering process of a predicted image in an image encoding device according to a fourth embodiment of the present invention. [Diagram 23] 13 is a flowchart relating to an orthogonal transform process in an image encoding device according to a fourth embodiment of the present invention. [Figure 24] FIG. 13 is a block diagram of a filter processing unit related to a predicted image and its surroundings in an image decoding device according to a fourth embodiment of the present invention. [Diagram 25] FIG. 13 is a block diagram showing an example of an image decoding device according to a fourth embodiment of the present invention. [Figure 26] 13(a) and 13(b) are block diagrams of the filter processing unit and its surroundings in an image encoding device and an image decoding device according to a fifth embodiment of the present invention, respectively. [Figure 27] 13 is a flowchart relating to a filtering process of a predicted image in an image encoding device or an image decoding device according to a fifth embodiment of the present invention. [Figure 28] FIG. 13 is an explanatory diagram relating to a filtering process of a predicted image in an image encoding device or an image decoding device according to a fifth embodiment of the present invention. [Figure 29]1A, 1B, 1C, and 1D are explanatory diagrams relating to intra prediction in conventional techniques. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] An image encoding device and an image decoding device according to each embodiment of the present invention will be described below in order.

[0030] [First embodiment] (Image encoding device) First, a filter processing unit 12 for a predicted image, which is a main component of the image encoding device 1 of the first embodiment according to the present invention, will be described with reference to Figs. 1 and 2, and a specific typical example of the image encoding device 1 will be described with reference to Figs. 3 to 5.

[0031] FIG. 1(a) is a block diagram of a filter processing unit 12 for a predicted image and its surroundings in an image encoding device 1 according to a first embodiment of the present invention, and FIG. 1(b) is an explanatory diagram showing an example of filtering of the predicted image by the filter processing unit 12.

[0032] As shown in FIG. 1(a), the image encoding device 1 of the first embodiment according to the present invention is configured to include an adjacent pixel non-reference prediction unit 11, a filter processing unit 12, a prediction residual signal generation unit 13, and an orthogonal transformation unit 14.

[0033] The adjacent pixel non-reference prediction unit 11 is a functional unit that performs signal prediction processing for generating a predicted image for a coding target region of an original image without using adjacent decoded signals (decoded adjacent signals). In this specification, such signal prediction is referred to as "adjacent pixel non-reference prediction", and adjacent pixel non-reference prediction includes, for example, inter prediction that performs motion compensation prediction between frames, intra block copy prediction that copies different decoded partial images of the same frame to generate a predicted image, and signal prediction that generates a predicted image by utilizing the correlation between component signals of a luminance signal and a chrominance signal at corresponding block positions of a certain frame (referred to as "cross component signal prediction" in this specification).

[0034] For example, in inter prediction, as shown in Figure 2(a), among multiple frames F1 to F6 of a certain GOP structure, if frame F1 is an I frame (intra frame), a predicted image may be generated by referring to past I frames and P frames, as in a P frame (predictive inter frame), or a predicted image may be generated by referring to multiple frames, such as past and future frames, as in a B frame (bi-predictive inter frame).

[0035] Moreover, in intra block copy prediction, as shown in FIG. 2(b), a predicted image Bkp is generated by referencing and copying a different decoded partial image Bkr of the same frame F.

[0036] 2(c), cross-component signal prediction utilizes correlation between component signals to generate a corrected predicted image Bkp of a color difference signal (e.g., U / V signal) by performing synthesis processing using weighted addition on a predicted image of a color difference signal (e.g., U / V signal) with reference to a locally decoded image Bkr of a luminance signal (e.g., Y signal) at a corresponding block position in a certain frame F. For details regarding such cross-component signal prediction, please refer to JP2014-158270A.

[0037] These non-neighboring pixel reference predictions have in common that, while they have a decoded area adjacent to the block in the original image to be coded, they perform signal prediction without utilizing the high degree of correlation between those adjacent pixels.

[0038] The filter processing unit 12 in the first embodiment is a functional unit that generates a prediction image made up of new prediction signals by performing low-pass filter processing on the prediction signal, using decoded signals (decoded adjacent signals) adjacent to the left and above the prediction image, out of the pixel signals (prediction signal) within a block of the prediction image generated by the adjacent pixel non-reference prediction unit 11, to generate a prediction image made up of new prediction signals, and outputs the prediction image to the prediction residual signal generation unit 13.

[0039] For example, as shown in FIG. 1(b), assuming that a block Bkp of a predicted image consisting of a 4×4 pixel signal p corresponding to a block Bko of the encoding target area of ​​the original image has been generated by the adjacent pixel non-reference prediction unit 11, the filter processing unit 12 uses the decoded adjacent signals So around the block boundaries of the block Bkp of the predicted image adjacent to the left and above the predicted image to perform low-pass filter processing, such as horizontal and vertical smoothing filters, on the leftmost and topmost pixel signals (prediction signals) in the predicted image as signals Sf of the filtering target area, thereby generating a predicted image consisting of a new prediction signal.

[0040] The prediction residual signal generation unit 13 calculates the error between each pixel signal (prediction signal) of the block Bkp of the predicted image obtained from the filter processing unit 12 and each pixel signal (original signal) of the block (block to be encoded) Bko of the encoding target area of ​​the original image, and outputs the error to the orthogonal transformation unit 14 as a prediction residual signal.

[0041] The orthogonal transform unit 14 performs a predetermined orthogonal transform process on the prediction residual signal input from the prediction residual signal generating unit 13, and generates a signal of the transform coefficient. For example, the predetermined orthogonal transform process may be an orthogonal transform process used in an encoding method such as DCT or DST, or an integer orthogonal transform process defined in H.264 or H.265 that approximates the orthogonal transform process to an integer, as long as it complies with the encoding method to be used.

[0042] The image coding device 1 of the first embodiment performs quantization and entropy coding on the signal of the transform coefficients and outputs the result to the outside. In the entropy coding process, the signal is converted into a code by arithmetic coding such as CABAC together with various coding parameters, and the video can be transmitted. The coding parameters can include selectable parameters such as inter prediction parameters, intra prediction parameters, block size parameters (block partition parameters) related to block partitioning, and quantization parameters related to quantization processing, and can be transmitted.

[0043] Next, an inter-prediction unit 11a using inter-prediction will be described as a representative example of the adjacent pixel non-reference prediction unit 11, and a configuration and an example of its operation of an image encoding device 1 that applies filtering to a prediction signal of a block of the predicted image by a filtering processing unit 12 will be described with reference to Figures 3 to 5.

[0044] Fig. 3 is a block diagram showing an example of an image encoding device 1 according to a first embodiment of the present invention. The image encoding device 1 shown in Fig. 3 includes a preprocessing unit 10, an inter prediction unit 11a, a filtering unit 12, a prediction residual signal generating unit 13, an orthogonal transform unit 14, a quantization unit 15, an inverse quantization unit 16, an inverse orthogonal transform unit 17, a decoded image generating unit 18, an in-loop filter unit 19, a frame memory 20, an intra prediction unit 21, a motion vector calculation unit 22, an entropy encoding unit 23, and a predicted image selecting unit 24.

[0045] The pre-processing unit 10 divides an original image for each frame of input video data into coding target blocks of a predetermined block size, and outputs the blocks to the prediction residual signal generating unit 13 in a predetermined order.

[0046] The prediction residual signal generator 13 calculates the error of each pixel signal of the block of the predicted image obtained from the filter processor 12 relative to each pixel signal of the block to be coded, and outputs the error to the orthogonal transformer 14 as a block-by-block prediction residual signal.

[0047] The orthogonal transform unit 14 performs a predetermined orthogonal transform process on the prediction residual signal input from the prediction residual signal generating unit 13, and generates a signal of the transform coefficients.

[0048] The quantization unit 15 performs a predetermined quantization process on the transform coefficient signal obtained from the orthogonal transformation unit 14 , and outputs the result to the entropy coding unit 23 and the inverse quantization unit 16 .

[0049] The inverse quantization unit 16 performs inverse quantization processing on the quantized transform coefficient signal obtained from the quantization unit 15 , and outputs the result to the inverse orthogonal transformation unit 17 .

[0050] The inverse orthogonal transform unit 17 performs inverse orthogonal transform processing on the signal of the transform coefficients after inverse quantization obtained from the inverse quantization unit 16 to reconstruct a prediction residual signal, and outputs the reconstructed signal to the decoded image generation unit 18 .

[0051] The decoded image generation unit 18 adds the prediction residual signal restored by the inverse orthogonal transformation unit 17 to the block of the predicted image predicted by the intra prediction unit 21 or the inter prediction unit 11a and obtained from the filter processing unit 12 to generate a block of a locally decoded image, and outputs it to the in-loop filter unit 19.

[0052] The in-loop filter unit 19 performs in-loop filtering such as an adaptive loop filter (ALF), a sample adaptive offset (SAO), or a deblocking filter on the blocks of the locally decoded image obtained from the decoded image generation unit 18, and outputs the result to a frame memory 20. Filter parameters related to these filtering processes are output to an entropy coding unit 23 as one of the coding parameters that serve as auxiliary information for the coding process.

[0053] The frame memory 20 stores blocks of the locally decoded image obtained through the in-loop filter unit 19, and holds them as reference images that can be used by the intra prediction unit 21, the inter prediction unit 11a, and the motion vector calculation unit 22.

[0054] When the prediction image selection unit 24 performs signal selection using only signals within the frame to be predicted, the intra prediction unit 21 performs DC prediction, planar prediction, or directional prediction on the pixel signals (original signals) within the block to be coded of the original image, using pixel signals of coded and decoded blocks adjacent to the left or above the block to be coded, which are stored as reference images in the frame memory 20, and generates a block of predicted images made up of pixel signals (prediction signals) predicted by extrapolation, and outputs the generated block to the filter processing unit 12.

[0055] Incidentally, an intra prediction parameter for identifying DC prediction, planar prediction, or directional prediction used in the intra prediction unit 21 is output to the entropy coding unit 23 as one of the coding parameters serving as supplementary information for the coding process.

[0056] When the prediction image selection unit 24 selects a signal to perform inter-frame motion compensation prediction on a P / B frame, the inter prediction unit 11a generates a block of a prediction image by motion compensating the pixel signal (original signal) in the block to be coded of the original image using the motion vector provided by the motion vector calculation unit 22 to generate a block of a prediction image and outputs the block to the filter processing unit 12.

[0057] The motion vector calculation unit 22 searches for a position in the reference image data stored in the frame memory 20 that is most similar to the block to be coded in the original image, using a block matching technique or the like, calculates a value indicating the spatial shift as a motion vector, and outputs the motion vector to the inter prediction unit 11a.

[0058] The inter prediction parameters used in the inter prediction including the motion vector calculated by the motion vector calculation unit 22 are output to the entropy coding unit 23 as one of the coding parameters serving as supplementary information for the coding process.

[0059] The entropy coding unit 23 performs entropy coding processing on the output signal from the quantization unit 15 and various coding parameters, and outputs a stream signal of coded video data.

[0060] An example of the operation of the filter processing unit 12 will be described with reference to Fig. 4 and Fig. 5. As shown in Fig. 4, when a predicted image is input from the intra prediction unit 21 or the inter prediction unit 11a (step S1), the filter processing unit 12 identifies whether or not the predicted image is a neighboring pixel non-reference prediction, that is, in this example, whether or not the predicted image is an inter prediction based on a signal selection by the predicted image selection unit 24 (step S2). The same applies when the neighboring pixel non-reference prediction is the above-mentioned intra block copy prediction or cross component signal prediction.

[0061] Next, when the filter processing unit 12 determines that the predicted image is not adjacent pixel non-reference prediction, that is, in this example, intra prediction (step S2: No), it determines whether or not to perform filter processing by a predetermined processing method (step S5). When filter processing is not performed on the predicted image of intra prediction (step S5: No), the filter processing unit 12 outputs the predicted signal to the prediction residual signal generating unit 13 without performing filter processing on the predicted signal (step S6). On the other hand, when filter processing is performed on the predicted image of intra prediction (step S5: Yes), the filter processing unit 12 proceeds to step S3. Note that the filter processing on the predicted image of intra prediction can be performed in the same manner as the currently specified H.265, and is not directly related to the gist of the present invention, so the explanation of step S3 will be an example of performing filter processing on the predicted image of inter prediction.

[0062] When the filter processing unit 12 determines that the predicted image is a prediction without reference to adjacent pixels, that is, in this example, an inter prediction (step S2: Yes), it selects pixel signals (prediction signals) of a predetermined filter processing target area from the predicted image (step S3). For example, as illustrated in Fig. 5, among the pixel signals (prediction signals) in a block of a 4x4 predicted image, the leftmost and uppermost pixel signals (prediction signals) of the predicted image are determined as a filter processing target area Sf.

[0063] Next, the filter processing unit 12 generates a prediction image made of a new prediction signal by performing low-pass filtering on the pixel signals (prediction signals) of the selected filter processing target region using decoded signals (decoded adjacent signals) adjacent to the left and above the predicted image, and outputs the prediction image to the prediction residual signal generating unit 13 (step S4). For example, as illustrated in Fig. 5, the filter processing unit 12 performs smoothing filtering with a predetermined weighting factor on each of the horizontal filter region, vertical filter region, and corner filter region in the filter processing target region Sf among the pixel signals (prediction signals) in the 4x4 predicted image block Bkp. and outputs the result to the prediction residual signal generator 13.

[0064] The filtering process by the filtering unit 12 may be a low-pass filtering process, and filtering processes other than a smoothing filter may be applied, and the area of ​​the decoded adjacent signal and the weighting coefficients used for the filtering process do not have to be limited to the example shown in Fig. 5. For example, in the case of a large-sized coding block, filtering may be applied to multiple rows or columns. The filter type, filtering target area, and weighting coefficients related to the filtering process by the filtering unit 12 may be configured to be transmitted as auxiliary information for the coding process, but they do not need to be transmitted if they are determined in advance between the transmitter and the receiver (between the coding and decoding).

[0065] By performing filtering on the predicted image by the filter processing unit 12 in this manner, regardless of the type of orthogonal transform processing performed by the orthogonal transform unit 14, the signal error that occurs between the leftmost and topmost pixel signals (prediction signals) in the predicted image and the adjacent decoded signals is reduced, thereby improving the video encoding efficiency.

[0066] (Image Decoding Device) Next, the filter processing unit 54 and its peripheral functional blocks, which are the main components of the image decoding device 5 of the first embodiment of the present invention, will be described with reference to FIG. 6, and a specific typical example of the image decoding device 5 will be described with reference to FIG. 7.

[0067] 6 is a block diagram of a filter processing unit 54 related to a predicted image and its surroundings in the image decoding device 5 according to the first embodiment of the present invention. As shown in FIG. 6, the image decoding device 5 according to the first embodiment of the present invention is configured to include an inverse orthogonal transform unit 52, an adjacent pixel non-reference prediction unit 53, a filter processing unit 54, and a decoded image generation unit 55.

[0068] The inverse orthogonal transform unit 52 receives a stream signal transmitted from the image encoding device 1 side, performs an inverse orthogonal transform process on the transform coefficients restored through entropy decoding process and inverse quantization process, and outputs the resulting prediction residual signal to the decoded image generation unit 55.

[0069] The neighboring pixel non-reference prediction unit 53 is a functional unit corresponding to the neighboring pixel non-reference prediction unit 11 on the image encoding device 1 side, and performs signal prediction processing to generate a predicted image without using adjacent decoded signals (decoded neighboring signals). Examples of neighboring pixel non-reference prediction include inter prediction that performs motion compensation prediction between frames, intra block copy prediction that generates a predicted image by copying different decoded partial images of the same frame, and cross component signal prediction that generates a predicted image by utilizing the correlation between component signals of luminance signals and chrominance signals at corresponding block positions of a certain frame.

[0070] The filter processing unit 54 of the first embodiment is a functional unit corresponding to the filter processing unit 12 on the image encoding device 1 side, and uses decoded signals (decoded adjacent signals) adjacent to the left and above the predicted image, respectively, from among the pixel signals (prediction signals) in a block of the predicted image generated by the adjacent pixel non-reference prediction unit 53, to perform low-pass filter processing on the prediction signal to generate a prediction image made up of a new prediction signal, and outputs the generated prediction image to the decoded image generation unit 55.

[0071] The decoded image generation unit 55 is a functional unit that adds a prediction residual signal restored by the inverse orthogonal transformation unit 52 to a block of a predicted image obtained from the filter processing unit 54 described later, to generate a block of a decoded image consisting of a decoded signal.

[0072] That is, the filter processing unit 12 in the image encoding device 1 filters the predicted image. The prediction residual signal generated through the filtering process is transmitted with the video encoding efficiency improved, and the image decoding device 5 can efficiently restore the transmitted video. As in the filtering process of ...

[0073] Next, an inter prediction unit 53a using inter prediction will be described as a representative example of the adjacent pixel non-reference prediction unit 53, and a configuration and operation example of an image decoding device 5 that applies filtering to a prediction signal by the inter prediction using a filtering processing unit 54 will be described with reference to FIG. 7.

[0074] Fig. 7 is a block diagram showing an example of an image decoding device 5 according to the first embodiment of the present invention. The image decoding device 5 shown in Fig. 7 includes an entropy decoding unit 50, an inverse quantization unit 51, an inverse orthogonal transformation unit 52, an inter prediction unit 53a, a filter processing unit 54, a decoded image generation unit 55, an in-loop filter unit 56, a frame memory 57, an intra prediction unit 58, and a predicted image selection unit 59.

[0075] The entropy decoding unit 50 receives a stream signal transmitted from the image encoding device 1 side, and outputs various encoding parameters obtained by performing an entropy decoding process corresponding to the entropy encoding process of the image encoding device 1 to each functional block, and also outputs quantized transform coefficients to the inverse quantization unit 51. As the various encoding parameters, for example, filter parameters, intra prediction parameters, and inter prediction parameters are output to an in-loop filter unit 56, an intra prediction unit 58, and an inter prediction unit 53a, respectively.

[0076] The inverse quantization unit 51 performs inverse quantization processing corresponding to the quantization processing of the image encoding device 1 on the quantized transformation coefficients obtained from the entropy decoding unit 50 , and outputs the transformation coefficients to an inverse orthogonal transformation unit 52 .

[0077] The inverse orthogonal transform unit 52 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing of the image encoding device 1 on the transform coefficients obtained from the inverse quantization unit 51 to restore a prediction residual signal, and outputs the signal to a decoded image generation unit 55.

[0078] The decoded image generation unit 55 adds the prediction residual signal restored by the inverse orthogonal transformation unit 52 to a block of a predicted image predicted by the intra prediction unit 58 or the inter prediction unit 53a and generated by applying a low-pass filter by the filter processing unit 54, to generate a block of a decoded image, and outputs the generated block of a decoded image to the in-loop filter unit 56.

[0079] The in-loop filter unit 56 performs a filtering process corresponding to the in-loop filtering process on the image encoding device 1 side, and outputs the result to a frame memory 57 .

[0080] The frame memory 57 stores blocks of the decoded image obtained through the in-loop filter unit 56, and holds them as reference images that can be used by the intra prediction unit 58 and the inter prediction unit 53a.

[0081] When the prediction image selection unit 59 performs signal selection using only signals within the frame to be predicted, the intra prediction unit 58 uses intra prediction parameters and pixel signals of decoded blocks adjacent to the left and above the block to be decoded, which are stored as reference images in the frame memory 57, to perform prediction processing corresponding to that of the intra prediction unit 21 on the image encoding device 1 side to generate a predicted image block and output it to the filter processing unit 54.

[0082] When the prediction image selection unit 59 selects a signal to perform inter-frame motion compensation prediction on a P / B frame, the inter prediction unit 53a uses the motion vector included in the inter prediction parameters to motion compensate the data stored as a reference image in the frame memory 20 to generate a predicted image block, and outputs the block to the filter processing unit 54.

[0083] It is possible to configure the system so that a frame is formed from the decoded data stored in the frame memory 57 and is output as a decoded image.

[0084] The filter processing unit 54 operates in the same manner as that illustrated and described with reference to Fig. 4 and Fig. 5. That is, in the filter processing according to the present invention in the filter processing unit 54, when the prediction image is prediction without reference to adjacent pixels, that is, in this example, inter prediction, the filter processing selects pixel signals (prediction signals) of a predetermined filter processing target area from the prediction image, and generates a prediction image consisting of a new prediction signal by performing low-pass filter processing on the pixel signals (prediction signals) of the selected filter processing target area using decoded signals (decoded adjacent signals) adjacent to the left and upper sides of the prediction image, and outputs the prediction image consisting of a new prediction signal to the decoded image generating unit 55. For example, as illustrated in Fig. 5, the filter processing unit 54 performs smoothing filter processing of a predetermined weighting factor on each of the horizontal filter area, vertical filter area, and corner filter area in the filter processing target area Sf of the pixel signals (prediction signals) in the block Bkp of the prediction image, using the decoded adjacent signals So around the block boundary of the block Bkp of the prediction image, and outputs the result to the decoded image generating unit 55.

[0085] According to the image encoding device 1 and image decoding device 5 of the first embodiment configured as above, the signal error occurring between the prediction signal of the predicted image and the adjacent decoded block signal is reduced, improving the encoding efficiency, so that it is possible to realize an image encoding device and an image decoding device of a video encoding method with high encoding efficiency. That is, it is possible to further reduce the residual component in the prediction residual signal, and to improve the encoding efficiency by reducing the amount of information to be encoded and transmitted.

[0086] Second Embodiment Next, the filter processing units 12 and 54 in the image encoding device 1 and image decoding device 5 according to the second embodiment of the present invention will be described with reference to Fig. 8. The filter processing units 12 and 54 in the first embodiment have been described as an example in which, out of pixel signals (prediction signals) in a block of a predicted image generated by adjacent pixel non-reference prediction, decoded signals (decoded adjacent signals) adjacent to the left and above the predicted image are used to perform low-pass filter processing on the prediction signals to generate a predicted image made of a new predicted signal.

[0087] On the other hand, the filter processing unit 12, 54 of this embodiment is configured to perform correlation judgment to determine whether or not to apply low-pass filter processing to the prediction signal to be subjected to the low-pass filter processing, using the decoded adjacent signal referred to in the low-pass filter processing among pixel signals (prediction signals) in a block of a prediction image generated by adjacent pixel non-reference prediction, and to determine whether or not to apply low-pass filter processing according to the result of this correlation judgment. Note that similar components are described with the same reference numbers. Therefore, since the filter processing unit 12, 54 of this embodiment can be applied to the image encoding device 1 and the image decoding device 5 illustrated in Fig. 3 and Fig. 7, respectively, only the processing contents related to the filter processing unit 12, 54 of this embodiment will be described here, and further detailed description thereof will be omitted.

[0088] Fig. 8 is a block diagram of the filter processing units 12, 54 in the image encoding device 1 or image decoding device 5 of this embodiment. Fig. 9 is a flowchart relating to the filter processing 12, 54 of the predicted image in the image encoding device 1 or image decoding device 5 of this embodiment.

[0089] The filter processing units 12 and 54 of this embodiment include a horizontal correlation determination unit 101, a vertical correlation determination unit 102, a filter processing determination unit 103, and a filter processing execution unit 104. The operations of these functional blocks will be described with reference to the processing example shown in Fig. 9. Note that the filter processing units 12 and 54 of this embodiment can also be configured to perform processing regardless of whether the prediction is neighboring pixel non-reference prediction, for example, whether the prediction is inter prediction or intra prediction, but here, an example of performing filter processing on a prediction image of neighboring pixel non-reference prediction according to the present invention will be described.

[0090] When the filter processing units 12 and 54 receive the predicted image (step S11), the horizontal correlation determination unit 101 and the vertical correlation determination unit 102 execute correlation determination processing for each filter processing target region individually determined (step S12).

[0091] That is, the horizontal correlation determination unit 101 determines the correlation with the decoded adjacent signal adjacent to the left side of the predicted image (step S13), and if it is determined that the horizontal correlation is high (step S13: Yes), it decides to perform horizontal filter processing using the decoded adjacent signal (step S14), and if it is determined that the horizontal correlation is low (step S13: No), it outputs to the filter processing determination unit 103 a message indicating that horizontal filter processing will not be performed.

[0092] Similarly, the vertical correlation determination unit 102 determines the correlation with the decoded adjacent signal adjacent to the upper side of the predicted image (step S15), and if it determines that the vertical correlation is high (step S15: Yes), it decides to perform vertical filter processing using the decoded adjacent signal (step S16), and if it determines that the vertical correlation is low (step S15: No), it outputs to the filter processing determination unit 103 a message that vertical filter processing will not be performed.

[0093] The filter processing determination unit 103 determines whether or not it has received a signal to determine that both horizontal and vertical filter processing are to be performed (step S17). If it has received a signal to determine that both horizontal and vertical filter processing are to be performed (step S17: Yes), it determines that both horizontal and vertical filter processing and 3-tap corner filter processing are to be performed (step S18). If it has received a signal to not perform either or both of the horizontal and vertical filter processing (step S17: No), it determines that only horizontal direction filter processing is to be performed when horizontal correlation is high, and only vertical direction filter processing is to be performed when vertical correlation is high, and also determines that the corner filter is to perform horizontal or vertical filter processing when only one of horizontal and vertical filter processing is performed (step S19).

[0094] 9 is merely an example and may be configured in any order. Therefore, the filter processing determination unit 103 performs correlation determination for each filter processing target area individually defined in the horizontal and vertical directions, determines whether the filter processing is to be "executed in the horizontal direction only", "executed in the vertical direction only", "executed in all of the horizontal, vertical, and angular directions", or "not executed in any of the horizontal, vertical, or angular directions", and outputs the result to the filter processing execution unit 104.

[0095] The filter processing execution unit 104 performs filter processing according to the determination result of the filter processing determination unit 103 and outputs the predicted image after the filter processing (step S20). When the determination result of the filter processing determination unit 103 is "no horizontal, vertical, or angular processing is performed", the predicted image input to the filter processing units 12 and 54 is output as is.

[0096] For example, the horizontal direction correlation determination unit 101 obtains the difference between adjacent pixels for each of the four horizontal direction determination areas A1 to A4 shown in FIG. 10(a), and determines that the correlation is high if the sum of the differences in the horizontal direction determination areas A1 to A4 is equal to or less than a predetermined threshold, and determines that the correlation is low if not. Similarly, the vertical direction correlation determination unit 102 obtains the difference between adjacent pixels for each of the four vertical direction determination areas B1 to B4 shown in FIG. 10(b), and determines that the correlation is high if the sum of the differences in the vertical direction determination areas B1 to B4 is equal to or less than a predetermined threshold, and determines that the correlation is low if not. Then, by applying a filter process in a direction with high correlation as shown in FIG. 5, the filter process can be "performed only in the horizontal direction", "performed only in the vertical direction", "performed in all horizontal, vertical, and corner directions", or "not performed in any of the horizontal, vertical, and corner directions". Then, when performing a filter process only in the horizontal direction or the vertical direction, a horizontal or vertical filter is applied to a pixel (prediction signal) located at a corner of the block of the prediction image.

[0097] It should be noted that the correlation determination region and the filter processing target region do not have to be the same, but making them the same simplifies the processing. It should also be noted that various techniques are envisaged for the specific processing methods of the correlation determination process and the filter processing, and the above is merely one example.

[0098] By configuring the filter processing units 12 and 54 as in this embodiment and applying them to the image encoding device 1 and the image decoding device 5, respectively, it is possible to improve the encoding efficiency while suppressing the resulting deterioration in image quality.

[0099] Third Embodiment (Image encoding device) Next, the filter processing unit 12 related to the predicted image and the orthogonal transform selection control unit 25, which are the main components of the image encoding device 1 according to the third embodiment of the present invention, will be described with reference to Fig. 1(b) and Fig. 11, and two specific examples of the image encoding device 1 will be described with reference to Fig. 5, Fig. 12 to Fig. 15. The following description will be given with a focus on the differences from the above-mentioned embodiments. That is, in this embodiment, the same components as those in the above-mentioned embodiments are given the same reference numbers, and further detailed description thereof will be omitted.

[0100] FIG. 11(a) is a block diagram of the filter processing unit 12 and the orthogonal transform selection control unit 25 related to the predicted image in the image encoding device 1 of this embodiment, and FIG. 11(b) is an explanatory diagram showing an example of filtering of the predicted image by the filter processing unit 12.

[0101] As shown in FIG. 11(a), the image encoding device 1 of this embodiment is configured to include an adjacent pixel non-reference prediction unit 11, a filter processing unit 12, a prediction residual signal generation unit 13, an orthogonal transformation unit 14, and an orthogonal transformation selection control unit 25.

[0102] The filter processing unit 12 in this embodiment is a functional unit that, under the control of the orthogonal transform selection control unit 25, generates a prediction image made up of a new prediction signal by performing low-pass filter processing on the prediction signal using decoded signals (decoded adjacent signals) adjacent to the left and above the prediction image, out of the pixel signals (prediction signal) within a block of the prediction image generated by the adjacent pixel non-reference prediction unit 11, to generate the prediction image made up of a new prediction signal, and outputs the prediction image to the prediction residual signal generation unit 13.

[0103] The orthogonal transform unit 14, under the control of the orthogonal transform selection control unit 25, performs one of two types of orthogonal transform processing on the prediction residual signal input from the prediction residual signal generating unit 13, and generates a signal of the transform coefficients.

[0104] For example, the two types of orthogonal transform processing can be orthogonal transform processing in which the ends of the transform base are closed (e.g., DST (Discrete Sine Transform)), or orthogonal transform processing in which the ends of the transform base are open (e.g., DCT (Discrete Cosine Transform)). There is no limitation as to whether the processing has real number precision or integer precision, but the processing can be, for example, integer orthogonal transform processing defined in H.264 or H.265 that approximates integers, as long as it complies with the encoding method used.

[0105] Here, an example of the basis waveform of the orthogonal transform is shown in Fig. 11(b) and Fig. 11(c). The examples shown in Fig. 11(b) and Fig. 11(c) illustrate the transform basis waveforms (u=0-3 in DCT, u=1-4 in DST) from low to high frequencies as the patterns of each frequency component of cosine and sine in the DCT transform basis (Fig. 11(b)) and the DST transform basis (Fig. 11(c)) for the four-point transform basis (N=4) available for orthogonal transform of a block of a 4x4 prediction residual signal. As shown in Fig. 11(b) and Fig. 11(c), the main difference other than u=0 in DCT and u=4 in DST is the phase, and it can be seen that each transform basis (e.g. u=2) of the corresponding frequency has the same inter-pixel correlation (same basis amplitude) at the same frequency, but the phase is shifted by π / 2. In either transform basis waveform, the DCT realizes an orthogonal transform process in which the end points have large values ​​and are open, as shown in Fig. 11(b), whereas the DST realizes an orthogonal transform process in which the end points have small values ​​and are closed, as shown in Fig. 11(c).

[0106] Note that "orthogonal transform processing with closed ends of the transform base" refers to one end of a block of a prediction residual signal that is adjacent to the prediction reference block being closed, and may be, for example, asymmetric DST type 7 of the transform base with the other end being open, as shown in Figure 11(c).

[0107] When the orthogonal transform selection control unit 25 causes the filter processing unit 12 to perform low-pass filter processing on pixel values ​​(prediction signals) located at block boundaries of the prediction image based on a predetermined evaluation criterion during adjacent pixel non-reference prediction, the orthogonal transform selection control unit 25 instructs the orthogonal transform unit 14 to use an orthogonal transform process (e.g., DST) with a closed end of the transform base for the block of the prediction residual signal obtained from the prediction residual signal generation unit 13 based on the block of the prediction image after the low-pass filter processing, and when the low-pass filter processing is not performed on the block of the prediction image based on the predetermined evaluation criterion, the orthogonal transform selection control unit 25 instructs the orthogonal transform unit 14 to use an orthogonal transform process (e.g., DCT) with an open end of the transform base for the block of the prediction residual signal, thereby selecting and controlling the orthogonal transform type. The orthogonal transform selection control unit 25 outputs a transform type identification signal indicating which of the two types of orthogonal transform processing has been applied to the decoding side as one of the coding parameters.

[0108] As the predetermined evaluation criterion in the orthogonal transform selection control unit 25, two embodiments will be described.

[0109] Although details will be described later, in the first embodiment, the orthogonal transform selection control unit 25 causes the filter processing unit 12 to select a signal of a filter processing target area as shown in Fig. 1(b), instructs the filter processing unit 12 to tentatively execute low-pass filter processing using decoded adjacent signals, and acquires each predicted image with / without the filter processing as prediction block information (est1) from the filter processing unit 12. Next, the orthogonal transform selection control unit 25 compares each predicted image with / without the filter processing from this prediction block information (est1), and when it is determined that the influence of the end point of the prediction area (i.e., the prediction residual signal on the pixel signal located at the outer periphery of the block of the prediction image) is relatively large with respect to the block size (4x4 is illustrated in this example, but the block size does not matter), causes the filter processing unit 12 to output the predicted image after the filter processing is executed to the prediction residual signal generation unit 13. Then, the orthogonal transform selection control unit 25 controls the orthogonal transform unit 14 to instruct it to use an orthogonal transform process (e.g., DST) in which the ends of the transform base are closed for the block of the prediction residual signal obtained from the prediction residual signal generating unit 13 based on the block of the predicted image after this low-pass filter processing.

[0110] On the other hand, the orthogonal transform selection control unit 25 compares each of the predicted images obtained from the filter processing unit 12 with and without the filter processing, and when it is determined that the influence of the end points of the prediction region (i.e., the prediction residual signal on the pixel signals located on the outer periphery of the block of the predicted image) is relatively small with respect to the block size (4×4 is illustrated in this example, but the block size does not matter), it causes the filter processing unit 12 to output the predicted image not subjected to the filter processing to the prediction residual signal generation unit 13. Then, the orthogonal transform selection control unit 25 controls to instruct the orthogonal transform unit 14 to use the orthogonal transform processing (e.g., DCT) with the end of the transform base opened for the block of the prediction residual signal obtained from the prediction residual signal generation unit 13 based on the block of the predicted image not subjected to the low-pass filter processing.

[0111] Here, various evaluation methods are assumed for whether the influence of the end points of the prediction region is relatively large with respect to the block size (4×4 is illustrated in this example, but the block size is not important), and for example, it may be judged by comparing the ratio between the variance value of the density distribution of the end points of the prediction region and the variance value of the density distribution of the entire block size and whether it is equal to or higher than a predetermined level. Thus, in the first embodiment, the orthogonal transform selection control unit 25 determines whether or not to execute the filter process based on the criterion for judging whether the filter process effect by the filter process is equal to or higher than a predetermined level, and further determines the type of orthogonal transform process to be applied depending on whether or not the filter process is executed, and controls the filter processing unit 12 and the orthogonal transform unit 14.

[0112] In addition, in the second embodiment, the orthogonal transform selection control unit 25 instructs the filter processing unit 12 to select a signal in the region to be filtered as shown in FIG. 1(b), instructs the filter processing unit 12 to tentatively execute low-pass filter processing using the decoded adjacent signals, and acquires and compares rate-distortion information (est2) of the generated coding amount (R) and the coding distortion amount (D) in the cases where an orthogonal transform process (e.g., DST) with closed ends of the transform base is performed when the filter processing is performed, followed by quantization processing and entropy coding processing, and an orthogonal transform process (e.g., DCT) with open ends of the transform base is performed when the filter processing is not performed, followed by quantization processing and entropy coding processing, and selects the combination with the better RD cost (e.g., a combination of performing filter processing and DST and a combination of not performing filter processing and DCT).

[0113] By using this type of RD optimization, if the combination of "execution of filtering process and orthogonal transform process with closed ends of the transform base" is found to be superior, the orthogonal transform selection control unit 25 controls the filter processing unit 12 to output the predicted image after the filtering process to the prediction residual signal generation unit 13, and instructs the orthogonal transform unit 14 to use the orthogonal transform process with closed ends of the transform base (e.g., DST) for the block of the prediction residual signal obtained from the prediction residual signal generation unit 13 based on the block of the predicted image after the low-pass filter process.

[0114] On the other hand, if the combination of "no filtering performed-orthogonal transform processing with the ends of the transform base opened" is superior, the orthogonal transform selection control unit 25 controls the filter processing unit 12 to output a predicted image with no filtering performed to the prediction residual signal generation unit 13, and instructs the orthogonal transform unit 14 to use an orthogonal transform processing (e.g., DCT) with the ends of the transform base opened for the block of the prediction residual signal obtained from the prediction residual signal generation unit 13 based on the block of the predicted image with no low-pass filtering performed.

[0115] As described above, in the second embodiment, the orthogonal transform selection control unit 25 determines whether or not to perform filter processing based on RD optimization, and further determines the type of orthogonal transform processing to be applied depending on whether or not to perform filter processing, and controls the filter processing unit 12 and the orthogonal transform unit 14.

[0116] Finally, the image coding device 1 of this embodiment performs quantization and entropy coding on the signal of the transform coefficient obtained from the orthogonal transform unit 14, and outputs the result to the outside. In the entropy coding process, the signal is converted into a code by arithmetic coding such as CABAC together with various coding parameters, and the video can be transmitted. Note that the coding parameters can include, in addition to the transformation type identification signal according to the present invention, selectable parameters such as inter prediction parameters, intra prediction parameters, block size parameters (block partition parameters) related to block partitioning, and quantization parameters related to quantization processing, and can be transmitted.

[0117] More specifically, below, an inter prediction unit 11a using inter prediction is used as a representative example of the adjacent pixel non-reference prediction unit 11, and examples of the configuration of an image encoding device 1 to which each of the above-mentioned embodiments is applied are described as a predetermined evaluation criterion in the orthogonal transform selection control unit 25.

[0118] (First Example) Fig. 12 is a block diagram showing a first example of the image coding device 1 of this embodiment. The image coding device 1 shown in Fig. 12 includes a preprocessing unit 10, an inter prediction unit 11a, a filter processing unit 12, a prediction residual signal generating unit 13, an orthogonal transformation unit 14, a quantization unit 15, an inverse quantization unit 16, an inverse orthogonal transformation unit 17, a decoded image generating unit 18, an in-loop filter unit 19, a frame memory 20, an intra prediction unit 21, a motion vector calculation unit 22, an entropy coding unit 23, a prediction image selecting unit 24, and an orthogonal transformation selection control unit 25.

[0119] The orthogonal transform unit 14, under the control of the orthogonal transform selection control unit 25, performs a predetermined orthogonal transform process on the prediction residual signal input from the prediction residual signal generating unit 13, and generates a signal of the transform coefficients.

[0120] In the first embodiment, the filter processing unit 12 is controlled to execute or not execute low-pass filter processing under the control of the orthogonal transform selection control unit 25, and when the low-pass filter processing is executed, among pixel signals (prediction signals) in a block of a prediction image generated by the inter prediction unit 11a, which is a neighboring pixel non-reference prediction, decoded signals (decoded neighboring signals) adjacent to the left and upper sides of the prediction image are used to perform low-pass filter processing on the prediction signal to generate a prediction image made of a new prediction signal, and output the prediction image to the prediction residual signal generation unit 13. When the low-pass filter processing is not executed, the prediction image generated by the inter prediction unit 11a is output to the prediction residual signal generation unit 13. Note that, when prediction other than the neighboring pixel non-reference prediction such as intra prediction, the filter processing unit 12 is assumed to follow a predetermined method for whether or not to execute the filter processing.

[0121] The operation of the orthogonal transform selection control unit 25 in the first embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart related to the processing of the orthogonal transform selection control unit 25 in the first embodiment of the image encoding device 1 of this embodiment. First, upon receiving a notification from, for example, the filter processing unit 12 or the predicted image selection unit 24, the orthogonal transform selection control unit 25 determines whether or not the predicted image processed by the filter processing unit 12 is adjacent pixel non-reference prediction (step S1).

[0122] When prediction is performed without reference to adjacent pixels (step S1: Yes), the orthogonal transform selection control unit 25 instructs the filter processing unit 12 to select a signal in the area to be filtered, and to temporarily execute low-pass filter processing using the decoded adjacent signal (step S2).

[0123] For example, as illustrated in FIG. 5, the filter processing unit 12 determines the leftmost and topmost pixel signals (prediction signals) of the prediction image as a filter processing target area Sf among the pixel signals (prediction signals) in a block of a 4×4 prediction image. Next, the filter processing unit 12 generates a prediction image made of a new prediction signal by performing low-pass filter processing on the pixel signals (prediction signals) of the selected filter processing target area using decoded signals (decoded adjacent signals) adjacent to the left and top of the prediction image. For example, as illustrated in FIG. 5, the filter processing unit 12 performs smoothing filter processing with a predetermined weighting factor on each of the horizontal filter area, vertical filter area, and corner filter area in the filter processing target area Sf among the pixel signals (prediction signals) in the block Bkp of the 4×4 prediction image to generate a prediction image after the filter processing.

[0124] The filtering process by the filtering unit 12 may be a low-pass filtering process, and filtering processes other than a smoothing filter may be applied, and the area of ​​the decoded adjacent signal and the weighting coefficients used for the filtering process do not have to be limited to the example shown in Fig. 5. For example, in the case of a large-sized coding block, filtering may be applied to multiple rows or columns. The filter type, filtering target area, and weighting coefficients related to the filtering process by the filtering unit 12 may be configured to be transmitted as auxiliary information for the coding process, but they do not need to be transmitted if they are determined in advance between the transmitter and the receiver (between the coding and decoding).

[0125] Next, referring to FIG. 13, the orthogonal transform selection control unit 25 acquires each predicted image for which the filtering process is performed / not performed from the filtering unit 12 as predicted block information (est1) (step S3).

[0126] Next, the orthogonal transform selection control unit 25 compares each predicted image with / without performing the filter process from the prediction block information (est1), and determines whether the effect of the filter process is above a predetermined level based on whether the influence of the end points of the prediction area (i.e., the prediction residual signal for the pixel signals located on the periphery of the block of the predicted image) is relatively large with respect to the block size (4x4 is exemplified in this example, but the block size is not important) (step S4).

[0127] When it is determined that the influence of the end points of the prediction region is relatively large compared to the block size (step S4: Yes), the orthogonal transform selection control unit 25 outputs the predicted image after the filter processing from the filter processing unit 12 to the prediction residual signal generation unit 13, and instructs the orthogonal transform unit 14 to use an orthogonal transform process (e.g., DST) with closed ends of the transform base for the block of the prediction residual signal obtained from the prediction residual signal generation unit 13 based on the block of the predicted image after the low-pass filter processing (step S5).

[0128] On the other hand, when it is determined that the influence of the end points of the prediction region is relatively small compared to the block size (step S4: No), the orthogonal transform selection control unit 25 outputs a predicted image not subjected to the filter processing from the filter processing unit 12 to the prediction residual signal generation unit 13, and instructs the orthogonal transform unit 14 to use an orthogonal transform process (e.g., DCT) in which the ends of the transform base are open for the block of the prediction residual signal obtained from the prediction residual signal generation unit 13 based on this block of the predicted image not subjected to the low-pass filter processing (step S6).

[0129] In addition, when prediction is performed other than prediction without reference to adjacent pixels, such as intra prediction (step S1: No), the orthogonal transform selection control unit 25 controls the filter processing unit 12 and the orthogonal transform unit 14 to determine whether or not to perform filter processing and to select the type of orthogonal transform according to a predetermined method (step S7).

[0130] As described above, in the first embodiment, the orthogonal transform selection control unit 25 controls the filter processing unit 12 and the orthogonal transform unit 14 to determine whether or not to perform filter processing and the type of orthogonal transform processing to be applied, based on a criterion of whether or not the filter processing effect of the filter processing is equal to or higher than a predetermined level. Then, the orthogonal transform selection control unit 25 outputs a transform type identification signal indicating which of the two types of orthogonal transform processing has been applied to the decoding side as one of the coding parameters via the entropy coding unit 23.

[0131] As described above, based on the criterion of whether or not the filtering effect of the filtering process is equal to or greater than a predetermined level, the filter processing unit 12 and the orthogonal transform unit 14 are controlled by determining whether or not to perform filtering and the type of orthogonal transform process to be applied, thereby making it possible to reduce the residual components in the prediction residual signal and improve the coding efficiency by reducing the amount of information to be coded and transmitted.

[0132] In particular, even when prediction is performed without reference to adjacent pixels, the correlation between adjacent pixel signals is utilized for the block of prediction residual signal obtained based on the block of the predicted image after the filtering process, and an orthogonal transform process (e.g., DST) in which the ends of the transform base are closed is used, so that not only the phase but also the characteristics of the transform base corresponding to the end points of the prediction region are matched, thereby making it possible to more significantly improve coding efficiency.

[0133] (Second Example) Next, a second embodiment will be described. Fig. 14 is a block diagram showing a second embodiment of the image encoding device 1 according to the present embodiment. In Fig. 14, the same components as those in Fig. 12 are given the same reference numerals, and further detailed description thereof will be omitted.

[0134] In the second embodiment, the filter processing unit 12 is controlled to execute or not execute low-pass filter processing under the control of the orthogonal transform selection control unit 25, as in the first embodiment, and when the low-pass filter processing is executed, among pixel signals (prediction signals) in a block of a prediction image generated by the inter prediction unit 11a, which is a neighboring pixel non-reference prediction, decoded signals (decoded neighboring signals) adjacent to the left and upper sides of the prediction image are used to perform low-pass filter processing on the prediction signal to generate a prediction image made of a new prediction signal, and output the prediction image to the prediction residual signal generation unit 13. When the low-pass filter processing is not executed, the prediction image generated by the inter prediction unit 11a is output to the prediction residual signal generation unit 13. Note that, when prediction other than the neighboring pixel non-reference prediction such as intra prediction, the filter processing unit 12 is assumed to follow a predetermined method for whether or not to execute the filter processing.

[0135] The operation of the orthogonal transform selection control unit 25 in the second embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart related to the processing of the orthogonal transform selection control unit 25 in the image encoding device 1 of this embodiment in the second embodiment. First, upon receiving a notification from, for example, the filter processing unit 12 or the predicted image selection unit 24, the orthogonal transform selection control unit 25 determines whether or not the predicted image processed by the filter processing unit 12 is adjacent pixel non-reference prediction (step S11).

[0136] When prediction is performed without reference to adjacent pixels (step S11: Yes), the orthogonal transform selection control unit 25 instructs the filter processing unit 12 to select a signal of the region to be filtered as shown in FIG. 1(b) and to temporarily execute low-pass filter processing using the decoded adjacent signals (step S12).

[0137] Next, the orthogonal transform selection control unit 25, when performing the filter processing of the filter processing unit 12, causes the orthogonal transform unit 14 to output a predicted image after the filter processing from the filter processing unit 12 to the prediction residual signal generation unit 13 using an orthogonal transform processing (e.g., DST) in which the ends of the transform base are closed, and performs an orthogonal transform on a block of the prediction residual signal obtained from the prediction residual signal generation unit 13 based on the block of the predicted image after the low-pass filter processing, and performs quantization processing and entropy coding processing by the quantization unit 15 and the entropy coding unit 23, and outputs a transform base to the orthogonal transform unit 14 when the filter processing of the filter processing unit 12 is not performed. A predicted image not subjected to the low-pass filter process is output from the filter processing unit 12 to the prediction residual signal generation unit 13 using an orthogonal transform process (e.g., DCT) with open ends, and a block of the prediction residual signal obtained from the prediction residual signal generation unit 13 is orthogonally transformed based on the block of the predicted image not subjected to the low-pass filter process. When quantization process and entropy coding process are performed by the quantization unit 15 and the entropy coding unit 23, rate-distortion information (est2) of the generated coding amount (R) and the coding distortion amount (D) are obtained from the quantization unit 15 and the entropy coding unit 23, respectively (step S13).

[0138] Next, the orthogonal transform selection control unit 25 compares the rate-distortion information (est2) to determine which combination (e.g., the combination of performing filtering and DST and the combination of not performing filtering and DCT) is superior in terms of RD cost (step S14).

[0139] When it is determined that performing the filter processing and using an orthogonal transform processing with closed ends of the transform base (e.g., DST) is superior in terms of RD cost (step S14: Yes), the predicted image after the filter processing is performed is output from the filter processing unit 12 to the prediction residual signal generation unit 13, and the orthogonal transform unit 14 is instructed to use an orthogonal transform processing with closed ends of the transform base (e.g., DST) for the block of the prediction residual signal obtained from the prediction residual signal generation unit 13 based on the block of the predicted image after this low-pass filter processing (step S15).

[0140] On the other hand, when it is determined that not performing the filter process and using an orthogonal transform process (e.g., DCT) with the ends of the transform base open is superior in terms of RD cost (step S14: No), a predicted image without the filter process is output from the filter processing unit 12 to the prediction residual signal generation unit 13, and the orthogonal transform unit 14 is instructed to use an orthogonal transform process (e.g., DCT) with the ends of the transform base open for the block of the prediction residual signal obtained from the prediction residual signal generation unit 13 based on the block of the predicted image without the low-pass filter process (step S16).

[0141] In addition, when prediction is performed other than prediction without reference to adjacent pixels, such as intra prediction (step S11: No), the orthogonal transform selection control unit 25 controls the filter processing unit 12 and the orthogonal transform unit 14 to determine whether or not to perform filter processing and to perform the orthogonal transform in accordance with a predetermined method (step S17).

[0142] As described above, in the second embodiment, the orthogonal transform selection control unit 25 determines whether or not to perform filtering and the type of orthogonal transform processing to be applied based on the result selected by RD optimization, and controls the filtering processing unit 12 and the orthogonal transform unit 14. Then, the orthogonal transform selection control unit 25 outputs a transform type identification signal indicating which of the two types of orthogonal transform processing has been applied as one of the coding parameters to the decoding side via the entropy coding unit 23.

[0143] As described above, the orthogonal transform selection control unit 25 determines whether or not to perform filter processing and the type of orthogonal transform processing to be applied based on the result selected by RD optimization, and controls the filter processing unit 12 and the orthogonal transform unit 14. This makes it possible to reduce the residual components in the prediction residual signal and improve the encoding efficiency by reducing the amount of information to be encoded and transmitted.

[0144] In particular, even when making predictions without referring to adjacent pixels, the execution of the filtering process is permitted, and the correlation between adjacent pixel signals is utilized for the block of prediction residual signals obtained based on the block of predicted images after the filtering process. In addition, an orthogonal transform process (e.g., DST) in which the ends of the transform base are closed is used, so that not only the phase but also the characteristics of the transform base corresponding to the end points of the prediction region are matched, thereby enabling a more significant improvement in coding efficiency.

[0145] (Image Decoding Device) Next, the peripheral functional blocks of the filter processing unit 54 and the inverse orthogonal transform selection control unit 60, which are the main components of the image decoding device 5 of this embodiment and which are related to the present invention, will be described with reference to FIG. 16, and a specific typical example of the image decoding device 5 will be described with reference to FIG. 17.

[0146] 16 is a block diagram of the filter processing unit 54 related to a predicted image and the inverse orthogonal transform selection control unit 60 and the surroundings in the image decoding device 5 of this embodiment. As shown in FIG. 16, the image decoding device 5 of this embodiment is configured to include an inverse orthogonal transform unit 52, an adjacent pixel non-reference prediction unit 53, a filter processing unit 54, a decoded image generation unit 55, and an inverse orthogonal transform selection control unit 60.

[0147] The inverse orthogonal transform unit 52, under the control of the inverse orthogonal transform selection control unit 60, receives a stream signal transmitted from the image encoding device 1 side, performs an inverse orthogonal transform process on the transform coefficients restored through the entropy decoding process and the inverse quantization process, and outputs the resulting prediction residual signal to the decoded image generation unit 55.

[0148] The filter processing unit 54 is a functional unit corresponding to the filter processing unit 12 on the image encoding device 1 side, and under the control of the inverse orthogonal transform selection control unit 60, execution and non-execution of low-pass filter processing are controlled, and when the low-pass filter processing is executed, among pixel signals (prediction signals) in a block of a prediction image generated by the adjacent pixel non-reference prediction unit 53, decoded signals (decoded adjacent signals) adjacent to the left and upper sides of the prediction image are used to perform low-pass filter processing on the prediction signal to generate a prediction image made of a new prediction signal, and output the prediction image to the decoded image generation unit 55. When the low-pass filter processing is not executed, the prediction image generated by the adjacent pixel non-reference prediction unit 53 is output to the decoded image generation unit 55. Note that, when making predictions other than adjacent pixel non-reference predictions such as intra prediction, the filter processing unit 54 corresponds to the encoding side, and whether or not to execute filter processing is in accordance with a predetermined method.

[0149] During adjacent pixel non-reference prediction, the inverse orthogonal transform selection control unit 60 refers to a transform type identification signal obtained from the image encoding device 1 side, and when it indicates that an orthogonal transform process with closed ends of the transform base (e.g., DST) has been used for the transform coefficient to be processed, it controls the selection by instructing the inverse orthogonal transform unit 52 to apply an inverse orthogonal transform process with closed ends of the transform base (e.g., IDST) to the restored transform coefficient, and also instructs the filter processing unit 54 to execute the corresponding low-pass filter process.

[0150] On the other hand, when the transform type identification signal obtained from the image encoding device 1 indicates that an orthogonal transform process (e.g., DCT) in which the ends of the transform base are opened has been used, the inverse orthogonal transform selection control unit 60 controls the selection by instructing the inverse orthogonal transform unit 52 to apply an inverse orthogonal transform process (e.g., IDCT) in which the ends of the transform base are opened to the restored transform coefficients, and instructs the filter processing unit 54 not to execute the corresponding low-pass filter process.

[0151] That is, the prediction residual signal generated by the filter processing unit 12 in the image encoding device 1 through the filter processing on the predicted image is transmitted in a state in which the video encoding efficiency is improved through the orthogonal transform processing (e.g., DST) in which the end of the transform base is closed, and the image decoding device 5 can efficiently restore the video related to this transmission. Note that, in the filter processing unit 54, as in the case of the filter processing unit 12, the filter processing may be a low-pass filter processing, and a filter other than a smoothing filter may be applied, and the area and weighting coefficient of the decoded adjacent signal used for the filter processing may be appropriately determined. Then, when the filter type, the filter processing target area, and the weighting coefficient related to the filter processing by the filter processing unit 12 from the image encoding device 1 side are previously determined between transmission and reception (between encoding and decoding) or are transmitted as auxiliary information of the encoding processing, the filter processing unit 54 of the image decoding device 5 also performs the filter processing in accordance with this, thereby improving the accuracy of the decoded signal.

[0152] Next, an inter prediction unit 53a using inter prediction will be described as a representative example of the adjacent pixel non-reference prediction unit 53, and an example configuration of an image decoding device 5 will be described that applies an inverse orthogonal transform selection control unit 60 that selects and controls the inverse orthogonal transform unit 52 and the filter processing unit 54 for the prediction signal by the inter prediction.

[0153] Fig. 17 is a block diagram showing an example of the image decoding device 5 of this embodiment. The image decoding device 5 shown in Fig. 17 includes an entropy decoding unit 50, an inverse quantization unit 51, an inverse orthogonal transformation unit 52, an inter prediction unit 53a, a filter processing unit 54, a decoded image generation unit 55, an in-loop filter unit 56, a frame memory 57, an intra prediction unit 58, a predicted image selection unit 59, and an inverse orthogonal transformation selection control unit 60.

[0154] The inverse orthogonal transform unit 52, under the control of the inverse orthogonal transform selection control unit 60, restores the prediction residual signal by performing inverse orthogonal transform processing corresponding to the orthogonal transform processing of the image encoding device 1 on the transform coefficients obtained from the inverse quantization unit 51, and outputs the restored prediction residual signal to the decoded image generation unit 55.

[0155] The filter processing unit 54 operates in the same manner as that illustrated and described with reference to Fig. 5 under the control of the inverse orthogonal transform selection control unit 60. That is, in the filter processing according to this embodiment in the filter processing unit 54, when the predicted image is a prediction without reference to adjacent pixels, that is, in this example, an inter prediction, the filter processing unit 54 selects a pixel signal (prediction signal) of a predetermined filter processing target region from the predicted image, and generates a predicted image made of a new prediction signal by performing low-pass filter processing on the pixel signal (prediction signal) of the selected filter processing target region using a decoded signal (decoded adjacent signal) adjacent to the left or upper side of the predicted image. For example, as illustrated in FIG. 5 , under the control of the inverse orthogonal transform selection control unit 60, the filter processing unit 54 uses the decoded adjacent signal So around the block boundary of the block Bkp of the predicted image to perform smoothing filter processing with a predetermined weighting coefficient on each of the horizontal filter region, vertical filter region, and corner filter region in the filter processing target region Sf of the pixel signals (prediction signals) within the block Bkp of the predicted image, and outputs the result to the decoded image generation unit 55.

[0156] The operation of the inverse orthogonal transform selection control unit 60 will be described with reference to Fig. 18. Fig. 18 is a flowchart related to the processing of the inverse orthogonal transform selection control unit 60 of one example in the image decoding device 5 of this embodiment. First, the inverse orthogonal transform selection control unit 60 grasps whether or not the current processing state is adjacent pixel non-reference prediction upon receiving a notification from the predicted image selection unit 59 (step S21).

[0157] Next, when non-reference prediction of adjacent pixels is performed (step S21: Yes), the inverse orthogonal transform selection control unit 60 refers to the transform type identification signal obtained from the image encoding device 1 and determines whether an orthogonal transform process in which the ends of the transform base are closed for the transform coefficients (e.g., DST) or an orthogonal transform process in which the ends of the transform base are open for the transform coefficients (e.g., DCT) has been used (step S22).

[0158] Next, when the inverse orthogonal transform selection control unit 60 determines that an orthogonal transform process in which the ends of the transform base are closed (e.g., DST) has been used for the transform coefficients (step S23: Yes), it instructs the filter processing unit 54 to execute the low-pass filter process, and also controls the selection by instructing the inverse orthogonal transform unit 52 to apply an inverse orthogonal transform process in which the ends of the transform base are closed (e.g., IDST) to the transform coefficients restored via the inverse quantization unit 51, and causes the inverse orthogonal transform unit 52 to generate a prediction residual signal to be added to the predicted image obtained via the filter processing unit 54 (step S24).

[0159] On the other hand, when it is determined that an orthogonal transform process (e.g., DCT) in which the ends of the transform base are opened has been used for the transform coefficients (step S23: No), the inverse orthogonal transform selection control unit 60 instructs the filter processing unit 54 not to execute the low-pass filter process, and also controls the selection by instructing the inverse orthogonal transform unit 52 to apply an inverse orthogonal transform process (e.g., IDCT) in which the ends of the transform base are opened to the transform coefficients restored via the inverse quantization unit 51, and causes the inverse orthogonal transform unit 52 to generate a prediction residual signal to be added to the predicted image obtained via the filter processing unit 54 (step S25).

[0160] In addition, when prediction is performed other than prediction without reference to adjacent pixels, such as intra prediction (step S21: No), the inverse orthogonal transform selection control unit 60 controls the inverse orthogonal transform unit 52 and the filter processing unit 54 to determine whether or not to perform filter processing and to perform the orthogonal transform in accordance with a predetermined method (step S26).

[0161] As described above, the inverse orthogonal transform selection control unit 60 refers to the transform type identification signal obtained from the image encoding device 1, thereby determining the type of inverse orthogonal transform processing to be applied and whether or not to perform filter processing, and controls the inverse orthogonal transform unit 52 and the filter processing unit 54. This makes it possible to reduce the residual components in the prediction residual signal on the image encoding device 1 side and perform decoding with a smaller amount of information, thereby improving the encoding efficiency in a series of processes on the encoding side and decoding side.

[0162] In particular, even when prediction is performed without reference to adjacent pixels, in correspondence with the encoding device 1 side, an inverse orthogonal transform process (e.g., IDST) in which the ends of the transform base are closed is used for the block of the prediction residual signal obtained based on the block of the predicted image after the filtering process, so that not only the phase but also the characteristics of the transform base corresponding to the end points of the prediction region are matched, and encoding efficiency can be improved more significantly.

[0163] According to the image encoding device 1 and image decoding device 5 of this embodiment configured as described above, the signal error occurring between the prediction signal of the predicted image and the adjacent decoded block signal is reduced, improving the encoding efficiency, thereby realizing an image encoding device and an image decoding device for a video encoding method with high encoding efficiency.

[0164] [Fourth embodiment] (Image encoding device) Next, a filter processing unit 12 relating to a predicted image, which is a main component of an image encoding device 1 according to a fourth embodiment of the present invention, will be described with reference to Figures 19 and 20, and a specific typical example of the image encoding device 1 will be described with reference to Figures 4, 21 and 22. The following description will focus on the differences from the above-mentioned embodiments. That is, in this embodiment, components similar to those in the above-mentioned embodiments are given the same reference numbers, and further detailed description thereof will be omitted.

[0165] In general, the image encoding device 1 of this embodiment applies, for example, low-pass filter processing to the predicted image used for inter prediction, similarly to intra prediction, by using the encoded and decoded adjacent signals to reduce the predicted residual signal for the leftmost and uppermost regions of the predicted image of inter prediction, and also divides the block of the predicted residual signal into blocks, and applies either DST or DCT according to the directionality of the low-pass filter processing. Note that, in the current H.265 standard, the block size of the predicted image to which DST can be applied is limited to only small ones (e.g., 4×4). Therefore, in the image encoding device 1 of this embodiment, in inter prediction of a block of a larger predicted image, the low-pass filter processing is applied to this predicted image, and then block division is performed up to a predetermined specified block size (e.g., a block size to which the application of DST is permitted in the standard), and then it is configured to determine whether to apply DST or DCT without a flag, and execute the corresponding orthogonal transform processing.

[0166] FIG. 19(a) is a block diagram of the filter processing unit 12 and its surroundings for the predicted image in the image encoding device 1 of this embodiment, and FIG. 19(b) is an explanatory diagram showing an example of filtering of the predicted image by the filter processing unit 12.

[0167] 19(a), the image encoding device 1 of this embodiment is configured to include a neighboring pixel non-reference prediction unit 11, a filter processing unit 12, a prediction residual signal generation unit 13, and an orthogonal transformation unit 14. The orthogonal transformation unit 14 includes a block division unit 141 and an orthogonal transformation selection application unit 142.

[0168] The filter processing unit 12 in this embodiment is a functional unit that generates a prediction image made up of a new prediction signal by performing low-pass filter processing on the prediction signal using decoded signals (decoded adjacent signals) adjacent to the left and above the prediction image, out of the pixel signals (prediction signal) within a block of the prediction image generated by the adjacent pixel non-reference prediction unit 11, and outputs the prediction image to the prediction residual signal generation unit 13.

[0169] For example, as shown in FIG. 19(b), assuming that a block Bkp of a predicted image consisting of an 8×8 pixel signal p corresponding to a block Bko of the encoding target area of ​​the original image has been generated by the adjacent pixel non-reference prediction unit 11, the filter processing unit 12 uses the decoded adjacent signal So around the block boundary of the block Bkp of the predicted image that is adjacent to the predicted image to perform low-pass filtering using smoothing filters (indicated by double arrows in the figure) such as a horizontal filter, vertical filter, and angular filter as the signal Sf of the filtering target area to generate a predicted image consisting of a new predicted signal.

[0170] More specifically, in the example shown in FIG. 19(b), for the smoothing filter target pixel on the upper side of the predicted image block Bkp, smoothing filter processing is performed using the decoded adjacent signal (pixel signal of the local decoded image) located at the same coordinate in the horizontal direction, and for the smoothing filter target pixel on the left side, smoothing filter processing is performed using the decoded adjacent signal (pixel signal of the local decoded image) located at the same coordinate in the vertical direction. For the target pixel located on the upper side and the left side, smoothing filter processing is performed using the pixels of the upper side and the left side local decoded image. In the example shown in FIG. 19(b), a 3-tap smoothing filter processing using two reference pixels is applied. For example, a low-pass filter processing such as 1 / 4

[0121] is applied. In addition, the number of taps and reference pixels of the low-pass filter processing are not limited to this example.

[0171] The orthogonal transform unit 14 includes a block division unit 141 and an orthogonal transform selection application unit 142. The block division unit 141 divides the blocks of the prediction residual signal input from the prediction residual signal generation unit 13 into a pre-specified block shape and outputs the blocks to the orthogonal transform selection application unit 142. The orthogonal transform selection application unit 142 selects and applies, for each block of the block-divided prediction residual signal, a plurality of types of orthogonal transform processing (a combination of vertical / horizontal DST and vertical / horizontal DCT) according to the position to which the filter processing by the filter processing unit 12 was applied.

[0172] For example, as shown in FIG. 19(b), the block division unit 141 divides a block of a prediction residual signal for a block Bkp of a prediction image consisting of an 8×8 pixel signal p into four blocks of 4×4 prediction residual signals (upper left, upper right, lower left, and lower right), and outputs the blocks to the orthogonal transform selection application unit 142. At this time, the orthogonal transform selection application unit 142 applies a plurality of types of orthogonal transform processing (a combination of vertical / horizontal DST and vertical / horizontal DCT) according to the position where the smoothing filter processing is applied to the divided blocks of the prediction residual signal located at the upper end and left end including the pixel position where the smoothing filter processing is applied, among the divided blocks divided by the block division unit 141, so that the orthogonal transform processing can be handled by utilizing the high correlation with the block of the decoded adjacent signal, and applies the orthogonal transform processing of vertical and horizontal DCT to the other divided blocks. Note that the vertical direction means the vertical direction, and the horizontal direction means the horizontal direction. Here, the orthogonal transform processing consisting of vertical DST and horizontal DCT is referred to as the first orthogonal transform processing, the orthogonal transform processing consisting of horizontal DST and vertical DCT is referred to as the second orthogonal transform processing, the orthogonal transform processing consisting of vertical and horizontal DST is referred to as the third orthogonal transform processing, and the orthogonal transform processing consisting of vertical and horizontal DCT is referred to as the fourth orthogonal transform processing.

[0173] In the example shown in FIG. 19(b), the block of the prediction residual signal for the block Bkp of the prediction image consisting of 8×8 pixel signals p is divided into four blocks (upper left, upper right, lower left and lower right) of the prediction residual signal of 4×4, but for the block of the prediction residual signal corresponding to the block of the prediction image having a block size larger than the predetermined block size (4×4 in this example) doubled vertically and horizontally (8×8 in this example), it is preferable to divide the divided blocks adjacent to the decoded adjacent signal (divided blocks located at the top and left ends in this example) to the specified block size (4×4 in this example) and the other blocks to the largest possible block size, and then appropriately perform re-division according to the characteristics of the image to be coded. For example, according to the characteristics of the image to be coded, as shown in FIG. 20(a), the divided blocks located at the top and left ends of the block of the prediction residual signal with a block size of 32×32 can be divided to 4×4, and the divided blocks with the block size can be enlarged as they move away from the top and left ends. By dividing the blocks in this manner, the subsequent orthogonal transform selection and application unit 142 can select and apply an orthogonal transform suitable for the signal characteristics of the divided block of the prediction residual signal that includes the pixel position to be processed by the smoothing filter.

[0174] For example, as shown in FIG. 20(a), when the pixel to which the smoothing filter process is applied is at the top of the divided blocks of the divided prediction residual signal (i.e., when the application direction of the smoothing filter process is vertical), the orthogonal transform selection application unit 142 applies the first orthogonal transform process of vertical DST and horizontal DCT to the divided block Bkt1 of the prediction residual signal located at the top (see FIG. 20(b)), and when the pixel to which the smoothing filter process is applied is at the leftmost side (i.e., when the application direction of the smoothing filter process is horizontal), the orthogonal transform selection application unit 142 applies the second orthogonal transform process of horizontal DST and vertical DCT to the divided block Bkt2 of the prediction residual signal located at the leftmost side (see FIG. 20(c)). Then, when the pixel to which the smoothing filter process is applied is at the top and at the leftmost side (i.e., when the application direction of the smoothing filter process is the application direction of the corner filter process), the orthogonal transform selection application unit 142 applies the third orthogonal transform process of vertical and horizontal DST to the divided block Bkt3 located in the corner region. For the divided block Bkt4 of the prediction residual signal where there is no pixel to which the smoothing filter process has been applied, the fourth orthogonal transform process of DCT is applied in the vertical and horizontal directions. The orthogonal transform process is also performed in the same manner when the block is divided as shown in Fig. 19(b). This makes it possible to handle transform coefficients that make maximum use of the high correlation with the blocks of the decoded adjacent signals.

[0175] Next, an inter-prediction unit 11a using inter-prediction will be described as a representative example of the adjacent pixel non-reference prediction unit 11, and a configuration and operation example of an image encoding device 1 that applies filtering to a prediction signal of a block of the predicted image by a filtering processing unit 12 will be described with reference to Figures 21 to 23.

[0176] Fig. 21 is a block diagram showing an example of the image coding device 1 of this embodiment. The image coding device 1 shown in Fig. 21 includes a preprocessing unit 10, an inter prediction unit 11a, a filter processing unit 12, a prediction residual signal generating unit 13, an orthogonal transformation unit 14, a quantization unit 15, an inverse quantization unit 16, an inverse orthogonal transformation unit 17, a decoded image generating unit 18, an in-loop filter unit 19, a frame memory 20, an intra prediction unit 21, a motion vector calculation unit 22, an entropy coding unit 23, and a predicted image selecting unit 24.

[0177] The orthogonal transform unit 14 performs a predetermined orthogonal transform process on the prediction residual signal input from the prediction residual signal generation unit 13, and generates a signal of the transform coefficient. In particular, the orthogonal transform unit 14 includes a block division unit 141 and an orthogonal transform selection application unit 142. The block division unit 141 divides the blocks of the prediction residual signal input from the prediction residual signal generation unit 13 into a block shape designated in advance and outputs the blocks to the orthogonal transform selection application unit 142. The orthogonal transform selection application unit 142 applies a plurality of types of orthogonal transform processes (a combination of vertical / horizontal DST and vertical / horizontal DCT) to each divided block of the block-divided prediction residual signal according to the position where the filter process by the filter processing unit 12 was applied. Block division parameters related to these block division processes are output to the entropy coding unit 23 (and the inverse orthogonal transform unit 17) as one of the coding parameters that are supplementary information of the coding process.

[0178] The inverse orthogonal transform unit 17 restores a prediction residual signal by performing inverse orthogonal transform processing on the signal of the transform coefficients after inverse quantization obtained from the inverse quantization unit 16, and outputs the restored prediction residual signal to the decoded image generation unit 18. In particular, the inverse orthogonal transform unit 17 determines the type of orthogonal transform processing for the transform coefficients divided into blocks, as in the case of FIG. 20, performs inverse orthogonal transform processing corresponding to each orthogonal transform processing, combines each divided block of the prediction residual signal obtained so as to be able to be processed in parallel, and reconstructs a block corresponding to the block size of the prediction image, and outputs the reconstructed block to the decoded image generation unit 18.

[0179] An example of the operation of the filter processing unit 12 will be described with reference to Fig. 22. As shown in Fig. 22, when a predicted image is input from the intra prediction unit 21 or the inter prediction unit 11a (step S1), the filter processing unit 12 identifies whether or not the predicted image is a neighboring pixel non-reference prediction, that is, in this example, whether or not the predicted image is an inter prediction based on a signal selection by the predicted image selection unit 24 (step S2). The same applies when the neighboring pixel non-reference prediction is the above-mentioned intra block copy prediction or cross component signal prediction.

[0180] Next, when the filter processing unit 12 determines that the predicted image is not adjacent pixel non-reference prediction, that is, in this example, intra prediction (step S2: No), it determines whether or not to perform filter processing by a predetermined processing method (step S5). When filter processing is not performed on the predicted image of intra prediction (step S5: No), the filter processing unit 12 outputs the predicted signal to the prediction residual signal generating unit 13 without performing filter processing on the predicted signal (step S6). On the other hand, when filter processing is performed on the predicted image of intra prediction (step S5: Yes), the filter processing unit 12 proceeds to step S3. Note that the filter processing on the predicted image of intra prediction can be performed in the same manner as the currently specified H.265, and is not directly related to the gist of the present invention, so the explanation of step S3 will be an example of performing filter processing on the predicted image of inter prediction.

[0181] When the filter processing unit 12 determines that the predicted image is a prediction without reference to adjacent pixels, that is, in this example, an inter prediction (step S2: Yes), it selects pixel signals (prediction signals) of a predetermined filter processing target area from the predicted image (step S3). For example, in the example shown in FIG. 19(b), among the pixel signals (prediction signals) in a block of an 8×8 predicted image, the leftmost and uppermost pixel signals (prediction signals) of the predicted image are determined as a filter processing target area Sf.

[0182] Next, the filter processing unit 12 generates a prediction image consisting of a new prediction signal by performing low-pass filter processing on the pixel signal (prediction signal) of the selected filter processing target area using decoded signals (decoded adjacent signals) adjacent to the left and above the prediction image, and outputs the prediction image to the prediction residual signal generation unit 13 (step S4).

[0183] The filtering process by the filtering unit 12 may be a low-pass filtering process, and filtering other than a smoothing filter may be applied, and the area of ​​the decoded adjacent signal and the weighting coefficient used for the filtering process do not need to be limited to the example shown in Fig. 19(b). The filter type, filtering target area, and weighting coefficient related to the filtering process by the filtering unit 12 may be configured to be transmitted as auxiliary information for the encoding process, but they do not need to be transmitted if they are determined in advance between the transmitter and the receiver (between encoding and decoding).

[0184] By performing filtering on the predicted image by the filter processing unit 12 in this manner, regardless of the type of orthogonal transform processing performed by the orthogonal transform unit 14, the signal error that occurs between the leftmost and topmost pixel signals (prediction signals) in the predicted image and the adjacent decoded signals is reduced, thereby improving the video encoding efficiency.

[0185] Next, an example of the operation of the orthogonal transform unit 14 will be described with reference to Fig. 23. As shown in Fig. 23, the orthogonal transform unit 14 divides the blocks of the prediction residual signal input from the prediction residual signal generating unit 13 into a block shape specified in advance by the block dividing unit 141 (step S11). At this time, for a block of a prediction image having a block size larger than 8x8 (for example, a prediction image having a block size of 32x32), the block dividing unit 141 divides the divided blocks of the prediction residual signal located at the upper end and the left end into a specified block size (4x4 in this example) and the other blocks into the largest possible block size, and then appropriately divides them again according to the characteristics of the image to be coded.

[0186] Next, when the orthogonal transform unit 14 performs orthogonal transform processing on the divided block of the divided prediction residual signal using the orthogonal transform selection application unit 142, it determines whether or not the divided block includes a pixel position to which filtering was applied by the filter processing unit 12 (i.e., whether or not the prediction residual signal is of a block including the topmost or leftmost pixel position of the predicted image) (step S12).If it is determined that the divided block does not include the pixel position to which filtering was applied (step S12: No), it applies a fourth orthogonal transform processing of DCT in the vertical and horizontal directions to the divided block of the prediction residual signal (step S15).

[0187] On the other hand, when it is determined that the divided block includes a pixel position to which filtering was applied by the filtering processing unit 12 (step S12: Yes), it is determined whether only vertical filtering was applied by the filtering processing unit 12 (i.e., whether the prediction residual signal is of a block including only the topmost pixel of the predicted image) (step S13), and if only vertical filtering was applied (step S13: Yes), a first orthogonal transform process of vertical DST and horizontal DCT is applied to the divided block of the prediction residual signal (step S16).

[0188] Next, if the vertical filtering process has not been applied alone (step S12: No), it is determined whether the filtering process by the filter processing unit 12 has been applied alone (i.e., whether the prediction residual signal is of a block including only the leftmost pixels of the predicted image) (step S14), and if the horizontal filtering process has been applied alone (step S14: Yes), a second orthogonal transform process of horizontal DST and vertical DCT is applied to the divided block of the prediction residual signal (step S17).

[0189] Then, when it is determined that horizontal and vertical filtering has been applied by the filter processing unit 12 (i.e., when the prediction residual signal is of a block that includes the topmost and leftmost pixel position of the predicted image) (step S14: No), a third orthogonal transform process in the vertical and horizontal directions DST is applied to the divided block of the prediction residual signal (step S18).

[0190] In Fig. 23, the orthogonal transform process is described as being performed sequentially in order to facilitate understanding of the process, but in practice it can be performed in parallel, which makes it possible to handle transform coefficients that make maximum use of the high correlation between blocks of decoded adjacent signals.

[0191] (Image Decoding Device) Next, the filter processing unit 54 and its peripheral functional blocks, which are the main components of the image decoding device 5 of this embodiment and which are related to the present invention, will be described with reference to FIG. 24, and a specific typical example of the image decoding device 5 will be described with reference to FIG. 25.

[0192] Fig. 24 is a block diagram of a filter processing unit 54 related to a predicted image and its surroundings in the image decoding device 5 of this embodiment. As shown in Fig. 24, the image decoding device 5 of this embodiment is configured to include an inverse orthogonal transform unit 52, a neighboring pixel non-reference prediction unit 53, a filter processing unit 54, and a decoded image generation unit 55. The inverse orthogonal transform unit 52 includes an inverse orthogonal transform selection application unit 521 and a block reconstruction unit 522.

[0193] The inverse orthogonal transform unit 52 receives a stream signal transmitted from the image encoding device 1 side, performs inverse orthogonal transform processing on the transform coefficients restored through the entropy decoding processing and the inverse quantization processing, and outputs the obtained prediction residual signal to the decoded image generation unit 55. In particular, the inverse orthogonal transform unit 52 includes an inverse orthogonal transform selection application unit 521 and a block reconstruction unit 522. The inverse orthogonal transform selection application unit 521 determines the type of orthogonal transform processing on the transform coefficients divided into blocks by the image encoding device 1 side in the same manner as in FIG. 23, performs inverse orthogonal transform processing corresponding to the orthogonal transform processing in the image encoding device 1, and outputs the divided blocks of the prediction residual signal to the block reconstruction unit 522. The block reconstruction unit 522 combines the divided blocks of the prediction residual signal obtained so as to be able to be processed in parallel, reconstructs a block corresponding to the block size of the prediction image, and outputs the reconstructed block to the decoded image generation unit 55.

[0194] Next, an inter-prediction unit 53a using inter-prediction will be described as a representative example of the adjacent pixel non-reference prediction unit 53, and a configuration and operation example of an image decoding device 5 that applies filtering to a prediction signal by the inter-prediction using a filtering processing unit 54 will be described with reference to FIG. 25.

[0195] Fig. 25 is a block diagram showing an example of the image decoding device 5 of this embodiment. The image decoding device 5 shown in Fig. 25 includes an entropy decoding unit 50, an inverse quantization unit 51, an inverse orthogonal transformation unit 52, an inter prediction unit 53a, a filter processing unit 54, a decoded image generation unit 55, an in-loop filter unit 56, a frame memory 57, an intra prediction unit 58, and a predicted image selection unit 59.

[0196] The inverse orthogonal transform unit 52 restores a prediction residual signal by performing inverse orthogonal transform processing corresponding to the orthogonal transform processing of the image encoding device 1 on the transform coefficients obtained from the inverse quantization unit 51, and outputs the restored prediction residual signal to the decoded image generation unit 55. In particular, the inverse orthogonal transform unit 52 includes an inverse orthogonal transform selection application unit 521 and a block reconstruction unit 522. The inverse orthogonal transform selection application unit 521 performs inverse orthogonal transform processing on the transform coefficients divided into blocks by the image encoding device 1 side, and outputs divided blocks of the prediction residual signal to the block reconstruction unit 522. The block reconstruction unit 522 combines the divided blocks of the prediction residual signal obtained so as to be able to be processed in parallel, to form a block corresponding to the block size of the prediction image, and outputs the block to the decoded image generation unit 55.

[0197] The filter processing unit 54 operates in the same manner as that illustrated and described with reference to Fig. 22. That is, in the filter processing according to the present invention in the filter processing unit 54, when the prediction image is prediction without reference to adjacent pixels, that is, in this example, inter prediction, a pixel signal (prediction signal) of a predetermined filter processing target area is selected from the prediction image, and a low-pass filter process is performed on the pixel signal (prediction signal) of the selected filter processing target area using a decoded signal (decoded adjacent signal) adjacent to the left or upper side of the prediction image, thereby generating a prediction image made of a new prediction signal, and outputting the prediction image to the decoded image generating unit 55. For example, as illustrated in Fig. 19(b), the filter processing unit 54 performs smoothing filter processing of a predetermined weighting factor on each of the horizontal filter area, vertical filter area, and corner filter area in the filter processing target area Sf of the pixel signal (prediction signal) in the block Bkp of the prediction image, using the decoded adjacent signal So around the block boundary of the block Bkp of the prediction image, and outputs the result to the decoded image generating unit 55.

[0198] According to the image encoding device 1 and image decoding device 5 of the present embodiment configured as above, the signal error occurring between the prediction signal of the predicted image and the adjacent decoded block signal is reduced, improving the encoding efficiency, so that it is possible to realize an image encoding device and an image decoding device of a video encoding method with high encoding efficiency. That is, it is possible to reduce the residual component in the prediction residual signal, and the encoding efficiency can be improved by reducing the amount of information to be encoded and transmitted.

[0199] Fifth embodiment Next, the configuration of the filter processing units 12, 54 and the surroundings in the image encoding device 1 and image decoding device 5 according to the fifth embodiment of the present invention will be described with reference to Fig. 26. The filter processing units 12, 54 of the fourth embodiment described above have been described as generating a predicted image made of a new predicted signal by performing low-pass filter processing on the predicted signal using only "decoded signals (decoded adjacent signals)" adjacent to the left and above the predicted image, out of pixel signals (predicted signals) in a block of a predicted image generated by adjacent pixel non-reference prediction.

[0200] On the other hand, the filter processing units 12 and 54 of this embodiment are configured to obtain a local decoded image for each divided block based on the block division processing of the block division unit 141, and each time this local decoded image is obtained, replace the pixel signals (prediction signals) within the block of the predicted image generated by adjacent pixel non-reference prediction, and perform low-pass filter processing according to each block size, in block size units of the divided blocks of the prediction residual signal, using decoded signals (decoded adjacent signals) adjacent to the left and above the predicted image, and the local decoded image for each divided block, among the pixel signals (prediction signals) within the block of the replaced predicted image.

[0201] 26(a) and (b), the filter processing units 12 and 54 are configured to obtain a local decoded image for each divided block based on the block division processing of the block division unit 141, and each time they obtain this local decoded image, they replace the pixel signal (prediction signal) in the block of the predicted image generated by adjacent pixel non-reference prediction to update the predicted image, and perform low-pass filter processing according to each block size on this updated predicted image using the decoded signal (decoded adjacent signal) adjacent to the left and above and the local decoded image for each divided block, which differs from the configurations shown in FIG. 19(a) and FIG. 24, respectively, but the operation of the other components is similar.

[0202] As an example of the operation, the block reconstruction unit 522 in the inverse orthogonal transform unit 52 shown in FIG. 26(b) is different from the configuration shown in FIG. 24 in that, when determining the position of the division block to be processed and reconstructing it to a block size corresponding to the predicted image, the area other than the division block to be processed is complemented with dummy data. This makes it possible to configure the unit so that a local decoded image for each division block can be obtained. Alternatively, a separate loop may be configured so that a local decoded image for each division block can be obtained. Therefore, the inverse orthogonal transform unit 17 in the image encoding device 1 of this embodiment may be configured in the same way as the inverse orthogonal transform unit 52 of this embodiment. Note that, similarly to the fourth embodiment, the frame memory 20 can hold the local decoded image for each division block so that it can be used as a reference signal.

[0203] Fig. 27 is a flowchart relating to the filter processing 12, 54 of the predicted image in the image encoding device 1 or the image decoding device 5 of this embodiment. Also, Fig. 28 is an explanatory diagram relating to the filter processing 12, 54 of the predicted image in the image encoding device 1 or the image decoding device 5 of this embodiment.

[0204] First, when the filter processing unit 12, 54 receives a predicted image, it acquires a local decoded image for each divided block based on the block division process of the block division unit 141, and generates a predicted image by replacing and updating the predicted image (step S21). For example, as illustrated in Fig. 28, a block Bkp of an 8x8 predicted image is successively replaced and updated by a block BkL of a locally decoded image of a block size divided into four in the order shown by the white arrows, and the following process shows the operation when the predicted image is updated by a locally decoded image for each divided block (for example, "STEP 1" in the figure).

[0205] Next, the filter processing unit 12, 54 identifies whether the predicted image is a prediction without reference to neighboring pixels, for example, an inter prediction (step S22). Note that the same applies when the prediction without reference to neighboring pixels is the above-mentioned intra block copy prediction or cross component signal prediction.

[0206] Next, when the filter processing unit 12, 54 determines that the predicted image is not adjacent pixel non-reference prediction, for example, intra prediction (step S22: No), it determines whether or not to perform filter processing by a predetermined processing method (step S25). When filter processing is not performed on the predicted image of intra prediction (step S25: No), the filter processing unit 12, 54 outputs the predicted signal to the prediction residual signal generating unit 13 or the decoded image generating unit 55 without performing filter processing on the predicted signal (step S26). On the other hand, when filter processing is performed on the predicted image of intra prediction (step S25: Yes), the filter processing unit 12, 54 proceeds to step S23. Note that the filter processing on the predicted image of intra prediction may be performed in the same manner as the currently specified H.265, and is not directly related to the gist of the present invention, so the explanation of step S23 will be an example of performing filter processing on the predicted image of inter prediction.

[0207] When the filter processing unit 12, 54 determines that the prediction image is a prediction without reference to adjacent pixels, that is, in this example, an inter prediction (step S22: Yes), it selects pixel signals (prediction signals) of a predetermined filter processing target area from the prediction image (step S23). For example, as illustrated in Fig. 28, in this example, in units of 4x4 divided blocks, the leftmost and uppermost pixel signals (prediction signals) are determined as the filter processing target area Sf.

[0208] Next, the filter processing units 12 and 54 use the decoded signals (decoded adjacent signals) adjacent to the left and / or above for each divided block and the local decoded images to perform low-pass filter processing on the pixel signals (prediction signals) of the selected filter processing target area to generate a prediction image consisting of a new prediction signal, and output the prediction image to the prediction residual signal generation unit 13 and the decoded image generation unit 55 (step S24).

[0209] In this way, by applying low-pass filter processing using the decoded signal (decoded adjacent signal) adjacent to the left and / or above of the predicted image and the local decoded image adjacent to the left and / or above for each divided block, the pixel signals of an 8x8 predicted image block Bkp are smoothed not only to the top and left, but also at the boundary positions of the divided blocks, which in this example form a cross, as illustrated in Figure 28 (for example, "STEP 2" shown).

[0210] By configuring the filter processing units 12 and 54 as in this embodiment and applying them to the image encoding device 1 and the image decoding device 5, respectively, it is possible to improve the encoding efficiency while suppressing deterioration in image quality caused thereby. In particular, even for signals of divided blocks (blocks located inside the predicted image when the predicted image is divided into blocks) other than the divided blocks located on the leftmost or topmost sides of the predicted image that may have a large value of the prediction residual signal, the values ​​of the leftmost and topmost prediction residual signals for each divided block can be reduced.

[0211] In this embodiment, all divided blocks include pixel positions to be filtered, so that DST in the vertical or horizontal direction is always applied. For this reason, in this embodiment, as in the fourth embodiment, the block shape when dividing the blocks is preferably such that the divided blocks located at the top and left ends of the corresponding prediction residual signal block have a prescribed block size (4×4 in this example), and the other blocks are divided to have the largest possible block size, and then appropriately re-divided according to the characteristics of the image to be encoded. There is no need to divide the blocks to have a larger block size as they move away from the top and left ends, and all are divided to a prescribed size (4×4 in this example) to which DST can be applied, and DST can be applied according to the processing order shown in FIG.

[0212] Furthermore, comparing the present embodiment with the above-mentioned fourth embodiment, the fourth embodiment has the advantage that each divided block of the prediction residual signal can be processed independently, enabling parallel processing, whereas the present embodiment can sequentially use locally decoded images in divided block units, which is expected to result in greater improvements in coding efficiency.

[0213] In addition, since the block shape for block division can be variably set for each block to be coded and the block division parameter can be transmitted as one of the coding parameters, an embodiment can also be made by combining the fourth embodiment and this embodiment.

[0214] The image encoding device 1 and the image decoding device 5 of each embodiment can be caused to function as a computer, and a program for causing the computer to realize each component of the present invention is stored in a memory provided inside or outside the computer. Under the control of a central processing unit (CPU) or the like provided in the computer, a program describing the processing contents for realizing the function of each component can be read from the memory as appropriate to cause the computer to realize the function of each component of the image encoding device 1 and the image decoding device 5 of each embodiment. Here, the function of each component may be realized by a part of hardware.

[0215] Although the present invention has been described above by taking specific examples of embodiments, the present invention is not limited to the above examples and can be modified in various ways without departing from the technical concept thereof. For example, in the above-mentioned embodiments, the block sizes of the coding target area, the predicted image, and the orthogonal transform processing target are the same, but the present invention can be similarly applied to a case where the block size of the predicted image is smaller than that of the coding target area, or the block size of the orthogonal transform processing target is smaller than that of the predicted image. In addition, the orthogonal transform processing can be directly used in the coding processing depending on the application, or can be a different one.

[0216] In addition, in the examples of each of the above-mentioned embodiments, the image decoding device 5 decodes transform coefficients related to a prediction residual signal encoded based on a prediction image that has been subjected to filtering by the corresponding image encoding device 1. However, if the application does not require the accuracy of restoring the block ends relative to the original image, the image decoding device 5 according to the present invention can also decode signals that have been encoded without filtering using the same processing. [Industrial Applicability]

[0217] INDUSTRIAL APPLICABILITY According to the present invention, it is possible to realize an image encoding device and an image decoding device for a video encoding method with high encoding efficiency, which is useful for applications in which it is desired to improve the encoding efficiency of video transmission. [Explanation of symbols]

[0218] 1 Image encoding device 5. Image Decoding Device 10 Pretreatment section 11 Neighboring pixel non-reference prediction unit 11a Inter prediction section 12 Filter processing section 13 Prediction residual signal generator 14 Orthogonal transformation section 15 Quantization section 16 Inverse quantization section 17 Inverse orthogonal transform section 18 Decoded image generation unit 19 In-loop filter section 20 Frame Memory 21 Intra prediction unit 22 Motion Vector Calculation Unit 23 Entropy coding unit 24 Prediction image selection unit 25 Orthogonal transform selection control section 50 Entropy Decoding Unit 51 Inverse quantization section 52 Inverse orthogonal transform unit 53 Neighboring pixel non-reference prediction unit 53a Inter prediction section 54 Filter processing section 55 Decoded image generation unit 56 In-loop filter section 57 Frame Memory 58 Intra Prediction Unit 59 Prediction image selection unit 60 Inverse orthogonal transform selection control section 101 Horizontal correlation determination unit 102 Vertical correlation determination unit 103 Filter processing decision unit 104 Filter processing execution unit 141 Block division section 142 Orthogonal transform selection application unit 521 Inverse orthogonal transform selection application unit 522 Block Reconstruction Unit

Claims

1. 1. An image decoding device that decodes a signal that is coded by dividing a frame constituting a moving image into blocks, comprising: a prediction unit that performs signal prediction on each pixel signal in a block unit to generate a predicted image in a block unit including a predicted signal; an acquisition unit for acquiring a control identification signal indicating a type of selection control selected by an encoding side; a filtering unit that generates a new block of the predicted image by filtering the predicted signal using decoded neighboring signals that are neighboring the block of the predicted image based on the control identification signal; an inverse quantization unit that performs a corresponding inverse quantization process on the quantized transform coefficients without a flag indicating the type of transform process selected on the encoding side to restore the transform coefficient signal; an inverse transform unit that performs an inverse transform process on the transform coefficient signal, the inverse transform process being determined based on the control identification signal, to reconstruct a prediction residual signal on a block-by-block basis; Image decoding device.

2. 1. An image decoding method for decoding a signal that has been coded by dividing a frame constituting a moving image into blocks, comprising the steps of: generating a block-by-block predicted image including a predicted signal by performing signal prediction on each pixel signal of the block; obtaining a control identification signal indicating a type of selection control selected by an encoding side; generating a new block of the predicted image by filtering a prediction signal located at a boundary of the block of the predicted image using a decoded neighboring signal adjacent to the block of the predicted image; a step of performing a corresponding inverse quantization process on the quantized transform coefficients without a flag indicating the type of transform process selected on the encoding side to restore the transform coefficient signal; and performing an inverse transform process on the transform coefficient signal, the inverse transform process being determined based on the control identification signal, thereby reconstructing a prediction residual signal on a block-by-block basis. Image decoding method.

Images