Video encoding device and video decoding device

Abstract

To provide a video encoding device that performs motion compensation predictive encoding on a video.SOLUTION: A video encoding device includes an acquisition section / selection unit that determines the available blocks that are blocks with motion vectors and the number of the available blocks from coded blocks adjacent to a block to be coded, and selects one selection block from the coded blocks and the available blocks, and a variable length encoder that includes a selection information encoder that encodes selection information specifying a selection block using a code table according to the number of available blocks, and performs motion compensation predictive encoding on the block to be coded using the motion vector of the selected block.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a video encoding device and a video decoding device that calculates motion vectors from encoded and decoded images and performs motion compensation prediction. [Background technology]

[0002] One of the techniques used in video coding is motion compensation prediction, in which a motion vector is calculated using a target image to be newly coded in a video coding device and a locally decoded image already obtained, and a predicted image is generated by performing motion compensation using this motion vector.

[0003] One method of calculating a motion vector in motion compensation prediction is a direct mode in which a predicted image is generated using a motion vector of a block to be coded, which is derived from a motion vector of a block that has already been coded (see Japanese Patent No. 4020789 and U.S. Patent No. 7233621). In the direct mode, the motion vector is not coded, so the amount of coding of the motion vector information can be reduced. The direct mode is adopted in H.264 / AVC. Summary of the Invention

[0004] In the direct mode, when predicting and generating a motion vector for a block to be coded, the motion vector is generated by a fixed method of calculating the motion vector from the median value of the motion vectors of coded blocks adjacent to the block to be coded. Therefore, the degree of freedom in calculating the motion vector is low. In addition, when a motion vector calculation method for selecting one from multiple coded blocks is used to increase the degree of freedom, the position of the selected coded block must always be sent as motion vector selection information. This leads to an increase in the amount of code.

[0005] An object of the present invention is to provide a video encoding device and a video decoding device that select one of the encoded blocks to increase the degree of freedom in calculating a motion vector, while reducing additional information of the motion vector selection information.

[0006] One aspect of the present invention provides a video coding device that performs motion compensation predictive coding of video, and includes an acquisition unit that obtains available blocks, which are blocks having motion vectors, and the number of the available blocks from already-coded blocks adjacent to a block to be coded, a selection unit that selects one selected block from the already-coded blocks and the available blocks, a selection information coding unit that codes selection information that identifies the selected block using a code table corresponding to the number of available blocks, and an image coding unit that performs motion compensation predictive coding of the block to be coded using the motion vector of the selected block.

[0007] Another aspect of the present invention provides a video decoding device that performs motion compensation prediction decoding of video, the video decoding device comprising: a selection information decoding unit that switches code tables depending on the number of available blocks that are adjacent to a block to be decoded and have motion vectors, and decodes selection information; a motion vector selection unit that selects one motion vector from the available blocks indicated by the selection information decoded by the selection information decoding unit; and an image decoding unit that performs motion compensation prediction decoding of an image to be decoded using the motion vector selected by the motion vector selection unit. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram of a video encoding device according to an embodiment of the present invention; [Diagram 2] Flowchart showing the processing procedure of the video encoding device [Diagram 3] Flowchart showing the processing procedure of the acquisition unit / selection unit [Figure 4A] A diagram explaining the method of discrimination based on block size [Figure 4B]A diagram explaining the method of discrimination based on block size [Figure 4C] A diagram explaining the method of discrimination based on block size [Diagram 5] FIG. 1 is a diagram for explaining a method of determining whether unidirectional or bidirectional prediction is used. [Figure 6] Flowchart showing the processing procedure of the selection information encoding unit [Figure 7] An example of an index of selected information [Figure 8] An example of a code table for selection information [Figure 9] Syntax structure overview [Figure 10] Macroblock Layer Data Structure [Figure 11] 1 is a block diagram of a video decoding device according to an embodiment of the present invention; [Figure 12] Flowchart showing the processing procedure of the video decoding device DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] A video coding device according to an embodiment will be described with reference to Fig. 1. A subtractor 101 is configured to calculate a difference between an input video signal 11 and a predicted image signal 15, and output a prediction error signal 12. An output terminal of the subtractor 101 is connected to a variable-length coder 111 via an orthogonal transformer 102 and a quantizer 103. The orthogonal transformer 102 performs orthogonal transform on the prediction error signal 12 from the subtractor 101 to generate orthogonal transform coefficients, and the quantizer 103 quantizes the orthogonal transform coefficients and outputs quantized orthogonal transform coefficient information 13. The variable-length coder 111 performs variable-length coding on the quantized orthogonal transform coefficient information 13 from the quantizer 103.

[0011] The output terminal of the quantizer 103 is connected to an adder 106 via an inverse quantizer 104 and an inverse orthogonal transformer 105. The inverse quantizer 104 inverse quantizes the quantized orthogonal transform coefficient information 13 and converts it into orthogonal transform coefficients. The inverse orthogonal transformer 105 converts the orthogonal transform coefficients into a prediction error signal. The adder 106 adds the prediction error signal of the inverse orthogonal transformer 105 and a predicted image signal 15 to generate a locally decoded image signal 14. The output terminal of the adder 105 is connected to a motion compensation predictor 108 via a frame memory 107.

[0012] The frame memory 107 stores the local decoded image signal 14. The setting unit 114 sets a motion compensation prediction mode (prediction mode) of the block to be coded. The prediction mode includes unidirectional prediction using one reference image and bidirectional prediction using two reference images. The unidirectional prediction includes L0 prediction and L1 prediction of AVC. The motion compensation predictor 108 includes a predictor 109 and an acquisition unit / selection unit 110. The acquisition unit / selection unit 110 obtains available blocks, which are blocks having a motion vector, and the number of the available blocks from the coded blocks adjacent to the block to be coded, and selects one selected block from the available blocks. The motion compensation predictor 108 generates a predicted image signal 15 from the local decoded image signal 14 and the input video signal 11 from the frame memory 107. The acquisition unit / selection unit 110 selects one block (selected block) from the adjacent blocks adjacent to the block to be coded. For example, a block having an appropriate motion vector from the adjacent blocks is selected as the selected block. The acquisition / selection unit 110 selects the motion vector of the selected block as the motion vector 16 to be used in motion compensation prediction, and sends it to the predictor 109. The acquisition / selection unit 110 also generates selection information 17 for the selected block, and sends it to the variable-length coder 111.

[0013] The variable length coder 111 has a selection information coding unit 112. The selection information coding unit 112 performs variable length coding on the selection information 17 while switching the code table so that the code table has entries equal in number to the number of available coded blocks. An available block is a block having a motion vector among coded blocks adjacent to the block to be coded. A multiplexer 113 multiplexes the coded quantized orthogonal transform coefficient information and the selection information, and outputs coded data.

[0014] The operation of the video encoding apparatus having the above configuration will be described with reference to the flow chart of FIG.

[0015] First, a prediction error signal 12 is generated (S11). In generating the prediction error signal 12, a motion vector is selected, and a predicted image is generated using the selected motion vector. The difference between the predicted image signal, i.e., the predicted image signal 15, and the input video signal 11 is calculated by a subtractor 101, and the prediction error signal 12 is generated.

[0016] The prediction error signal 12 is orthogonally transformed by the orthogonal transformer 102 to generate orthogonal transform coefficients (S12). The orthogonal transform coefficients are quantized by the quantizer 103 (S13). The quantized orthogonal transform coefficient information is inversely quantized by the inverse quantizer 104 (S14), and then inversely orthogonally transformed by the inverse orthogonal transformer 105 to obtain a reconstructed prediction error signal (S15). The adder 106 adds the reconstructed prediction error signal and the predicted image signal 15 to generate a locally decoded image signal 14 (S16). The locally decoded image signal 14 is stored in the frame memory 107 (as a reference image) (S17), and the locally decoded image signal read from the frame memory 107 is input to the motion compensation predictor 108.

[0017] A predictor 109 of the motion compensation predictor 108 performs motion compensation prediction of a local decoded image signal (reference image) using a motion vector 16 to generate a predicted image signal 15. The predicted image signal 15 is sent to a subtractor 101 to take the difference with the input video signal 11, and is also sent to an adder 106 to generate a local decoded image signal 14.

[0018] The acquisition unit / selection unit 110 selects one motion vector to be used for motion compensation prediction from the adjacent blocks, sends the selected motion vector 16 to the predictor 109, and generates selection information 17. The selection information 17 is sent to the selection information encoding unit 112. When selecting a motion vector from the adjacent blocks, it is possible to select an appropriate motion vector that reduces the amount of code.

[0019] The orthogonal transform coefficient information 13 quantized by the quantizer 103 is also input to the variable length coder 111, where it is subjected to variable length coding (S18). The acquisition unit / selection unit 110 outputs the selection information 16 used in the motion compensation prediction, and inputs it to the selection information coding unit 112. In the selection information coding unit 112, the code table is switched so that the code table has entries equal in number to the number of available blocks, which are coded blocks adjacent to the coding target block and have a motion vector, and the selection information 17 is variable length coded. The quantized orthogonal transform coefficient information and the selection information from the variable length coder 111 are multiplexed by the multiplexer 113, and a bit stream of coded data 18 is output (S19). The coded data 18 is sent to a storage system or a transmission line, not shown.

[0020] 2, the flow of steps S14 to S17 can be interchanged with the flow of steps S18 and S19. That is, the variable length coding step S18 and the multiplexing step S19 are performed following the quantization step S13, and the inverse quantization step S14 to the storage step S17 can be performed in conjunction with the multiplexing step S19.

[0021] Next, the operation of the acquisition / selection unit 110 will be described with reference to the flowchart shown in FIG.

[0022] First, the frame memory 107 is referenced to search for available block candidates, which are coded blocks having motion vectors adjacent to the block to be coded (S101). After the available block candidates are searched for, the block sizes of the motion compensation prediction of these available block candidates are determined (S102). Next, it is determined whether the available block candidates are unidirectional or bidirectional predictive (S103). Based on the determination result and the prediction mode of the block to be coded, available blocks are extracted from the available block candidates. One selection block is selected from the extracted available blocks, and information identifying the selection block is obtained as selection information (S104).

[0023] Next, the determination of the block size (S102) will be described with reference to FIGS. 4A to 4C.

[0024] The adjacent blocks used in this embodiment are the blocks located to the left, top left, top, and top right of the encoding target block. Therefore, if the encoding target block is located at the top left of the frame, there is no available block adjacent to the encoding target block, so the present invention cannot be applied to this encoding target block. If the encoding target block is at the top edge of the screen, the available block is only the left block, and if the encoding target block is at the left edge of the screen but not the right edge, the available blocks are the two blocks above and to the top right of the encoding target block.

[0025] When the macroblock size is 16x16, there are four block sizes for motion compensation prediction of adjacent blocks, 16x16, 16x8, 8x16, and 8x8, as shown in Figs. 4A to 4C. Considering these four types, there are 20 types of adjacent blocks that can be available blocks, as shown in Figs. 4A to 4C. That is, there are four types of 16x16 size shown in Fig. 4A, ten types of 16x8 and 8x16 sizes shown in Fig. 4B, and six types of 8x8 size shown in Fig. 4C. In determining the block size (S102), available blocks are searched for according to the block size from among these 20 types of blocks. For example, when the size of the available block is only 16x16, the available blocks determined by this block size are four types of 16x16 size blocks as shown in Fig. 4A. That is, the available blocks are the upper left block of the block to be coded, the upper block of the block to be coded, the block to the left of the block to be coded, and the upper right block of the block to be coded. Also, when the macroblock size is expanded to 16x16 or more, the available blocks can be the same as when the macroblock size is 16x16. For example, when the macroblock size is 32x32, there are four block sizes of the motion compensation prediction of the adjacent blocks, 32x32, 32x16, 16x32, and 16x16, and there are 20 types of adjacent blocks that can be the available blocks.

[0026] Next, the determination of unidirectional or bidirectional prediction (S103) performed by the acquisition / selection unit 110 will be described with reference to FIG.

[0027] For example, suppose that the block size is limited to 16x16, and the unidirectional or bidirectional prediction of the adjacent blocks with respect to the encoding target block is as shown in FIG. 5. In the determination of unidirectional or bidirectional prediction (S103), available blocks are searched for according to the prediction direction. For example, the adjacent blocks including the prediction direction L0 are set as available blocks determined in the prediction direction. That is, the blocks above, to the left, and to the right of the encoding target block shown in FIG. 5(a) are available blocks determined in the prediction direction. In this case, the upper left block of the encoding target block is not used. If the adjacent blocks including the prediction direction L1 are set as available blocks determined in the prediction method, the upper left and upper blocks of the encoding target block shown in FIG. 5(b) are available blocks determined in the prediction direction. In this case, the left and upper right blocks of the encoding target block are not used. If the adjacent blocks including the prediction direction L0 / L1 are set as available blocks determined in the prediction method, only the blocks above the encoding target block shown in FIG. 5(c) are available blocks determined in the prediction direction. In this case, the blocks to the left, top left, and top right of the current block are not used. Note that the prediction direction L0 (L1) corresponds to the prediction direction of L0 prediction (L1 prediction) in AVC.

[0028] Next, the selection information encoding unit 112 will be described with reference to the flowchart shown in FIG.

[0029] Among the adjacent blocks adjacent to the block to be coded, an available block that is a coded block having a motion vector is searched for, and available block information determined by block size and unidirectional or bidirectional prediction is obtained (S201). Using this available block information, a code table is switched according to the number of available blocks as shown in Fig. 8 (S202). Using the switched code table, the selection information 17 sent from the acquisition unit / selection unit 110 is variable-length coded (S203).

[0030] Next, an example of an index of selection information will be described with reference to FIG.

[0031] When there is no available block as shown in Fig. 7(a), the present invention is not applicable to this block, so no selection information is sent. When there is one available block as shown in Fig. 7(b), the motion vector of the available block used for motion compensation of the current block is uniquely determined, so no selection information is sent. When there are two available blocks as shown in Fig. 7(c), selection information of index 0 or 1 is sent. When there are three available blocks as shown in Fig. 7(d), selection information of index 0, 1 or 2 is sent. When there are four available blocks as shown in Fig. 7(e), selection information of index 0, 1, 2 or 3 is sent.

[0032] As an example of how to index available blocks, an example in which available blocks are indexed in the order of left, top left, top, and top right of the block to be coded is shown in Fig. 7. In other words, the blocks to be used, excluding unused blocks, are indexed with consecutive numbers.

[0033] Next, the code table of the selection information 17 will be described with reference to FIG.

[0034] The selection information encoding unit 112 switches between code tables depending on the number of available blocks (S202). As described above, the selection information 17 needs to be encoded when there are two or more available blocks.

[0035] First, when there are two available blocks, indices 0 and 1 are required, and the code table is as shown on the left side of Figure 8. When there are three available blocks, indices 0, 1, and 2 are required, and the code table is as shown in the center of Figure 8. When there are four available blocks, indices 0, 1, 2, and 3 are required, and the code table is as shown on the right side of Figure 8. These code tables are switched depending on the number of available blocks.

[0036] Next, a method for encoding the selection information will be described.

[0037] 9 shows an outline of the syntax structure used in this embodiment. The syntax mainly consists of three parts. High Level Syntax 801 contains syntax information for layers higher than slices. Slice Level Syntax 804 specifies information required for each slice, and Macroblock Level Syntax 807 specifies variable-length coded error signals and mode information required for each macroblock.

[0038] Each of these syntaxes is composed of more detailed syntaxes, and the High Level Syntax 801 is composed of sequence and picture level syntaxes such as Sequence parameter set syntax 802 and Picture parameter set syntax 803. The Slice Level Syntax 804 is composed of Slice header syntax 405 and Slice data syntax 406. Furthermore, the Macroblock Level Syntax 807 is composed of macroblock layer syntax 808 and macroblock prediction syntax 809.

[0039] The syntax information required in this embodiment is macroblock layer syntax 808, and the syntax will be explained below.

[0040] 10(a)(b), available_block_num indicates the number of available blocks, and if this is 2 or more, selection information needs to be coded. Furthermore, mvcopy_flag is a flag indicating whether or not to use a motion vector of an available block in motion compensation prediction, and if the number of available blocks is 1 or more and this flag is 1, the motion vector of the available block can be used in motion compensation prediction. Furthermore, mv_select_info indicates selection information, and the code table is as described above.

[0041] Figure 10(a) shows the syntax when encoding selection information after mb_type. For example, when the block size is only 16x16, if mb_type is other than 16x16, mvcopy_flag and mv_select_info do not need to be encoded. If mb_type is 16x16, mvcopy_flag and mv_select_info are encoded.

[0042] Figure 10(b) shows the syntax when the selection information is coded before mb_type. For example, if mvcopy_flag is 1, there is no need to code mb_type. If mv_copy_flag is 0, code mb_type.

[0043] In this embodiment, the encoding scan order may be any order. For example, the present invention is also applicable to line scans, Z scans, and the like.

[0044] A video decoding device according to another embodiment will be described with reference to FIG.

[0045] The coded data 18 output from the video coding device of Fig. 1 is input to a demultiplexer 201 of the video decoding device as coded data 21 to be decoded via a storage system or a transmission system. The demultiplexer 201 demultiplexes the coded data 21 and separates the coded data 21 into quantized orthogonal transform coefficient information and selection information. The output terminal of the demultiplexer 201 is connected to a variable length decoder 202. The variable length decoder 202 decodes the quantized orthogonal transform coefficient information and the selection information. The output terminal of the variable length decoder 202 is connected to an inverse quantizer 204 and an inverse orthogonal transformer 205, which are connected to an adder 206. The inverse quantizer 204 inverse quantizes the quantized orthogonal transform coefficient information and converts it into orthogonal transform coefficients. The inverse orthogonal transformer 205 inversely orthogonally transforms the orthogonal transform coefficients to generate a prediction error signal. The adder 206 adds the prediction error signal to a prediction image signal from a prediction image generator 207 to generate a video signal.

[0046] The predicted image generator 207 includes a predictor 208 and a selection unit 209. The selection unit 209 selects a motion vector based on the selection information 23 decoded by the selection information decoder 203 of the variable length decoder 202, and sends the selected motion vector 25 to the predictor 208. The predictor 208 motion-compensates a reference image stored in a frame memory 210 based on the motion vector 25, and generates a predicted image.

[0047] The operation of the video decoding apparatus having the above configuration will be described with reference to the flow chart of FIG.

[0048] The encoded data 21 is demultiplexed by the demultiplexer 201 (S31) and decoded by the variable length decoder 202 to generate quantized orthogonal transform coefficient information 22 (S32). The selection information decoder 203 also checks the states of adjacent blocks adjacent to the block to be decoded, and switches the code table as shown in Fig. 8 according to the number of available blocks, which are adjacent encoded blocks having motion vectors, in the same manner as the selection information encoder 112 of the encoding device to perform decoding. As a result, the selection information 23 is output (S33).

[0049] Quantized orthogonal transform coefficient information 22, which is information output from the variable decoder 202, is sent to an inverse quantizer 204, and selection information 23, which is information output from the selection information decoding unit 203, is sent to a selection unit 209.

[0050] The quantized orthogonal transform coefficient information 22 is inverse quantized by the inverse quantizer 204 (S34), and then inverse orthogonally transformed by the inverse orthogonal transformer 205 (S35). This results in a prediction error signal 24. The adder 206 adds the prediction image signal to the prediction error signal 24, thereby reproducing a video signal 26 (S36). The reproduced video signal 27 is stored in the frame memory 210 (S37).

[0051] A predicted image generator 207 generates a predicted image 26 using the motion vector of an available block, which is a decoded block adjacent to the block to be decoded and has a motion vector, selected by the decoded selection information 23. A selector 209 checks the state of the adjacent block, and selects one motion vector to be used for motion compensation prediction from the available block information of the adjacent block and the selection information 23 decoded by the selection information decoder 203 in the same manner as the acquisition / selector 110 of the encoding device. A predictor 208 generates a predicted image 26 using this selected motion vector 25, and the predicted image 26 is sent to an adder 206 to obtain a video signal 27.

[0052] According to the present invention, by encoding the selection information according to the number of available blocks, the selection information can be sent using an appropriate encoding table, and additional information for the selection information can be reduced.

[0053] Furthermore, by using the motion vectors of the available blocks for motion compensation prediction of the current block to be coded, it is possible to reduce additional information related to the motion vector information.

[0054] Furthermore, by selecting an appropriate motion vector calculation method from available blocks instead of using a fixed method, the degree of freedom in motion vector calculation is increased compared to the direct mode.

[0055] The techniques of the present invention described in the embodiments of the present invention can be executed by a computer, and can also be distributed by storing them on recording media such as magnetic disks (flexible disks, hard disks, etc.), optical disks (CD-ROMs, DVDs, etc.), and semiconductor memories as programs that can be executed by a computer.

[0056] Furthermore, the present invention is not limited to the above-described embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the gist of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined. [Industrial Applicability]

[0057] The device of the present invention is used for image compression processing in communication, storage and broadcasting.

Claims

1. a first decoding unit that decodes a flag indicating whether or not a motion vector of an available block is to be used in motion compensation prediction; an acquisition unit that acquires an available block having a motion vector from a plurality of already-decoded blocks adjacent to a block to be decoded; a setting unit that sets indexes to the available blocks in an order according to a position of the available blocks with respect to the block to be decoded; a second decoding unit for decoding an index that identifies the selected block; a selection unit that selects the selected block from the available blocks according to the set index and the decoded index; a third decoding unit that performs motion compensation prediction decoding on the current block by using the motion vector of the selected block; A video decoding device comprising:

2. decoding a flag indicating whether to use a motion vector of the available block in motion compensation prediction; obtaining an available block having a motion vector from a plurality of decoded blocks adjacent to the block to be decoded; setting indexes for the available blocks in an order according to a position of the available blocks relative to the current block; decoding an index identifying the selected block; selecting the selected block from the available blocks according to the set index and the decoded index; performing motion compensation prediction decoding on the current block using the motion vector of the selected block; A video decoding method comprising:

3. a first encoding unit that encodes a flag indicating whether or not a motion vector of an available block is to be used in motion compensation prediction; an acquisition unit that acquires available blocks having a motion vector from a plurality of encoded blocks adjacent to a current block; a setting unit that sets indexes to the available blocks in an order according to a position of the available blocks with respect to the current block; a selection unit for selecting one block from the available blocks; a second encoding unit that encodes the index set for the selected block; a third encoding unit that performs motion compensation prediction encoding on the current block by using the motion vector of the selected block; A video encoding device comprising:

4. encoding a flag indicating whether to use a motion vector of the available block in the motion compensation prediction; obtaining an available block having a motion vector from a plurality of coded blocks adjacent to the coding target block; setting indexes for the available blocks in an order according to positions of the available blocks relative to the current block; selecting a block from the available blocks; encoding the index set for the selected block; performing motion compensation predictive coding of the current block using the motion vector of the selected block; A video encoding method comprising:

Images