Video decoding apparatus, video coding apparatus, and video coding and decoding apparatus

Abstract

To improve coding efficiency.SOLUTION: A video decoding apparatus is configured to: decode a temporal prediction regulation flag indicating whether or not temporal motion vector prediction is used (S211); when the temporal prediction regulation flag indicates that the temporal motion vector prediction is used (Yes in S212), decode a first parameter for specifying a prediction direction for calculating a temporal predictive motion vector (S213); derive, based on the prediction direction specified by the first parameter, a plurality of first predictive motion vector candidates including the temporal predictive motion vector (S214); and derive, when the temporal prediction regulation flag indicates that the temporal motion vector prediction is not used (No in S212), a plurality of second predictive motion vector candidates not including the temporal predictive motion vector (S217).SELECTED DRAWING: Figure 21

Description

[Technical Field]

[0001] The present invention relates to a video encoding method and a video decoding method. [Background technology]

[0002] In video coding, the amount of information is generally compressed by utilizing spatial and temporal redundancies of video. Generally, a method for utilizing spatial redundancy involves transforming to the frequency domain. Furthermore, a method for utilizing temporal redundancy involves inter-picture prediction (hereinafter referred to as inter-prediction) coding (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] ITU-T Recommendation H.264 "Advanced video coding for generic audiovisual services", March 2010 Summary of the Invention [Problem to be solved by the invention]

[0004] In such video encoding and decoding methods, it is desirable to improve the encoding efficiency.

[0005] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a video encoding method and a video decoding method that can improve encoding efficiency. [Means for solving the problem]

[0006] In order to achieve the above object, a video decoding device according to one aspect of the present invention is a video decoding device that performs inter-prediction decoding using a motion vector on a current block included in a current picture, the video decoding device comprising: a control circuit; and a storage device electrically connected to the control circuit; wherein the control circuit decodes a temporal prediction restriction flag indicating whether or not temporal motion vector prediction using a temporal prediction motion vector that is a motion vector of a block included in a decoded picture other than the current picture will be used; when the temporal prediction restriction flag indicates that the temporal motion vector prediction will be used, the control circuit decodes a first parameter for specifying a prediction direction for calculating the temporal prediction motion vector; and, based on the prediction direction specified by the first parameter, performs inter-prediction decoding on a current block including the temporal prediction motion vector. and deriving a predicted motion vector candidate, using one of the plurality of first predicted motion vector candidates, to decode a motion vector to be used for inter-prediction decoding of the block to be decoded; if the temporal prediction restriction flag indicates that the temporal motion vector prediction is not used, deriving a plurality of second predicted motion vector candidates that do not include the temporal predicted motion vector, using one of the plurality of second predicted motion vector candidates, to decode a motion vector to be used for inter-prediction decoding of the block to be decoded; if the temporal prediction restriction flag indicates that the temporal motion vector prediction is not used, not decoding the first parameter, and including an alternative vector in place of the temporal predicted motion vector in the plurality of first predicted motion vector candidates or the plurality of second predicted motion vector candidates.

[0007] These general or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0008] The present invention can provide a video encoding method and a video decoding method that can improve encoding efficiency. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram showing motion vectors in temporal motion vector prediction. [Figure 2] FIG. 2 is a block diagram of the video encoding device according to the first embodiment. [Figure 3] FIG. 3 is a diagram showing an outline of a processing flow of the video coding method according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing examples of motion vector predictor candidates according to the first embodiment. [Figure 5] FIG. 5 is a diagram showing an example of a code table when variable-length coding a motion vector predictor index according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing a flow of determining a motion vector predictor candidate according to the first embodiment. [Figure 7] FIG. 7 is a conceptual diagram showing the read / write process for the colPic memory and the global vector storage unit according to the first embodiment. [Figure 8A] FIG. 8A is a diagram showing a detailed processing flow of step S11 in FIG. 3 according to the first embodiment. [Figure 8B] FIG. 8B is a diagram showing an example of a B-picture referenced by other pictures according to Embodiment 1. In FIG. [Figure 9] FIG. 9 is a diagram showing a detailed processing flow of step S17 in FIG. 3 according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing a detailed processing flow of steps S13 and S14 in FIG. 3 according to the first embodiment. [Figure 11A] FIG. 11A is a diagram showing an example of a method for deriving a motion vector predictor candidate using a forward reference motion vector according to Embodiment 1. FIG. [Figure 11B] FIG. 11B is a diagram showing an example of a method for deriving a motion vector predictor candidate using a backward reference motion vector according to Embodiment 1. In FIG. [Figure 12A]FIG. 12A is a diagram showing an example of a method for deriving a motion vector predictor candidate using a backward reference motion vector according to Embodiment 1. FIG. [Figure 12B] FIG. 12B is a diagram showing an example of a method for deriving a motion vector predictor candidate using a forward reference motion vector according to Embodiment 1. In FIG. [Figure 13] FIG. 13 is a diagram showing a processing flow of the video coding method according to the first embodiment. [Figure 14] FIG. 14 is a diagram showing a processing flow of the video coding method according to the first embodiment. [Figure 15] FIG. 15 is a block diagram of a video decoding device according to the second embodiment. [Figure 16] FIG. 16 is a diagram showing an outline of a processing flow of the video decoding method according to the second embodiment. [Figure 17] FIG. 17 is a diagram showing an example of syntax of a bitstream in the video decoding method according to the second embodiment. [Figure 18] FIG. 18 is a diagram showing a processing flow of the video decoding method according to the second embodiment. [Figure 19A] FIG. 19A is a diagram illustrating an example of syntax according to the second embodiment. [Figure 19B] FIG. 19B is a diagram illustrating an example of syntax according to the second embodiment. [Figure 20] FIG. 20 is a diagram illustrating an example of syntax according to the second embodiment. [Figure 21] FIG. 21 is a diagram showing a processing flow of the video decoding method according to the second embodiment. [Figure 22] FIG. 22 is a block diagram of a video coding device according to a variation of the first embodiment. [Figure 23] FIG. 23 is a flowchart showing the operation of the video coding method according to the variation of the first embodiment. [Figure 24] FIG. 24 is a diagram showing an example of an image including a base view and a dependent view according to a variation of the second embodiment. [Figure 25]FIG. 25 is a block diagram of a video decoding device according to a variation of the second embodiment. [Figure 26] FIG. 26 is a diagram showing the overall configuration of a content supply system that realizes a content distribution service. [Figure 27] FIG. 27 is a diagram showing the overall configuration of a digital broadcasting system. [Figure 28] FIG. 28 is a block diagram showing an example of the configuration of a television. [Figure 29] FIG. 29 is a block diagram showing an example of the configuration of an information reproducing / recording unit that reads and writes information from and to a recording medium that is an optical disc. [Figure 30] FIG. 30 is a diagram showing an example of the structure of a recording medium that is an optical disc. [Figure 31A] FIG. 31A is a diagram showing an example of a mobile phone. [Figure 31B] FIG. 31B is a block diagram showing an example of the configuration of a mobile phone. [Figure 32] FIG. 32 is a diagram showing the structure of multiplexed data. [Figure 33] FIG. 33 is a diagram showing a schematic diagram of how each stream is multiplexed in multiplexed data. [Figure 34] FIG. 34 shows in more detail how a video stream is stored in a PES packet sequence. [Figure 35] FIG. 35 is a diagram showing the structure of TS packets and source packets in multiplexed data. [Figure 36] FIG. 36 shows the data structure of a PMT. [Figure 37] FIG. 37 is a diagram showing the internal structure of the multiplexed data information. [Figure 38] FIG. 38 shows the internal structure of the stream attribute information. [Figure 39] FIG. 39 shows the steps for identifying video data. [Figure 40] FIG. 40 is a block diagram showing an example of the configuration of an integrated circuit that realizes the video encoding method and video decoding method according to each embodiment. [Figure 41] FIG. 41 is a diagram showing a configuration for switching the drive frequency. [Figure 42] FIG. 42 is a diagram showing steps for identifying video data and switching the drive frequency. [Figure 43] FIG. 43 is a diagram showing an example of a lookup table in which video data standards and drive frequencies are associated with each other. [Figure 44A] FIG. 44A is a diagram showing an example of a configuration in which modules of a signal processing unit are shared. [Figure 44B] FIG. 44B is a diagram showing another example of a configuration in which modules of a signal processing unit are shared. DETAILED DESCRIPTION OF THE INVENTION

[0010] (Findings that form the basis of the present invention) The present inventors have found that the following problems occur with the conventional techniques.

[0011] In inter-prediction coding, when coding a picture, a video encoder uses a coded picture that is either before or after the current picture in display order (display time order) as a reference picture. The video encoder then derives a motion vector for the current picture relative to the reference picture by motion estimation, and calculates a difference between predicted image data obtained by motion compensation based on the motion vector and the image data of the current picture. This removes redundancy in the temporal direction. In the motion estimation, the video encoder calculates a difference between a block to be coded in the current picture and a block in the reference picture, and selects the block in the reference picture with the smallest difference value as the reference block. The video encoder then estimates a motion vector using the current block and the reference block.

[0012] A video coding method known as H.264, which has already been standardized, uses three picture types, I-pictures, P-pictures, and B-pictures, to compress the amount of information. An I-picture is a picture that is not subjected to inter-prediction coding, i.e., is subjected to only intra-picture prediction (hereinafter referred to as intra-prediction) coding. A P-picture is a picture that is subjected to inter-prediction coding with reference to only one previously coded picture that is located before or after the current picture in display order. A B-picture is a picture that is subjected to inter-prediction coding with reference to two previously coded pictures that are located before or after the current picture in display order.

[0013] Furthermore, in a video coding method called H.264, a motion vector estimation mode is used as the inter-prediction coding mode for each block to be coded in a B picture, in which a difference value between predicted image data and image data of the block to be coded and a motion vector used to generate the predicted image data are coded. In the motion vector estimation mode, a video coding device can select, as the prediction direction, either bidirectional prediction, in which a predicted image is generated by referencing two previously coded pictures located before or after the picture to be coded, or unidirectional prediction, in which a predicted image is generated by referencing one previously coded picture located before or after the picture to be coded.

[0014] In addition, in a video coding method called H.264, a video coding device can select a coding mode called temporal motion vector prediction (temporal motion vector predictor mode or temporal direct mode) when deriving a motion vector in coding a B picture. Furthermore, a predicted motion vector (predictor motion vector candidate) generated by temporal motion vector prediction is called a temporal motion vector predictor.

[0015] An inter-prediction coding method in temporal motion vector prediction will be described with reference to Fig. 1. Fig. 1 is an explanatory diagram showing motion vectors in temporal motion vector prediction, and illustrates a case where block a in picture B2 is coded using temporal motion vector prediction.

[0016] In this case, the motion vector vb of block b, which is located at the same position as block a in picture P3, a reference picture after picture B2, is used. Motion vector vb is the motion vector used when block b was coded, and references picture P1. The video coding apparatus then obtains reference blocks from picture P1, which is a forward reference picture, and picture P3, which is a backward reference picture, using a motion vector parallel to motion vector vb, and codes block a by performing bidirectional prediction. That is, the motion vectors used when coding block a are motion vector va1 for picture P1 and motion vector va2 for picture P3.

[0017] However, in such temporal motion vector prediction, if information on a reference picture containing information such as a motion vector used in calculating a temporal motion vector predictor is lost, for example, due to packet loss in streaming distribution, the correct temporal motion vector predictor cannot be calculated, resulting in degradation of the decoded image. Furthermore, errors propagate to pictures that refer to the decoded image, which may result in the decoding process being halted. For example, if information on reference picture P3 in FIG. 1 is lost, the temporal motion vector predictor for picture B2 cannot be calculated. As a result, picture B2 may not be decoded correctly, causing the decoding process to be halted.

[0018] Therefore, in this embodiment, a moving image coding method and a moving image decoding method that can effectively prevent the propagation of errors in the decoding process will be described.

[0019] Furthermore, in such video encoding and decoding methods, it is desirable to improve the encoding efficiency.

[0020] Therefore, in this embodiment, a moving image coding method and a moving image decoding method that can improve coding efficiency will be described.

[0021] A video coding method according to one aspect of the present invention is a video coding method for inter-predictively coding a current block included in a current picture using a motion vector, the video coding method including: a flag coding step of coding a first flag indicating whether or not temporal motion vector prediction using a temporal motion vector predictor, the temporal motion vector being a motion vector of a block included in a coded picture other than the current picture, will be used; a parameter coding step of coding a first parameter for calculating the temporal motion vector predictor if the first flag indicates that the temporal motion vector prediction will be used; and a first candidate derivation step of deriving a plurality of first motion vector predictor candidates including the temporal motion vector predictor using the first parameter. a first motion vector encoding step of encoding a motion vector to be used for inter-predictive encoding of the block to be encoded using one of the plurality of first predicted motion vector candidates; a second candidate derivation step of deriving a plurality of second predicted motion vector candidates that do not include the temporal predicted motion vector if the first flag indicates that the temporal motion vector prediction is not used; and a second motion vector encoding step of encoding a motion vector to be used for inter-predictive encoding of the block to be encoded using one of the plurality of second predicted motion vector candidates, and the first parameter is not encoded if the first flag indicates that the temporal motion vector prediction is not used.

[0022] According to this, when temporal motion vector prediction is not used, the video coding method does not code the unnecessary first parameter, thereby improving coding efficiency.

[0023] For example, the first parameters may include parameters for identifying the coded picture.

[0024] For example, the first parameter may include a reference picture index for identifying the previously coded picture among a plurality of pictures indicated in a reference picture list used to code the current picture.

[0025] For example, the first parameter may include a flag indicating which of a plurality of reference picture lists used to encode the current picture is to be used to identify the previously encoded picture.

[0026] For example, in the first or second candidate derivation step, the plurality of first motion vector predictors or the plurality of second motion vector predictors may include an alternative vector to replace the temporal motion vector predictor.

[0027] This makes it possible for the video coding method to suppress a decrease in coding efficiency.

[0028] For example, the video encoding method is a method for encoding pictures belonging to a base view and a dependent view included in a multi-view video, and further includes a disparity vector generation step for generating a disparity vector corresponding to the disparity between the base view and the dependent view, and in the first candidate derivation step, if the picture to be encoded belongs to the dependent view and is the first picture in a GOP (Group Of Pictures), the disparity vector may be included as the alternative vector in the plurality of first predicted motion vectors.

[0029] This makes it possible for the video coding method to suppress a decrease in coding efficiency.

[0030] For example, the video encoding method may be a method for encoding pictures belonging to a base view and a dependent view included in a multi-view video, and may further include a disparity vector generation step for generating a disparity vector corresponding to the disparity between the base view and the dependent view, and in the second candidate derivation step, the disparity vector may be included as the alternative vector in the plurality of second predicted motion vectors.

[0031] This makes it possible for the video coding method to suppress a decrease in coding efficiency.

[0032] Furthermore, a video decoding method according to one aspect of the present invention is a video decoding method for performing inter-prediction decoding using a motion vector on a current block included in a current picture, the video decoding method including: a flag decoding step of decoding a first flag indicating whether or not temporal motion vector prediction using a temporal motion vector predictor that is a motion vector of a block included in a decoded picture other than the current picture will be used; a parameter decoding step of decoding a first parameter for calculating the temporal motion vector predictor when the first flag indicates that the temporal motion vector prediction will be used; and a first candidate decoding step of deriving the plurality of first motion vector predictor candidates including the temporal motion vector predictor using the first parameter. The method includes a complementary derivation step, a first motion vector decoding step of decoding a motion vector to be used for inter-prediction decoding of the block to be decoded using one of the plurality of first predicted motion vector candidates, a second candidate derivation step of deriving a plurality of second predicted motion vector candidates that do not include the temporal predicted motion vector if the first flag indicates that the temporal motion vector prediction is not used, and a second motion vector decoding step of decoding a motion vector to be used for inter-prediction decoding of the block to be decoded using one of the plurality of second predicted motion vector candidates, and the first parameter is not decoded if the first flag indicates that the temporal motion vector prediction is not used.

[0033] According to this, when temporal motion vector prediction is not used, the video decoding method does not decode the unnecessary first parameter, thereby improving coding efficiency.

[0034] For example, the first parameter may include a parameter for identifying the decoded picture.

[0035] For example, the first parameter may include a reference picture index for identifying the already-decoded picture among a plurality of pictures indicated in a reference picture list used to decode the current picture.

[0036] For example, the first parameter may include a flag indicating which of a plurality of reference picture lists used to decode the current picture is to be used to identify the coded picture.

[0037] For example, in the first or second candidate derivation step, the plurality of first motion vector predictors or the plurality of second motion vector predictors may include an alternative vector to replace the temporal motion vector predictor.

[0038] According to this, the video decoding method can suppress a decrease in coding efficiency.

[0039] For example, the video decoding method is a method for decoding pictures belonging to a base view and a dependent view included in a multi-view video, and further includes a disparity vector generation step for generating a disparity vector corresponding to the disparity between the base view and the dependent view, and in the first candidate derivation step, if the picture to be decoded belongs to the dependent view and is the first picture in a GOP (Group Of Pictures), the disparity vector may be included as the alternative vector in the plurality of first predicted motion vectors.

[0040] According to this, the video decoding method can suppress a decrease in coding efficiency.

[0041] For example, the video decoding method may be a method for decoding pictures belonging to a base view and a dependent view included in a multi-view video, and may further include a disparity vector generation step for generating a disparity vector corresponding to the disparity between the base view and the dependent view, and in the second candidate derivation step, the disparity vector may be included as the alternative vector in the plurality of second predicted motion vectors.

[0042] According to this, the video decoding method can suppress a decrease in coding efficiency.

[0043] Furthermore, a video encoding device according to one aspect of the present invention is a video encoding device including a control circuit and a storage device electrically connected to the control circuit, and configured to perform inter-prediction encoding using a motion vector of a current block included in a current picture to be encoded, wherein the control circuit includes a flag encoding step of encoding a first flag indicating whether or not temporal motion vector prediction using a temporal motion vector predictor that is a motion vector of a block included in an encoded picture other than the current picture to be encoded will be used; a parameter encoding step of encoding a first parameter for calculating the temporal motion vector predictor when the first flag indicates that the temporal motion vector prediction will be used; and a parameter encoding step of encoding a plurality of first parameters including the temporal motion vector predictor using the first parameter. The method executes a first candidate derivation step of deriving a predicted motion vector candidate, a first motion vector encoding step of encoding a motion vector to be used for inter-predictive encoding of the block to be encoded using one of the plurality of first predicted motion vector candidates, a second candidate derivation step of deriving a plurality of second predicted motion vector candidates that do not include the temporal predicted motion vector if the first flag indicates that the temporal motion vector prediction is not used, and a second motion vector encoding step of encoding a motion vector to be used for inter-predictive encoding of the block to be encoded, and does not encode the first parameter if the first flag indicates that the temporal motion vector prediction is not used.

[0044] According to this, when the video encoding device does not use temporal motion vector prediction, the video encoding device does not decode the unnecessary first parameter, thereby improving encoding efficiency.

[0045] Furthermore, a video decoding device according to one aspect of the present invention includes a control circuit and a storage device electrically connected to the control circuit, and performs inter-prediction decoding using motion vectors on a current block included in a current picture, wherein the control circuit includes a flag decoding step of decoding a first flag indicating whether or not temporal motion vector prediction using a temporal motion vector predictor that is a motion vector of a block included in a decoded picture other than the current picture will be used; a parameter decoding step of decoding, if the first flag indicates that the temporal motion vector prediction will be used, a first parameter for calculating the temporal motion vector predictor; and a parameter decoding step of decoding, using the first parameter, a plurality of first parameters including the temporal motion vector predictor. The method executes a first candidate derivation step of deriving a predicted motion vector candidate, a first motion vector decoding step of decoding a motion vector to be used for inter-prediction decoding of the block to be decoded using one of the plurality of first predicted motion vector candidates, a second candidate derivation step of deriving a plurality of second predicted motion vector candidates that do not include the temporal predicted motion vector if the first flag indicates that the temporal motion vector prediction is not used, and a second motion vector decoding step of decoding a motion vector to be used for inter-prediction decoding of the block to be decoded using one of the plurality of second predicted motion vector candidates, and does not decode the first parameter if the first flag indicates that the temporal motion vector prediction is not used.

[0046] According to this, when the video decoding device does not use temporal motion vector prediction, the video decoding device does not decode the unnecessary first parameter, thereby enabling the video decoding device to improve coding efficiency.

[0047] A video encoding / decoding device according to an aspect of the present invention includes the video encoding device and the video decoding device.

[0048] These general or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0049] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0050] It should be noted that the embodiments described below each illustrate a specific example of the present invention. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept are described as optional components.

[0051] (Embodiment 1) FIG. 2 is a block diagram showing the configuration of a video coding device that uses the video coding method according to the first embodiment.

[0052] As shown in FIG. 2, the video encoding device 100 includes a subtraction unit 101, an orthogonal transformation unit 102, a quantization unit 103, an inverse quantization unit 104, an inverse orthogonal transformation unit 105, an addition unit 106, a block memory 107, a frame memory 108, an intra prediction unit 109, an inter prediction unit 110, a switch 111, an inter prediction control unit 112, a picture type determination unit 113, a temporal predicted motion vector calculation unit 114, a colPic memory 115, a global vector storage unit 116, a co-located information determination unit 117, and a variable length encoding unit 118.

[0053] The subtraction unit 101 acquires an input image sequence including a block to be coded from outside the device, acquires a predicted block from the switch 111, generates a residual block by subtracting the predicted block from the block to be coded, and outputs the generated residual block to the orthogonal transformation unit 102.

[0054] The orthogonal transform unit 102 generates transform coefficients by transforming the residual block obtained from the subtraction unit 101 from the image domain to the frequency domain, and outputs the generated transform coefficients to the quantization unit 103. The quantization unit 103 quantizes the transform coefficients obtained from the quantization unit 103 to generate quantized coefficients, and outputs the generated quantized coefficients to the inverse quantization unit 104 and the variable-length coding unit 118.

[0055] The inverse quantization unit 104 restores transform coefficients by inverse quantizing the quantized coefficients obtained from the quantization unit 103, and outputs the restored transform coefficients to the inverse orthogonal transformation unit 105. The inverse orthogonal transformation unit 105 restores residual blocks by transforming the restored transform coefficients obtained from the inverse quantization unit 104 from the frequency domain to the image domain, and outputs the restored residual blocks to the addition unit 106.

[0056] The adder 106 reconstructs the current block to be coded by adding the reconstructed residual block obtained from the inverse orthogonal transformer 105 and the predicted block obtained from the switch 111, and outputs the reconstructed current block to the block memory 107 and the frame memory 108. The block memory 107 stores the reconstructed input image sequence in blocks. The frame memory 108 stores the reconstructed input image sequence in frames.

[0057] The picture type determination unit 113 determines whether to encode the input image sequence as an I-picture, a B-picture, or a P-picture, and generates picture type information indicating the determined picture type. The picture type determination unit 113 then outputs the generated picture type information to the switch 111, the inter-prediction control unit 112, the co-located information determination unit 117, and the variable-length coding unit 118.

[0058] The intra prediction unit 109 generates a prediction block by performing intra prediction on the block to be coded using the sequence of input images restored in block units stored in the block memory 107, and outputs the generated prediction block to the switch 111. The inter prediction unit 110 generates a prediction block by performing inter prediction on the block to be coded using the sequence of input images restored in frame units stored in the frame memory 108 and a motion vector derived by motion estimation, and outputs the generated prediction block to the switch 111.

[0059] The switch 111 outputs the predicted block generated by the intra prediction unit 109 or the predicted block generated by the inter prediction unit 110 to the subtraction unit 101 and the addition unit 106. For example, the switch 111 outputs the predicted block with the smaller coding cost of the two predicted blocks.

[0060] The co-located information determination unit 117 determines whether or not to prohibit the use of the co-located block. Then, the co-located information determination unit 117 generates a co-located use prohibition flag indicating the determination result for each picture, and outputs the generated co-located use prohibition flag to the temporal motion vector predictor calculation unit 114 and the variable length coding unit 118. This co-located use prohibition flag is included in a bitstream (typically, a picture header or a slice header). Note that in the present embodiment, the co-located use prohibition flag is used to prohibit the use of the co-located block, thereby prohibiting the calculation of a temporal motion vector predictor using information about the co-located block and preventing the addition of the co-located block to a motion vector predictor candidate, but this is not necessarily limited to this. For example, a flag (enable_temporal_mvp_flag) or the like indicating whether or not to directly allow a temporal motion vector predictor using information about the co-located block may be prepared, and this flag may be included in a bitstream (typically, a picture header or a slice header).

[0061] Furthermore, the co-located information determination unit 117 determines, as the co-located block, either a block included in a picture located before the current picture to be coded in display order (hereinafter referred to as a forward reference block) or a block included in a picture located after the current picture to be coded (hereinafter referred to as a backward reference block). In other words, the forward reference block is a block included in a reference picture identified by the reference picture list L0. The backward reference block is a block included in a reference picture identified by the reference picture list L1. Note that, although the present embodiment has shown an example in which the reference picture list L0 includes a forward reference block and the reference picture list L1 includes a backward reference block, this is not necessarily the case. For example, as in the case in which the reference picture list L0 includes a forward reference block and the reference picture list L1 includes a forward reference block, both reference picture lists may include reference blocks in the same direction, or two reference picture lists may include reference blocks in opposite directions.

[0062] Then, the co-located information determination unit 117 generates a co-located reference direction flag (collocated_from_l0_flag) indicating the determination result for each picture, and outputs the generated co-located reference direction flag to the temporal motion vector predictor calculation unit 114 and the variable length coding unit 118. This co-located reference direction flag is included in a bitstream (typically, a picture header or a slice header). For example, if the value of collocated_from_l0_flag is 1, a co-located block is calculated from the reference picture list L0, and if the value of collocated_from_0_flag is 0, a co-located block is calculated from the reference picture list L1. Note that if the co-located usage prohibition flag is set to a value indicating "prohibited" (for example, if the value of enable_temporal_mvp_flag is 0), the co-located reference direction flag (collocated_from_l0_flag) may be omitted. A detailed method will be described later.

[0063] Here, the co-located block is a block in a picture different from the current picture containing the current block to be coded, and is located at the same position in the picture as the current block to be coded. For example, among multiple pictures included in a reference picture list determined according to the value of the co-located reference direction flag, a picture whose reference picture index value is the value of the co-located reference picture index (collocated_ref_idx) is used as the co-located picture. Then, a block in the co-located picture that is located at the same position as the current block to be coded is used as the co-located block.

[0064] Here, the co-located reference picture index (collocated_ref_idx) is an index value that specifies a co-located picture from among multiple pictures included in a reference picture list determined according to the value of the co-located reference direction flag. The collocated_ref_idx is included in a bitstream (typically, a picture header or a slice header).

[0065] For example, when specifying a picture corresponding to reference picture index 0 of reference picture list L1 as a co-located picture, the co-located information determination unit 117 sets the co-located reference direction flag (collocated_from_l0_flag) to a value of 0 and sets the co-located reference picture index collocated_ref_idx to a value of 0. Note that, when a value indicating "prohibited" is set in the co-located use prohibition flag (for example, when the value of enable_temporal_mvp_flag is 0), the co-located information determination unit 117 may omit the co-located reference picture index (collocated_ref_idx). A detailed method will be described later. Note that the positions of the current block and the co-located block within each picture do not need to strictly match. For example, the co-located information determination unit 117 may set a block surrounding (adjacent to) a block at the same position of the current block in a picture other than the current picture as the co-located block.

[0066] The temporal predicted motion vector calculation unit 114 derives predicted motion vector candidates, which are candidates for predicted motion vectors, using colPic information such as motion vectors of co-located blocks stored in the colPic memory 115, or the overall motion vector of the colPic picture stored in the overall vector storage unit, depending on the value of the co-located use prohibition flag obtained from the co-located information determination unit 117.

[0067] Specifically, when the co-located prohibition flag is on (prohibited), the temporal motion vector predictor calculation unit 114 adds the global motion vector (alternative vector) read from the global vector storage unit 116 to the motion vector predictor candidates. On the other hand, when the co-located prohibition flag is off (permitted), the temporal motion vector predictor calculation unit 114 adds the temporal motion vector predictor calculated using colPic information read from the colPic memory 115 to the motion vector predictor candidates.

[0068] Furthermore, the temporal motion vector predictor calculation unit 114 assigns the value of the motion vector predictor index to the motion vector predictor added as a candidate. Then, the temporal motion vector predictor calculation unit 114 outputs the motion vector predictor added as a candidate and the motion vector predictor index to the inter prediction control unit 112. On the other hand, if the co-located block does not have a motion vector, the temporal motion vector predictor calculation unit 114 either stops deriving a motion vector by temporal motion vector prediction, or derives a vector with a motion amount of 0 as a motion vector predictor candidate. Furthermore, the temporal motion vector predictor calculation unit 114 outputs the overall motion vector to the variable length coding unit 118.

[0069] The inter prediction control unit 112 determines to encode the motion vector using, from among a plurality of motion vector predictor candidates, a motion vector predictor having the smallest error with respect to the motion vector derived by motion estimation. Here, the error indicates, for example, a difference value between the motion vector predictor candidate and the motion vector derived by motion estimation.

[0070] Furthermore, the inter prediction control unit 112 identifies, for each block, a predicted motion vector index corresponding to the determined predicted motion vector. Then, the inter prediction control unit 112 outputs the predicted motion vector index and a difference value between the motion vector and the predicted motion vector to the variable length coding unit 118. The inter prediction control unit 112 also transfers colPic information including the motion vector and the like of the current block to be coded to the colPic memory 115. The inter prediction control unit 112 also transfers the motion vector and the like of the coding block to the global vector storage unit 116.

[0071] The colPic memory 115 stores colPic information including the motion vector of the current block to be coded, for use as a predicted motion vector when coding the next picture. The global vector storage unit 116 calculates a global motion vector from the motion vectors of multiple current blocks to be coded included in the picture, and stores the global motion vector for use as a predicted motion vector when coding the next picture.

[0072] The variable length coding unit 118 generates a bitstream by variable length coding the quantization coefficients obtained from the quantization unit 103, the predicted motion vector index and the difference value between the motion vector and the predicted motion vector obtained from the inter prediction control unit 112, the picture type information obtained from the picture type determination unit 113, the co-located use prohibition flag (or enable_temporal_mvp_flag), the co-located reference direction flag (collocated_from_l0_flag), and the co-located reference picture index (collocated_ref_idx) obtained from the co-located information determination unit 117, and the overall motion vector obtained from the temporal predicted motion vector calculation unit 114.

[0073] FIG. 3 is a diagram showing an outline of a processing flow of the video coding method according to the first embodiment.

[0074] When deriving a predicted motion vector candidate using temporal motion vector prediction, the co-located information determination unit 117 determines co-located information including a co-located use prohibition flag, a co-located reference direction flag, and a co-located reference picture index, etc., using a method described below (S11).

[0075] Next, the temporal motion vector predictor calculation unit 114 determines whether the co-located use prohibition flag is on (prohibited) (or the value of enable_temporal_mvp_flag is 0) (S12). If the determination result is true (Yes in S12), the temporal motion vector predictor calculation unit 114 reads the global motion vector from the global vector storage unit 116 and attaches the read global motion vector to header information such as a picture header (S13).

[0076] Next, the temporal motion vector predictor calculation unit 114 adds the global motion vector to the motion vector predictor candidates as an alternative vector for the temporal motion vector predictor (S14). In addition, the temporal motion vector predictor calculation unit 114 assigns a motion vector predictor index value to the motion vector predictor added to the candidates.

[0077] On the other hand, if the co-located use prohibition flag is off (No in S12) (or the value of enable_temporal_mvp_flag is 1), the temporal motion vector predictor calculation unit 114 reads colPic information including the reference motion vector of the co-located block from the colPic memory according to the co-located information, calculates a temporal motion vector predictor using the reference motion vector of the co-located block, and adds the calculated temporal motion vector predictor to the motion vector predictor candidates (S17). In addition, the temporal motion vector predictor calculation unit 114 assigns a value of the motion vector predictor index to the motion vector predictor added to the candidates.

[0078] Generally, when the value of predicted motion vector index is small, the amount of information required is small.On the other hand, when the value of predicted motion vector index is large, the amount of information required is large.Therefore, if small predicted motion vector index is assigned to the predicted motion vector that is likely to be the motion vector with higher accuracy, the coding efficiency will be improved.

[0079] Next, the inter prediction unit 110 performs inter prediction using the motion vector derived by motion estimation to generate a prediction block of the current block to be coded. Then, the subtraction unit 101, the orthogonal transformation unit 102, the quantization unit 103, and the variable length coding unit 118 code the current block to be coded using the prediction block generated by the inter prediction unit 110.

[0080] Furthermore, the inter prediction control unit 112 encodes the motion vector using a motion vector predictor candidate that is the motion vector predictor candidate with the smallest error from the motion vector among multiple motion vector predictor candidates. The inter prediction control unit 112, for example, calculates, as an error, a difference value between each of the multiple motion vector predictor candidates and the motion vector derived by motion estimation, and determines, from the calculated errors, the motion vector predictor candidate with the smallest error as the motion vector predictor to be used when encoding the motion vector.

[0081] Then, the inter prediction control unit 112 outputs the predicted motion vector index corresponding to the selected predicted motion vector and error information between the motion vector and the predicted motion vector to the variable length coding unit 118. The variable length coding unit 118 performs variable length coding on the predicted motion vector index and error information acquired from the inter prediction control unit 112, and includes the variable length coded predicted motion vector index and error information in a bitstream (S15).

[0082] Next, the inter prediction control unit 112 stores colPic information including the motion vectors and the like used in the inter prediction in the colPic memory 115. The colPic memory 115 stores the motion vectors of the reference pictures, the reference picture index values, the prediction directions, and the like in order to calculate the temporal prediction motion vector of the current block to be coded. The inter prediction control unit 112 also stores the motion vectors and the like used in the inter prediction in the global vector storage unit 116 (S16).

[0083] FIG. 4 shows examples of motion vector predictor candidates. Motion vector A (MV_A) is the motion vector of adjacent block A located to the left of the current block to be coded. Motion vector B (MV_B) is the motion vector of adjacent block B located above the current block to be coded. Motion vector C (MV_C) is the motion vector of adjacent block C located above and to the right of the current block to be coded. Median (MV_A, MV_B, MV_C) indicates the median value of motion vectors A, B, and C. Here, the median value is derived using, for example, the following equations 1 to 3.

[0084]

number

number

number

[0085] The value of the motion vector predictor index is 0 corresponding to the Median (MV_A, MV_B, MV_C), 1 corresponding to the motion vector A, 2 corresponding to the motion vector B, 3 corresponding to the motion vector C, and 4 corresponding to the temporal motion vector predictor (or alternative vector). Note that the method of assigning motion vector predictor indexes is not limited to this example.

[0086] 5 shows an example of a code table used when variable-length coding a motion vector predictor index. In the example of FIG. 5, codes with shorter code lengths are assigned in ascending order of the values ​​of the motion vector predictor index. In this way, a small motion vector predictor index is assigned to a motion vector predictor candidate that is likely to have good prediction accuracy, thereby improving coding efficiency.

[0087] Figure 6 is a diagram showing a flow of determining a motion vector predictor candidate in the inter prediction control unit 112. According to the flow shown in Figure 6, the inter prediction control unit 112 determines the motion vector predictor candidate having the smallest error with respect to the motion vector derived by motion estimation as the motion vector predictor to be used when encoding the motion vector. Then, error information between the motion vector and the motion vector predictor and a motion vector predictor index indicating the determined motion vector predictor are variable-length coded and included in the bitstream.

[0088] Specifically, first, the inter prediction control unit 112 initializes a motion vector predictor candidate index mvp_idx and a minimum motion vector error (S21). Next, the inter prediction control unit 112 compares the motion vector predictor candidate index mvp_idx with the number of motion vector predictor candidates (the number of records in the table shown in FIG. 4) (S22).

[0089] If mvp_idx<the number of motion vector predictor candidates (Yes in S22), the inter prediction control unit 112 calculates a motion vector error (error information) of the motion vector predictor candidate identified by the current mvp_idx value among the multiple motion vector predictor candidates (S23). For example, the inter prediction control unit 112 calculates the motion vector error by subtracting the motion vector predictor candidate with motion vector predictor index=0 in FIG. 4 from the motion vector used in encoding the current block.

[0090] Next, the inter prediction control unit 112 compares the motion vector error calculated in step S23 with the minimum motion vector error (S24). If the motion vector error is less than the minimum motion vector error (Yes in S24), the inter prediction control unit 112 sets (overwrites) the motion vector error calculated in step S23 as the minimum motion vector error, and sets (overwrites) the current mvp_idx as the predicted motion vector index (S25). On the other hand, if the motion vector error is greater than or equal to the minimum motion vector error (No in S24), step S25 is skipped.

[0091] Then, the inter prediction control unit 112 increments mvp_idx by 1 (S26) and repeatedly executes the above processes (steps S22 to S26) the number of times equal to the number of motion vector predictor candidates. Then, when mvp_idx equals the number of motion vector predictor candidates (No in S22), the inter prediction control unit 112 outputs the values ​​set in the minimum motion vector error and the motion vector predictor index to the variable length coding unit 118, and ends the processing in FIG. 6 (S27).

[0092] Fig. 7 is a conceptual diagram showing the process of reading and writing data from and to the colPic memory 115 and the global vector storage unit 116 shown in Fig. 2. In Fig. 7, a motion vector mvCol1 in prediction direction 1 and a motion vector mvCol2 in prediction direction 2 of a co-located block included in the co-located picture colPic are stored in the colPic memory 115 and the global vector storage unit 116.

[0093] Here, when the block to be coded is coded, colPic information including the motion vector stored in the colPic memory 115, or the global motion vector in the global vector storage unit 116, is read out in accordance with the co-located use prohibition flag and added to the predicted motion vector candidates.

[0094] The predicted motion vector candidate is used to encode the motion vector of the current block to be encoded.In this embodiment 1, an example will be described in which the prediction direction 1 is forward reference and the prediction direction 2 is backward reference, but the prediction direction 1 may be backward reference and the prediction direction 2 may be forward reference, or both the prediction direction 1 and the prediction direction 2 may be forward reference or backward reference.In this embodiment 1, an example will be shown in which the prediction direction 1 is prediction using the reference picture list L0 and the prediction direction 2 is prediction using the reference picture list L1.

[0095] The global vector storage unit 116 stores a global motion vector calculated from motion vectors of a plurality of blocks to be coded that constitute a picture to be coded. For example, the global motion vector is an average value for each prediction direction of motion vectors during inter-prediction coding of the entire picture to be coded. Note that, in the first embodiment, an example has been shown in which the average value of the motion vectors of a plurality of blocks to be coded that constitute a picture to be coded is used as the global vector, but this is not necessarily limited to this.

[0096] For example, the global motion vector may be an intermediate value or a weighted average value of motion vectors of the blocks to be coded that constitute the current picture when inter-predictive coding is performed. Alternatively, the global motion vector may be the motion vector that appears most frequently among the motion vectors of the blocks to be coded that constitute the current picture when inter-predictive coding is performed. Alternatively, the global motion vector may be the motion vector that references the closest picture in display order among the motion vectors of the blocks to be coded that constitute the current picture when inter-predictive coding is performed.

[0097] Fig. 8A is a diagram showing a detailed processing flow of step S11 in Fig. 3. Fig. 8A will be described below.

[0098] First, the co-located information determination unit 117 determines whether or not to perform temporal motion vector prediction using a co-located block on a current picture to be coded (S31). Then, the co-located information determination unit 117 generates a co-located use prohibition flag (or enable_temporal_mvp_flag) for each picture, which indicates whether or not use of the co-located block is permitted (temporal motion vector prediction), and outputs the generated co-located use prohibition flag to the variable-length coding unit 118.

[0099] For example, in order to suppress propagation of decoding errors due to temporal motion vector prediction during streaming distribution, it is conceivable to turn on the co-located use prohibition flag at certain intervals. One example of how to achieve this is to provide a counter that counts the number of coded pictures to be coded, and if the number of coded pictures is smaller than a certain threshold, set the co-located use prohibition flag to off, and if the number of coded pictures becomes equal to or greater than the threshold, set the co-located use prohibition flag to on and reset the counter to 0.

[0100] Also, for example, a method of suppressing decoding error propagation can be considered, in which the co-located use prohibition flag is set to ON for pictures that can be referenced (for example, P pictures and B pictures referenced by other pictures), and the co-located use prohibition flag is set to OFF for non-reference pictures that cannot be referenced (for example, B pictures that are not referenced by other pictures). In this way, by setting the co-located use prohibition flag to ON for pictures that are referenced by other pictures, it is possible to effectively suppress propagation of decoding errors.

[0101] Next, the co-located information determination unit 117 determines either a forward reference block or a backward reference block as the co-located block (S32). For example, the co-located information determination unit 117 selects, as the co-located block, one of a co-located block (forward reference block) included in a forward reference picture and a co-located block (backward reference block) included in a backward reference picture, whichever is included in a picture that is closer to the current picture in display order to be coded. Then, the co-located information determination unit 117 generates, for each picture (or slice), a co-located reference direction flag indicating whether the co-located block is a forward reference block or a backward reference block, and a co-located reference picture index for specifying the co-located picture, and outputs the generated co-located reference direction flag and co-located reference picture index to the variable-length coding unit 118.

[0102] Figure 8B shows an example of a B-picture referenced by other pictures. In the example of Figure 8B, a reference structure having multiple layers is defined. The first picture in the stream is set to be an I-picture, and pictures other than the first I-picture are set to B-pictures. Furthermore, a picture belonging to a higher level layer among multiple layers references a picture belonging to a layer at the same level or a picture belonging to a layer at a lower level.

[0103] For example, in FIG. 8B, picture B1 belonging to layer 3 references picture I0 belonging to layer 0 and picture Br2 belonging to layer 2. Furthermore, picture Bf8 belonging to layer 0, which has the lowest level, references picture I0 belonging to the same layer. Here, pictures belonging to layer 0, which has the lowest level, reference only pictures that precede them in display order. In such a reference structure, it may be possible to turn on the co-located prohibition flag for pictures belonging to layer 0 that are likely to be referenced by other pictures.

[0104] Fig. 9 is a detailed processing flow of step S17 in Fig. 3. Fig. 9 will be described below.

[0105] First, the temporal motion vector predictor calculation unit 114 reads colPic information including a reference motion vector for prediction direction 1 and a reference motion vector for prediction direction 2 from the colPic memory 115 (S41). Next, the temporal motion vector predictor calculation unit 114 determines whether the co-located block included in the colPic information has two or more motion vectors (S42). That is, the temporal motion vector predictor calculation unit 114 determines whether the co-located block has a forward reference motion vector (mvL0) and a backward reference motion vector (mvL1).

[0106] If it is determined that the co-located block has two or more motion vectors (Yes in S42), the temporal motion vector predictor calculation unit 114 determines whether the co-located block is a backward reference block (S43). That is, the temporal motion vector predictor calculation unit 114 determines whether the picture including the co-located block is located after the current picture in display order.

[0107] Next, when it is determined that the co-located block is a backward reference block (Yes in S43), the temporal motion vector predictor calculation unit 114 derives a temporal motion vector predictor by temporal motion vector prediction using the forward reference motion vector of the co-located block (motion vector mvL0 for the reference picture in the reference picture list L0) (S44).Then, the temporal motion vector predictor calculation unit 114 adds the temporal motion vector predictor calculated in step S44 to the motion vector predictor candidates (S45).

[0108] On the other hand, if it is determined that the co-located block is a forward reference block (No in S43), the temporal predicted motion vector calculation unit 114 derives a temporal predicted motion vector by temporal motion vector prediction using the backward reference motion vector of the co-located block (motion vector mvL1 for the reference picture in the reference picture list L1) (S46), and adds the derived temporal predicted motion vector to the predicted motion vector candidates (S45).

[0109] On the other hand, when it is determined that the co-located block has only one of a forward reference motion vector and a backward reference motion vector (No in S42), the temporal motion vector predictor calculation unit 114 determines whether the co-located block has a forward reference motion vector (S47).When it is determined that the co-located block has a forward reference motion vector (Yes in S47), the temporal motion vector predictor calculation unit 114 derives a temporal motion vector predictor of the block to be coded using the forward reference motion vector of the co-located block (S48), and adds the derived temporal motion vector predictor to the motion vector predictor candidates (S45).

[0110] On the other hand, if it is determined that the co-located block does not have a forward reference motion vector (No in S47), the temporal motion vector predictor calculation unit 114 determines whether the co-located block has a backward reference motion vector (S49).If it is determined that the co-located block has a backward reference motion vector (Yes in S49), the temporal motion vector predictor calculation unit 114 derives a temporal motion vector predictor of the current block to be coded using the backward reference motion vector (S50), and adds the derived temporal motion vector predictor to candidate motion vector predictors (S45).

[0111] On the other hand, when it is determined that the co-located block does not have a backward reference motion vector (No in S49), the temporal motion vector predictor calculation unit 114 does not add the temporal motion vector predictor to the motion vector predictor candidates, and ends the processing in Fig. 9 (S51). Alternatively, instead of the processing of step S51, the temporal motion vector predictor calculation unit 114 may add a motion vector with a motion amount of 0 as the temporal motion vector predictor of the co-located block to the motion vector predictor candidates.

[0112] 9, the temporal motion vector predictor calculation unit 114 determines whether or not the co-located block has a forward reference motion vector in step S47, and determines whether or not the co-located block has a backward reference motion vector in step S49, but the flow is not limited to this. For example, the temporal motion vector predictor calculation unit 114 may first determine whether or not the co-located block has a backward reference motion vector, and then determine whether or not the co-located block has a forward reference motion vector.

[0113] Fig. 10 is a detailed processing flow of steps S13 and S14 in Fig. 3. Fig. 10 will be described below.

[0114] First, the temporal motion vector predictor calculation unit 114 reads at least one of the global motion vector information including the global motion vector of prediction direction 1 and the global motion vector of prediction direction 2 from the global vector storage unit 116 (S61). Next, the temporal motion vector predictor calculation unit 114 determines whether the global motion vector information has two or more motion vectors (S62). That is, the temporal motion vector predictor calculation unit 114 determines whether the forward reference motion vector (mvL0) and the backward reference motion vector (mvL1) are included in the global motion vector information.

[0115] If it is determined that the global motion vector information has two or more motion vectors (Yes in S62), the temporal motion vector predictor calculation unit 114 determines whether the co-located reference direction is a backward reference block (S63).If it is determined that the co-located reference direction is a backward reference block (Yes in S63), the temporal motion vector predictor calculation unit 114 selects a forward reference motion vector included in the global motion vector information (S64).

[0116] Then, the temporal motion vector predictor calculation unit 114 adds the selected global motion vector to header information such as a picture header (outputs it to the variable-length coding unit 118), and adds it to the motion vector predictor candidates for the block to be coded (S65). Note that the temporal motion vector predictor calculation unit 114 adds, to the header information, information specifying the reference pictures referenced by the selected global motion vector (more specifically, referenced by multiple motion vectors used to calculate the global motion vector). This information is used in the scaling process described later with reference to Figures 11A to 12B.

[0117] On the other hand, if it is determined that the co-located reference direction is a forward reference block (No in S63), the temporal motion vector predictor calculation unit 114 selects a backward reference motion vector included in the global motion vector information (S66).Then, the temporal motion vector predictor calculation unit 114 adds the selected global motion vector to header information such as a picture header, and adds it to the motion vector predictor candidates of the current block to be coded (S65).

[0118] Also, if it is determined that the overall motion vector information has only one of a forward reference motion vector and a backward reference motion vector (No in S62), the temporal predicted motion vector calculation unit 114 determines whether the overall motion vector information has a forward reference motion vector (S67).

[0119] If it is determined that the global motion vector information has a forward reference motion vector (Yes in S67), the temporal motion vector predictor calculation unit 114 selects the forward reference motion vector included in the global motion vector information (S68).Then, the temporal motion vector predictor calculation unit 114 adds the selected global motion vector to header information such as a picture header, and also adds it to the candidate motion vector predictors of the current block to be coded (S65).

[0120] On the other hand, if it is determined that the global motion vector information does not have a forward reference motion vector (No in S67), the temporal motion vector predictor calculation unit 114 determines whether or not the global motion vector information has a backward reference motion vector (S69). If it is determined that the global motion vector information has a backward reference motion vector (Yes in S69), the temporal motion vector predictor calculation unit 114 selects a backward reference motion vector included in the global motion vector information (S70). Then, the temporal motion vector predictor calculation unit 114 adds the selected global motion vector to header information such as a picture header, and adds it to the motion vector predictor candidates for the current block to be coded (S65).

[0121] On the other hand, when it is determined that the global motion vector information does not have a backward reference motion vector (No in S69), the temporal motion vector predictor calculation unit 114 does not add the temporal motion vector predictor to the motion vector predictor candidates, or sets the global motion vector to 0 (S71). Then, the temporal motion vector predictor calculation unit 114 adds the set global motion vector to header information such as a picture header, and adds it to the motion vector predictor candidates of the current block to be coded (S65).

[0122] 10, the temporal motion vector predictor calculation unit 114 determines whether or not the global motion vector has a forward reference motion vector in step S67, and determines whether or not the global motion vector has a backward reference motion vector in step S69, but the flow is not limited to this. For example, the temporal motion vector predictor calculation unit 114 may first determine whether or not the global motion vector has a backward reference motion vector, and then determine whether or not the global motion vector has a forward reference motion vector.

[0123] 10, steps S63 to S66 have been described as an example in which the temporal motion vector predictor calculation unit 114 determines whether to select the global motion vector mvL0 or mvL1 based on the co-located reference direction flag, but this is not limiting. For example, the temporal motion vector predictor calculation unit 114 may select the global motion vector mvL0 as the candidate motion vector predictor for the reference picture list L0, and may select the global motion vector mvL1 as the candidate motion vector predictor for the reference picture list L1. This eliminates the need to add a co-located reference direction flag to the header when the global motion vector is used, thereby further improving coding efficiency.

[0124] Next, the scaling method when adding a temporal motion vector predictor to a motion vector predictor candidate will be described in detail. Note that the scaling method when adding a global motion vector to a motion vector predictor candidate is the same except that the global motion vector is used as input instead of the motion vector of the co-located block, so the description will be omitted.

[0125] Figure 11A shows that when co-located block is backward reference block and has forward reference motion vector and backward reference motion vector, forward reference motion vector is used by temporal motion vector prediction to derive predicted motion vector candidate (temporal predicted motion vector).Specifically, forward reference motion vector is used by the following formula 4 to derive predicted motion vector candidate (TemporalMV).

[0126] TemporalMV=mvL0×(B2-B0) / (B4-B0) (Formula 4)

[0127] Here, (B2-B0) indicates the time difference information between the display times of pictures B2 and B0. Similarly, (B4-B0) indicates the time difference information between the display times of pictures B4 and B0.

[0128] 11B shows a method for deriving a motion vector predictor candidate (temporal motion vector predictor) by temporal motion vector prediction using a backward reference motion vector. Specifically, a motion vector predictor candidate is derived by the following Equation 5 using the backward reference motion vector.

[0129] TemporalMV=mvL1×(B2-B0) / (B4-B8) ···(Formula 5)

[0130] Figure 12A shows that when co-located block is forward reference block, and has forward reference motion vector and backward reference motion vector, by using backward reference motion vector, temporal motion vector prediction, derive predicted motion vector candidate (temporal predicted motion vector).Specifically, by using backward reference motion vector, derive predicted motion vector candidate by the following formula 6:

[0131] TemporalMV=mvL1×(B6-B8) / (B4-B8) (Equation 6)

[0132] 12B shows a method for deriving a motion vector predictor candidate (temporal motion vector predictor) by temporal motion vector prediction using a forward reference motion vector. A motion vector predictor candidate is derived by the following Equation 7 using a backward reference motion vector.

[0133] TemporalMV=mvL0×(B6-B8) / (B4-B0) (Equation 7)

[0134] As described above, in the first embodiment, the video coding device 100 turns off temporal motion vector prediction using motion vectors for each unit of coding processing of reference pictures at regular intervals, and instead adds the overall motion vector of the reference pictures to the header information and codes the motion vector of the current picture using the scaled overall vector. This makes it possible to prevent the propagation of decoding errors while suppressing a decrease in coding efficiency.

[0135] More specifically, when the co-located use prohibition flag is on, the video encoding device 100 adds the global vector read from the global vector storage unit 116 to the motion vector predictor candidate for the current block to be encoded and attaches it to header information such as a picture header. This allows the video decoding device to decode the bitstream without decoding errors even if a reference picture is lost during decoding. In this way, error propagation is suppressed.

[0136] Furthermore, when the co-located use prohibition flag is off, the video encoding device 100 can select an optimal predicted motion vector for the current block to be encoded according to the co-located reference direction flag, thereby improving compression efficiency. In particular, when the co-located block is a forward reference block, the video encoding device 100 can reduce prediction errors by using a backward reference motion vector. The backward reference motion vector is a motion vector that points from a picture including the co-located block to a reference picture in the direction of the picture including the current block to be encoded, and is highly likely to be close to the optimal motion vector. This reduces prediction errors.

[0137] On the other hand, the forward reference motion vector is a motion vector in the opposite direction from the picture including the co-located block to the picture including the current block to be coded, and is less likely to be close to the optimal motion vector. This results in a large prediction error. Similarly, when the co-located block is a backward reference block, the forward reference motion vector is more likely to be close to the optimal motion vector. This results in a small prediction error.

[0138] In this embodiment 1, when a co-located block has two or more motion vectors, the video encoding device 100 switches the motion vector of the co-located block used to calculate the temporal predicted motion vector of the block to be encoded depending on whether the co-located block is a backward reference block or a forward reference block, but this is not limited to this.

[0139] For example, the video encoding device 100 may calculate a temporal motion vector predictor using a motion vector that references a reference picture that is temporally close to the picture including the co-located block (a motion vector with a short temporal distance). Here, the temporal distance is determined according to, for example, the number of pictures between the picture including the co-located block and the reference picture referenced by the co-located block in display order.

[0140] Furthermore, in the first embodiment, when a co-located block has two or more motion vectors, the video encoding device 100 switches the motion vector of the co-located block used to calculate the temporal predicted motion vector of the encoding target block depending on whether the co-located block is a backward reference block or a forward reference block, but is not limited to this. For example, the video encoding device 100 may calculate the temporal predicted motion vector using the motion vector with the smaller magnitude of the two motion vectors of the co-located block. Here, the magnitude of the motion vector means the absolute value of the motion vector, etc.

[0141] Furthermore, in the first embodiment, when the co-located use prohibition flag is on, the video encoding device 100 adds the global vector read from the global vector storage unit 116 to the motion vector predictor candidate as an alternative vector for the temporal motion vector predictor. However, this is not limited to this. For example, the video encoding device 100 may always add a motion vector with a value of 0 as the global motion vector to the motion vector predictor candidate (i.e., add a motion vector with a motion amount of 0 to the motion vector predictor candidate as an alternative vector). In this case, the video encoding device 100 may not attach the global motion vector to header information or the like. Furthermore, when the co-located use prohibition flag is on, the video encoding device 100 may always not add the temporal motion vector predictor to the motion vector predictor candidate. By not adding the temporal motion vector predictor to the motion vector predictor candidate, it is possible to improve encoding efficiency.

[0142] Furthermore, in the first embodiment, the video coding device 100 adds the co-located use prohibition flag to all pictures, but may add the flag only to specific pictures. For example, the video coding device 100 adds the co-located use prohibition flag only to pictures referenced by other pictures (P pictures, B pictures referenced by other pictures, and pictures belonging to the lowest layer in a reference structure having multiple layers), and does not add the co-located use prohibition flag to pictures not referenced by other pictures. In this way, by adding the co-located use prohibition flag only to specific pictures, the video coding device 100 can improve coding efficiency and suppress decoding error propagation.

[0143] In the first embodiment, the video encoding device 100 adds a co-located use prohibition flag to each picture, but the co-located use prohibition flag may be added to each slice made up of multiple blocks. Adding a co-located use prohibition flag to each slice can improve the prediction accuracy of the global vector.

[0144] Furthermore, in the first embodiment, the video encoding device 100 adds a co-located use prohibition flag to all pictures, but may determine not to add a temporal motion vector predictor to motion vector predictor candidates based on the picture type without adding a co-located use prohibition flag. For example, for a picture referenced by other pictures (a P picture, a B picture referenced by other pictures, or a picture belonging to the lowest layer in a reference structure having multiple layers), the video encoding device 100 may add a global vector to motion vector predictor candidates without adding a temporal motion vector predictor. In this way, by determining whether to add a temporal motion vector predictor to motion vector predictor candidates based on the picture type, the co-located use prohibition flag can be omitted, thereby enabling improved encoding efficiency.

[0145] Furthermore, when the temporal motion vector predictor is not included in the motion vector predictor candidates, the video encoding device 100 does not include an unnecessary flag in the bitstream, thereby improving encoding efficiency. A specific example will be described with reference to Fig. 13. Fig. 13 is a detailed flow chart of S11 in Fig. 3, showing a modified example of the flow for determining co-located information.

[0146] First, the video encoding device 100 determines the value of the co-located use prohibition flag in the same manner as in Fig. 8A, and encodes the co-located use prohibition flag indicating the determined value (S101). Note that here, an example will be described in which a flag (enable_temporal_mvp_flag) indicating whether or not to permit a temporal motion vector predictor using information on the co-located block is used as the co-located use prohibition flag.

[0147] Next, the video encoding device 100 determines whether enable_teporal_mvp_flag is 1 (S102). If enable_teporal_mvp_flag is 1 (Yes in S102), the video encoding device 100 determines the values ​​of the co-located reference direction flag and the co-located reference picture index and encodes them respectively in the same manner as in S32 of Fig. 8A (S103 and S104).

[0148] On the other hand, when enable_teporal_mvp_flag is 0 (No in S102), the video encoding device 100 does not encode the co-located reference direction flag and the co-located reference picture index. In this way, when the temporal motion vector predictor is not included in the motion vector predictor candidates (when enable_teporal_mvp_flag is 0), the video encoding device 100 does not add the unnecessary co-located reference direction flag and the co-located reference picture index to the bitstream. This allows the video encoding device 100 to improve encoding efficiency.

[0149] As described above, the video encoding device 100 according to this embodiment performs the video encoding process shown in FIG.

[0150] The video encoding device 100 performs inter-prediction encoding of a current block included in a current picture using a motion vector.

[0151] First, the video encoding device 100 generates a first flag (co-located use prohibition flag) indicating whether or not temporal motion vector prediction using a temporal motion vector predictor, which is a motion vector of a block included in a coded picture other than the current picture to be coded, is used (whether temporal motion vector prediction is permitted or prohibited), and encodes the generated first flag (S111).The video encoding device 100 also adds the encoded first flag to a bitstream.

[0152] Next, the video encoding device 100 determines whether the first flag indicates that temporal motion vector prediction is used (enabled) (S112).

[0153] If the first flag indicates that temporal motion vector prediction is used (enabled) (Yes in S112), the video encoding device 100 generates first parameters for calculating a temporal motion vector predictor and encodes the generated first parameters (S113).The video encoding device 100 then adds the encoded first parameters to a bitstream.

[0154] Specifically, the first parameters include parameters for identifying a coded picture (co-located picture) that is a reference target of the motion vector. More specifically, the first parameters include a reference picture index (collocated_ref_idx) for identifying the coded picture (co-located picture) from among multiple pictures indicated in a reference picture list used to code the current picture. The first parameters also include a flag (collocated_from_l0_flag) indicating which reference picture list is to be used to identify the coded picture (co-located picture) from among multiple reference picture lists used to code the current picture.

[0155] Here, each reference picture list indicates multiple reference pictures (coded pictures). In each reference picture list, the multiple reference pictures are indicated by reference picture indices. The video coding device 100 selects a reference picture list indicated by the flag (collocated_from_l0_flag) from the multiple reference picture lists, and identifies a picture having the reference picture index (collocated_ref_idx) from the multiple reference pictures included in the selected reference picture list as the coded picture (co-located picture).

[0156] Next, the video encoding device 100 generates a temporal motion vector predictor using the first parameter, and derives a plurality of first motion vector predictor candidates including the generated temporal motion vector predictor (S114).

[0157] Next, the video encoding device 100 uses one of the multiple first motion vector predictor candidates to encode a motion vector used in inter-prediction encoding of the current block (S115). Specifically, the video encoding device 100 selects, from the multiple first motion vector predictor candidates, a first motion vector predictor candidate that has the smallest difference from the motion vector used in inter-prediction encoding of the current block, and encodes the motion vector using the selected first motion vector predictor candidate. More specifically, the video encoding device 100 encodes the difference between the selected first motion vector predictor candidate and the motion vector.

[0158] Furthermore, the video encoding device 100 encodes the current block by inter-coding using the motion vector, and then adds the encoded motion vector (difference) and the current block to a bitstream.

[0159] On the other hand, if the first flag indicates that temporal motion vector prediction is not used (prohibited) (No in S112), the video encoding device 100 does not encode the first parameter (S116). That is, the video encoding device 100 does not generate the first flag. Furthermore, the video encoding device 100 does not add the first flag to the bitstream.

[0160] Furthermore, the video encoding device 100 derives a plurality of second motion vector predictor candidates that do not include a temporal motion vector predictor (S117).

[0161] Next, the video encoding device 100 encodes a motion vector to be used for inter-predictive encoding of the current block by using one of the second motion vector predictor candidates (S118). Note that a specific example of this process is the same as the process in the case where the first motion vector predictor candidates are replaced with the second motion vector predictor candidates in step S115 above.

[0162] Note that step S111 above is executed by a flag coding unit included in the video coding device 100. Steps S113 and S116 are executed by a parameter coding unit included in the video coding device 100. Steps S114 and S117 are executed by a candidate derivation unit included in the video coding device 100. Steps S115 and S118 are executed by a motion vector coding unit included in the video coding device 100.

[0163] Here, the function of the flag coding unit is realized by, for example, the co-located information determination unit 117 and the variable length coding unit 118 shown in Fig. 2. Furthermore, the function of the parameter coding unit is realized by the co-located information determination unit 117 and the variable length coding unit 118 shown in Fig. 2. The function of the candidate derivation unit is realized by the inter prediction control unit 112 and the temporal predicted motion vector calculation unit 114 shown in Fig. 2. The function of the motion vector coding unit is realized by the subtraction unit 101, the inter prediction unit 110, the inter prediction control unit 112, the variable length coding unit 118, etc. shown in Fig. 2.

[0164] Furthermore, the first flag (co-located prohibition flag) and the first parameters (collocated_from_l0_flag and collocated_ref_idx) are generated and coded for each picture or slice. That is, the first flag and the first parameters are included in the picture header or slice header of the bit stream. Note that the first flag and the first parameters may be generated for different units (pictures or slices). For example, the first flag may be generated for each picture, and the first parameters may be generated for each slice.

[0165] Furthermore, at least one of the first flag and the first parameter may be generated and coded for each set of pictures, i.e., at least one of the first flag and the first parameter may be included in a PPS (Picture Parameter Set) or an SPS (Sequence Parameter Set) of the bitstream.

[0166] Furthermore, the first flag may be hierarchically included in multiple units among a slice, a picture, and multiple pictures (sequences). For example, the video encoding device 100 generates, for each picture, a first flag indicating whether temporal motion vector prediction is used for the picture. Furthermore, if the first flag indicates that temporal motion vector prediction is used, the video encoding device 100 further generates, for each slice included in the picture, a second flag indicating whether temporal motion vector prediction is used for the slice. On the other hand, if the first flag indicates that temporal motion vector prediction is not used, the video encoding device 100 does not generate a second flag for each slice. Note that the video encoding device 100 may generate a second flag for each slice included in the picture only if the first flag indicates that temporal motion vector prediction is not used. Alternatively, the video encoding device 100 may generate the first flag for each multiple pictures and generate the second flag for each picture or each slice.

[0167] (Embodiment 2) In this embodiment, a video decoding device 200 that decodes a bitstream generated by the video encoding device 100 will be described.

[0168] FIG. 15 is a block diagram showing a configuration of a video decoding device 200 that uses the video decoding method according to the second embodiment.

[0169] In this second embodiment, a block included in a picture located before the current picture in display order (a reference picture identified by the reference picture list L0) is called a forward reference block, and a block included in a picture located after the current picture in display order (a reference picture identified by the reference picture list L1) is called a backward reference block.

[0170] As shown in FIG. 15, the video decoding device 200 includes a variable length decoding unit 201, an inverse quantization unit 202, an inverse orthogonal transformation unit 203, an addition unit 204, a block memory 205, a frame memory 206, an intra prediction unit 207, an inter prediction unit 208, a switch 209, an inter prediction control unit 210, a temporal prediction motion vector calculation unit 211, and a colPic memory 212.

[0171] The variable length decoding unit 201 variable-length decodes the input bitstream to obtain picture type information, a predicted motion vector index, a co-located use prohibition flag (or enable_temporal_mvp_flag), a co-located reference direction flag (collocated_from_l0_flag), a co-located reference picture index (collocated_ref_idx), a global motion vector, and a quantization coefficient. The variable length decoding unit 201 outputs the picture type information to the switch 209 and the inter prediction control unit 210, outputs the predicted motion vector index to the inter prediction control unit 210, outputs the co-located use prohibition flag (or enable_temporal_mvp_flag), the co-located reference direction flag (collocated_from_l0_flag), the co-located reference picture index (collocated_ref_idx), and the global motion vector to the temporal predicted motion vector calculation unit 211, and outputs the quantization coefficient to the inverse quantization unit 202.

[0172] The inverse quantization unit 202 restores transform coefficients by inverse quantizing the quantized coefficients obtained from the variable length decoding unit 201, and outputs the restored transform coefficients to the inverse orthogonal transformation unit 203. The inverse orthogonal transformation unit 203 restores residual blocks by transforming the restored transform coefficients obtained from the inverse quantization unit 202 from the frequency domain to the image domain, and outputs the restored residual blocks to the addition unit 204.

[0173] The adder 204 reconstructs a decoded block by adding the reconstructed residual block obtained from the inverse orthogonal transformer 203 and the predicted block obtained from the switch 209. The adder 204 then outputs a decoded image sequence including this reconstructed decoded block to the outside of the device, and also stores the decoded image sequence in the block memory 205 and the frame memory 206.

[0174] The block memory 205 stores the decoded image sequence obtained from the adder 204 in units of blocks. The frame memory 206 stores the decoded image sequence obtained from the adder 204 in units of frames.

[0175] The intra prediction unit 207 generates a predicted block of the block to be decoded by performing intra prediction using the sequence of decoded images in block units stored in the block memory 205, and outputs the generated predicted block to the switch 209. The inter prediction unit 208 generates a predicted block of the block to be decoded by performing inter prediction using the sequence of decoded images in frame units stored in the frame memory 206, and outputs the generated predicted block to the switch 209. The switch 209 outputs the predicted block generated by the intra prediction unit 207 or the predicted block generated by the inter prediction unit 208 to the adder 204.

[0176] When the co-located use prohibition flag acquired from the variable length decoding unit 201 is off, the temporal motion vector predictor calculation unit 211 derives a motion vector predictor candidate (temporal motion vector predictor) for temporal motion vector prediction using colPic information such as the motion vector of the co-located block stored in the colPic memory 212. On the other hand, when the co-located use prohibition flag is on, the temporal motion vector predictor calculation unit 211 adds the overall motion vector acquired from the variable length decoding unit 201 to the motion vector predictor candidate.

[0177] Furthermore, the temporal motion vector predictor calculation unit 211 assigns a motion vector predictor index to the motion vector predictor added to the candidates. Then, the temporal motion vector predictor calculation unit 211 outputs the motion vector predictor and the motion vector predictor index to the inter prediction control unit 210.

[0178] In addition, if the co-located block does not have a motion vector, the temporal predicted motion vector calculation unit 211 may stop deriving a motion vector by temporal motion vector prediction, or may add a motion vector with a motion amount of 0 to the predicted motion vector candidates.

[0179] The inter prediction control unit 210 identifies a predicted motion vector corresponding to the predicted motion vector index acquired from the variable length decoding unit 201 from among a plurality of candidate predicted motion vectors. The inter prediction control unit 210 then calculates a motion vector to be used during inter prediction by adding error information between the motion vector and the predicted motion vector to the identified predicted motion vector. The inter prediction control unit 210 also stores colPic information including the motion vector of the current block to be decoded in the colPic memory 212.

[0180] FIG. 16 shows an outline of the processing flow of the video decoding method according to the second embodiment.

[0181] First, the variable length decoding unit 201 decodes the co-located use prohibition flag on a picture-by-picture basis (S81). Next, the variable length decoding unit 201 determines whether the co-located use prohibition flag is off (S82). If the co-located use prohibition flag is off (Yes in S82), the variable length decoding unit 201 decodes the co-located reference direction flag and the co-located reference picture index (collocated_ref_idx) on a picture-by-picture basis (S83). Then, the variable length decoding unit 201 outputs the decoded co-located use prohibition flag, co-located reference direction flag, and co-located reference picture index to the temporal motion vector predictor calculation unit 211.

[0182] Next, in a manner similar to that of Figure 9, the temporal predicted motion vector calculation unit 211 reads colPic information including the reference motion vector of the co-located block from the colPic memory 212 according to the co-located information, generates a temporal predicted motion vector using the reference motion vector of the co-located block, and adds the generated temporal predicted motion vector to the predicted motion vector candidates (S84).

[0183] On the other hand, if the co-located use prohibition flag is on (No in S82), the temporal predicted motion vector calculation unit 211 obtains the overall motion vector stored in header information such as a picture header from the variable length decoding unit 201, and adds the obtained overall motion vector to the predicted motion vector candidates (S87).

[0184] Next, the inter prediction control unit 210 selects a motion vector predictor corresponding to the decoded motion vector predictor index from among the multiple motion vector predictor candidates (S85). The inter prediction control unit 210 also derives a motion vector by adding prediction error information to the selected motion vector predictor, and outputs the derived motion vector to the inter prediction unit 208. The inter prediction unit 208 then generates a prediction block for the current block by inter prediction using the derived motion vector.

[0185] Next, the inter prediction control unit 210 saves colPic information including the motion vector used in the inter prediction in the colPic memory 212 (S86). The colPic memory 212 stores the motion vector of the reference picture, the reference picture index value, the prediction direction, and the like, in order to calculate the temporal motion vector predictor of the current block.

[0186] Note that when a reference block has two or more reference motion vectors, the method of selecting a reference motion vector for calculating a temporal motion vector predictor is not limited to the method based on the co-located block reference direction flag. For example, the video decoding device 200 may calculate the temporal distance of the reference motion vectors and use a reference motion vector with a shorter temporal distance. Here, the temporal distance is calculated based on the number of pictures between a reference picture including the reference block and a picture referenced by the reference picture in display time. Also, for example, the video decoding device 200 may calculate the magnitude of the reference motion vector and use a motion vector derived using a reference motion vector with a smaller magnitude as the temporal motion vector predictor.

[0187] Fig. 17 shows an example of the syntax of a bitstream in the video decoding method according to Embodiment 2. In Fig. 17, forbid_collocated_flag represents a co-located use prohibition flag, tmv_x represents the horizontal component of the global motion vector, tmv_y represents the vertical component of the global motion vector, and collocated_from_l0_flag represents a co-located reference direction flag.

[0188] As shown in FIG. 17, when the co-located use prohibition flag (forbid_collocated_flag) is 1, the global motion vector (tmv_x, tmv_y) is attached to the bitstream and added to the motion vector predictor candidates.

[0189] Furthermore, when the co-located use prohibition flag (forbid_collocated_flag) is 0, a co-located reference direction flag (collocated_from_l0_flag) is attached to the bitstream. Then, a co-located block is determined according to the co-located reference direction flag, and a temporal predicted motion vector is calculated using the reference motion vector of the co-located block. Note that here, when collocated_from_l0_flag is 1, it indicates that the co-located block is a forward reference block, and when it is 0, it indicates that the co-located block is a backward reference block, but this is not necessarily limited to this.

[0190] In the second embodiment, the video decoding device 200 uses the global motion vector decoded from header information or the like when the co-located use prohibition flag is on, but may always add a global motion vector of value 0 to motion vector predictor candidates in accordance with the encoding method. In this case, since the global motion vector is not attached to header information or the like, its decoding process is omitted. Furthermore, the video decoding device 200 may always avoid adding a temporal motion vector predictor to motion vector predictor candidates when the co-located use prohibition flag is on.

[0191] As described above, in the first and second embodiments, the video encoding device 100 periodically turns off temporal motion vector prediction using the motion vector for each unit of coding processing of the reference picture, and instead adds the overall motion vector of the reference picture to the header information. The video encoding device 100 then uses this to encode the motion vector of the current picture, thereby generating a bitstream that prevents the propagation of decoding errors while suppressing a decrease in coding efficiency. Furthermore, the video decoding device 200 can appropriately decode the bitstream generated in this manner.

[0192] More specifically, when the co-located use prohibition flag is on, the video encoding device 100 adds the global vector read from the global vector storage unit 116 to the motion vector predictor candidates for the current block to be encoded, and also attaches it to header information such as a picture header. This allows the video decoding device 200 to decode the bitstream without decoding errors, even if a reference picture is lost during decoding. In this way, the video decoding device 200 can appropriately decode a bitstream with error propagation suppressed.

[0193] In addition, when the co-located use prohibition flag is off, the video decoding device 200 becomes able to properly decode a bitstream in which the optimal predicted motion vector for the block to be coded is selected according to the co-located reference direction flag.

[0194] In the first and second embodiments, the video encoding device 100 uses the global vector read from the global vector storage unit 116 when the co-located use prohibition flag is on, but may always add a global motion vector with a value of 0 to the motion vector predictor candidates. Also, the video encoding device 100 may always avoid adding a temporal motion vector predictor to the motion vector predictor candidates when the co-located use prohibition flag is on. With this configuration, it is possible to reduce the decoding process in the video decoding device 200.

[0195] Furthermore, in the second embodiment, the video decoding device 200 decodes the co-located use prohibition flags of all pictures, but may decode only the co-located use prohibition flags of specific pictures. For example, the video decoding device 200 decodes only the co-located use prohibition flags of pictures referenced by other pictures (P pictures, B pictures referenced by other pictures, and pictures belonging to the lowest layer in a reference structure having multiple layers), and does not decode the co-located use prohibition flags of pictures not referenced by other pictures. In this way, by decoding only the co-located use prohibition flags of specific pictures, it is possible to reduce the decoding process and suppress decoding error propagation.

[0196] In addition, in the second embodiment, the video decoding device 200 decodes the co-located use prohibition flag for each picture, but the co-located use prohibition flag may be decoded for each slice made up of multiple blocks. By decoding the co-located use prohibition flag for each slice, it is possible to improve the prediction accuracy of the global vector.

[0197] Furthermore, in the second embodiment, the video decoding device 200 decodes the co-located disable flags of all pictures, but may not add a temporal motion vector predictor to motion vector predictor candidates based on the picture type. For example, for a picture referenced by another picture (a P picture, a B picture referenced by another picture, or a picture belonging to the lowest layer in a reference structure having multiple layers), the video decoding device 200 may add a global motion vector to motion vector predictor candidates without adding a temporal motion vector predictor to the motion vector predictor candidates. In this way, by determining whether to add a temporal motion vector predictor or a global motion vector to the motion vector predictor candidates based on the picture type, it is possible to reduce the decoding process and improve coding efficiency.

[0198] Furthermore, when a temporal motion vector predictor is not included in the motion vector predictor candidates, the video encoding device 100 does not include an unnecessary flag in the bitstream, thereby generating a bitstream with improved encoding efficiency. Furthermore, the video decoding device 200 can appropriately decode this bitstream. A specific example will be described with reference to Fig. 18. Fig. 18 is a diagram showing a modified example of the flow for decoding co-located information.

[0199] First, the video decoding device 200 decodes the co-located use prohibition flag (S201). Here, an example will be described in which a flag (enable_temporal_mvp_flag) indicating whether or not a temporal motion vector predictor using information on a co-located block is permitted is decoded.

[0200] Next, the video decoding device 200 determines whether enable_teporal_mvp_flag is 1 (S202). If enable_teporal_mvp_flag is 1 (Yes in S202), the video decoding device 200 decodes the co-located reference direction flag and the co-located reference picture index (S203 and S204).

[0201] On the other hand, when enable_teporal_mvp_flag is 0 (No in S202), the video decoding device 200 does not decode the co-located reference direction flag and the co-located reference picture index. In this way, when the temporal motion vector predictor is not included in the motion vector predictor candidates (when enable_teporal_mvp_flag is 0), the video encoding device 100 can generate a bitstream with improved coding efficiency by not adding the unnecessary co-located reference direction flag and co-located reference picture index to the bitstream. Furthermore, the video decoding device 200 can appropriately decode this bitstream.

[0202] Figures 19A and 19B show examples of syntax when a co-located use prohibition flag (enable_temporal_mvp_flag) indicating whether or not a temporal prediction motion vector using information about a co-located block is allowed is added to a PPS (Picture Parameter Set), and a co-located reference direction flag (collocated_from_l0_flag) and a co-located reference picture index (collocated_ref_idx) are added to a slice header.

[0203] When the value of the co-located use prohibition flag is 1, calculation of a temporal motion vector predictor is permitted using co-located information. When the value of the co-located use prohibition flag is 0, calculation of a temporal motion vector predictor is prohibited using co-located information.

[0204] If the value of the co-located reference direction flag is 1, a co-located picture is selected from the reference picture list of prediction direction 1. If the value of the co-located reference direction flag is 0, a co-located picture is selected from the reference picture list of prediction direction 1.

[0205] Of the multiple pictures included in the reference picture list determined according to the value of collocated_from_l0_flag, a picture whose reference picture index is collocated_ref_idx is selected as the co-located picture.

[0206] Figure 20 also shows an example syntax when a co-located disable flag (enable_temporal_mvp_flag), a co-located reference direction flag (collocated_from_l0_flag), and a co-located reference picture index (collocated_ref_idx), which indicate whether or not a temporal prediction motion vector using information about a co-located block is allowed, are all added to the slice header.

[0207] As shown in Figures 19B and 20, when the value of enable_temporal_mvp_flag is 0, collocated_from_l0_flag and collocated_ref_idx are not added to the bitstream.

[0208] As described above, the video decoding device 200 according to this embodiment performs the video decoding process shown in FIG.

[0209] The video decoding device 200 performs inter-prediction decoding on a current block included in a current picture using a motion vector.

[0210] First, the video decoding device 200 decodes a first flag (co-located use prohibition flag) indicating whether or not temporal motion vector prediction using a temporal motion vector predictor, which is a motion vector of a block included in a decoded picture other than the current picture, is used (whether temporal motion vector prediction is permitted or prohibited) (S211). That is, the video decoding device 200 obtains the coded first flag from the bitstream and decodes the coded first flag to obtain the first flag.

[0211] Next, the video decoding device 200 determines whether the first flag indicates that temporal motion vector prediction is used (enabled) (S212).

[0212] If the first flag indicates that temporal motion vector prediction is used (enabled) (Yes in S212), the video decoding device 200 decodes first parameters for calculating a temporal motion vector predictor (S213). Specifically, the video decoding device 200 obtains the coded first parameters from the bitstream and decodes the obtained coded first parameters to obtain the first parameters. The first parameters include parameters for identifying a decoded picture (co-located picture) that is a reference target of the motion vector. More specifically, the first parameters include a reference picture index (collocated_ref_idx) for identifying the decoded picture (co-located picture) among multiple pictures indicated in a reference picture list used to decode the current picture. The first parameters also include a flag (collocated_from_l0_flag) indicating which reference picture list is to be used to identify the decoded picture (co-located picture) among multiple reference picture lists used to decode the current picture.

[0213] Next, the video decoding device 200 determines a temporal motion vector predictor using the first parameter, and derives a plurality of first motion vector predictor candidates including the determined temporal motion vector predictor (S214).

[0214] Next, the video decoding device 200 decodes a motion vector to be used in inter-prediction decoding of the current block using one of the first motion vector predictor candidates (S215). Specifically, the video decoding device 200 acquires an encoded motion vector (difference value) from the bitstream. Then, the video decoding device 200 generates a motion vector difference value by decoding the encoded motion vector (difference value). Next, a motion vector is generated using one of the first motion vector predictor candidates and the motion vector difference value.

[0215] Furthermore, the video decoding device 200 decodes the current block by inter-decoding using the motion vector. Specifically, the video decoding device 200 acquires the coded current block (differential value) from the bitstream. Then, the video decoding device 200 generates the differential value of the current block by decoding the coded current block (differential value). Next, the video decoding device 200 reconstructs the current block using the motion vector and the differential value of the current block.

[0216] On the other hand, if the first flag indicates that temporal motion vector prediction is not used (prohibited) (No in S212), the video decoding device 200 does not decode the first parameter (S216). That is, the video decoding device 200 does not acquire the first parameter from the bitstream.

[0217] Next, the video decoding device 200 derives a plurality of second motion vector predictor candidates that do not include a temporal motion vector predictor (S217).

[0218] Next, the video decoding device 200 uses one of the plurality of second motion vector predictor candidates to inter-decode the current block included in the current picture (S218). Note that a specific example of this process is the same as the process when the plurality of first motion vector predictor candidates are replaced with the plurality of second motion vector predictor candidates in the above step S215.

[0219] Note that step S211 above is executed by a flag decoding unit included in the video decoding device 200. Furthermore, steps S213 and S216 are executed by a parameter decoding unit included in the video decoding device 200. Steps S214 and S217 are executed by a candidate derivation unit included in the video decoding device 200. Steps S215 and S218 are executed by a motion vector decoding unit included in the video decoding device 200.

[0220] Here, the function of the flag decoding unit is realized by, for example, the variable length decoding unit 201 shown in Fig. 15, etc. Furthermore, the function of the parameter decoding unit is realized by the variable length decoding unit 201 shown in Fig. 15, etc. The function of the candidate derivation unit is realized by the inter prediction control unit 210 and temporal predicted motion vector calculation unit 211 shown in Fig. 15. The function of the motion vector decoding unit is realized by the variable length decoding unit 201, inter prediction unit 208, inter prediction control unit 210, etc. shown in Fig. 15.

[0221] (Variation) Next, a video coding device 300 according to a variation of Embodiment 1 will be described with reference to Fig. 22. Fig. 22 is a block diagram of a video coding device 300 according to a variation of Embodiment 1. Note that a detailed description of commonalities with Embodiment 1 will be omitted, and the description will focus on differences.

[0222] As shown in Fig. 22, the video encoding device 300 includes a first encoding unit 310 that generates a base bitstream by encoding a base view, and a second encoding unit 320 that generates a dependent bitstream by encoding a dependent view. Note that Fig. 22 shows an example in which the video encoding device 300 outputs the base bitstream and the dependent bitstream as independent streams, but the present invention is not limited to this, and the video encoding device 300 may output a single bitstream in which the base bitstream and the dependent bitstream are combined.

[0223] The basic configuration of the first encoding unit 310 and the second encoding unit 320 is the same as that of the video encoding device 100 shown in Fig. 2. However, in addition to the functions of the video encoding device 100, the second encoding unit 320 has a function of referring to the frame memory 108 of the first encoding unit 310, etc.

[0224] Next, a video coding method according to a variation of Embodiment 1 will be described with reference to Figures 23 and 24. Figure 23 is a flowchart showing the operation of the video coding method according to the variation of Embodiment 1. Figure 24 is a diagram showing examples of pictures belonging to a base view and a dependent view.

[0225] The basic view is a set of multiple pictures I, as shown in FIG. 11 , P 12 , P 13 , P 14 , I 15 , P 16 , P 17 In addition, among the pictures belonging to the basic view, the first picture I of the GOP (Group Of Pictures) 11 , I 15 is an I picture, and other pictures P 12 , P 13 , P 14 , P 16 , P 17 is a P picture. Note that the base view is coded and decoded by referring only to pictures belonging to the base view (that is, by intra-prediction coding or inter-prediction coding).

[0226] Also, the dependent view may be a set of multiple pictures P 21 , P 22 , P 23 , P 24 , P 25 , P 26 , P 27 In addition, all pictures P belonging to the dependent view 21 , P 22 , P 23 , P 24 , P 25 , P 26 , P 27 is a P picture. Note that the dependent view is coded and decoded by referring to a picture belonging to the base view corresponding to the picture to be processed (i.e., inter-view predictive coding) in addition to pictures belonging to the dependent view.

[0227] Furthermore, the base view and the dependent view are images of an object viewed from different viewpoints. That is, corresponding pictures (pictures with the same timestamp added) of the base view and the dependent view have horizontal parallax. The second encoding unit 320 can encode each picture belonging to the dependent view by using an image belonging to the base view corresponding to the picture to be processed as a reference picture. Hereinafter, the operation of the temporal motion vector predictor calculation unit 114 of the second encoding unit 320 will be described with reference to FIG. 23.

[0228] First, the temporal motion vector predictor calculation unit 114 determines whether or not a temporal motion vector predictor can be acquired when encoding the current block to be encoded (S91). If a temporal motion vector predictor cannot be acquired (Yes in S91), the temporal motion vector predictor calculation unit 114 includes a disparity vector, which will be described later, in the motion vector predictor candidate (S92). On the other hand, if a temporal motion vector predictor can be acquired (No in S91), the temporal motion vector predictor calculation unit 114 includes the temporal motion vector predictor in the motion vector predictor candidate (S93).

[0229] Here, the case where a temporal motion vector predictor cannot be obtained means, for example, when the encoding target block is a picture P 21 , P 25 The first picture of a GOP is P 21 , P 25 cannot refer to pictures that precede it in display order. That is, if the coding order and display order are the same, then picture P 21 , P 25 can refer to the corresponding picture I of the base view 11 , I 15 Only.

[0230] However, Picture I 11 , I 15is an I-picture, and therefore does not have motion vector information. In such a case, the temporal motion vector predictor calculation unit 114 includes the disparity vector stored in the global vector storage unit 116 in the motion vector predictor candidate as an alternative vector for the temporal motion vector predictor, and also includes the disparity vector in the header information of the dependent bitstream.

[0231] Here, the disparity vector is a vector corresponding to the disparity between the base view and the dependent view. Specifically, the inter prediction control unit 112 of the second encoding unit 320 outputs motion vectors obtained when inter-view predictive encoding is performed on each block constituting a current picture to be encoded of the dependent view (i.e., motion vectors obtained when encoding is performed using a corresponding picture of the base view as a reference picture) to the global vector storage unit 116. Then, the global vector storage unit 116 stores, as a disparity vector, an average value, a median value, a mode value, or the like of the motion vectors acquired from the inter prediction control unit 112 on a picture-by-picture basis.

[0232] In step S92 of FIG. 23, the temporal motion vector predictor calculation unit 114 calculates the picture P 25 As the disparity vector of the picture P 25 The first picture P of the GOP immediately preceding the GOP to which 21 The disparity vector calculated by (picture I 11 Alternatively, the disparity vector using the immediately preceding coded picture P 24 The disparity vector (picture P 14 (disparity vector with reference picture) may be selected.

[0233] 23, a specific example of a case where a temporal motion vector predictor cannot be acquired is not limited to the above example, and may be a case where the co-located use prohibition flag of the current picture to be coded is on. The co-located use prohibition flag is the same as that described in the first embodiment, and therefore will not be described again.

[0234] In this way, the present invention can also be applied to the case where base views and dependent views constituting multi-view video are coded. That is, by switching between including a temporal motion vector predictor and including a disparity vector, which is an alternative vector for the temporal motion vector predictor, in the candidate motion vector predictors when coding a picture to be coded that belongs to a dependent view, it is possible to prevent propagation of decoding errors while suppressing degradation in coding efficiency.

[0235] Next, a video decoding device 400 according to a variation of Embodiment 2 will be described with reference to Fig. 25. Fig. 25 is a block diagram of the video decoding device 400 according to a variation of Embodiment 2. Note that a detailed description of commonalities with Embodiment 2 will be omitted, and the description will focus on differences.

[0236] As shown in Fig. 25, the video decoding device 400 includes a first decoding unit 410 that generates a base view by decoding a base bitstream, and a second decoding unit 420 that generates a dependent view by decoding a dependent bitstream. Note that Fig. 25 shows an example in which an independent base bitstream and a dependent bitstream are separately input to the video decoding device 400, but this is not limiting. A single bitstream combining the base bitstream and the dependent bitstream may be input and split into the base bitstream and the dependent bitstream within the video decoding device 400.

[0237] The basic configuration of the first decoding unit 410 and the second decoding unit 420 is the same as that of the video decoding device 200 shown in Fig. 15. However, the second decoding unit 420 has a function of referring to the frame memory 206 of the first decoding unit 410, etc., in addition to the functions of the video decoding device 200. In other words, the video decoding device 400 decodes the base bitstream and dependent bitstream coded by the video coding device 300.

[0238] The second decoding unit 420 of the video decoding device 400 can switch between including the temporal motion vector predictor stored in the colPic memory 212 or the disparity vector included in the header information of the dependent bitstream as one of the motion vector predictor candidates for the block to be decoded. Note that the operation of the temporal motion vector predictor calculation unit 211 included in the second decoding unit 420 is similar to the processing in FIG. 23 .

[0239] Although the video encoding device and video decoding device according to the embodiment have been described above, the present invention is not limited to these embodiments.

[0240] Furthermore, each processing unit included in the video encoding device and video decoding device according to the above embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0241] Furthermore, the integration is not limited to LSI, but may be realized by dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays), which can be programmed after LSI fabrication, or reconfigurable processors, which allow the connections and settings of circuit cells within LSIs to be reconfigured, may also be used.

[0242] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0243] In other words, the video encoding device and the video decoding device each include a control circuitry and a storage device electrically connected to the control circuitry (accessible from the control device). The control circuitry includes at least one of dedicated hardware and a program execution unit. If the control circuitry includes a program execution unit, the storage device stores a software program to be executed by the program execution unit.

[0244] Furthermore, the present invention may be the above-mentioned software program, or a non-transitory computer-readable recording medium on which the above-mentioned program is recorded. Needless to say, the above-mentioned program can be distributed via a transmission medium such as the Internet.

[0245] Furthermore, all the numbers used above are merely examples for the purpose of specifically explaining the present invention, and the present invention is not limited to the numbers used as examples.

[0246] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0247] The order in which the steps included in the above video encoding method or video decoding method are executed is merely an example for specifically explaining the present invention, and an order other than the above may be used. Furthermore, some of the steps may be executed simultaneously (in parallel) with other steps.

[0248] While the video encoding device and video decoding device according to one or more aspects of the present invention have been described above based on the embodiments, the present invention is not limited to these embodiments. As long as they do not deviate from the spirit of the present invention, various modifications conceivable by those skilled in the art to the present embodiments, or configurations constructed by combining components of different embodiments, may also be included within the scope of one or more aspects of the present invention.

[0249] (Embodiment 3) By recording a program for implementing the video coding method (image coding method) or video decoding method (image decoding method) shown in each of the above embodiments on a storage medium, it becomes possible to easily perform the processes shown in each of the above embodiments on an independent computer system. The storage medium may be a magnetic disk, optical disk, magneto-optical disk, IC card, semiconductor memory, or any other medium capable of recording a program.

[0250] Furthermore, here, we will explain application examples of the video coding method (image coding method) and video decoding method (image decoding method) shown in each of the above embodiments, and a system using the same. The system is characterized by having an image coding / decoding device consisting of an image coding device using the image coding method and an image decoding device using the image decoding method. Other components of the system can be appropriately changed depending on the situation.

[0251] 26 is a diagram showing the overall configuration of a content supply system ex100 that provides a content distribution service. The area where communication services are provided is divided into cells of a desired size, and base stations ex106, ex107, ex108, ex109, and ex110, which are fixed wireless stations, are installed in each cell.

[0252] This content supply system ex100 is connected to the Internet ex101 via an Internet service provider ex102, a telephone network ex104, and base stations ex106 to ex110, and devices such as a computer ex111, a PDA (Personal Digital Assistant) ex112, a camera ex113, a mobile phone ex114, and a game console ex115.

[0253] However, the content supply system ex100 is not limited to the configuration shown in Fig. 26, and any combination of elements may be connected. Also, each device may be directly connected to the telephone network ex104 without going through base stations ex106 to ex110, which are fixed wireless stations. Also, each device may be directly connected to each other via short-range wireless or the like.

[0254] The camera ex113 is a device capable of shooting moving images, such as a digital video camera, and the camera ex116 is a device capable of shooting still images and moving images, such as a digital camera. The mobile phone ex114 may be any of a GSM (registered trademark) (Global System for Mobile Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, an LTE (Long Term Evolution) system, an HSPA (High Speed ​​Packet Access) mobile phone, or a PHS (Personal Handyphone System) system.

[0255] In the content supply system ex100, a camera ex113 and the like are connected to a streaming server ex103 via a base station ex109 and a telephone network ex104, thereby enabling live streaming and the like. In live streaming, a user shoots content (e.g., video of a live music concert) using the camera ex113, and encodes the content as described in the above embodiments (i.e., functions as an image encoding device according to an aspect of the present invention) and transmits the content to the streaming server ex103. Meanwhile, the streaming server ex103 streams the transmitted content data to a requesting client. Examples of clients include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, a game console ex115, and the like that are capable of decoding the encoded data. Each device that receives the distributed data decodes and plays back the received data (i.e., functions as an image decoding device according to an aspect of the present invention).

[0256] The encoding process of the captured data may be performed by the camera ex113, by the streaming server ex103 that processes the data transmission, or by a mutually shared responsibility. Similarly, the decoding process of the distributed data may be performed by the client, by the streaming server ex103, or by a mutually shared responsibility. Furthermore, still images and / or video data captured by camera ex116, not limited to camera ex113, may be transmitted to the streaming server ex103 via computer ex111. In this case, the encoding process may be performed by the camera ex116, the computer ex111, or the streaming server ex103, or by a mutually shared responsibility.

[0257] Furthermore, these encoding and decoding processes are generally performed by the computer ex111 or an LSIex500 possessed by each device. The LSIex500 may be a single chip or may be configured with multiple chips. It is also possible to embed video encoding and decoding software on some kind of recording medium (CD-ROM, flexible disk, hard disk, etc.) that can be read by the computer ex111, etc., and perform the encoding and decoding processes using that software. Furthermore, if the mobile phone ex114 is equipped with a camera, video data captured by the camera may be transmitted. This video data is data that has been encoded and processed by the LSIex500 possessed by the mobile phone ex114.

[0258] The streaming server ex103 may also be a plurality of servers or computers that process, record, and distribute data in a distributed manner.

[0259] In this way, the content delivery system ex100 allows a client to receive and play back encoded data. In this way, the content delivery system ex100 allows a client to receive, decode, and play back information sent by a user in real time, enabling even users without special rights or equipment to realize personal broadcasting.

[0260] In addition to the example of the content supply system ex100, as shown in FIG. 27, a digital broadcasting system ex200 can also incorporate at least one of the video encoding device (image encoding device) or video decoding device (image decoding device) according to each of the above embodiments. Specifically, a broadcasting station ex201 communicates with a satellite ex202 via radio waves to generate multiplexed data, in which music data and other data are multiplexed onto video data. This video data is data encoded using the video encoding method described in each of the above embodiments (i.e., data encoded by an image encoding device according to one aspect of the present invention). Receiving this data, the broadcasting satellite ex202 transmits broadcasting radio waves, which are received by a home antenna ex204 capable of receiving satellite broadcasts. The received multiplexed data is decoded and played back by a device such as a television (receiver) ex300 or a set-top box (STB) ex217 (i.e., functions as an image decoding device according to one aspect of the present invention).

[0261] The video decoding device or video encoding device described in each of the above embodiments can also be implemented in a reader / recorder ex218 that reads and decodes multiplexed data recorded on a recording medium ex215 such as a DVD or Blu-ray, or encodes a video signal onto the recording medium ex215 and, in some cases, multiplexes it with an audio signal before writing it. In this case, the reproduced video signal is displayed on a monitor ex219, and the video signal can be reproduced in another device or system using the recording medium ex215 on which the multiplexed data is recorded. Alternatively, a video decoding device may be implemented in a set-top box ex217 connected to a cable television cable ex203 or a satellite / terrestrial broadcast antenna ex204, and the video may be displayed on the television monitor ex219. In this case, the video decoding device may be incorporated into the television rather than the set-top box.

[0262] 28 is a diagram showing a television (receiver) ex300 that uses the video decoding method and video encoding method described in each of the above embodiments. The television ex300 includes a tuner ex301 that acquires or outputs multiplexed data in which audio data is multiplexed onto video data via an antenna ex204 that receives the broadcasts or a cable ex203, a modulation / demodulation unit ex302 that demodulates the received multiplexed data or modulates it into multiplexed data to be transmitted externally, and a multiplexing / demultiplexing unit ex303 that separates the demodulated multiplexed data into video data and audio data or multiplexes the video data and audio data encoded by a signal processing unit ex306.

[0263] The television ex300 also has a signal processing unit ex306 having an audio signal processing unit ex304 and a video signal processing unit ex305 (which function as an image encoding device or an image decoding device according to an embodiment of the present invention) that decode the audio data and the video data, respectively, or encode the respective information, and an output unit ex309 having a speaker ex307 that outputs the decoded audio signal and a display unit ex308 such as a display that displays the decoded video signal.The television ex300 also has an interface unit ex317 that has an operation input unit ex312 that accepts user operation input, etc.The television ex300 also has a control unit ex310 that controls each unit overall, and a power supply circuit unit ex311 that supplies power to each unit. In addition to the operation input unit ex312, the interface unit ex317 may have a bridge ex313 connected to an external device such as a reader / recorder ex218, a slot unit ex314 for allowing a recording medium ex216 such as an SD card to be attached, a driver ex315 for connecting to an external recording medium such as a hard disk, a modem ex316 for connecting to a telephone network, etc. The recording medium ex216 is a non-volatile / volatile semiconductor memory element that stores information and allows it to be electrically recorded. The various units of the television ex300 are connected to each other via a synchronous bus.

[0264] First, a configuration in which the television ex300 decodes and plays back multiplexed data acquired from an external source via an antenna ex204 or the like will be described. The television ex300 receives user operation via a remote controller ex220 or the like, and, under the control of a control unit ex310 having a CPU or the like, separates the multiplexed data demodulated by a modulation / demodulation unit ex302 in a multiplexing / separation unit ex303. The television ex300 then decodes the separated audio data in an audio signal processing unit ex304 and decodes the separated video data in a video signal processing unit ex305 using the decoding method described in each of the above embodiments. The decoded audio and video signals are output to the outside from an output unit ex309. When outputting, it is preferable to temporarily store these signals in buffers ex318, ex319, or the like so that the audio and video signals are played back in sync. The television ex300 may also read the multiplexed data from recording media ex215, ex216, such as magnetic / optical discs or SD cards, rather than from broadcasts or the like. Next, a configuration in which the television ex300 encodes audio and video signals and transmits them externally or writes them to a recording medium or the like will be described. The television ex300 receives user operation from a remote controller ex220 or the like, and, under the control of the control unit ex310, encodes the audio signal in the audio signal processing unit ex304 and encodes the video signal in the video signal processing unit ex305 using the encoding method described in each of the above embodiments. The encoded audio and video signals are multiplexed by the multiplexing / demultiplexing unit ex303 and output externally. When multiplexing, these signals may be temporarily stored in buffers ex320, ex321, etc., so that the audio and video signals are synchronized. Note that multiple buffers ex318, ex319, ex320, and ex321 may be provided as shown, or one or more buffers may be shared. Furthermore, data may be stored in buffers other than those shown in the figure, for example, between the modulation / demodulation unit ex302 and the multiplexing / demultiplexing unit ex303, as a buffer to prevent system overflow and underflow.

[0265] Furthermore, in addition to acquiring audio data and video data from broadcasts, recording media, etc., the television ex300 may also be configured to accept AV input from a microphone or camera and perform encoding processing on the data acquired from them. Note that while the television ex300 has been described here as being configured to be capable of the above encoding processing, multiplexing, and external output, it may also be configured not to be able to perform these processes and only be capable of the above reception, decoding processing, and external output.

[0266] Furthermore, when multiplexed data is read from or written to a recording medium using the reader / recorder ex218, the above-mentioned decoding or encoding process may be performed by either the television ex300 or the reader / recorder ex218, or the television ex300 and the reader / recorder ex218 may share the process.

[0267] As an example, Figure 29 shows the configuration of the information reproducing / recording unit ex400 when reading or writing data from an optical disc. The information reproducing / recording unit ex400 includes the following elements ex401, ex402, ex403, ex404, ex405, ex406, and ex407. The optical head ex401 writes information by irradiating a laser spot onto the recording surface of the recording medium ex215, which is an optical disc, and reads the information by detecting the light reflected from the recording surface of the recording medium ex215. The modulation / recording unit ex402 electrically drives the semiconductor laser built into the optical head ex401 and modulates the laser light according to the recorded data. The reproduction / demodulation unit ex403 amplifies the reproduction signal obtained by electrically detecting the light reflected from the recording surface using a photodetector built into the optical head ex401, separates and demodulates the signal components recorded on the recording medium ex215, and reproduces the required information. The buffer ex404 temporarily stores information to be recorded on the recording medium ex215 and information reproduced from the recording medium ex215. The disk motor ex405 rotates the recording medium ex215. The servo control unit ex406 controls the rotation of the disk motor ex405, moves the optical head ex401 to a specified information track, and performs laser spot tracking. The system control unit ex407 controls the entire information reproduction / recording unit ex400. The system control unit ex407 performs the above read and write processes by using various information stored in the buffer ex404, generating and adding new information as needed, and recording and reproducing information through the optical head ex401 while coordinating the modulation recording unit ex402, reproduction demodulation unit ex403, and servo control unit ex406. The system control unit ex407 is composed of, for example, a microprocessor and performs these processes by executing read and write programs.

[0268] In the above description, the optical head ex401 is described as irradiating a laser spot, but it may be configured to perform higher density recording using near-field light.

[0269] FIG. 30 shows a schematic diagram of recording medium ex215, an optical disc. A spiral guide groove is formed on the recording surface of recording medium ex215, and address information indicating absolute positions on the disc is recorded in advance on information track ex230 by changing the shape of the groove. This address information includes information for identifying the position of recording block ex231, which is the unit of data recording. A recording or playback device can identify a recording block by reproducing information track ex230 and reading the address information. Recording medium ex215 also includes a data recording area ex233, an inner peripheral area ex232, and an outer peripheral area ex234. The data recording area ex233 is the area used for recording user data, while the inner peripheral area ex232 and outer peripheral area ex234, which are located either inner or outer than data recording area ex233, are used for specific purposes other than recording user data. The information reproducing / recording unit ex400 reads and writes encoded audio data, video data, or multiplexed data obtained by multiplexing these data, from the data recording area ex233 of such recording medium ex215.

[0270] The above explanation has been given using examples of optical discs such as single-layer DVDs and BDs, but the present invention is not limited to these and may be an optical disc with a multi-layer structure that allows recording on areas other than the surface. It may also be an optical disc with a structure that allows multidimensional recording / playback, such as recording information using light of various different wavelengths in the same location on the disc or recording different layers of information from various angles.

[0271] In addition, in the digital broadcasting system ex200, a car ex210 equipped with an antenna ex205 can receive data from a satellite ex202 or the like, and the video can be played on a display device such as a car navigation system ex211 installed in the car ex210. The car navigation system ex211 can be configured, for example, by adding a GPS receiving unit to the configuration shown in Fig. 28, and similar configurations can be considered for a computer ex111, a mobile phone ex114, and the like.

[0272] 31A is a diagram showing a mobile phone ex114 that uses the video decoding method and video encoding method described in the above embodiment. The mobile phone ex114 includes an antenna ex350 for transmitting and receiving radio waves to and from base station ex110, a camera unit ex365 capable of capturing video and still images, and a display unit ex358 such as an LCD display that displays decoded data of video captured by the camera unit ex365 and video received by the antenna ex350. The mobile phone ex114 also includes a main body unit having an operation key unit ex366, an audio output unit ex357 such as a speaker for outputting audio, an audio input unit ex356 such as a microphone for inputting audio, a memory unit ex367 for storing captured video, still images, recorded audio, or encoded or decoded data of received video, still images, email, etc., or a slot unit ex364 that serves as an interface with a recording medium for similarly storing data.

[0273] Furthermore, a configuration example of mobile phone ex114 will be described with reference to Fig. 31B. Mobile phone ex114 has a main control unit ex360 that comprehensively controls each unit of a main body unit including a display unit ex358 and an operation key unit ex366, and a power supply circuit unit ex361, an operation input control unit ex362, a video signal processing unit ex355, a camera interface unit ex363, an LCD (Liquid Crystal Display) control unit ex359, a modulation / demodulation unit ex352, a multiplexing / demultiplexing unit ex353, an audio signal processing unit ex354, a slot unit ex364, and a memory unit ex367, which are all connected to each other via a bus ex370.

[0274] When the end call and power key is turned on by the user, the power supply circuit unit ex361 starts up the mobile phone ex114 into an operable state by supplying power to each unit from the battery pack.

[0275] Based on the control of a main control unit ex360 having a CPU, ROM, RAM, etc., the mobile phone ex114 converts an audio signal collected by an audio input unit ex356 into a digital audio signal by an audio signal processing unit ex354 in a voice call mode, which undergoes spectrum spread processing by a modulation / demodulation unit ex352, digital-to-analog conversion processing and frequency conversion processing by a transmission / reception unit ex351, and then transmits the digital audio signal via an antenna ex350. Furthermore, the mobile phone ex114 amplifies received data received via the antenna ex350 in a voice call mode, performs frequency conversion processing and analog-to-digital conversion processing, performs spectrum despread processing by the modulation / demodulation unit ex352, converts the data into an analog audio signal by the audio signal processing unit ex354, and then outputs the data from an audio output unit ex357.

[0276] Furthermore, when sending an e-mail in data communication mode, the text data of the e-mail entered by operating the operation key unit ex366 or the like of the main unit is sent to the main control unit ex360 via the operation input control unit ex362. The main control unit ex360 performs spectrum spread processing on the text data in the modulation / demodulation unit ex352, performs digital-to-analog conversion processing and frequency conversion processing in the transmission / reception unit ex351, and then transmits the data to the base station ex110 via the antenna ex350. When receiving an e-mail, the received data is subjected to roughly the reverse processing and output to the display unit ex358.

[0277] When transmitting video, still images, or video and audio in the data communication mode, the video signal processing unit ex355 compresses and encodes the video signal supplied from the camera unit ex365 using the video encoding method described in each of the above embodiments (i.e., functions as an image encoding device according to one aspect of the present invention), and sends the encoded video data to the multiplexing / separating unit ex353. In addition, the audio signal processing unit ex354 encodes the audio signal collected by the audio input unit ex356 while the camera unit ex365 is capturing video, still images, etc., and sends the encoded audio data to the multiplexing / separating unit ex353.

[0278] The multiplexing / separation unit ex353 multiplexes the encoded video data supplied from the video signal processing unit ex355 and the encoded audio data supplied from the audio signal processing unit ex354 using a predetermined method, and the resulting multiplexed data is subjected to spectrum spreading processing in the modulation / demodulation unit (modulation / demodulation circuit unit) ex352, digital-to-analog conversion processing and frequency conversion processing in the transmission / reception unit ex351, and then transmitted via the antenna ex350.

[0279] When receiving video file data linked to a website or the like in data communication mode, or when receiving an email with video and / or audio attachments, the multiplexer / demultiplexer ex353 decodes the multiplexed data received via the antenna ex350 into a video data bitstream and an audio data bitstream. The multiplexer / demultiplexer ex353 then decodes the multiplexed data into a video data bitstream and an audio data bitstream via a synchronization bus ex370. The video signal processor ex355 decodes the video signal using a video decoding method corresponding to the video encoding method described in each of the above embodiments (i.e., functions as an image decoding device according to one aspect of the present invention). The display unit ex358 displays, via an LCD controller ex359, video and still images included in the video file linked to a website, for example. The audio signal processor ex354 decodes the audio signal, and audio is output from an audio output unit ex357.

[0280] Furthermore, like the television ex300, terminals such as the mobile phone ex114 can be implemented in three ways: a transmitting / receiving terminal with both an encoder and a decoder, a transmitting terminal with only an encoder, and a receiving terminal with only a decoder. Furthermore, in the digital broadcasting system ex200, it has been explained that multiplexed data in which music data and the like are multiplexed onto video data is received and transmitted, but the data may also be multiplexed with text data related to the video in addition to audio data, or it may be video data itself rather than multiplexed data.

[0281] In this way, it is possible to use the video encoding method or video decoding method shown in each of the above embodiments in any of the above-mentioned devices and systems, and by doing so, it is possible to obtain the effects described in each of the above embodiments.

[0282] Furthermore, the present invention is not limited to the above-described embodiment, and various modifications and alterations are possible without departing from the scope of the present invention.

[0283] (Fourth embodiment) It is also possible to generate video data by switching between the video encoding method or device shown in each of the above embodiments and a video encoding method or device conforming to a different standard, such as MPEG-2, MPEG4-AVC, or VC-1, as needed.

[0284] When multiple pieces of video data conforming to different standards are generated, it is necessary to select a decoding method corresponding to each standard when decoding. However, since it is not possible to identify which standard the video data to be decoded conforms to, a problem arises in that it is not possible to select an appropriate decoding method.

[0285] To solve this problem, multiplexed data, which is video data multiplexed with audio data, etc., is configured to include identification information that indicates which standard the video data conforms to. A specific configuration of multiplexed data including video data generated by the video encoding methods or devices described in the above embodiments is described below. The multiplexed data is a digital stream in MPEG-2 transport stream format.

[0286] FIG. 32 shows the structure of multiplexed data. As shown in FIG. 32, the multiplexed data is obtained by multiplexing one or more of a video stream, an audio stream, a presentation graphics stream (PG), and an interactive graphics stream. The video stream represents the main video and secondary video of a movie, the audio stream (IG) represents the main audio portion of the movie and the secondary audio mixed with the main audio, and the presentation graphics stream represents the subtitles of the movie. Here, the main video refers to the normal video displayed on the screen, and the secondary video refers to the video displayed on a small screen within the main video. The interactive graphics stream represents an interactive screen created by arranging GUI components on the screen. The video stream is encoded using the video encoding method or device described in each of the above embodiments or a video encoding method or device conforming to conventional standards such as MPEG-2, MPEG4-AVC, or VC-1. The audio stream is encoded using a format such as Dolby AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD, or Linear PCM.

[0287] Each stream included in the multiplexed data is identified by a PID. For example, 0x1011 is assigned to the video stream used for movie images, 0x1100 to 0x111F to the audio stream, 0x1200 to 0x121F to the presentation graphics, 0x1400 to 0x141F to the interactive graphics stream, 0x1B00 to 0x1B1F to the video stream used for movie secondary video, and 0x1A00 to 0x1A1F to the audio stream used for secondary audio to be mixed with the main audio.

[0288] 33 is a diagram showing how multiplexed data is multiplexed. First, a video stream ex235 consisting of multiple video frames and an audio stream ex238 consisting of multiple audio frames are converted into PES packet sequences ex236 and ex239, respectively, and then converted into TS packets ex237 and ex240. Similarly, presentation graphics stream ex241 and interactive graphics data ex244 are converted into PES packet sequences ex242 and ex245, respectively, and then converted into TS packets ex243 and ex246. Multiplexed data ex247 is constructed by multiplexing these TS packets into a single stream.

[0289] FIG. 34 shows in more detail how a video stream is stored in a PES packet sequence. The first row in FIG. 34 shows a video frame sequence of the video stream. The second row shows a PES packet sequence. As indicated by arrows yy1, yy2, yy3, and yy4 in FIG. 34, I-pictures, B-pictures, and P-pictures, which are multiple Video Presentation Units in the video stream, are divided into individual pictures and stored in the payload of PES packets. Each PES packet has a PES header, which stores a Presentation Time-Stamp (PTS), which is the display time of the picture, and a Decoding Time-Stamp (DTS), which is the decoding time of the picture.

[0290] Figure 35 shows the format of the TS packet that is finally written to the multiplexed data. TS packets are 188-byte fixed-length packets consisting of a 4-byte TS header containing information such as a PID that identifies the stream, and a 184-byte TS payload that stores the data. The PES packets are divided and stored in the TS payload. In the case of BD-ROM, a 4-byte TP_Extra_Header is added to the TS packet, forming a 192-byte source packet that is written to the multiplexed data. The TP_Extra_Header contains information such as an ATS (Arrival Time Stamp). The ATS indicates the start time of the transfer of the TS packet to the PID filter of the decoder. As shown in the lower part of Figure 35, source packets are lined up in the multiplexed data, and the number that increments from the beginning of the multiplexed data is called the SPN (Source Packet Number).

[0291] In addition to the individual streams (video, audio, subtitles, etc.), the TS packets contained in the multiplexed data also contain a Program Association Table (PAT), Program Map Table (PMT), and Program Clock Reference (PCR). The PAT indicates the PID of the PMT used in the multiplexed data, and the PAT's own PID is registered as 0. The PMT contains the PIDs of each stream (video, audio, subtitles, etc.) contained in the multiplexed data, as well as attribute information for the streams corresponding to each PID. It also contains various descriptors related to the multiplexed data. The descriptors include copy control information that indicates whether copying of the multiplexed data is permitted or prohibited. The PCR contains information about the Arrival Time Clock (ATC), which is the time axis of the ATS, and the System Time Clock (STC), which is the time axis of the PTS and DTS, and contains information about the STC time corresponding to the ATS at which the PCR packet is transferred to the decoder.

[0292] Figure 36 is a diagram explaining the data structure of a PMT in detail. At the beginning of a PMT is a PMT header that describes the length of the data contained in the PMT, among other things. This is followed by multiple descriptors related to the multiplexed data. The above-mentioned copy control information and other information are written as descriptors. After the descriptors are multiple stream information items related to each stream included in the multiplexed data. The stream information consists of stream descriptors that describe the stream type to identify the stream compression codec, the stream PID, and stream attribute information (frame rate, aspect ratio, etc.). There are as many stream descriptors as there are streams in the multiplexed data.

[0293] When recording on a recording medium, the multiplexed data is recorded together with a multiplexed data information file.

[0294] As shown in FIG. 37, the multiplexed data information file is management information for multiplexed data, has one-to-one correspondence with the multiplexed data, and is composed of multiplexed data information, stream attribute information, and an entry map.

[0295] As shown in Figure 37, the multiplexed data information consists of a system rate, a playback start time, and a playback end time. The system rate indicates the maximum transfer rate of the multiplexed data to the PID filter of the system target decoder, which will be described later. The interval between ATSs contained in the multiplexed data is set to be equal to or less than the system rate. The playback start time is set to the PTS of the first video frame of the multiplexed data, and the playback end time is set to the PTS of the last video frame of the multiplexed data plus the playback interval of one frame.

[0296] As shown in Figure 38, the stream attribute information for each stream included in the multiplexed data is registered for each PID. The attribute information has different information for each video stream, audio stream, presentation graphics stream, and interactive graphics stream. The video stream attribute information includes information such as the compression codec used to compress the video stream, the resolution of the individual picture data that make up the video stream, the aspect ratio, and the frame rate. The audio stream attribute information includes information such as the compression codec used to compress the audio stream, the number of channels included in the audio stream, the language it supports, and the sampling frequency. This information is used to initialize the decoder before playback on the player.

[0297] In this embodiment, the stream type included in the PMT of the multiplexed data is used. Furthermore, if multiplexed data is recorded on a recording medium, the video stream attribute information included in the multiplexed data information is used. Specifically, the video coding method or device shown in each of the above embodiments includes a step or means for setting, in the stream type included in the PMT or the video stream attribute information, unique information indicating that the video data is generated by the video coding method or device shown in each of the above embodiments. This configuration makes it possible to distinguish between video data generated by the video coding method or device shown in each of the above embodiments and video data that conforms to other standards.

[0298] FIG. 39 shows the steps of the video decoding method according to this embodiment. In step exS100, the stream type included in the PMT or the video stream attribute information included in the multiplexed data information is obtained from the multiplexed data. Next, in step exS101, it is determined whether the stream type or the video stream attribute information indicates that the multiplexed data was generated by the video coding method or device described in the above embodiments. If it is determined that the stream type or the video stream attribute information was generated by the video coding method or device described in the above embodiments, in step exS102, a reference picture or a motion vector is selected from the candidates using the video decoding method described in the above embodiments, and decoding is performed. If the stream type or the video stream attribute information indicates that the data complies with a conventional standard such as MPEG-2, MPEG4-AVC, or VC-1, decoding is performed using the video decoding method according to the conventional standard in step exS103. For example, if the attribute information indicates that the image conforms to the MPEG4-AVC standard, decoding is performed using a motion vector calculated from the motion vectors of spatially or temporally adjacent blocks, rather than selecting from multiple candidates.

[0299] In this way, by setting a new unique value in the stream type or video stream attribute information, it is possible to determine whether the video decoding method or device shown in each of the above embodiments can decode the data when decoding. Therefore, even when multiplexed data conforming to a different standard is input, an appropriate decoding method or device can be selected, enabling decoding without errors. Furthermore, the video encoding method or device or video decoding method or device shown in this embodiment can be used in any of the above-mentioned devices and systems.

[0300] (Embodiment 5) The video encoding method and device, and video decoding method and device shown in each of the above embodiments are typically realized by an LSI, which is an integrated circuit. As an example, FIG. 40 shows the configuration of a single-chip LSI ex500. LSI ex500 includes elements ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508, and ex509, which are described below, and each element is connected via a bus ex510. When the power supply is on, a power supply circuit unit ex505 supplies power to each unit, thereby activating them into an operable state.

[0301] For example, when performing encoding processing, the LSI ex500 inputs AV signals from the microphone ex117, camera ex113, etc. via the AV I / O ex509 under the control of a control unit ex501 including a CPU ex502, a memory controller ex503, a stream controller ex504, a drive frequency control unit ex512, etc. The input AV signals are temporarily stored in an external memory ex511 such as an SDRAM. Under the control of the control unit ex501, the stored data is divided into multiple batches as appropriate depending on the processing volume and processing speed and sent to the signal processing unit ex507, where the audio signal and / or video signal is encoded. Here, the video signal encoding processing is the encoding processing described in each of the above embodiments. The signal processing unit ex507 may further perform processing such as multiplexing the encoded audio data and the encoded video data, and output the resulting data to the outside from the stream I / O ex506. This output multiplexed data is transmitted to the base station ex107 or written to a recording medium ex215. When multiplexing, it is advisable to temporarily store the data in a buffer ex508 to ensure synchronization.

[0302] Although the memory ex511 has been described above as being external to the LSIex500, it may be included within the LSIex500. The buffer ex508 is not limited to one, and multiple buffers may be provided. Furthermore, the LSIex500 may be formed as a single chip or multiple chips.

[0303] Furthermore, in the above description, the control unit ex501 is described as having a CPU ex502, a memory controller ex503, a stream controller ex504, a drive frequency control unit ex512, etc., but the configuration of the control unit ex501 is not limited to this configuration. For example, the signal processing unit ex507 may further include a CPU. By providing a CPU inside the signal processing unit ex507, it is possible to further improve processing speed. As another example, the CPU ex502 may include the signal processing unit ex507, or a part of the signal processing unit ex507, such as an audio signal processing unit. In such a case, the control unit ex501 is configured to include a CPU ex502 that includes the signal processing unit ex507, or a part of it.

[0304] Although we have referred to it as an LSI here, it may also be called an IC, system LSI, super LSI, or ultra LSI depending on the level of integration.

[0305] Furthermore, the integrated circuit implementation is not limited to LSIs, but may be realized using dedicated circuits or general-purpose processors. Field programmable gate arrays (FPGAs) that can be programmed after LSI fabrication, or reconfigurable processors that allow the connections and settings of circuit cells within the LSI to be reconfigured, may also be used. Such programmable logic devices can typically execute the video encoding method or video decoding method described in each of the above embodiments by loading or reading from memory a program that constitutes software or firmware.

[0306] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology could be used to integrate functional blocks. The application of biotechnology is also a possibility.

[0307] (Sixth embodiment) When decoding video data generated by the video encoding method or device described in each of the above embodiments, the amount of processing is likely to increase compared to when decoding video data conforming to conventional standards such as MPEG-2, MPEG4-AVC, or VC-1. Therefore, it is necessary to set the drive frequency of the LSIex500 to a higher frequency than the drive frequency of the CPUex502 when decoding video data conforming to conventional standards. However, increasing the drive frequency raises the problem of increased power consumption.

[0308] To solve this problem, video decoding devices such as televisions ex300 and LSIs ex500 are configured to identify the standard to which video data conforms and switch the drive frequency according to the standard. FIG. 41 shows a configuration ex800 in this embodiment. If the video data was generated using the video encoding method or device described in each of the above embodiments, a drive frequency switching unit ex803 sets a high drive frequency. The unit then instructs a decoding processing unit ex801, which executes the video decoding method described in each of the above embodiments, to decode the video data. On the other hand, if the video data conforms to a conventional standard, the unit sets a low drive frequency compared to when the video data was generated using the video encoding method or device described in each of the above embodiments. The unit then instructs a decoding processing unit ex802, which conforms to the conventional standard, to decode the video data.

[0309] More specifically, the drive frequency switching unit ex803 is composed of the CPU ex502 and drive frequency control unit ex512 in FIG. 40. The decoding processing unit ex801 that executes the video decoding method described in each of the above embodiments and the decoding processing unit ex802 that complies with the conventional standard correspond to the signal processing unit ex507 in FIG. 40. The CPU ex502 identifies the standard to which the video data conforms. The driving frequency control unit ex512 sets the drive frequency based on the signal from the CPU ex502. The signal processing unit ex507 decodes the video data based on the signal from the CPU ex502. Here, the video data can be identified using, for example, the identification information described in the fourth embodiment. The identification information is not limited to that described in the fourth embodiment, and may be any information that can identify the standard to which the video data conforms. For example, if it is possible to identify the standard to which the video data conforms based on an external signal that identifies whether the video data is for use on a television or a disc, then the identification may be based on such an external signal. Furthermore, the selection of the drive frequency in the CPUex502 can be performed based on a lookup table that associates the video data standard with the drive frequency, as shown in Fig. 43. The lookup table is stored in the buffer ex508 or the internal memory of the LSI, and the CPUex502 can select the drive frequency by referring to this lookup table.

[0310] FIG. 42 shows steps for implementing the method of this embodiment. First, in step exS200, the signal processing unit ex507 acquires identification information from the multiplexed data. Next, in step exS201, the CPU ex502 identifies, based on the identification information, whether the video data was generated by the encoding method or device described in any of the above embodiments. If the video data was generated by the encoding method or device described in any of the above embodiments, in step exS202, the CPU ex502 sends a signal to the driving frequency control unit ex512 to set the driving frequency to a high level. The driving frequency control unit ex512 then sets the driving frequency to a high level. On the other hand, if the video data indicates that the video data complies with a conventional standard such as MPEG-2, MPEG4-AVC, or VC-1, in step exS203, the CPU ex502 sends a signal to the driving frequency control unit ex512 to set the driving frequency to a low level. The driving frequency control unit ex512 then sets the driving frequency to a lower level than when the video data was generated by the encoding method or device described in any of the above embodiments.

[0311] Furthermore, by changing the voltage applied to the LSIex500 or a device including the LSIex500 in conjunction with switching the drive frequency, it is possible to further enhance the power saving effect. For example, when the drive frequency is set low, it is conceivable to set the voltage applied to the LSIex500 or a device including the LSIex500 lower in response to this change than when the drive frequency is set high.

[0312] Furthermore, the method of setting the drive frequency is not limited to the above-described setting method, and may be such that a high drive frequency is set when the decoding processing volume is large, and a low drive frequency is set when the decoding processing volume is small. For example, if the processing volume required to decode video data conforming to the MPEG4-AVC standard is larger than the processing volume required to decode video data generated by the video encoding method or device described in each of the above-described embodiments, the drive frequency may be set in the opposite way to the above-described setting method.

[0313] Furthermore, the method of setting the drive frequency is not limited to a configuration that lowers the drive frequency. For example, if the identification information indicates that the video data is generated by the video encoding method or device described in each of the above embodiments, the voltage applied to the LSIex500 or a device including the LSIex500 can be set high. If the identification information indicates that the video data complies with conventional standards such as MPEG-2, MPEG4-AVC, or VC-1, the voltage applied to the LSIex500 or a device including the LSIex500 can be set low. As another example, if the identification information indicates that the video data is generated by the video encoding method or device described in each of the above embodiments, the drive of the CPUex502 can be suspended without stopping. If the identification information indicates that the video data complies with conventional standards such as MPEG-2, MPEG4-AVC, or VC-1, the drive of the CPUex502 can be suspended temporarily because there is sufficient processing capacity. Even if the identification information indicates that the video data is generated by the video encoding method or device described in each of the above embodiments, the drive of the CPUex502 can be suspended temporarily if there is sufficient processing capacity. In this case, it is conceivable to set the stop time shorter than when the video data indicates that it is video data that conforms to conventional standards such as MPEG-2, MPEG4-AVC, and VC-1.

[0314] In this way, by switching the drive frequency depending on the standard to which the video data conforms, it is possible to achieve power savings. Furthermore, if the LSIex500 or a device including the LSIex500 is driven by a battery, the power savings can also extend the battery life.

[0315] (Embodiment 7) The above-mentioned devices and systems, such as televisions and mobile phones, may receive multiple inputs of video data conforming to different standards. To ensure that the signal processing unit ex507 of the LSIex500 can decode such inputs, the signal processing unit ex507 must support multiple standards. However, using separate signal processing units ex507 for each standard increases the circuit size of the LSIex500 and increases costs.

[0316] To solve this problem, a configuration is provided in which a decoding processing unit for executing the video decoding method described in each of the above embodiments is partially shared with a decoding processing unit conforming to conventional standards such as MPEG-2, MPEG4-AVC, and VC-1. An example of this configuration is shown in ex900 of FIG. 44A. For example, the video decoding method described in each of the above embodiments and a video decoding method conforming to the MPEG4-AVC standard share some of the processing content, such as entropy coding, inverse quantization, deblocking filtering, and motion compensation. A possible configuration is to share a decoding processing unit ex902 conforming to the MPEG4-AVC standard for the common processing content, and use a dedicated decoding processing unit ex901 for other processing content unique to one aspect of the present invention that does not conform to the MPEG4-AVC standard. In particular, since one aspect of the present invention is characterized by inter-prediction, for example, a dedicated decoding processing unit ex901 is used for inter-prediction, and a decoding processing unit is shared for any or all of the other processing, such as entropy decoding, inverse quantization, deblocking filtering, and motion compensation. Regarding the sharing of the decoding processing unit, for common processing content, the decoding processing unit for executing the video decoding method shown in each of the above embodiments may be shared, and for processing content specific to the MPEG4-AVC standard, a dedicated decoding processing unit may be used.

[0317] Another example of partially sharing processing is shown in ex1000 in Figure 44B. In this example, a dedicated decoding processing unit ex1001 corresponding to processing content specific to one aspect of the present invention, a dedicated decoding processing unit ex1002 corresponding to processing content specific to another conventional standard, and a shared decoding processing unit ex1003 corresponding to processing content common to the video decoding method according to one aspect of the present invention and the video decoding method of another conventional standard are used. Here, the dedicated decoding processing units ex1001 and ex1002 are not necessarily specialized for processing content specific to one aspect of the present invention or another conventional standard, and may be capable of performing other general-purpose processing. The configuration of this embodiment can also be implemented using an LSI ex500.

[0318] In this way, by sharing a decoding processing unit for processing content that is common between a video decoding method according to one embodiment of the present invention and a video decoding method of a conventional standard, it is possible to reduce the circuit size of the LSI and reduce costs. [Industrial Applicability]

[0319] The present invention can be used in video encoding devices and video decoding devices, and can also be used in information display devices and imaging devices such as televisions, digital video recorders, car navigation systems, mobile phones, digital still cameras, and digital video cameras. [Explanation of symbols]

[0320] 100, 300 Video encoding device 101 Subtraction section 102 Orthogonal transformation unit 103 Quantization section 104, 202 Inverse quantization section 105, 203 Inverse orthogonal transform unit 106, 204 Addition section 107, 205 Block Memory 108, 206 frame memory 109, 207 Intra prediction section 110, 208 Inter prediction section 111, 209 switches 112, 210 Inter-prediction control unit 113 Picture type determination unit 114, 211 Temporal prediction motion vector calculation unit 115, 212 colPic memory 116 Global Vector Storage Unit 117 Co-located Information Determination Unit 118 Variable-length coding section 200, 400 Video Decoding Device 201 Variable length decoding unit 310 First Encoding Unit 320 Second Encoding Unit 410 First Decoding Unit 420 Second Decoding Unit

Claims

1. A video decoding device that performs inter-prediction decoding using a motion vector on a current block included in a current picture, the video decoding device comprising: a control circuit and a storage device electrically connected to the control circuit; The control circuit decoding a temporal prediction restriction flag indicating whether or not temporal motion vector prediction using a temporal motion vector predictor, which is a motion vector of a block included in a decoded picture other than the current picture to be decoded, is used; If the temporal prediction restriction flag indicates that the temporal motion vector prediction is used, decoding a first parameter for specifying a prediction direction for calculating the temporal motion vector predictor; deriving a plurality of first motion vector predictor candidates including the temporal motion vector predictor based on the prediction direction specified by the first parameter; decoding a motion vector to be used in inter prediction decoding of the current block by using one of the plurality of first motion vector predictor candidates; If the temporal prediction restriction flag indicates that the temporal motion vector prediction is not used, deriving a plurality of second motion vector predictor candidates that do not include the temporal motion vector predictor; decoding a motion vector to be used in inter prediction decoding of the current block by using one of the plurality of second motion vector predictor candidates; not decoding the first parameter if the temporal prediction restriction flag indicates that the temporal motion vector prediction is not used; The plurality of first motion vector predictor candidates or the plurality of second motion vector predictor candidates include an alternative vector that replaces the temporal motion vector predictor. Video decoding device.

2. 1. A video encoding device that performs inter-prediction encoding using a motion vector on a current block included in a current picture, the device comprising: a control circuit and a storage device electrically connected to the control circuit; The control circuit encoding a temporal prediction restriction flag indicating whether or not temporal motion vector prediction using a temporal motion vector predictor, which is a motion vector of a block included in a coded picture other than the coding target picture, is to be used; If the temporal prediction restriction flag indicates that the temporal motion vector prediction is used, encoding a first parameter for specifying a prediction direction for calculating the temporal motion vector predictor; deriving a plurality of first motion vector predictor candidates including the temporal motion vector predictor based on the prediction direction specified by the first parameter; encoding a motion vector to be used in inter-prediction coding of the current block by using one of the plurality of first motion vector predictor candidates; If the temporal prediction restriction flag indicates that the temporal motion vector prediction is not used, deriving a plurality of second motion vector predictor candidates that do not include the temporal motion vector predictor; encoding a motion vector to be used in inter-prediction coding of the current block by using one of the plurality of second motion vector predictor candidates; if the temporal prediction restriction flag indicates that the temporal motion vector prediction is not used, not encoding the first parameter; The plurality of first motion vector predictor candidates or the plurality of second motion vector predictor candidates include an alternative vector that replaces the temporal motion vector predictor. Video encoding device.

3. The video decoding device according to claim 1 ; The video encoding device according to claim 2, A video encoding / decoding device.

Images