Image processing equipment and methods

Abstract

The invention relates to an image processing device and an image processing method that can improve the efficiency of encoding while preventing the reduction of the efficiency of encoding processing. The image processing device comprises an encoding mode setter which sets, in units of hierarchically structured encoding units, whether uncompressed mode is selected as the encoding mode for encoding image data, uncompressed mode is the encoding mode in which image data is output as encoded data, and an encoder which encodes image data in units of hierarchically structured encoding units according to the mode set by the encoding mode setter. The invention is applicable to, for example, an image processing device.

Description

[0001] This disclosure relates to an image processing apparatus and method, and more particularly to an image processing apparatus and method that can improve coding efficiency while suppressing a reduction in the efficiency of coding processing.

[0002] In recent years, devices that handle image information digitally and, with the aim of highly efficient information transmission and storage, utilize the redundancy inherent in image information to compress it using orthogonal transformations such as discrete cosine transforms and motion compensation, such as MPEG (Moving Picture Experts Group), are becoming widespread in both information distribution by broadcasting stations and information reception in general households.

[0003] In particular, MPEG2 (ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission) 13818-2) is defined as a general-purpose image encoding scheme and is a standard that covers both interleaved and sequential scanned images, as well as standard-resolution and high-resolution images, and is currently widely used in a wide range of professional and consumer applications. By using the MPEG2 compression scheme, for example, a standard-resolution interleaved scanned image with 720 x 480 pixels can be assigned a code amount (bitrate) of 4 to 8 Mbps, and a high-resolution interleaved scanned image with 1920 x 1088 pixels can be assigned a code amount (bitrate) of 18 to 22 Mbps, thereby achieving high compression ratio and good image quality.

[0004] MPEG2 was primarily intended for high-quality encoding suitable for broadcasting, but it did not support encoding methods with lower bitrates, i.e., higher compression ratios, than MPEG1. With the spread of mobile devices, the need for such encoding methods is expected to increase in the future, and in response to this, the MPEG4 encoding method was standardized. Regarding the image encoding method, its standard was approved as an international standard in December 1998 as ISO / IEC 14496-2.

[0005] Furthermore, in recent years, standardization has been underway for a standard called H.26L (ITU-T (International Telecommunication Union Telecommunication Standardization Sector) Q6 / 16 VCEG (Video Coding Expert Group)), initially intended for video encoding for video conferencing. Compared to conventional encoding methods such as MPEG2 and MPEG4, H.26L requires more computation for encoding and decoding, but it is known to achieve higher encoding efficiency. Currently, as part of the MPEG4 activities, standardization is being carried out as the Joint Model of Enhanced-Compression Video Coding, which is based on H.26L but incorporates features not supported by H.26L to achieve even higher encoding efficiency.

[0006] In terms of the standardization schedule, it became an international standard in March 2003 under the name H.264 and MPEG-4 Part 10 (Advanced Video Coding, hereinafter referred to as AVC).

[0007] Furthermore, as an extension of this, the standardization of FRExt (Fidelity Range Extension), which includes encoding tools necessary for professional use such as RGB, 4:2:2, and 4:4:4, as well as 8x8DCT (Discrete Cosine Transform) and quantization matrices that were defined in MPEG-2, was completed in February 2005. As a result, AVC became an encoding method that could accurately represent film noise contained in movies, and it came to be used in a wide range of applications such as Blu-ray Discs.

[0008] However, in recent years there has been a growing need for even higher compression ratios, such as the ability to compress images with four times the resolution of high-definition images, or the ability to distribute high-definition images in environments with limited transmission capacity, such as the internet. For this reason, the Video Coding Expert Group (VCEG) under the ITU-T is continuing to study ways to improve coding efficiency.

[0009] By the way, currently, for the purpose of further improving the coding efficiency beyond AVC, the ITU-T and JCTVC (Joint Collaboration Team - Video Coding), a joint standardization body of ISO / IEC, are promoting the standardization of a coding method called HEVC (High Efficiency Video Coding) (see, for example, Non-Patent Document 1).

[0010] In this HEVC coding method, a coding unit (CU (Coding Unit)) is defined as a processing unit similar to the macroblock in AVC. The size of this CU is not fixed at 16×16 pixels like the macroblock in AVC, but is specified in the image compression information for each sequence.

[0011] The CU is hierarchically composed from the largest LCU (Largest Coding Unit) to the smallest SCU (Smallest Coding Unit). That is, generally, it can be considered that the LCU corresponds to the macroblock in AVC, and the CUs in the lower hierarchy than the LCU (CUs smaller than the LCU) correspond to the sub-macroblocks in AVC.

[0012] By the way, there is a coding method that has a coding mode for encoding and outputting image data and a non-coding mode for outputting image data without encoding, and selects whether to use the coding mode or the non-coding mode in macroblock units, and can mix the coding mode and the non-coding mode within one picture (see, for example, Patent Document 1). In the AVC coding method, an I_PCM mode for outputting image data without encoding is also supported as mb_type (see, for example, Patent Document 2). This is used to guarantee the real-time operation of the arithmetic coding process when the amount of information in the coded data exceeds that of the input image when the quantization parameter is set to a small value such as QP = 0. Also, by using I-PCM, it is possible to achieve lossless coding.

[0013] In addition, methods for increasing internal calculations have been proposed (see, for example, Non-Patent Document 2). This makes it possible to reduce internal calculation errors in processes such as orthogonal transformations and motion compensation, thereby improving coding efficiency.

[0014] Furthermore, a method has been proposed that includes an FIR filter within the motion compensation loop (see, for example, Non-Patent Document 3). In the encoding device, by determining the FIR filter coefficients using a Wiener filter to minimize the error with the input image, it is possible to minimize degradation in the reference image and improve the encoding efficiency of the output image compression information.

[0015] Patent No. 3992303 Patent No. 4240283

[0016] "Test Model under Consideration",JCTVC-B205,Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG112nd Meeting:Geneva,CH,21-28 July, 2010Takeshi Chujoh,Reiko Noda,"Internal bit depth increase except frame memory",VCEG-AF07,ITU - Telecommunications Standardization SectorSTUDY GROUP 16 Question 6Video Coding Experts Group (VCEG)32nd Meeting: San Jose, USA, 20-21 April, 2007Takeshi Chujoh,Goki Yasuda,Naofumi Wada,Takashi Watanabe,Tomoo Yamakage,"Block-based Adaptive Loop Filter",VCEG-AI18,ITU - Telecommunications Standardization SectorSTUDY GROUP 16 Question 6Video Coding Experts Group (VCEG)35th Meeting: Berlin, Germany, 16-18 July, 2008

[0017] However, in the case of encoding schemes that define CUs, such as HEVC, and perform various processing using those CUs as units, macroblocks in AVC are considered to correspond to LCUs. However, if I_PCM can only be set in LCU units, the processing unit is a maximum of 128 x 128 pixels, which could lead to an increase in unnecessary encoding processing and a decrease in the efficiency of the encoding process. For example, it could become difficult to guarantee real-time operation of CABAC.

[0018] Furthermore, the encoding schemes proposed in Non-Patent Document 2 and Non-Patent Document 3 are not included in the AVC encoding scheme, and support for the I_PCM mode was not disclosed.

[0019] This disclosure has been made in view of the above circumstances and aims to improve coding efficiency while suppressing a reduction in the efficiency of coding processing.

[0020] One aspect of the present disclosure is an image processing apparatus comprising: an encoding mode setting unit that sets whether to select an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, as an encoding mode when encoding the image data, for each encoding unit having a hierarchical structure; and an encoding unit that encodes the image data for each encoding unit according to the mode set by the encoding mode setting unit.

[0021] The system may further include a shift processing control unit that controls the encoding unit for which the uncompressed mode has been set by the encoding mode setting unit to skip the shift processing that expands the bit precision when encoding or decoding, and a shift processing unit that performs the shift processing on the image data for the encoding unit that is controlled by the shift processing control unit to perform the shift processing.

[0022] The system may further include: a filter processing control unit that controls the encoding unit for which the uncompressed mode has been set by the encoding mode setting unit to skip the filter processing that performs filtering on the locally decoded image; a filter coefficient calculation unit that calculates the filter coefficients for the filter processing using the image data of the encoding unit that is controlled by the filter processing control unit to perform the filter processing; and a filter processing unit that performs the filter processing for each block, which is the unit of the filter processing, using the filter coefficients calculated by the filter coefficient calculation unit.

[0023] The filter processing unit can perform the filter processing only on pixels included in the current block that is to be processed, which are controlled by the filter processing control unit to perform the filter processing.

[0024] The system may further include a filter identification information generation unit that generates filter identification information for each block, which is identification information indicating whether the filtering process is performed.

[0025] The filter processing unit can perform an adaptive loop filter, which is an adaptive filtering process using classification processing, on the locally decoded image.

[0026] The encoding mode setting can be set to the uncompressed mode if the amount of encoding of the encoded data obtained from the image data of the current encoding unit that is the target of the encoding process is less than or equal to the input data amount, which is the amount of data of the image data of the current encoding unit.

[0027] The system further comprises an input data amount calculation unit for calculating the input data amount, and the coding mode setting unit can compare the input data amount calculated by the input data amount calculation unit with the coding amount, with respect to the current coding unit.

[0028] The system may further include an identification information generation unit that generates identification information for each encoding unit indicating whether the uncompressed mode has been set by the encoding mode setting unit.

[0029] Another aspect of the present disclosure is an image processing method for an image processing apparatus, wherein an encoding mode setting unit sets whether to select an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, as the encoding mode when encoding the image data, for each encoding unit having a hierarchical structure, and the encoding unit encodes the image data for each encoding unit according to the set mode.

[0030] Another aspect of the present disclosure is an image processing apparatus comprising: an encoding mode determination unit that determines for each encoding unit having a hierarchical structure whether an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, has been selected as the encoding mode when encoding the image data; and a decoding unit that decodes the encoding result for each encoding unit according to the mode determined by the encoding mode determination unit.

[0031] The system may further include a shift processing control unit that controls the encoding unit to skip the shift processing that expands the bit precision when encoding or decoding, for encoding units that the uncompressed mode has been determined to be selected by the encoding mode determination unit, and a shift processing unit that performs the shift processing on the image data for encoding units that the shift processing control unit has controlled to perform the shift processing on.

[0032] The system includes a filter processing control unit that controls the encoding unit to skip the filtering process that performs filtering on the locally decoded image for encoding units in which the uncompressed mode has been selected, and a filter processing unit that performs the filtering process on the image data for each block which is the unit of the filtering process, wherein the filter processing unit can perform the filtering process only on pixels included in the current block which is the target of processing and which have been controlled by the filter processing control unit to perform the filtering process.

[0033] The filter processing unit can perform an adaptive loop filter, which is an adaptive filtering process using a classification process, on the locally decoded image.

[0034] The filter processing unit can perform the filter processing only when the filter identification information indicating whether the filter processing has been performed indicates that the filter processing has been performed on the image data of the current block that is the target of processing, and the filter processing control unit is controlled to perform the filter processing on all pixels included in the current block.

[0035] The encoding mode determination unit can determine whether the uncompressed mode has been selected for each encoding unit based on identification information indicating whether the uncompressed mode has been selected.

[0036] Another aspect of the present disclosure is an image processing method for an image processing apparatus, wherein an encoding mode determination unit determines for each encoding unit having a hierarchical structure whether an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, has been selected as the encoding mode when encoding the image data, and a decoding unit decodes the encoded data for each encoding unit according to the determined mode.

[0037] In one aspect of the present disclosure, when encoding image data, an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, is set for each encoding unit having a hierarchical structure, and the image data is encoded for each encoding unit according to the set mode.

[0038] In other aspects of the present disclosure, when encoding image data, it is determined for each encoding unit having a hierarchical structure whether an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, has been selected as the encoding mode, and the encoded data is decoded for each encoding unit according to the determined mode.

[0039] According to this disclosure, images can be processed. In particular, encoding efficiency can be improved while suppressing a reduction in the efficiency of the encoding process.

[0040] This is a block diagram of an image encoding device that outputs image compression information based on the AVC encoding scheme. This is a block diagram of an image decoding device that takes image compression information based on the AVC encoding scheme as input. This is a diagram showing examples of macroblock types. This is a diagram illustrating an example of the configuration of a coding unit. This is a diagram illustrating a method for increasing the bit amount in internal calculations. This is a diagram illustrating an adaptive loop filter. This is a block diagram showing an example of the main configuration of an image encoding device. This is a block diagram showing an example of the main configuration of the reversible encoding unit, loop filter, and PCM encoding unit of Figure 7. This is a block diagram showing an example of the main configuration of the PCM determination unit of Figure 8. This is a flowchart illustrating an example of the encoding process flow. This is a flowchart illustrating an example of the PCM encoding control process flow. This is a flowchart illustrating an example of the PCM encoding process flow. This is a flowchart illustrating an example of the reference image generation process flow. This is a flowchart illustrating an example of the loop filter process flow. This is a block diagram showing an example of the main configuration of an image decoding device. This is a block diagram showing an example of the main configuration of the reversible decoding unit, loop filter, and PCM decoding unit of Figure 15. This is a flowchart illustrating an example of the decoding process flow. This is a flowchart following Figure 17 illustrating an example of the decoding process flow. This is a flowchart illustrating an example of the loop filter process flow. This is a diagram illustrating an example of I_PCM information. This is a block diagram showing an example of the main configuration of a personal computer. This is a block diagram showing the main configurations of a television receiver. This is a block diagram showing the main configurations of a mobile phone. This is a block diagram showing the main configurations of a hard disk recorder. This is a block diagram showing the main configurations of a camera.

[0041] The following describes embodiments for implementing this technology. The description will be given in the following order. 1. First Embodiment (Image Encoding Device) 2. Second Embodiment (Image Decoding Device) 3. Third Embodiment (Personal Computer) 4. Fourth Embodiment (Television Receiver) 5. Fifth Embodiment (Mobile Phone) 6. Sixth Embodiment (Hard Disk Recorder) 7. Seventh Embodiment (Camera)

[0042] <1. First Embodiment> [AVC encoding image encoding device] Figure 1 shows the configuration of one embodiment of an image encoding device that encodes images using the H.264 and MPEG (Moving Picture Experts Group) 4 Part 10 (AVC (Advanced Video Coding)) encoding schemes.

[0043] The image encoding device 100 shown in Figure 1 is a device that encodes and outputs images using an encoding method based on the AVC standard. As shown in Figure 1, the image encoding device 100 has an A / D conversion unit 101, a screen sorting buffer 102, a calculation unit 103, an orthogonal transformation unit 104, a quantization unit 105, a reversible encoding unit 106, and a storage buffer 107. The image encoding device 100 also has an inverse quantization unit 108, an inverse orthogonal transformation unit 109, a calculation unit 110, a deblocking filter 111, a frame memory 112, a selection unit 113, an intra prediction unit 114, a motion prediction / compensation unit 115, a selection unit 116, and a rate control unit 117.

[0044] The A / D conversion unit 101 performs A / D conversion on the input image data and outputs it to the screen reordering buffer 102 for storage. The screen reordering buffer 102 rearranges the images of the stored display order frames in the order of frames for encoding according to the GOP (Group of Picture) structure. The screen reordering buffer 102 supplies the rearranged image frames to the calculation unit 103. The screen reordering buffer 102 also supplies the rearranged image frames to the intra prediction unit 114 and the motion prediction / compensation unit 115.

[0045] The calculation unit 103 subtracts the predicted image supplied from the intra prediction unit 114 or the motion prediction / compensation unit 115 via the selection unit 116 from the image read from the screen sorting buffer 102, and outputs the difference information to the orthogonal transformation unit 104.

[0046] For example, in the case of an image that is intra-coded, the calculation unit 103 subtracts the predicted image supplied from the intra-prediction unit 114 from the image read from the screen sorting buffer 102. Also, for example, in the case of an image that is inter-coded, the calculation unit 103 subtracts the predicted image supplied from the motion prediction and compensation unit 115 from the image read from the screen sorting buffer 102.

[0047] The orthogonal transformation unit 104 applies orthogonal transformations such as discrete cosine transform and Carunen-Loebe transform to the difference information supplied from the calculation unit 103, and supplies the resulting transformation coefficients to the quantization unit 105.

[0048] The quantization unit 105 quantizes the conversion coefficients output by the orthogonal transformation unit 104. The quantization unit 105 sets quantization parameters based on information about the target value of the code amount supplied from the rate control unit 117 and performs quantization. The quantization unit 105 supplies the quantized conversion coefficients to the reversible coding unit 106.

[0049] The reversible coding unit 106 applies reversible coding such as variable-length coding or arithmetic coding to the quantized conversion coefficients. Since the coefficient data is quantized under the control of the rate control unit 117, the code amount becomes (or approximates) the target value set by the rate control unit 117.

[0050] The reversible coding unit 106 obtains information indicating intra-prediction from the intra-prediction unit 114, and information indicating inter-prediction mode and motion vector information from the motion prediction / compensation unit 115. The information indicating intra-prediction (in-screen prediction) will also be referred to below as intra-prediction mode information. The information indicating information mode for inter-prediction (inter-screen prediction) will also be referred to below as inter-prediction mode information.

[0051] The reversible coding unit 106 encodes the quantized conversion coefficients and also incorporates various information such as filter coefficients, intra-prediction mode information, inter-prediction mode information, and quantization parameters as part of the header information of the encoded data (multiplexing). The reversible coding unit 106 supplies the encoded data obtained to the storage buffer 107 for storage.

[0052] For example, the reversible coding unit 106 performs reversible coding processing such as variable-length coding or arithmetic coding. Examples of variable-length coding include CAVLC (Context-Adaptive Variable Length Coding) as defined in the H.264 / AVC scheme. Examples of arithmetic coding include CABAC (Context-Adaptive Binary Arithmetic Coding).

[0053] The storage buffer 107 temporarily holds the encoded data supplied from the reversible encoding unit 106 and, at a predetermined timing, outputs it as an encoded image encoded in the H.264 / AVC format to, for example, a recording device or transmission line (not shown) in a later stage.

[0054] The conversion coefficients quantized in the quantization unit 105 are also supplied to the inverse quantization unit 108. The inverse quantization unit 108 inversely quantizes the quantized conversion coefficients in a manner corresponding to the quantization performed by the quantization unit 105. The inverse quantization unit 108 supplies the obtained conversion coefficients to the inverse orthogonal conversion unit 109.

[0055] The inverse orthogonal transformation unit 109 performs an inverse orthogonal transformation on the supplied transformation coefficients in a manner corresponding to the orthogonal transformation processing by the orthogonal transformation unit 104. The output after the inverse orthogonal transformation (restored difference information) is supplied to the calculation unit 110.

[0056] The calculation unit 110 adds the inverse orthogonal transformation result supplied from the inverse orthogonal transformation unit 109, i.e., the restored difference information, to the prediction image supplied from the intra prediction unit 114 or the motion prediction / compensation unit 115 via the selection unit 116, to obtain a locally decoded image (decoded image).

[0057] For example, if the difference information corresponds to an image that is subjected to intra-coding, the calculation unit 110 adds the predicted image supplied from the intra-prediction unit 114 to the difference information. Also, for example, if the difference information corresponds to an image that is subjected to inter-coding, the calculation unit 110 adds the predicted image supplied from the motion prediction and compensation unit 115 to the difference information.

[0058] The summation result is supplied to the deblock filter 111 or the frame memory 112.

[0059] The deblocking filter 111 removes block distortion from the decoded image by performing deblocking filtering as appropriate. The deblocking filter 111 supplies the filtering result to the frame memory 112. The decoded image output from the arithmetic unit 110 can be supplied to the frame memory 112 without going through the deblocking filter 111. In other words, the deblocking filtering of the deblocking filter 111 can be omitted.

[0060] The frame memory 112 stores the supplied decoded image and, at a predetermined timing, outputs the stored decoded image as a reference image to the intra prediction unit 114 or the motion prediction / compensation unit 115 via the selection unit 113.

[0061] For example, in the case of an image that is intra-coded, the frame memory 112 supplies the reference image to the intra-prediction unit 114 via the selection unit 113. Also, for example, in the case of inter-coded, the frame memory 112 supplies the reference image to the motion prediction and compensation unit 115 via the selection unit 113.

[0062] If the reference image supplied from the frame memory 112 is an image to be intra-coded, the selection unit 113 supplies the reference image to the intra-prediction unit 114. Also, if the reference image supplied from the frame memory 112 is an image to be inter-coded, the selection unit 113 supplies the reference image to the motion prediction and compensation unit 115.

[0063] The intra prediction unit 114 performs intra prediction (in-screen prediction) which generates a predicted image using the pixel values ​​in the picture to be processed supplied from the frame memory 112 via the selection unit 113. The intra prediction unit 114 performs this intra prediction in one of several pre-prepared modes (intra prediction modes).

[0064] In the H.264 image information coding scheme, intra 4x4 prediction mode, intra 8x8 prediction mode, and intra 16x16 prediction mode are defined for the luminance signal, and for the chrominance signal, it is possible to define a prediction mode independent of the luminance signal for each macroblock. For the intra 4x4 prediction mode, one intra prediction mode is defined for each 4x4 luminance block, and for the intra 8x8 prediction mode, one intra prediction mode is defined for each 8x8 luminance block. For the intra 16x16 prediction mode and for the chrominance signal, one prediction mode is defined for each macroblock.

[0065] The intra prediction unit 114 generates prediction images for all candidate intra prediction modes, evaluates the cost function value of each prediction image using the input image supplied from the screen sorting buffer 102, and selects the optimal mode. Once the intra prediction unit 114 has selected the optimal intra prediction mode, it supplies the prediction image generated in that optimal mode to the calculation unit 103 and the calculation unit 110 via the selection unit 116.

[0066] As described above, the intra prediction unit 114 also appropriately supplies information such as intra prediction mode information indicating the adopted intra prediction mode to the reversible coding unit 106.

[0067] The motion prediction and compensation unit 115 performs motion prediction (interpretation) on the image to be intercoded using the input image supplied from the screen sorting buffer 102 and the reference image supplied from the frame memory 112 via the selection unit 113, and performs motion compensation processing according to the detected motion vector to generate a predicted image (interpretation image information). The motion prediction and compensation unit 115 performs such interpretation in one of several pre-prepared modes (interpretation modes).

[0068] The motion prediction and compensation unit 115 generates prediction images for all candidate interpretation modes, evaluates the cost function value of each prediction image, and selects the optimal mode. The motion prediction and compensation unit 115 supplies the generated prediction images to the calculation unit 103 and the calculation unit 110 via the selection unit 116.

[0069] The motion prediction and compensation unit 115 also supplies the reversible coding unit 106 with interprediction mode information indicating the adopted interprediction mode and motion vector information indicating the calculated motion vector.

[0070] The selection unit 116 supplies the output of the intra prediction unit 114 to the calculation unit 103 or calculation unit 110 in the case of an image to be intra-coded, and supplies the output of the motion prediction / compensation unit 115 to the calculation unit 103 or calculation unit 110 in the case of an image to be inter-coded.

[0071] The rate control unit 117 controls the rate of the quantization operation of the quantization unit 105 based on the compressed image stored in the storage buffer 107 so that no overflow or underflow occurs.

[0072] [AVC encoding image decoding device] Figure 2 is a block diagram showing a main configuration example of an image decoding device that achieves image compression by orthogonal transformations such as discrete cosine transform or Carunen-Loebé transform and motion compensation. The image decoding device 200 shown in Figure 2 is a decoding device corresponding to the image encoding device 100 in Figure 1.

[0073] The encoded data encoded by the image encoding device 100 is supplied to the image decoding device 200 corresponding to the image encoding device 100 via any path, such as a transmission line or recording medium, and decoded.

[0074] As shown in Figure 2, the image decoding device 200 includes a storage buffer 201, a reversible decoding unit 202, an inverse quantization unit 203, an inverse orthogonal transformation unit 204, an arithmetic unit 205, a deblocking filter 206, a screen sorting buffer 207, and a D / A conversion unit 208. The image decoding device 200 also includes a frame memory 209, a selection unit 210, an intra prediction unit 211, a motion prediction and compensation unit 212, and a selection unit 213.

[0075] The storage buffer 201 stores the transmitted encoded data. This encoded data is encoded by the image encoding device 100. The reversible decoding unit 202 decodes the encoded data read from the storage buffer 201 at a predetermined timing using a method corresponding to the encoding method of the reversible encoding unit 106 in Figure 1.

[0076] Also, when the frame is intra-coded, intra-prediction mode information is stored in the header part of the coded data. The reversible decoding unit 202 also decodes this intra-prediction mode information and supplies the information to the intra-prediction unit 211. On the other hand, when the frame is inter-coded, motion vector information is stored in the header part of the coded data. The reversible decoding unit 202 also decodes this motion vector information and supplies the information to the motion prediction / compensation unit 212.

[0077] The inverse quantization unit 203 inverse-quantizes the coefficient data (quantized coefficients) obtained by decoding by the reversible decoding unit 202 in a method corresponding to the quantization method of the quantization unit 105 in FIG. 1. That is, the inverse quantization unit 203 performs inverse quantization of the quantized coefficients in the same method as the inverse quantization unit 108 in FIG. 1.

[0078] The inverse quantization unit 203 supplies the inverse-quantized coefficient data, that is, the orthogonal transform coefficients, to the inverse orthogonal transform unit 204. The inverse orthogonal transform unit 204 inverse-orthogonally transforms the orthogonal transform coefficients in a method corresponding to the orthogonal transform method of the orthogonal transform unit 104 in FIG. 1 (the same method as the inverse orthogonal transform unit 109 in FIG. 1) to obtain decoded residual data corresponding to the residual data before orthogonal transformation in the image coding device 100. For example, a fourth-order inverse orthogonal transform is performed.

[0079] The decoded residual data obtained by inverse orthogonal transformation is supplied to the arithmetic unit 205. Also, a predicted image is supplied to the arithmetic unit 205 from the intra-prediction unit 211 or the motion prediction / compensation unit 212 via the selection unit 213.

[0080] The arithmetic unit 205 adds the decoded residual data and the predicted image to obtain decoded image data corresponding to the image data before the predicted image is subtracted by the arithmetic unit 103 of the image coding device 100. The arithmetic unit 205 supplies the decoded image data to the deblocking filter 206.

[0081] After removing the block distortion of the supplied decoded image, the deblocking filter 206 supplies it to the screen rearrangement buffer 207.

[0082] The screen rearrangement buffer 207 rearranges the images. That is, the order of the frames rearranged by the screen rearrangement buffer 102 in FIG. 1 for the encoding order is rearranged to the original display order. The D / A conversion unit 208 D / A-converts the image supplied from the screen rearrangement buffer 207, outputs it to a display (not shown), and causes it to be displayed.

[0083] The output of the deblocking filter 206 is further supplied to the frame memory 209.

[0084] The frame memory 209, selection unit 210, intra prediction unit 211, motion prediction / compensation unit 212, and selection unit 213 respectively correspond to the frame memory 112, selection unit 113, intra prediction unit 114, motion prediction / compensation unit 115, and selection unit 116 of the image encoding device 100.

[0085] The selection unit 210 reads out the inter-processed image and the reference image from the frame memory 209 and supplies them to the motion prediction / compensation unit 212. Also, the selection unit 210 reads out the image used for intra prediction from the frame memory 209 and supplies it to the intra prediction unit 211.

[0086] Information indicating the intra prediction mode obtained by decoding the header information and the like is appropriately supplied from the reversible decoding unit 202 to the intra prediction unit 211. The intra prediction unit 211 generates a predicted image from the reference image acquired from the frame memory 209 based on this information, and supplies the generated predicted image to the selection unit 213.

[0087] The motion prediction / compensation unit 212 acquires information (prediction mode information, motion vector information, reference frame information, flag, and various parameters, etc.) obtained by decoding the header information from the reversible decoding unit 202.

[0088] The motion prediction / compensation unit 212 generates a predicted image from the reference image acquired from the frame memory 209 based on the information supplied from the reversible decoding unit 202, and supplies the generated predicted image to the selection unit 213.

[0089] The selection unit 213 selects the predicted image generated by the motion prediction / compensation unit 212 or the intra prediction unit 211, and supplies it to the arithmetic unit 205.

[0090] [Macroblock type] Incidentally, as shown in Patent Document 1, there is an encoding method that has an encoding mode for encoding and outputting image data, and an unencoded mode for outputting image data without encoding, and allows the user to select whether to use the encoding mode or the unencoded mode on a macroblock basis, and allows the encoding mode and the unencoded mode to coexist within a single picture. As shown in Patent Document 2, in the AVC encoding method as shown in Figure 3, one of the macroblock types (mb_type) is the I_PCM (Intra-block pulse code modulation) (uncompressed) mode which outputs image data without encoding.

[0091] This is used to ensure real-time operation of the arithmetic coding process when the amount of information in the coded data exceeds that of the input image, if the quantization parameter is set to a small value such as QP=0. Furthermore, lossless coding can be achieved by using I-PCM mode (uncompressed mode).

[0092] [Cost function] By the way, in AVC coding schemes, selecting the appropriate prediction mode is crucial for achieving higher coding efficiency.

[0093] An example of such a selection method is the method implemented in the H.264 / MPEG-4 AVC reference software called JM (Joint Model), which is available at http: / / iphome.hhi.de / suehring / tml / index.htm.

[0094] In JM, it is possible to select two mode determination methods, High Complexity Mode and Low Complexity Mode, as described below. In both cases, a cost function value for each prediction mode is calculated, and the prediction mode that minimizes this value is selected as the optimal mode for the block or macroblock.

[0095] The cost function in High Complexity Mode is given by equation (1) below.

[0096] Cost(Mode∈Ω)=D+λ*R...(1)

[0097] Here, Ω is the entire set of candidate modes for encoding the block or macroblock, D is the difference energy between the decoded image and the input image when encoded with the prediction mode Mode, λ is a Lagrange multiplier given as a function of the quantization parameters, and R is the total code amount when encoded with the mode Mode, including the orthogonal transformation coefficients.

[0098] In other words, in order to perform encoding in High Complexity Mode, it is necessary to perform a provisional encoding process once for all candidate modes (Mode) in order to calculate the above parameters D and R, which requires a higher amount of computation.

[0099] The cost function in Low Complexity Mode is given by equation (2) below.

[0100] Cost(Mode∈Ω)=D+QP2Quant(QP)*HeaderBit...(2)

[0101] Here, D is the difference energy between the predicted image and the input image, unlike in the High Complexity Mode. QP2Quant(QP) is given as a function of the quantization parameter QP, and HeaderBit is a code quantity relating to information belonging to the Header, such as motion vectors and modes, and does not include orthogonal transformation coefficients.

[0102] In other words, in Low Complexity Mode, prediction processing is required for each candidate mode, but decoding the image is not necessary, and therefore encoding processing is not required. For this reason, it can be implemented with less computation than in High Complexity Mode.

[0103] [Coding Unit] Next, we will explain the coding unit as defined in the HEVC coding scheme described in Non-Patent Document 1.

[0104] A Coding Unit (CU), also known as a Coding Tree Block (CTB), is a picture-level sub-region of an image that plays a similar role to a macroblock in AVC, and is a hierarchical coding unit. While macroblocks are fixed in size to 16x16 pixels, the size of a CU is not fixed and is specified in the image compression information for each sequence.

[0105] In particular, the largest CU is called the LCU (Largest Coding Unit), and the smallest CU is called the SCU (Smallest Coding Unit). For example, in the sequence parameter set (SPS) included in the image compression information, the size of these regions is specified, but each is limited to a square and a size that can be expressed as a power of 2.

[0106] Figure 4 shows an example of a coding unit as defined in HEVC. In the example in Figure 4, the size of the LCU is 128, and the maximum hierarchy depth is 5. When the split_flag value is "1", a CU of size N×N is split into N×N CUs, which are one level lower.

[0107] Furthermore, the CU is divided into Prediction Units (PUs), which are regions (partial regions of the image on a picture-by-picture basis) that serve as processing units for intra or interpretation, and into Transform Units (TUs), which are regions (partial regions of the image on a picture-by-picture basis) that serve as processing units for orthogonal transformations. Currently, in HEVC, in addition to 4x4 and 8x8, 16x16 and 32x32 orthogonal transformations can be used.

[0108] [IBDI] Incidentally, Non-Patent Document 2 proposes a method for increasing internal processing (IBDI (Internal bit depth increase except frame memory)) as shown in Figure 5. In this method, as shown in Figure 5, the bit depth of the data is extended from, for example, 8 bits to 12 bits in the quantization process, reversible coding process, dequantization process, filtering process, prediction process, and reversible decoding process of the encoding and decoding devices. This makes it possible to reduce internal calculation errors in processes such as orthogonal transformations and motion compensation, and to improve encoding efficiency.

[0109] [BALF] Incidentally, Non-Patent Document 3 proposes a method, as shown in Figure 5, in which an FIR filter is placed within the motion compensation loop, and adaptive loop filtering (BALF (Block-based Adaptive Loop Filter)) is performed using that filter. In the encoding device, by determining the FIR filter coefficients using a Wiener filter to minimize the error with the input image, it is possible to minimize degradation in the reference image and improve the encoding efficiency of the output image compression information.

[0110] [Efficiency of encoding processing] Incidentally, in encoding schemes like HEVC, where CUs are defined and various processing is performed using these CUs as units, macroblocks in AVC can be considered equivalent to LCUs. However, since CUs have a hierarchical structure as shown in Figure 4, the size of the top-level LCU is generally set to be larger than that of an AVC macroblock, for example, 128 x 128 pixels.

[0111] Therefore, in this encoding scheme as well, if the I_PCM mode is set on an LCU basis, as in the case of AVC, the processing unit becomes larger than in the case of AVC, for example, to a 128 x 128 pixel unit.

[0112] The modes for intra-prediction and inter-prediction are determined by calculating and comparing cost function values ​​as described above. That is, prediction and coding are performed in all modes, the cost function value for each is calculated, the optimal mode is selected, and coded data is generated in that optimal mode.

[0113] However, when the I_PCM mode is adopted, the encoded data generated in the optimal mode is discarded, and the input image (unencoded data) is used as the encoding result. Therefore, when the I_PCM mode is selected, all processing required to generate encoded data in the optimal mode becomes unnecessary. In other words, as the unit of control for selecting the I_PCM mode increases, the amount of unnecessary processing increases even further. In other words, as mentioned above, if the LCU is required to choose whether or not to adopt the I_PCM mode, there is a risk that the efficiency of the encoding process will decrease even further. Therefore, for example, there is a risk that it will become difficult to guarantee the real-time operation of CABAC.

[0114] Furthermore, the aforementioned technologies such as IBDI and BALF are not included in the AVC encoding scheme, and it was unclear how to control these processes when the I_PCM mode is adopted.

[0115] In this implementation, the selection of the I_PCM mode (uncompressed mode) can be controlled in more detail, thereby improving encoding efficiency while suppressing a reduction in the efficiency of the encoding process. In addition, the execution of IBDI and BALF can be appropriately controlled according to the selection of the I_PCM mode, thereby further suppressing a reduction in the efficiency of the encoding process.

[0116] [Image encoding device] Figure 7 is a block diagram showing a typical configuration of an image coding device.

[0117] The image encoding device 300 shown in Figure 7 is basically the same as the image encoding device 100 in Figure 1 and encodes image data. As shown in Figure 7, the image encoding device 300 has an A / D conversion unit 301, a screen reordering buffer 302, an adaptive left shift unit 303, a calculation unit 304, an orthogonal transformation unit 305, a quantization unit 306, a reversible encoding unit 307, and a storage buffer 308. The image encoding device 300 also has an inverse quantization unit 309, an inverse orthogonal transformation unit 310, a calculation unit 311, a loop filter 312, an adaptive right shift unit 313, a frame memory 314, an adaptive left shift unit 315, a selection unit 316, an intra prediction unit 317, a motion prediction / compensation unit 318, a selection unit 319, and a rate control unit 320.

[0118] The image encoding device 300 further includes a PCM encoding unit 321.

[0119] The A / D conversion unit 301 performs A / D conversion on the input image data, similar to the case of the A / D conversion unit 101. The A / D conversion unit 301 supplies the converted image data (digital data) to the screen reordering buffer 302 for storage. The screen reordering buffer 302, similar to the case of the screen reordering buffer 102, rearranges the images of the stored display order frames in the order of the frames for encoding according to the GOP (Group of Picture) structure. The screen reordering buffer 302 supplies the rearranged image frames to the adaptive left shift unit 303.

[0120] The screen reordering buffer 302 also supplies the reversible encoding unit 307 and the PCM encoding unit 321 with the reordered frames.

[0121] The adaptive left shift unit 303 is controlled by the PCM encoding unit 321 and shifts the image data read from the screen sorting buffer 302 to the left, increasing its bit depth by a predetermined number of bits (for example, 4 bits). For example, the adaptive left shift unit 303 increases the bit depth of the image data read from the screen sorting buffer 302 from 8 bits to 12 bits. By increasing the bit depth in this way, the accuracy of internal calculations in each process such as orthogonal transformation processing, quantization processing, reversible encoding processing, and prediction processing can be improved and errors can be suppressed.

[0122] This left shift amount (bit amount) is arbitrary and may be a fixed value or variable. Also, this left shift process may be omitted (skipped) by the control of the PCM encoding unit 321.

[0123] The adaptive left shift unit 303 supplies the image data after the left shift processing to the calculation unit 304 (or, if the processing is omitted (skipped), the image data output from the screen rearrangement buffer 302 as is). The adaptive left shift unit 303 also supplies the image data to the intra prediction unit 317 and the motion prediction / compensation unit 318.

[0124] The calculation unit 304 subtracts the predicted image supplied from the intra prediction unit 317 or the motion prediction / compensation unit 318 via the selection unit 319 from the image supplied from the adaptive left shift unit 303, similar to the case of the calculation unit 103. The calculation unit 304 outputs the difference information to the orthogonal transformation unit 305.

[0125] For example, in the case of an image that is intra-coded, the calculation unit 304 subtracts the predicted image supplied by the intra-prediction unit 317 from the image supplied by the adaptive left-shift unit 303. Also, for example, in the case of an image that is inter-coded, the calculation unit 304 subtracts the predicted image supplied by the motion prediction and compensation unit 318 from the image supplied by the adaptive left-shift unit 303.

[0126] The orthogonal transformation unit 305 performs orthogonal transformations such as discrete cosine transform and Carunen-Loebe transform on the difference information supplied from the calculation unit 304, similar to the case of the orthogonal transformation unit 104. The method of this orthogonal transformation is arbitrary. The orthogonal transformation unit 305 supplies the transformation coefficients to the quantization unit 306.

[0127] The quantization unit 306 quantizes the conversion coefficients supplied from the orthogonal transformation unit 305, similar to the case of the quantization unit 105. The quantization unit 306 sets quantization parameters based on information regarding the target value of the code amount supplied from the rate control unit 320 and performs the quantization. Note that this quantization method is arbitrary. The quantization unit 306 supplies the quantized conversion coefficients to the reversible coding unit 307.

[0128] The reversible coding unit 307, similar to the case of the reversible coding unit 106, applies reversible coding such as variable-length coding and arithmetic coding to the conversion coefficients quantized in the quantization unit 306. Since the coefficient data is quantized under the control of the rate control unit 320, this code value becomes (or approximates) the target value set by the rate control unit 320.

[0129] When the I_PCM mode is selected by the PCM encoding unit 321, the reversible encoding unit 307 uses the input image (unencoded data) supplied from the screen reordering buffer 302 as the encoding result (i.e., encoding is actually omitted (skipped)).

[0130] The reversible coding unit 307 also obtains information indicating the intra-prediction mode from the intra-prediction unit 317, and information indicating the inter-prediction mode and motion vector information from the motion prediction and compensation unit 318. Furthermore, the reversible coding unit 307 obtains the filter coefficients used in the loop filter 312.

[0131] The reversible coding unit 307 encodes these filter coefficients, information indicating the modes of intra-prediction and inter-prediction, and various other information such as quantization parameters, in the same manner as the reversible coding unit 106, and incorporates them as part of the header information of the encoded data (multiplexed). The reversible coding unit 307 supplies the encoded data obtained (including unencoded data in the case of I_PCM mode) to the storage buffer 308 for storage.

[0132] For example, in the reversible coding unit 307, reversible coding processing such as variable-length coding or arithmetic coding is performed, similar to the case of the reversible coding unit 106. Examples of variable-length coding include CAVLC (Context-Adaptive Variable Length Coding) as defined in the H.264 / AVC scheme. Examples of arithmetic coding include CABAC (Context-Adaptive Binary Arithmetic Coding). Of course, the reversible coding unit 307 may also perform coding using methods other than these.

[0133] The storage buffer 308, similar to the storage buffer 107, temporarily holds the encoded data (including unencoded data in the case of I_PCM mode) supplied from the reversible encoding unit 307. At a predetermined timing, the storage buffer 308 outputs the stored encoded data to, for example, a subsequent recording device (recording medium) or transmission line (not shown).

[0134] The conversion coefficients quantized in the quantization unit 306 are also supplied to the inverse quantization unit 309. The inverse quantization unit 309 inversely quantizes the quantized conversion coefficients in the same way as in the inverse quantization unit 108, using a method corresponding to the quantization by the quantization unit 306. This inverse quantization method can be any method that corresponds to the quantization process by the quantization unit 306. The inverse quantization unit 309 supplies the obtained conversion coefficients to the inverse orthogonal transformation unit 310.

[0135] The inverse orthogonal transformation unit 310 inversely transforms the transformation coefficients supplied from the inverse quantization unit 309 in a manner corresponding to the orthogonal transformation process by the orthogonal transformation unit 305, similar to the case of the inverse orthogonal transformation unit 109. This inverse orthogonal transformation method can be any method that corresponds to the orthogonal transformation process by the orthogonal transformation unit 305. The inversely orthogonal transformed output (restored difference information) is supplied to the calculation unit 311.

[0136] The calculation unit 311, similar to the calculation unit 110, adds the inverse orthogonal transformation result supplied from the inverse orthogonal transformation unit 310, i.e., the restored difference information, to the prediction image supplied from the intra prediction unit 317 or the motion prediction / compensation unit 318 via the selection unit 319, to obtain a locally decoded image (decoded image).

[0137] For example, if the difference information corresponds to an image that is subjected to intra-coding, the calculation unit 311 adds the predicted image supplied from the intra-prediction unit 317 to the difference information. Also, for example, if the difference information corresponds to an image that is subjected to inter-coding, the calculation unit 311 adds the predicted image supplied from the motion prediction and compensation unit 318 to the difference information.

[0138] The summation result (decoded image) is supplied to the loop filter 312 or the adaptive right shift unit 313.

[0139] The loop filter 312 includes a deblocking filter and an adaptive loop filter, and performs appropriate filtering on the decoded image supplied from the arithmetic unit 311. For example, the loop filter 312 removes block distortion from the decoded image by performing deblocking filtering on the decoded image in the same way as the deblocking filter 111. Alternatively, for example, the loop filter 312 is controlled by the PCM encoding unit 321 to improve image quality by performing loop filtering on the result of the deblocking filtering (the decoded image from which block distortion has been removed) using a Wiener filter. Note that this adaptive loop filtering may be omitted (skipped) by the control of the PCM encoding unit 321.

[0140] The loop filter 312 may perform arbitrary filtering on the decoded image. The loop filter 312 may also, if necessary, supply the filter coefficients used in the filtering process to the reversible encoding unit 307 and have it encoded.

[0141] The loop filter 312 supplies the filtering result (the decoded image after filtering) to the adaptive right shift unit 313. As described above, the decoded image output from the calculation unit 311 can be supplied to the adaptive right shift unit 313 without going through the loop filter 312. In other words, the filtering by the loop filter 312 can be omitted (skipped).

[0142] The adaptive right shift unit 313 is controlled by the PCM encoding unit 321 and shifts the image data supplied from the arithmetic unit 311 or the loop filter 312 to the right, reducing its bit depth by a predetermined number of bits (for example, 4 bits). In other words, the adaptive right shift unit 313 shifts to the right by the number of bits that were shifted to the left by the adaptive left shift unit 303, returning the bit depth of the image data to its state before the left shift (the state at the time it was read from the screen sorting buffer 302).

[0143] For example, the adaptive right shift unit 313 reduces the bit depth of the image data supplied from the arithmetic unit 311 or the loop filter 312 from 12 bits to 8 bits. By reducing the bit depth in this way, the amount of data in the image data stored in the frame memory can be reduced.

[0144] This right shift amount (bit amount) is arbitrary as long as it matches the left shift amount in the adaptive left shift unit 303. That is, it may be a fixed value or it may be variable. Also, this right shift process may be omitted (skipped) by the control of the PCM encoding unit 321.

[0145] The adaptive right shift unit 313 supplies the image data after the right shift processing to the frame memory 314 (or, if the processing is omitted (skipped), the image data output from the calculation unit 311 or the loop filter 312 as is).

[0146] The frame memory 314 stores the supplied decoded image, similar to the frame memory 112, and outputs the stored decoded image as a reference image to the adaptive left shift unit 315 at a predetermined timing.

[0147] The adaptive left shift unit 315 is a processing unit similar to the adaptive left shift unit 303, controlled by the PCM encoding unit 321, and appropriately shifts the image data (reference image) read from the frame memory 314 to the left, increasing its bit depth by a predetermined number of bits (for example, 4 bits).

[0148] For example, if it is not in I_PCM mode, the input image data is left-shifted in the adaptive left-shift unit 303. Then, the adaptive left-shift unit 315, in accordance with the control of the PCM encoding unit 321, left-shifts the reference image data read from the frame memory 314 and increases the bit depth by the same number of bits as in the case of the adaptive left-shift unit 303 (for example, changing the bit depth from 8 bits to 12 bits).

[0149] The adaptive left shift unit 315 then supplies the image data after the left shift process to the selection unit 316. By increasing the bit depth in this way, the bit depth of the reference image can be matched to the bit depth of the input image, and the reference image can be added to the input image. In addition, the accuracy of internal calculations such as prediction processing can be improved and errors can be suppressed.

[0150] In contrast, for example, in I_PCM mode, the input image data is not left-shifted in the adaptive left-shift unit 303. Therefore, the adaptive left-shift unit 315 supplies the reference image read from the frame memory 314 to the selection unit 316 without increasing the bit depth, in accordance with the control of the PCM encoding unit 321.

[0151] In the case of intra prediction, the selection unit 316 supplies the reference image supplied from the adaptive left shift unit 315 to the intra prediction unit 317, similar to the case of the selection unit 113. Also, in the case of inter prediction, the selection unit 316 supplies the reference image supplied from the adaptive left shift unit 315 to the motion prediction and compensation unit 318, similar to the case of the selection unit 113.

[0152] The intra prediction unit 317 performs intra prediction (in-screen prediction) which generates a predicted image using a reference image supplied from the adaptive left shift unit 315 via the selection unit 316. The intra prediction unit 317 performs this intra prediction in one of several pre-prepared modes (intra prediction modes). The intra prediction unit 317 can also perform this intra prediction in any mode other than the modes defined in the AVC coding scheme.

[0153] The intra prediction unit 317 generates prediction images for all candidate intra prediction modes, evaluates the cost function value of each prediction image using the input image supplied from the adaptive left shift unit 303, and selects the optimal mode. Once the intra prediction unit 317 has selected the optimal intra prediction mode, it supplies the prediction image generated in that optimal mode to the calculation unit 304 and the calculation unit 311 via the selection unit 319.

[0154] As described above, the intra prediction unit 317 appropriately supplies information such as intra prediction mode information indicating the adopted intra prediction mode to the reversible encoding unit 307 for encoding.

[0155] The motion prediction and compensation unit 318 performs motion prediction (interpretation) on the image to be intercoded using the input image supplied from the adaptive left shift unit 303 and the reference image supplied from the adaptive left shift unit 315 via the selection unit 316, and performs motion compensation processing according to the detected motion vector to generate a predicted image (interpretation image information). The motion prediction and compensation unit 318 performs such interpretation in one of several pre-prepared modes (interpretation modes). The motion prediction and compensation unit 318 can also perform this interpretation in any mode other than the modes defined in the AVC coding scheme.

[0156] The motion prediction and compensation unit 318 generates predicted images in all candidate interpretation modes, evaluates the cost function value of each predicted image, and selects the optimal mode. Once the motion prediction and compensation unit 318 has selected the optimal interpretation mode, it supplies the predicted images generated in that optimal mode to the calculation unit 304 and the calculation unit 311 via the selection unit 319.

[0157] The motion prediction and compensation unit 318 also supplies interpretation mode information indicating the adopted interpretation mode and motion vector information indicating the calculated motion vector to the reversible encoding unit 307 for encoding.

[0158] The selection unit 319, as with the selection unit 116, supplies the output of the intra prediction unit 317 to the calculation unit 304 or calculation unit 311 in the case of an image to be intra-coded, and supplies the output of the motion prediction / compensation unit 318 to the calculation unit 304 or calculation unit 311 in the case of an image to be inter-coded.

[0159] The rate control unit 320 controls the rate of the quantization operation of the quantization unit 306 based on the amount of code in the encoded data stored in the storage buffer 308, so as to prevent overflow or underflow.

[0160] The rate control unit 320 also supplies the amount of code (amount of code generated) of the encoded data stored in the storage buffer 308 to the PCM encoding unit 321.

[0161] The PCM encoding unit 321 compares the amount of code supplied from the rate control unit 320 with the amount of data of the input image supplied from the screen sorting buffer 302, and selects whether or not to adopt I_PCM mode. In this case, the PCM encoding unit 321 makes this selection in units of CU smaller than LCU. That is, the PCM encoding unit 321 controls whether or not it is in I_PCM mode in more detail.

[0162] The PCM encoding unit 321 controls the operation of the reversible encoding unit 307, the adaptive left shift unit 303, the adaptive right shift unit 313, the adaptive left shift unit 315, and the loop filter 312 according to the selection result.

[0163] [Lossless coding unit, PCM coding unit, and loop filter] Figure 8 is a block diagram showing the main configuration examples of the reversible coding unit 307, the PCM coding unit 321, and the loop filter 312 in Figure 7.

[0164] As shown in Figure 8, the reversible coding unit 307 has a NAL (Network Abstraction Layer) coding unit 331 and a CU coding unit 332.

[0165] The NAL encoding unit 331 encodes NALs such as sequence parameter sets (SPS) and picture parameter sets (PPS) based on user instructions and specifications input via a user interface (not shown). The NAL encoding unit 331 supplies the encoded NALs (NAL data) to the storage buffer 308 and adds them to the encoded VCL (Video Coding Layer) CU data supplied from the CU encoding unit 332 to the storage buffer 308.

[0166] The CU encoding unit 332 is controlled by the PCM encoding unit 321 (based on the On / Off control signal supplied from the PCM encoding unit 321) and performs VCL encoding. For example, if the I_PCM mode is not selected by the PCM encoding unit 321 (if the PCM encoding unit 321 supplies a control signal indicating "On"), the CU encoding unit 332 encodes the quantized orthogonal transformation coefficients for each CU. The CU encoding unit 332 supplies the encoded data (CU data) for each encoded CU to the storage buffer 308.

[0167] Also, for example, if the I_PCM mode is selected by the PCM encoding unit 321 (when the PCM encoding unit 321 supplies a control signal indicating "Off"), the CU encoding unit 332 supplies the input pixel values ​​supplied from the screen sorting buffer 302 to the storage buffer 308 as the encoding result (CU data).

[0168] The CU encoding unit 332 also encodes a flag (I_PCM_flag) supplied from the PCM encoding unit 321 that indicates whether or not the encoding mode is I_PCM mode, and supplies it to the storage buffer 308 as CU data. Furthermore, the CU encoding unit 332 encodes information related to filtering, such as adaptive filter flags and filter coefficients supplied from the loop filter 312, and supplies it to the storage buffer 308 as CU data.

[0169] The encoding method used by the CU encoding unit 332 is arbitrary (e.g., CABAC or CAVLC). The NAL data and CU data supplied to the storage buffer 308 are combined and stored.

[0170] In practice, the PCM encoding unit 321 controls whether or not to select the I_PCM mode using the code amount of the encoded data, which is encoded using the orthogonal transformation coefficients quantized in the CU encoding unit 332.

[0171] Therefore, for example, if the I_PCM mode is not selected, the encoded data already supplied to the storage buffer 308 is used as is as the encoded result of the quantized orthogonal transformation coefficients of the CU. Therefore, the CU encoding unit 332 only needs to encode additional information such as I_PCM_flag.

[0172] In contrast, for example, when the I_PCM mode is selected, the CU encoding unit 332 supplies the input pixel values ​​of the CU supplied from the screen sorting buffer 302 to the storage buffer 308 as the encoded result (CU data). Therefore, in this case, the encoded data of the CU that has already been supplied (encoded data in which the quantized orthogonal transformation coefficients have been encoded) is discarded. In other words, all processing related to the generation of that encoded data becomes redundant.

[0173] As shown in Figure 8, the PCM encoding unit 321 has an I_PCM_flag generation unit 341 and a PCM determination unit 342.

[0174] The I_PCM_flag generation unit 341 generates an I_PCM_flag and determines its value according to the decision of the PCM determination unit 342. The I_PCM_flag generation unit 341 supplies the generated I_PCM_flag to the CU encoding unit 332 of the reversible encoding unit 307. For example, if the PCM determination unit 342 selects the I_PCM mode, the I_PCM_flag generation unit 341 sets the value of the I_PCM_flag to a value that indicates the I_PCM mode has been selected (for example, "1") and supplies the I_PCM_flag to the CU encoding unit 332. Furthermore, for example, if the PCM determination unit 342 does not select the I_PCM mode, the I_PCM_flag generation unit 341 sets the value of I_PCM_flag to a value indicating that the I_PCM mode was not selected (for example, "0"), and supplies that I_PCM_flag to the CU encoding unit 332.

[0175] The PCM determination unit 342 determines whether or not to set the encoding mode to I_PCM mode. The PCM determination unit 342 obtains the amount of data of the input pixel values ​​supplied from the screen sorting buffer 302, compares it with the amount of generated code supplied from the rate control unit 320, and decides whether or not to select I_PCM mode based on the comparison result. The PCM determination unit 342 controls the operation according to the selection result by supplying On / Off control signals indicating the selection result to the CU encoding unit 332 and the I_PCM_flag generation unit 341.

[0176] For example, if the amount of data of the input pixel value is greater than the amount of code generated, the PCM determination unit 342 does not select the I_PCM mode. In this case, the PCM determination unit 342 supplies a control signal indicating "On" to the CU encoding unit 332, causing the CU encoding unit 332 to encode the quantized orthogonal transformation coefficients. The PCM determination unit 342 also supplies a control signal indicating "On" to the I_PCM_flag generation unit 341, causing it to generate an I_PCM_flag of a value (for example, "0") indicating that the I_PCM mode was not selected.

[0177] In contrast, for example, if the amount of data of the input pixel value is less than or equal to the amount of code generated, the PCM determination unit 342 selects the I_PCM mode. In this case, the PCM determination unit 342 supplies a control signal indicating "Off" to the CU encoding unit 332, causing it to output the input pixel value as the encoded result (CU data). The PCM determination unit 342 also supplies a control signal indicating "Off" to the I_PCM_flag generation unit 341, causing it to generate an I_PCM_flag with a value (for example, "1") indicating that the I_PCM mode has been selected.

[0178] The PCM determination unit 342 can make the decision of whether or not to select such I_PCM mode not only for LCU but also for CU of all sizes (of any hierarchy) set in the sequence parameter set. This makes it possible to perform the process of limiting the generation of many bits at low QP by using I_PCM mode on a smaller CU basis, for example, and not only can the control of the code amount (including the data amount of uncoded data by I_PCM mode) be made in more detail, but redundant processing that occurs when I_PCM mode is used can also be reduced.

[0179] The PCM determination unit 342 also controls the IBDI according to the selection result by supplying On / Off control signals indicating the selection result to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315. In other words, the PCM determination unit 342 controls the adaptive shift device so that it does not perform the process of increasing or decreasing bit precision when the CU is in I_PCM mode.

[0180] For example, if I_PCM mode is not selected, the PCM determination unit 342 supplies a control signal indicating "On" to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315, causing them to perform left shift processing and right shift processing, thereby expanding the bit precision in internal processing.

[0181] In contrast, if I_PCM mode is selected, for example, the PCM determination unit 342 supplies a control signal indicating "Off" to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315, causing the left shift and right shift processes to be omitted (skipped) and preventing the expansion of bit precision in the internal processing.

[0182] In I_PCM mode, the input image pixel values ​​themselves are transmitted to the image compression information, so no calculation errors occur, and increasing the bit calculation precision in this case would be redundant processing. The PCM determination unit 342 can eliminate such redundant processing by performing the processing as described above.

[0183] Furthermore, the PCM determination unit 342 controls the adaptive loop filter processing (BALF) according to the selection result by supplying an On / Off control signal indicating the selection result to the loop filter 312. In other words, the PCM determination unit 342 controls the loop filter 312 so that adaptive loop filter processing is not performed when the CU is in I_PCM mode.

[0184] For example, if the I_PCM mode is not selected, the PCM determination unit 342 supplies a control signal indicating "On" to the loop filter 312, so that adaptive loop filtering is performed. Conversely, if the I_PCM mode is selected, for example, the PCM determination unit 342 supplies a control signal indicating "Off" to the loop filter 312, so that adaptive loop filtering is omitted (skipped).

[0185] In I_PCM mode, the input image pixel values ​​themselves are transmitted as image compression information, so no degradation occurs, and applying adaptive loop filtering to this would be redundant. The PCM determination unit 342 can eliminate such redundant processing by performing the processing as described above.

[0186] As shown in Figure 8, the loop filter 312 includes a deblocking filter 351, a pixel sorting unit 352, a filter coefficient calculation unit 353, and a filtering unit 354.

[0187] The deblocking filter 351 removes block distortion by performing a deblocking filter operation on the decoded image (pixel value before deblocking filter) supplied from the calculation unit 311, similar to the case of the deblocking filter 111.

[0188] Even if the CU to be processed is processed in I_PCM mode, it does not necessarily mean that the CUs adjacent to that CU are also processed in I_PCM mode. Therefore, even if the CU is processed in I_PCM mode, block distortion may occur. For this reason, the deblocking filter process is performed regardless of whether the CU is in I_PCM mode or not.

[0189] The deblocking filter 351 supplies the result of its filtering (pixel value after deblocking filter) to the pixel sorting unit 352.

[0190] The pixel sorting unit 352 sorts each filter processing result (deblocked pixel value) into pixel values ​​that undergo adaptive loop filtering and pixel values ​​that do not undergo adaptive loop filtering, according to the value of the On / Off control signal supplied from the PCM determination unit 342.

[0191] For example, if a control signal indicating "On" is supplied from the PCM determination unit 342, the pixel sorting unit 352 sorts the deblocked pixel value of that CU into a pixel value that undergoes adaptive loop filtering. Conversely, if a control signal indicating "Off" is supplied from the PCM determination unit 342, the pixel sorting unit 352 sorts the deblocked pixel value of that CU into a pixel value that does not undergo adaptive loop filtering.

[0192] The pixel sorting unit 352 supplies the pixel value (deblocked pixel value) of each sorted pixel to the filter coefficient calculation unit 353.

[0193] The filter coefficient calculation unit 353 calculates the filter coefficients (FIR filter coefficients) of the adaptive loop filter for the pixel values ​​that have been sorted from the supplied deblocked pixel values ​​to be processed using a Wiener filter, in order to minimize the error with the input image. In other words, the filter coefficient calculation unit 353 calculates the filter coefficients excluding pixels that are processed in I_PCM mode.

[0194] The filter coefficient calculation unit 353 supplies the deblocked pixel value and the calculated filter coefficient to the filtering unit 354.

[0195] The filtering unit 354 uses the supplied filter coefficients to perform adaptive loop filtering on the pixel values ​​that have been sorted for adaptive loop filtering. The filtering unit 354 supplies the pixel values ​​after this filtering, along with the pixel values ​​that have been sorted for which adaptive loop filtering is not performed, to the adaptive right shift unit 313 as the pixel values ​​after adaptive filtering.

[0196] The filtering unit 354 also generates an adaptive filter flag (on / off_flag), which is filter identification information indicating whether or not filtering has been performed, for each predetermined block, which is set independently of the CU. The method for setting the value of this adaptive filter flag is arbitrary.

[0197] For example, if adaptive loop filtering is performed on some or all pixels within the block being processed (current block), this adaptive filter flag may be set to a value indicating that filtering has been performed (e.g., "1"). Also, for example, if adaptive loop filtering has not been performed on all pixels within the block, this adaptive filter flag may be set to a value indicating that filtering has not been performed (e.g., "0"). The value of the adaptive filter flag may also be set based on other criteria.

[0198] The filtering unit 354 supplies the generated adaptive filter flag to the CU encoding unit 332 of the reversible encoding unit 307, has it encoded, and provides it to the decoding side. However, if the value of the adaptive filter flag is a value that indicates that no filtering has been performed (for example, "0"), this provision to the decoding side can be omitted (the adaptive filter flag is not provided to the decoding side).

[0199] For example, if the value of the adaptive filter flag is a value indicating that no filtering has been performed (e.g., "0"), and the encoding scheme of the reversible encoding unit 307 (CU encoding unit 332) is VLC, the filtering unit 354 omits supplying this adaptive filter flag (does not provide it to the decoding side). Also, for example, if the value of the adaptive filter flag is a value indicating that no filtering has been performed (e.g., "0"), and the encoding scheme of the reversible encoding unit 307 (CU encoding unit 332) is CABAC, the filtering unit 354 supplies this adaptive filter flag to the CU encoding unit 332 of the reversible encoding unit 307 (provides it to the decoding side).

[0200] This is because, in the case of VLC, it is possible to achieve higher coding efficiency when there is little input information, whereas in the case of CABAC, when the same input information is repeated, a bias occurs in the probability during arithmetic coding, which makes it possible to achieve higher coding efficiency.

[0201] Furthermore, the filtering unit 354 supplies the filter coefficients used in the adaptive loop filtering process to the CU encoding unit 332 of the reversible encoding unit 307, has them encoded, and provides them to the decoding side.

[0202] [PCM decision section] Figure 9 is a block diagram showing the main configuration example of the PCM determination unit 342 in Figure 8.

[0203] As shown in Figure 9, the PCM determination unit 342 includes an input data amount calculation unit 361, a PCM determination unit 362, an encoding control unit 363, an adaptive shift control unit 364, and a filter control unit 365.

[0204] The input data amount calculation unit 361 calculates the input data amount, which is the amount of input pixel value data supplied from the screen sorting buffer 302 for the CU, and supplies the calculated input data amount to the PCM determination unit 362.

[0205] The PCM determination unit 362 acquires the generated code amount (generated bits) supplied from the rate control unit 320, compares it with the input data amount supplied from the input data amount calculation unit 361, and determines whether or not to select the I_PCM mode for the CU based on the comparison result. In other words, the PCM determination unit 362 determines whether or not to select the I_PCM mode for each CU at any given level. The PCM determination unit 362 supplies the determination result to the encoding control unit 363, the adaptive shift control unit 364, and the filter control unit 365.

[0206] The coding control unit 363 supplies On / Off control signals to the CU coding unit 332 and the I_PCM_flag generation unit 341 based on the determination result (identification information indicating whether or not the I_PCM mode has been selected) supplied from the PCM determination unit 362.

[0207] Accordingly, the encoding control unit 363 can control the encoding mode for each CU at any level. This allows the encoding control unit 363 to control the amount of encoding (including the amount of data of unencoded data by the I_PCM mode) in more detail, and also reduces redundant processing when the I_PCM mode is selected.

[0208] The adaptive shift control unit 364 supplies On / Off control signals to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315 based on the determination result (information indicating whether or not the I_PCM mode is selected) supplied from the PCM determination unit 362.

[0209] Accordingly, the adaptive shift control unit 364 can control the bit depth in its internal calculations so as not to be expanded when in I_PCM mode. This allows the adaptive shift control unit 364 to reduce redundant processing.

[0210] The filter control unit 365 supplies an On / Off control signal to the pixel sorting unit 352 based on the determination result (information indicating whether or not the I_PCM mode has been selected) supplied from the PCM determination unit 362.

[0211] Accordingly, the filter control unit 365 can be controlled to prevent the execution of adaptive loop filtering when in I_PCM mode. This allows the filter control unit 365 to reduce redundant processing.

[0212] As described above, the image encoding device 300 can reduce redundant processing and suppress a decrease in the efficiency of the encoding process. Furthermore, the image encoding device 300 can select the I_PCM mode (uncompressed mode) in more detail (for each small data unit) and improve the encoding efficiency. Therefore, the image encoding device 300 can improve the encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

[0213] [Encoding process flow] Next, we will explain the flow of each process performed by the image encoding device 300 as described above. First, we will explain an example of the encoding process flow with reference to the flowchart in Figure 10.

[0214] In step S301, the A / D conversion unit 301 performs A / D conversion on the input image. In step S302, the screen sorting buffer 302 stores the A / D converted images and sorts the pictures from the display order to the encoding order.

[0215] In step S303, the adaptive left shift unit 303 performs an adaptive left shift on the input image based on the control of the PCM encoding unit 321. In step S304, the adaptive left shift unit 315 performs an adaptive left shift on the reference image.

[0216] In step S305, the intra prediction unit 317 performs intra prediction processing in intra prediction mode using the reference image that was shifted to the left in step S304. In step S306, the motion prediction / compensation unit 318 performs inter motion prediction processing to perform motion prediction and motion compensation in inter prediction mode using the reference image that was shifted to the left in step S304.

[0217] In practice, the process of left-shifting the bit depth of the reference image may be performed in the intra-prediction process or the inter-motion prediction process when the reference image is read from the frame memory 314.

[0218] In step S307, the selection unit 319 determines the optimal mode based on the cost function values ​​output from the intra prediction unit 317 and the motion prediction / compensation unit 318. In other words, the selection unit 319 selects either the predicted image generated by the intra prediction unit 317 or the predicted image generated by the motion prediction / compensation unit 318.

[0219] Furthermore, selection information indicating which of these prediction images has been selected is supplied to the intra prediction unit 317 and the motion prediction / compensation unit 318 that selected the prediction image. If the prediction image of the optimal intra prediction mode is selected, the intra prediction unit 317 supplies intra prediction mode information indicating the optimal intra prediction mode, etc., to the reversible coding unit 307. If the prediction image of the optimal inter prediction mode is selected, the motion prediction / compensation unit 318 outputs information indicating the optimal inter prediction mode and, if necessary, information corresponding to the optimal inter prediction mode, to the reversible coding unit 307. Examples of information corresponding to the optimal inter prediction mode include motion vector information, flag information, and reference frame information.

[0220] In step S308, the calculation unit 304 calculates the difference between the image whose bit depth has been shifted to the left by the processing in step S303 and the predicted image selected by the processing in step S307. The predicted image is supplied to the calculation unit 304 via the selection unit 319 from the motion prediction and compensation unit 318 in the case of interpretation, and from the intraprediction unit 317 in the case of intraprediction.

[0221] The difference data has a reduced data size compared to the original image data. Therefore, the amount of data can be compressed compared to encoding the image as is.

[0222] In step S309, the orthogonal transformation unit 305 performs an orthogonal transformation on the difference information generated by the processing in step S308. Specifically, an orthogonal transformation such as a discrete cosine transform or a Carunen-Lobe transform is performed, and the transformation coefficients are output.

[0223] In step S310, the quantization unit 306 quantizes the orthogonal transformation coefficients obtained in the process of step S309.

[0224] In step S311, the reversible coding unit 307 encodes the conversion coefficients quantized by the processing in step S310. That is, reversible coding such as variable-length coding or arithmetic coding is performed on the difference image.

[0225] The reversible coding unit 307 encodes the quantization parameters calculated in step S310 and adds them to the encoded data. The reversible coding unit 307 also encodes information regarding the mode of the predicted image selected by the processing in step S307 and adds it to the encoded data obtained by encoding the difference image. In other words, the reversible coding unit 307 also encodes the optimal intra-prediction mode information supplied from the intra-prediction unit 317, or information corresponding to the optimal intra-prediction mode supplied from the motion prediction / compensation unit 318, and adds it to the encoded data.

[0226] Furthermore, the reversible encoding unit 307 encodes the filter coefficients and flag information obtained from the loop filter 312 and adds them to the encoded data. In addition, the reversible encoding unit 307 also encodes the NAL data.

[0227] In step S312, the storage buffer 308 stores the encoded data output from the reversible encoding unit 307. The encoded data stored in the storage buffer 308 is read out as appropriate and transmitted to the decoding side via the transmission line or recording medium.

[0228] In step S313, the rate control unit 320 calculates the amount of code (generated code amount) of the encoded data stored in the storage buffer 308 by the processing in step S312, and based on this, controls the rate of the quantization operation of the quantization unit 306 so that no overflow or underflow occurs. The rate control unit 320 also supplies the generated code amount to the PCM encoding unit 321.

[0229] In step S314, the PCM encoding unit 321 performs PCM encoding control processing using the generated code amount calculated in step S313. In step S315, the reversible encoding unit 307 performs PCM encoding processing according to the control of the PCM encoding unit 321.

[0230] In step S316, the inverse quantization unit 309 to the frame memory 314 locally decode the difference information quantized by the processing in step S310 and execute a reference image generation process to generate a reference image.

[0231] When the processing in step S316 is completed, the encoding process is terminated. This encoding process is executed repeatedly, for example, for each CU.

[0232] [PCM encoding control processing] Next, with reference to the flowchart in Figure 11, an example of the flow of the PCM encoding control process performed in step S314 of Figure 10 will be described.

[0233] When the PCM coding control process is started, in step S331, the PCM determination unit 362 of the PCM determination unit 342 acquires the generated code amount of the coded data of the quantized orthogonal transformation coefficients of the CU supplied from the rate control unit 320.

[0234] In step S332, the input data amount calculation unit 361 calculates the input data amount of the input pixel value of the CU.

[0235] In step S333, the PCM determination unit 362 compares the amount of code obtained in step S331 with the amount of input data calculated in step S332 and determines whether or not to perform encoding in I_PCM mode.

[0236] In step S334, the I_PCM_flag generation unit 341 generates an I_PCM_flag based on the On / Off control signal supplied by the coding control unit 363, which indicates the determination result of step S333.

[0237] In step S335, the coding control unit 363 controls the coding of CU data by supplying an On / Off control signal to the CU coding unit 332 that indicates the determination result of step S333.

[0238] In step S336, the adaptive shift control unit 364 controls the adaptive shift process by supplying On / Off control signals indicating the determination result of step S333 to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315.

[0239] In step S337, the coding control unit 363 controls the adaptive loop filter processing by supplying an On / Off control signal indicating the determination result of step S333 to the pixel sorting unit 352 of the loop filter 312.

[0240] When the processing in step S337 is completed, the PCM determination unit 342 terminates the PCM encoding control processing and returns the process to step S314 in Figure 10, and executes the processing from step S315 onwards.

[0241] [PCM Encoding Process Flow] Next, with reference to the flowchart in Figure 12, an example of the PCM encoding process performed in step S315 of Figure 10 will be described.

[0242] When the PCM encoding process is started, in step S351, the CU encoding unit 332 determines whether or not to perform encoding in I_PCM mode. If the PCM encoding control process described above controls the CU encoding to perform encoding in I_PCM mode, the CU encoding unit 332 proceeds to step S352. In step S352, the CU encoding unit 332 selects the input pixel value of the CU as the encoding result. The CU encoding unit 332 discards the CU data of the CU in the storage buffer 308 and stores its input pixel value in the storage buffer 308.

[0243] When the processing in step S352 is completed, the CU encoding unit 332 proceeds to step S353. Also, if it is determined in step S351 that encoding in I_PCM mode is not to be performed, the CU encoding unit 332 proceeds to step S353.

[0244] In step S353, the CU encoding unit 332 encodes the I_PCM_flag generated by the PCM encoding control process described above and stores it in the storage buffer 308.

[0245] When the processing in step S353 is completed, the CU encoding unit 332 terminates the PCM encoding process and returns the process to step S315 in Figure 10, and executes the processes from step S316 onwards.

[0246] [Flowchart of Reference Image Generation Process] Next, with reference to the flowchart in Figure 13, an example of the flow of the reference image generation process performed in step S316 of Figure 10 will be described.

[0247] When the reference image generation process is started, the adaptive left shift unit 315 determines in step S371 whether the I_PCM mode has been selected based on the control of the adaptive shift control unit 364. If the I_PCM mode is selected, the bit depth expansion in the internal calculation is not performed, so the prediction process is restarted. In other words, if it is determined that the I_PCM mode has been selected, the adaptive left shift unit 315 proceeds to step S372 without performing a left shift operation on the reference image.

[0248] In step S372, the intra prediction unit 317 performs intra prediction processing using a reference image whose bit depth has not been left-shifted. In step S373, the motion prediction and compensation unit 318 performs inter-motion prediction processing using a reference image whose bit depth has not been left-shifted.

[0249] In step S374, the selection unit 319 determines the optimal mode based on the cost function values ​​output from the intra prediction unit 317 and the motion prediction / compensation unit 318. In other words, the selection unit 319 selects either the predicted image generated by the intra prediction unit 317 or the predicted image generated by the motion prediction / compensation unit 318.

[0250] Furthermore, selection information indicating which of these prediction images has been selected is supplied to the intra prediction unit 317 and the motion prediction / compensation unit 318 that selected the prediction image. If the prediction image of the optimal intra prediction mode is selected, the intra prediction unit 317 supplies intra prediction mode information indicating the optimal intra prediction mode, etc., to the reversible coding unit 307. If the prediction image of the optimal inter prediction mode is selected, the motion prediction / compensation unit 318 supplies information indicating the optimal inter prediction mode and, if necessary, information corresponding to the optimal inter prediction mode, to the reversible coding unit 307. Examples of information corresponding to the optimal inter prediction mode include motion vector information, flag information, and reference frame information.

[0251] When I_PCM mode is selected, the bit depth is not expanded in the internal calculation. In other words, the left shift processing on the reference image by the adaptive left shift unit 303 is omitted (skipped). In step S375, the calculation unit 304 calculates the difference between the input image whose bit depth has not been left shifted and the predicted image selected by the processing in step S374. The predicted image is supplied to the calculation unit 304 via the selection unit 319 from the motion prediction / compensation unit 318 in the case of inter prediction, and from the intra prediction unit 317 in the case of intra prediction.

[0252] In step S376, the orthogonal transformation unit 305 orthogonally transforms the difference information generated by the processing in step S375. Specifically, orthogonal transformations such as discrete cosine transform and Carunen-Loebe transform are performed, and transformation coefficients are output. In step S377, the quantization unit 306 quantizes the orthogonal transformation coefficients obtained by the processing in step S376.

[0253] Once the orthogonal transformation coefficients are quantized, the quantization unit 306 proceeds to step S378, where a reference image is generated using the quantized orthogonal transformation coefficients (data that has not been left-shifted) in step S377.

[0254] In contrast, if it is determined in step S371 that it is not in I_PCM mode, the adaptive left shift unit 315 skips the processes in steps S372 to S377 and proceeds to step S378. In other words, if it is not in I_PCM mode, the reference image is generated using the quantized orthogonal transformation coefficients (left-shifted data) in step S310 of Figure 10.

[0255] In step S378, the inverse quantization unit 309 inversely quantizes the quantized orthogonal transformation coefficients (also called quantization coefficients) using characteristics corresponding to the characteristics of the quantization unit 306. In step S379, the inverse orthogonal transformation unit 310 inversely transforms the orthogonal transformation coefficients obtained by the process in step S378 using characteristics corresponding to the characteristics of the orthogonal transformation unit 305.

[0256] In step S380, the calculation unit 311 adds the predicted image to the locally decoded difference information to generate a locally decoded image (an image corresponding to the input to the calculation unit 304). For example, in I_PCM mode, the calculation unit 311 adds the predicted image that has not been left-shifted to the difference information that has not been left-shifted to generate a decoded image that has not been left-shifted. Also, for example, in a mode other than I_PCM mode, the calculation unit 311 adds the predicted image that has been left-shifted to the difference information that has been left-shifted to generate a decoded image that has been left-shifted.

[0257] In step S381, the loop filter 312 performs loop filtering on the local decoded image obtained by the processing in step S380 based on the control of the filter control unit 365, and appropriately performs loop filtering including deblocking filtering and adaptive loop filtering.

[0258] In step S382, the adaptive right shift unit 313 determines whether the I_PCM mode is selected based on the control of the adaptive shift control unit 364. If it is determined that the I_PCM mode is not selected, the adaptive right shift unit 313 proceeds to step S383.

[0259] If it is not I_PCM mode, the decoded image has undergone bit depth expansion in internal calculations, so the adaptive right shift unit 313 adaptively performs a right shift with respect to the bit depth of the filtered result (decoded image) obtained by the loop filtering process in step S381 in step S383. When the processing in step S383 is completed, the adaptive right shift unit 313 proceeds to step S384.

[0260] Also, if it is determined in step S382 that the I_PCM mode has been selected, the adaptive right shift unit 313 proceeds to step S384 without performing the right shift process.

[0261] In step S384, the frame memory 314 stores the decoded image. When the processing in step S384 is completed, the frame memory 314 finishes the reference image generation process, returns the process to step S316 in Figure 10, and finishes the encoding process.

[0262] [Flow of Loop Filtering Process] Next, with reference to the flowchart in Figure 14, an example of the loop filtering process performed in step S381 of Figure 13 will be explained.

[0263] When loop filtering is started, the deblocking filter 351 of the loop filter 312 performs deblocking filtering on the decoded image (pixel value before deblocking) supplied from the calculation unit 311 in step S401.

[0264] In step S402, the pixel sorting unit 352 is controlled by the filter control unit 365 of the PCM determination unit 342 to sort each pixel of the decoded image based on whether or not it is in I_PCM mode.

[0265] In step S403, the filter coefficient calculation unit 353 calculates filter coefficients for the pixels that have been sorted to be filtered (pixels to be processed). In step S404, the filtering unit 354 performs adaptive filtering on the pixels to be processed using the filter coefficients calculated in step S403.

[0266] In step S405, the filtering unit 354 sets an adaptive filter flag for the block to be processed, and supplies the adaptive filter flag and filter coefficients to the CU encoding unit 332 for encoding.

[0267] When the processing in step S405 is completed, the loop filter 312 terminates the loop filtering process and returns the process to step S381 in Figure 13, and executes the processing from step S382 onwards.

[0268] By performing each process as described above, the image encoding device 300 can improve encoding efficiency while suppressing a reduction in the efficiency of the encoding process.

[0269] <2. Second Embodiment> [Image Decoding Device] Figure 15 is a block diagram showing an example of the main configuration of an image decoding device. The image decoding device 500 shown in Figure 15 is basically the same as the image decoding device 200 in Figure 2, and decodes encoded data in which image data has been encoded.

[0270] The image decoding device 500 shown in Figure 15 is a decoding device corresponding to the image encoding device 300 in Figure 7. The encoded data encoded by the image encoding device 300 is supplied to this image decoding device 500 via any path, such as a transmission line or recording medium, and decoded.

[0271] As shown in Figure 15, the image decoding device 500 includes a storage buffer 501, a reversible decoding unit 502, an inverse quantization unit 503, an inverse orthogonal transformation unit 504, an arithmetic unit 505, a loop filter 506, an adaptive right shift unit 507, a screen rearrangement buffer 508, and a D / A conversion unit 509. The image decoding device 500 also includes a frame memory 510, an adaptive left shift unit 511, a selection unit 512, an intra prediction unit 513, a motion prediction and compensation unit 514, and a selection unit 515.

[0272] The image decoding device 500 further includes a PCM decoding unit 516.

[0273] The storage buffer 501 stores the transmitted encoded data, similar to the storage buffer 201. This encoded data is encoded by the image encoding device 300.

[0274] The reversible decoding unit 502 reads the encoded data from the storage buffer 501 at a predetermined timing and decodes it in a manner corresponding to the encoding method of the reversible encoding unit 307 in Figure 15. At that time, the reversible decoding unit 502 supplies the I_PCM_flag contained in the encoded data to the PCM decoding unit 516 to determine whether or not it is in I_PCM mode (uncompressed mode).

[0275] In I_PCM mode, the CU data acquired from the storage buffer 501 is unencoded data, so the reversible decoding unit 502 supplies the CU data to the inverse quantization unit 503 in accordance with the control of the PCM decoding unit 516.

[0276] If it is not in I_PCM mode, the CU data acquired from the storage buffer 501 is encoded data, so the reversible decoding unit 502 decodes the CU data according to the control of the PCM decoding unit 516 and supplies the decoding result to the inverse quantization unit 503.

[0277] For example, if the CU is intra-encoded, intra-prediction mode information is stored in the header portion of the encoded data. The reversible decoding unit 502 also decodes this intra-prediction mode information and supplies that information to the intra-prediction unit 513. In contrast, if the CU is inter-encoded, for example, motion vector information and inter-prediction mode information are stored in the header portion of the encoded data. The reversible decoding unit 502 also decodes this motion vector information and inter-prediction mode information and supplies that information to the motion prediction and compensation unit 514.

[0278] When not in I_PCM mode, the inverse quantization unit 503, similar to the case of the inverse quantization unit 203, inversely quantizes the coefficient data (quantization coefficients) supplied from the reversible decoding unit 502 in a manner corresponding to the quantization method of the quantization unit 306 in Figure 7. In other words, the inverse quantization unit 503 performs inverse quantization of the quantization coefficients in the same manner as the inverse quantization unit 309 in Figure 7. The inverse quantization unit 503 supplies the inversely quantized coefficient data, i.e., orthogonal transformation coefficients, to the inverse orthogonal transformation unit 504.

[0279] In addition, when in I_PCM mode, the inverse quantization unit 503 supplies the CU data (unencoded image data) supplied from the reversible decoding unit 502 to the inverse orthogonal transform unit 504.

[0280] If the inverse orthogonal transformation unit 504 is not in I_PCM mode, it performs an inverse orthogonal transformation on the orthogonal transformation coefficients in the same way as the inverse orthogonal transformation unit 204, using a method corresponding to the orthogonal transformation method of the orthogonal transformation unit 305 in Figure 7 (the same method as the inverse orthogonal transformation unit 310 in Figure 7). Through this inverse orthogonal transformation process, the inverse orthogonal transformation unit 504 obtains decoded residual data corresponding to the residual data before orthogonal transformation in the image encoding device 300. For example, a fourth-order inverse orthogonal transformation is performed. The inverse orthogonal transformation unit 504 supplies the decoded residual data obtained by the inverse orthogonal transformation to the calculation unit 505.

[0281] In addition, when in I_PCM mode, the inverse orthogonal transform unit 504 supplies CU data (unencoded image data) supplied from the inverse quantization unit 503 to the calculation unit 505.

[0282] The calculation unit 505 is also supplied with predicted images from the intra prediction unit 513 or the motion prediction / compensation unit 514 via the selection unit 515.

[0283] If the operation is not in I_PCM mode, the calculation unit 505 adds the decoded residual data and the predicted image, similar to the case of the calculation unit 205, to obtain decoded image data corresponding to the image data before the predicted image was subtracted by the calculation unit 304 of the image encoding device 300. The calculation unit 505 supplies the decoded image data to the loop filter 506.

[0284] In addition, when in I_PCM mode, the calculation unit 505 supplies the CU data (unencoded image data) supplied from the inverse orthogonal transform unit 504 to the loop filter 506. In this case, since the CU data is not residual information, it is not necessary to add it to the predicted image.

[0285] The loop filter 506 is controlled by the PCM decoding unit 516 and appropriately applies loop filtering processing, including deblocking filtering and adaptive loop filtering, to the decoded image supplied from the calculation unit 505.

[0286] More specifically, the loop filter 506 removes block distortion from the decoded image by performing a deblocking filter process on the decoded image in the same manner as the deblocking filter 206. Then, in accordance with the control of the PCM decoding unit 516, the loop filter 506 improves image quality by performing a loop filtering process on the result of the deblocking filter process (the decoded image from which block distortion has been removed) using a Wiener filter.

[0287] The loop filter 506 may perform arbitrary filtering on the decoded image. Alternatively, the loop filter 506 may perform filtering using filter coefficients supplied from the image encoding device 300 in Figure 7.

[0288] The loop filter 506 supplies the filtering result (decoded image after filtering) to the adaptive right shift unit 507.

[0289] The adaptive right shift unit 507 is a processing unit similar to the adaptive right shift unit 313 (Figure 7), controlled by the PCM decoding unit 516, and when not in I_PCM mode, it shifts the decoded image data supplied from the loop filter 506 to the right, reducing its bit depth by a predetermined number of bits (for example, 4 bits) (for example, reducing the bit depth from 12 bits to 8 bits). In other words, the adaptive right shift unit 507 shifts to the right by the number of bits that were shifted to the left in the image encoding device 300, returning the bit depth of the image data to the state before it was shifted to the left (the state at the time it was read from the screen sorting buffer 302 (Figure 7)).

[0290] The adaptive right shift unit 507 supplies the image data after the right shift processing to the screen rearrangement buffer 508 and the frame memory 510.

[0291] This right shift amount (bit amount) is arbitrary as long as it matches the left shift amount in the image encoding device 300 (adaptive left shift unit 303). That is, it may be a fixed value or it may be variable. For example, the shift amount may be predetermined (shared in advance between the image encoding device 300 and the image decoding device 500), or the image decoding device 500 may be able to calculate the shift amount in the image encoding device 300, or information indicating the shift amount may be provided from the image encoding device 300.

[0292] In addition, this right shift process may be omitted (skipped) by the control of the PCM decoding unit 516. For example, in I_PCM mode, the image data is not left-shifted in the image encoding device 300. Therefore, in this case, the adaptive right shift unit 507 is controlled by the PCM decoding unit 516 and supplies the decoded image data supplied from the loop filter 506 to the screen reordering buffer 508 and the frame memory 510 without shifting it to the right.

[0293] The screen reordering buffer 508 rearranges the images in the same way as the screen reordering buffer 207. That is, the order of the frames that were rearranged by the screen reordering buffer 302 in Figure 7 for encoding order is rearranged to the original display order. The screen reordering buffer 508 supplies the decoded image data of each frame to the D / A conversion unit 509 in the rearranged order.

[0294] The D / A conversion unit 509, in the same manner as the D / A conversion unit 208, performs D / A conversion on the frame images supplied from the screen rearrangement buffer 508, outputs them to a display (not shown), and displays them.

[0295] The frame memory 510 stores the decoded image supplied from the adaptive right shift unit 507, and at a predetermined timing, supplies the stored decoded image as a reference image to the adaptive left shift unit 511.

[0296] The adaptive left shift unit 511 is a processing unit similar to the adaptive left shift unit 315, controlled by the PCM decoding unit 516, and appropriately shifts the image data (reference image) read from the frame memory 510 to the left, increasing its bit depth by a predetermined number of bits (for example, 4 bits).

[0297] For example, if it is not in I_PCM mode, the decoded image data supplied from the inverse orthogonal transform unit 504 to the arithmetic unit 505 is image data that has been left-shifted in the image encoding device 300 (for example, with a bit depth of 12 bits). The adaptive left-shift unit 511 then increases the bit depth of the reference image read from the frame memory 510 by a predetermined number of bits according to the control of the PCM decoding unit 516 (for example, changing the bit depth from 8 bits to 12 bits).

[0298] The adaptive left shift unit 511 then supplies the image data after the left shift process to the selection unit 512. By increasing the bit depth in this way, the bit depth of the reference image can be matched to the bit depth of the decoded image, and the reference image can be added to the decoded image. In addition, the accuracy of internal calculations such as prediction processing can be improved and errors can be suppressed.

[0299] In contrast, for example, in I_PCM mode, the decoded image data supplied from the inverse orthogonal transform unit 504 to the arithmetic unit 505 is image data that has not been left-shifted in the image encoding device 300 (for example, with a bit depth of 8 bits). Therefore, the adaptive left-shift unit 511 supplies the reference image read from the frame memory 510 to the selection unit 512 without increasing the bit depth, in accordance with the control of the PCM decoding unit 516.

[0300] In the case of intra prediction, the selection unit 512 supplies the reference image supplied from the adaptive left shift unit 511 to the intra prediction unit 513. Also, in the case of inter prediction, the selection unit 512 supplies the reference image supplied from the adaptive left shift unit 511 to the motion prediction and compensation unit 514.

[0301] The intra prediction unit 513 is appropriately supplied with information indicating the intra prediction mode obtained by decoding the header information from the reversible decoding unit 502. The intra prediction unit 513 performs intra prediction using the reference image acquired from the frame memory 510 in the intra prediction mode used in the intra prediction unit 317, and generates a predicted image. In other words, the intra prediction unit 513 can also perform this intra prediction in any mode other than the modes defined in the AVC coding scheme, similar to the intra prediction unit 317.

[0302] The intra prediction unit 513 supplies the generated prediction image to the selection unit 515.

[0303] The motion prediction and compensation unit 514 is supplied with information obtained by decoding the header information (prediction mode information, motion vector information, reference frame information, flags, and various parameters, etc.) from the reversible decoding unit 502.

[0304] The motion prediction and compensation unit 514 performs interprediction using the reference image acquired from the frame memory 510 in the interprediction mode used in the motion prediction and compensation unit 318, and generates a predicted image. In other words, the motion prediction and compensation unit 514 can also perform this intraprediction in any mode other than the modes defined in the AVC coding scheme, just like the motion prediction and compensation unit 318.

[0305] The motion prediction and compensation unit 514 supplies the generated predicted image to the selection unit 515.

[0306] The selection unit 515, similar to the selection unit 213, selects a predicted image generated by the motion prediction / compensation unit 514 or the intra prediction unit 513 and supplies it to the calculation unit 505.

[0307] The PCM decoding unit 516 controls the reversible decoding unit 502, the loop filter 506, the adaptive right shift unit 507, and the adaptive left shift unit 511 based on the I_PCM_flag supplied from the reversible decoding unit 502.

[0308] [Reversible decoding unit, PCM decoding unit, and loop filter] Figure 16 is a block diagram showing the main configuration examples of the reversible decoding unit 502, PCM decoding unit 516, and loop filter 506 shown in Figure 15.

[0309] As shown in Figure 16, the reversible decoding unit 502 has a NAL decoding unit 531 and a CU decoding unit 532. The NAL decoding unit 531 decodes the encoded NAL data supplied from the storage buffer 501 and supplies the decoded NAL data to the CU decoding unit 532.

[0310] The CU decoding unit 532 decodes the encoded CU data supplied from the storage buffer 501 based on the NAL data supplied from the NAL decoding unit 531.

[0311] When the CU decoding unit 532 decodes the CU data and obtains the I_PCM_flag, it supplies it to the PCM decoding unit 516. The CU decoding unit 532 decodes the encoded CU data according to the control of the PCM decoding unit 516 based on the I_PCM_flag (based on the On / Off control signal supplied from the PCM encoding unit 321).

[0312] For example, if I_PCM_flag is a value indicating that it is not in I_PCM mode, and a control signal indicating "On" is obtained from the PCM decoding unit 516, the CU decoding unit 532 decodes the encoded CU data supplied from the storage buffer 501 based on the NAL data supplied from the NAL decoding unit 531, and supplies the quantized orthogonal transformation coefficients to the inverse quantization unit 503.

[0313] In contrast, for example, if I_PCM_flag is a value indicating that it is in I_PCM mode, and a control signal indicating "Off" is obtained from the PCM decoding unit 516, the CU decoding unit 532 supplies the CU data (unencoded image data) supplied from the storage buffer 501 to the inverse quantization unit 503.

[0314] The CU decoding unit 532 supplies information such as filter coefficients and adaptive filter flags obtained by decoding the CU data to the loop filter 506.

[0315] As shown in Figure 8, the PCM decoding unit 516 has an I_PCM_flag buffer 541 and a PCM control unit 542.

[0316] The I_PCM_flag buffer 541 stores the I_PCM_flag supplied from the CU decoding unit 532 of the reversible decoding unit 502 and supplies it to the PCM control unit 542 at a predetermined timing.

[0317] The PCM control unit 542 determines whether the I_PCM mode was selected during encoding in the image encoding device 300 based on the value of I_PCM_flag obtained from the I_PCM_flag buffer 541. Based on the determination result, the PCM control unit 542 supplies an On / Off control signal to the CU decoding unit 532 to control its operation.

[0318] For example, if it is determined that the I_PCM mode is not selected, the PCM control unit 542 supplies a control signal indicating "On" to the CU decoding unit 532, causing the CU decoding unit 532 to decode the encoded CU data, and then supplies the quantized orthogonal transformation coefficients obtained by the decoding to the inverse quantization unit 503.

[0319] In response to this, if it is determined that the I_PCM mode has been selected, for example, the PCM control unit 542 supplies a control signal indicating "Off" to the CU decoding unit 532, causing the CU data, which is unencoded data, to be supplied to the inverse quantization unit 503 as output pixel values.

[0320] With this control, the CU decoding unit 532 can appropriately decode the encoded data supplied from the image encoding device 300. In other words, the CU decoding unit 532 can decode more appropriately even when the I_PCM mode is controlled in units smaller than LCU.

[0321] The PCM control unit 542 also supplies On / Off control signals to the adaptive right shift unit 507 and the adaptive left shift unit 511 based on the determination result of the I_PCM mode, and controls their operation.

[0322] For example, if it is determined that the I_PCM mode is not selected, the PCM control unit 542 supplies a control signal indicating "On" to the adaptive right shift unit 507 and the adaptive left shift unit 511, causing them to execute left shift processing and right shift processing, and enabling bit precision expansion in internal processing.

[0323] In response to this, if it is determined that the I_PCM mode has been selected, for example, the PCM control unit 542 supplies a control signal indicating "Off" to the adaptive right shift unit 507 and the adaptive left shift unit 511, causing the left shift and right shift processes to be omitted (skipped) and preventing the expansion of bit precision in the internal processing.

[0324] With this control, the PCM control unit 542 can appropriately eliminate redundant processing even when the I_PCM mode is controlled in units smaller than LCU.

[0325] Furthermore, the PCM control unit 542 controls the operation of the loop filter 506 by supplying On / Off control signals to the loop filter 506 based on the determination result of the I_PCM mode. For example, if it is determined that the I_PCM mode is not selected, the PCM control unit 542 supplies an On control signal to the loop filter 506. Conversely, if it is determined that the I_PCM mode is selected, for example, the PCM control unit 542 supplies an Off control signal to the loop filter 506.

[0326] With this control, the PCM control unit 542 can more effectively eliminate redundant processing even when the I_PCM mode is controlled in units smaller than LCU.

[0327] As shown in Figure 8, the loop filter 506 has a deblocking filter 551, a pixel sorting unit 552, and a filtering unit 553.

[0328] The deblocking filter 551 removes block distortion by performing a deblocking filter operation on the decoded image (pixel value before deblocking filter) supplied from the calculation unit 505, similar to the case of the deblocking filter 206.

[0329] Similar to the case of the image encoding device 300, the deblocking filter process is performed regardless of whether the CU is in I_PCM mode or not. The deblocking filter 551 supplies the filtering result (pixel value after deblocking filter) to the pixel sorting unit 552.

[0330] The pixel sorting unit 552 sorts each filter processing result (deblocked pixel value) into pixel values ​​that undergo adaptive loop filtering and pixel values ​​that do not undergo adaptive loop filtering, according to the value of the On / Off control signal supplied from the PCM control unit 542.

[0331] For example, if a control signal indicating "On" is supplied from the PCM control unit 542, the pixel sorting unit 552 sorts the deblocked pixel value of that CU into a pixel value that undergoes adaptive loop filtering. Conversely, if a control signal indicating "Off" is supplied from the PCM control unit 542, the pixel sorting unit 552 sorts the deblocked pixel value of that CU into a pixel value that does not undergo adaptive loop filtering.

[0332] The pixel sorting unit 552 supplies the pixel value (deblocked pixel value) of each sorted pixel to the filtering unit 553.

[0333] The filtering unit 553 performs adaptive loop filtering on pixel values ​​that have been sorted to undergo adaptive loop filtering based on the adaptive loop filtering flag supplied from the CU decoding unit 532.

[0334] Adaptive loop filtering is performed for each predetermined block, which is set independently of the CU. If the value of the adaptive loop filtering flag for the block is a value that indicates that filtering was not performed on the encoding side (for example, "0"), the filtering unit 553 omits (skips) the adaptive loop filtering for that block and supplies the pixel value after deblock filtering to the adaptive right shift unit 507 as the pixel value after adaptive filtering.

[0335] By the way, the method for setting the adaptive loop filter flag depends on the specifications of the image encoding device 300. Therefore, depending on the method for setting the adaptive loop filter flag by the image encoding device 300, even if the value of the adaptive loop filter flag for a block is a value that indicates that filtering has been performed on the encoding side (for example, "1"), there is a possibility that pixels in I_PCM mode are included in that block.

[0336] Therefore, if there are pixel values ​​in the block that have been sorted to not undergo adaptive loop filtering, the filtering unit 553 will omit (skip) the adaptive loop filtering for the block, even if the value of the adaptive loop filtering flag for the block is a value that indicates that filtering has been performed on the encoding side (for example, "1"), and will supply the supplied deblocked pixel values ​​to the adaptive right shift unit 507 as the adaptively filtered pixel values.

[0337] In other words, the filtering unit 553 performs adaptive loop filtering on a block only if the value of the adaptive loop filtering flag for that block is a value that indicates filtering has been performed on the encoding side (for example, "1"), and all pixels in that block are sorted to be subjected to adaptive loop filtering.

[0338] When the filtering unit 553 performs adaptive loop filtering, it performs adaptive loop filtering using the filter coefficients supplied from the CU decoding unit 532 (the filter coefficients used in the adaptive loop filtering on the encoding side).

[0339] The filtering unit 553 supplies the pixel values ​​that have undergone adaptive loop filtering as the adaptively filtered pixel values ​​to the adaptive right shift unit 507.

[0340] With this control, the PCM control unit 542 can more appropriately eliminate redundant processing even when the I_PCM mode (uncompressed mode) is controlled at a CU unit smaller than LCU.

[0341] As described above, the image decoding device 500 can improve encoding efficiency while suppressing a reduction in the efficiency of the encoding process through the processing of each processing unit.

[0342] [Decryption process flow] Next, we will explain the flow of each process performed by the image decoding device 500 as described above. First, we will explain an example of the decoding process flow with reference to the flowcharts in Figures 17 and 18.

[0343] When the decoding process is started, in step S501, the storage buffer 501 stores the transmitted image (encoded data). In step S502, the CU decoding unit 532 obtains an adaptive filter flag from the encoded data stored in step S501. In step S503, the CU decoding unit 532 obtains an I_PCM_flag from the encoded data stored in step S501. The I_PCM_flag buffer 541 stores the I_PCM_flag.

[0344] In step S504, the PCM control unit 542 determines whether the encoding mode of the encoded data (CU data) accumulated in step S501 is I_PCM mode (i.e., unencoded data) based on the value of I_PCM_flag stored in the I_PCM_flag buffer 541.

[0345] If it is determined that the encoding mode is not I_PCM mode, the PCM control unit 542 proceeds to step S505. In step S505, the reversible decoding unit 502 performs reversible decoding processing to decode the encoded data (CU data) accumulated in step S501 and obtain quantized orthogonal transformation coefficients, filter coefficients, etc.

[0346] In step S506, the inverse quantization unit 503 inversely quantizes the quantized orthogonal transformation coefficients obtained in step S505. In step S507, the inverse orthogonal transformation unit 504 inversely transforms the orthogonal transformation coefficients inversely quantized in step S506 to generate decoded image data.

[0347] In step S508, the adaptive left shift unit 511 acquires a reference image corresponding to the CU from the frame memory 510 and performs a left shift operation on the reference image in accordance with the control of the PCM decoding unit 516.

[0348] In step S509, the intra prediction unit 513 or the motion prediction / compensation unit 514 performs prediction processing using the reference image that was left-shifted in step S508 and generates a predicted image.

[0349] In step S510, the calculation unit 505 adds the predicted image generated in step S509 by the intra prediction unit 513 or the motion prediction / compensation unit 514 to the difference information obtained in step S507.

[0350] In step S511, the loop filter 506 performs loop filtering on the summation result of step S510.

[0351] In step S512, the adaptive right shift unit 507 performs a right shift operation on the loop filter processing result of step S511 in accordance with the control of the PCM decoding unit 516.

[0352] In step S513, the frame memory 510 stores the decoded image data as a reference image.

[0353] In step S514, the screen reordering buffer 508 rearranges the frames of the decoded image data. That is, the order of the frames of the decoded image data, which have been rearranged for encoding by the screen reordering buffer 302 (Figure 7) of the image encoding device 300, is rearranged to the original display order.

[0354] In step S515, the D / A conversion unit 509 performs D / A conversion on the decoded image data, which was rearranged in step S514. This decoded image data is output to a display (not shown), and the image is displayed.

[0355] Also, if it is determined in step S504 that the encoding mode is I_PCM mode, the PCM control unit 542 proceeds to step S521 in Figure 18. In step S521, the reversible decoding unit 502 performs reversible decoding and uses the encoded data (uncompressed data) accumulated in step S501 as the encoding result (output pixel value).

[0356] In step S522, the loop filter 506 performs loop filtering on the output pixel values ​​obtained in step S521. When the processing in step S522 is completed, the loop filter 506 returns to step S513 in Figure 17 and executes the subsequent processing.

[0357] [Loop Filter Processing Flow] Next, with reference to the flowchart in Figure 19, an example of the loop filtering process performed in step S511 in Figure 17 and step S522 in Figure 18 will be explained.

[0358] When loop filtering is started, the deblocking filter 551 performs deblocking filtering on the pre-deblocking pixel values ​​obtained in step S510 or step S521 in step S541.

[0359] In step S542, the filtering unit 553 obtains the filter coefficients. In step S543, the pixel sorting unit 552 determines whether or not to perform adaptive loop filtering based on the value of the adaptive loop filtering flag. If it is determined that adaptive loop filtering should be performed, the pixel sorting unit 552 proceeds to step S544.

[0360] In step S544, the pixel sorting unit 552 sorts the deblocked pixel values ​​according to the control of the PCM control unit 542, depending on whether or not it is in I_PCM mode.

[0361] In step S545, the filtering unit 545 performs adaptive loop filtering on the deblocked pixel values ​​that have been sorted for adaptive loop filtering, using the filter coefficients obtained in step S542. When the processing in step S545 is completed, the filtering unit 545 finishes the loop filtering and returns the process to step S511 in Figure 17 or step S522 in Figure 18, and proceeds to step S512 in Figure 17 or step S513 in Figure 17.

[0362] Also, if it is determined in step S543 of Figure 19 that adaptive loop filtering should not be performed, the pixel sorting unit 552 terminates the loop filtering and returns the process to step S511 of Figure 17 or step S522 of Figure 18, and proceeds to step S512 of Figure 17 or step S513 of Figure 17.

[0363] By performing each process as described above, the image decoding device 500 can improve encoding efficiency while suppressing a reduction in the efficiency of the encoding process.

[0364] [Location of I_PCM information] As explained above, by enabling the selection control of the I_PCM mode on a CU basis, it is possible to encode only some of the CUs within an LCU in I_PCM mode, for example, as shown in Figure 20A. Figure 20A shows the structure of CUs within one LCU. The LCU shown in Figure 20A consists of seven CUs, CU0 to CU6. Each number indicates the processing order within that LCU. In the example in Figure 20A, the CUs shown in gray (CU1, CU3, CU6) are assumed to be encoded in I_PCM mode.

[0365] In such cases, as shown in Figure 20B, I_PCM information, which is information about the I_PCM inside the LCU, may be added to the beginning of the LCU data in the code stream.

[0366] The I_PCM information includes, for example, flag information indicating whether the LCU contains a CU encoded in I_PCM mode. In the example of Figure 20, since CU1, CU3, and CU6 are CUs encoded in I_PCM mode, this flag information is set to a value (e.g., "1") indicating that the LCU contains a CU encoded in I_PCM mode.

[0367] In this way, by the image encoding device 300 adding information about I_PCM within the LCU to the beginning of the LCU, the image decoding device 500 can easily determine whether or not there is a CU encoded in I_PCM mode within the LCU before decoding the LCU.

[0368] In addition, the I_PCM information includes, for example, information indicating the CUs that are encoded in I_PCM mode and included in the LCU. In the example of Figure 20, CU1, CU3, and CU6 are CUs that are encoded in I_PCM mode, so the I_PCM information includes information indicating these CUs (CU1, CU3, CU6).

[0369] In this way, by the image encoding device 300 adding information about I_PCM within the LCU to the beginning of the LCU, the image decoding device 500 can easily determine which CUs within the LCU were encoded in I_PCM mode (uncompressed mode) before decoding the LCU.

[0370] In the above, information other than image data, such as I_PCM_flag, adaptive filter flag, filter coefficients, and I_PCM information, has been described as being provided from the image encoding device 300 to the image decoding device 500 as needed. However, this information may be added to any position in the encoded data. For example, it may be added to the beginning of CU or LCU, added to the slice header, or stored in the sequence parameter set (SPS) or picture parameter set (PPS), etc. Also, for example, this information may be stored in a parameter set such as SEI (Suplemental Enhancement Information) (for example, the header of a sequence or picture).

[0371] Furthermore, this information may be transmitted to the decryption side separately from the encoded data. In that case, it is necessary to clarify the correspondence between this information and the encoded data (so that the decryption side can understand it), but the method of doing so is arbitrary. For example, a separate table information showing the correspondence may be created, or link information indicating the corresponding data may be embedded in each other's data.

[0372] Alternatively, this information may be shared in advance between the image encoding device 300 and the image decoding device 500. In that case, the transmission of this information may be omitted.

[0373] <3. Third Embodiment> [Personal Computer] The series of processes described above can be executed by hardware or by software. In this case, for example, it may be configured as a personal computer as shown in Figure 21.

[0374] In Figure 21, the CPU (Central Processing Unit) 601 of the personal computer 600 executes various processes according to the program stored in the ROM (Read Only Memory) 602 or the program loaded from the storage unit 613 into the RAM (Random Access Memory) 603. The RAM 603 also stores data that is necessary for the CPU 601 to execute various processes.

[0375] The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output interface 610 is also connected to this bus 604.

[0376] The input / output interface 610 is connected to an input unit 611 consisting of a keyboard, mouse, etc., an output unit 612 consisting of a display such as a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display), and speakers, a storage unit 613 consisting of a hard disk, etc., and a communication unit 614 consisting of a modem, etc. The communication unit 614 performs communication processing via a network including the Internet.

[0377] A drive 615 is also connected to the input / output interface 610 as needed, and removable media 621 such as magnetic disks, optical disks, magneto-optical disks, or semiconductor memory are appropriately mounted there, and computer programs read from them are installed in the storage unit 613 as needed.

[0378] When the above-described series of processes are to be executed by software, the programs that make up that software are installed from a network or storage medium.

[0379] This recording medium, as shown in Figure 21, for example, is not only composed of removable media 621 such as magnetic disks (including flexible disks), optical disks (including CD-ROMs (Compact Disc - Read Only Memory), DVDs (Digital Versatile Discs)), magneto-optical disks (including MDs (Mini Discs)), or semiconductor memory on which programs are recorded, which are distributed to the user separately from the main body of the device, but also consists of ROMs 602 on which programs are recorded, or hard disks included in the storage unit 613, which are pre-installed in the main body of the device and distributed to the user.

[0380] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when a call is made.

[0381] In addition, in this specification, the step of describing a program to be recorded on a recording medium includes not only processes that are performed chronologically in the order described, but also processes that are not necessarily performed chronologically, but are performed in parallel or individually.

[0382] In this specification, a system refers to an entire apparatus composed of multiple devices (apparatus).

[0383] Furthermore, the configuration described above as a single device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described above as multiple devices (or processing units) may be combined and configured as a single device (or processing unit). In addition, it is also possible to add configurations other than those described above to the configuration of each device (or each processing unit). Furthermore, if the overall system configuration and operation are substantially the same, a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit). In other words, the embodiments of this technology are not limited to the embodiments described above, and various modifications are possible without departing from the gist of this technology.

[0384] For example, the reversible coding unit 307, the loop filter 312, and the PCM coding unit 321 shown in Figure 8 may each be configured as independent devices. Alternatively, the NAL coding unit 331, the CU coding unit 332, the I_PCM_flag generation unit 341, the PCM determination unit 342, the deblocking filter 351, the pixel sorting unit 352, the filter coefficient calculation unit 353, and the filtering unit 354 shown in Figure 8 may each be configured as independent devices.

[0385] Furthermore, the input data amount calculation unit 361, PCM determination unit 362, coding control unit 363, adaptive shift control unit 364, and filter control unit shown in Figure 9 may each be configured as independent devices.

[0386] Furthermore, these processing units may be combined in any way to form independent devices. Of course, they may be combined with any of the processing units shown in Figures 7 to 9, or with processing units not shown.

[0387] The same applies to the image decoding device 500. For example, the reversible decoding unit 502, the loop filter 506, and the PCM decoding unit 516 shown in Figure 15 may each be configured as independent devices. Alternatively, the NAL decoding unit 531, the CU decoding unit 532, the I_PCM_flag buffer 541, the PCM control unit 542, the deblocking filter 551, the pixel sorting unit 552, and the filtering unit 553 shown in Figure 16 may each be configured as independent devices.

[0388] Furthermore, these processing units may be combined in any way to form independent devices. Of course, they may be combined with any processing units shown in Figures 15 and 16, or with processing units not shown.

[0389] Furthermore, for example, the image encoding and decoding devices described above can be applied to any electronic device. Examples are described below.

[0390] <4. Fourth Embodiment> [Television receiver] Figure 22 is a block diagram showing a typical configuration of a television receiver using an image decoding device 500.

[0391] The television receiver 1000 shown in Figure 22 includes a terrestrial tuner 1013, a video decoder 1015, a video signal processing circuit 1018, a graphics generation circuit 1019, a panel driving circuit 1020, and a display panel 1021.

[0392] The terrestrial tuner 1013 receives the terrestrial analog broadcast wave signal via the antenna, demodulates it, acquires the video signal, and supplies it to the video decoder 1015. The video decoder 1015 decodes the video signal supplied from the terrestrial tuner 1013 and supplies the resulting digital component signal to the video signal processing circuit 1018.

[0393] The video signal processing circuit 1018 performs predetermined processing, such as noise reduction, on the video data supplied from the video decoder 1015, and supplies the resulting video data to the graphic generation circuit 1019.

[0394] The graphics generation circuit 1019 generates video data of programs to be displayed on the display panel 1021, as well as image data obtained through processing based on applications supplied via the network, and supplies the generated video data and image data to the panel drive circuit 1020. The graphics generation circuit 1019 also generates video data (graphics) for displaying screens used by the user for selecting items, etc., and supplies the resulting video data to the panel drive circuit 1020, as appropriate.

[0395] The panel drive circuit 1020 drives the display panel 1021 based on the data supplied from the graphics generation circuit 1019, and displays the program's video and the various screens described above on the display panel 1021.

[0396] The display panel 1021 consists of an LCD (Liquid Crystal Display) or the like, and displays the program's video and other content according to the control of the panel drive circuit 1020.

[0397] The television receiver 1000 also includes an audio A / D (Analog / Digital) conversion circuit 1014, an audio signal processing circuit 1022, an echo cancellation / speech synthesis circuit 1023, an audio amplification circuit 1024, and a speaker 1025.

[0398] The terrestrial tuner 1013 acquires not only the video signal but also the audio signal by demodulating the received broadcast wave signal. The terrestrial tuner 1013 supplies the acquired audio signal to the audio A / D conversion circuit 1014.

[0399] The audio A / D conversion circuit 1014 performs A / D conversion processing on the audio signal supplied from the terrestrial tuner 1013 and supplies the resulting digital audio signal to the audio signal processing circuit 1022.

[0400] The audio signal processing circuit 1022 performs predetermined processing, such as noise reduction, on the audio data supplied from the audio A / D conversion circuit 1014, and supplies the obtained audio data to the echo cancellation / speech synthesis circuit 1023.

[0401] The echo cancellation / speech synthesis circuit 1023 supplies the audio data supplied from the audio signal processing circuit 1022 to the audio amplification circuit 1024.

[0402] The audio amplification circuit 1024 performs D / A conversion and amplification processing on the audio data supplied from the echo cancellation / speech synthesis circuit 1023, adjusts it to a predetermined volume, and then outputs the audio from the speaker 1025.

[0403] Furthermore, the television receiver 1000 also has a digital tuner 1016 and an MPEG decoder 1017.

[0404] The digital tuner 1016 receives broadcast wave signals of digital broadcasts (terrestrial digital broadcasting, BS (Broadcasting Satellite) / CS (Communications Satellite) digital broadcasting) via an antenna, demodulates them, obtains MPEG-TS (Moving Picture Experts Group-Transport Stream), and supplies it to the MPEG decoder 1017.

[0405] The MPEG decoder 1017 decrypts the scramble applied to the MPEG-TS supplied from the digital tuner 1016 and extracts a stream containing the data of the program to be played (viewed). The MPEG decoder 1017 decodes the audio packets that make up the extracted stream and supplies the obtained audio data to the audio signal processing circuit 1022, and also decodes the video packets that make up the stream and supplies the obtained video data to the video signal processing circuit 1018. The MPEG decoder 1017 also supplies the EPG (Electronic Program Guide) data extracted from the MPEG-TS to the CPU 1032 via a path not shown.

[0406] The television receiver 1000 uses the image decoding device 500 described above as an MPEG decoder 1017 for decoding video packets. The MPEG-TS transmitted from broadcasting stations, etc., is encoded by the image encoding device 300.

[0407] The MPEG decoder 1017, similar to the image decoding device 500, appropriately decodes encoded data in units smaller than LCU by controlling the selection of the I_PCM mode. Therefore, the MPEG decoder 1017 can reduce redundant processing during encoding and reduce redundant information contained in the encoded data. As a result, the MPEG decoder 1017 can improve encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

[0408] The video data supplied from the MPEG decoder 1017 is processed in the video signal processing circuit 1018 in the same way as the video data supplied from the video decoder 1015, and the generated video data is appropriately superimposed in the graphic generation circuit 1019. The resulting image is then supplied to the display panel 1021 via the panel drive circuit 1020 and displayed.

[0409] The audio data supplied from the MPEG decoder 1017 is processed in the audio signal processing circuit 1022 in the same way as the audio data supplied from the audio A / D conversion circuit 1014, and is then supplied to the audio amplification circuit 1024 via the echo cancellation / speech synthesis circuit 1023, where D / A conversion and amplification processing are performed. As a result, audio adjusted to a predetermined volume is output from the speaker 1025.

[0410] The television receiver 1000 also has a microphone 1026 and an A / D conversion circuit 1027.

[0411] The A / D conversion circuit 1027 receives the user's voice signal captured by the microphone 1026 provided on the television receiver 1000 for voice conversation, performs A / D conversion processing on the received voice signal, and supplies the resulting digital voice data to the echo cancellation / voice synthesis circuit 1023.

[0412] When the A / D conversion circuit 1027 supplies voice data of the user (user A) of the television receiver 1000, the echo cancellation / speech synthesis circuit 1023 performs echo cancellation on the voice data of user A, synthesizes it with other voice data, and outputs the resulting voice data from the speaker 1025 via the voice amplification circuit 1024.

[0413] Furthermore, the television receiver 1000 also has an audio codec 1028, an internal bus 1029, an SDRAM (Synchronous Dynamic Random Access Memory) 1030, a flash memory 1031, a CPU 1032, a USB (Universal Serial Bus) I / F 1033, and a network I / F 1034.

[0414] The A / D conversion circuit 1027 receives the user's voice signal captured by the microphone 1026 provided on the television receiver 1000 for voice conversation, performs A / D conversion processing on the received voice signal, and supplies the resulting digital voice data to the voice codec 1028.

[0415] The audio codec 1028 converts the audio data supplied from the A / D conversion circuit 1027 into data in a predetermined format for transmission over the network, and supplies it to the network interface 1034 via the internal bus 1029.

[0416] The network interface 1034 is connected to the network via a cable attached to the network terminal 1035. The network interface 1034 transmits audio data supplied from the audio codec 1028 to other devices connected to the network, for example. The network interface 1034 also receives audio data transmitted from other devices connected via the network via the network terminal 1035 and supplies it to the audio codec 1028 via the internal bus 1029.

[0417] The audio codec 1028 converts the audio data supplied from the network interface 1034 into data in a predetermined format and supplies it to the echo cancellation / speech synthesis circuit 1023.

[0418] The echo cancellation / speech synthesis circuit 1023 performs echo cancellation on the audio data supplied from the audio codec 1028, and outputs the resulting audio data, obtained by combining it with other audio data, through the audio amplification circuit 1024 to the speaker 1025.

[0419] The SDRAM 1030 stores various data necessary for the CPU 1032 to perform processing.

[0420] The flash memory 1031 stores programs executed by the CPU 1032. The programs stored in the flash memory 1031 are read by the CPU 1032 at predetermined timings, such as when the television receiver 1000 is started up. The flash memory 1031 also stores EPG data acquired via digital broadcasting, and data acquired from a predetermined server via the network.

[0421] For example, the flash memory 1031 stores an MPEG-TS file containing content data acquired from a predetermined server via a network under the control of the CPU 1032. The flash memory 1031 supplies the MPEG-TS file to the MPEG decoder 1017 via the internal bus 1029 under the control of the CPU 1032, for example.

[0422] The MPEG decoder 1017 processes the MPEG-TS in the same way as the MPEG-TS supplied from the digital tuner 1016. In this way, the television receiver 1000 can receive content data consisting of video and audio via the network, decode it using the MPEG decoder 1017, and display the video or output the audio.

[0423] The television receiver 1000 also has a light receiving unit 1037 that receives infrared signals transmitted from the remote controller 1051.

[0424] The light receiving unit 1037 receives infrared light from the remote controller 1051 and outputs a control code representing the content of the user operation obtained by demodulation to the CPU 1032.

[0425] The CPU 1032 executes a program stored in the flash memory 1031 and controls the overall operation of the television receiver 1000 according to control codes supplied from the light receiving unit 1037. The CPU 1032 and the various parts of the television receiver 1000 are connected via paths not shown.

[0426] The USB I / F 1033 transmits and receives data with external devices of the television receiver 1000, which are connected via a USB cable attached to the USB terminal 1036. The network I / F 1034 connects to the network via a cable attached to the network terminal 1035 and also transmits and receives data other than audio data with various devices connected to the network.

[0427] The television receiver 1000, by using the image decoding device 500 as the MPEG decoder 1017, can improve the encoding efficiency of the data while suppressing a reduction in the efficiency of the encoding process when generating broadcast wave signals received via an antenna or content data acquired via a network.

[0428] <5. Fifth Embodiment> [Mobile phone] Figure 23 is a block diagram showing a typical configuration of a mobile phone using an image encoding device 300 and an image decoding device 500.

[0429] The mobile phone 1100 shown in Figure 23 has a main control unit 1150, a power supply circuit unit 1151, an operation input control unit 1152, an image encoder 1153, a camera I / F unit 1154, an LCD control unit 1155, an image decoder 1156, a multiplexing / decoupling unit 1157, a recording / playback unit 1162, a modulation / demodulation circuit unit 1158, and an audio codec 1159, all of which are connected to each other via a bus 1160.

[0430] The mobile phone 1100 also has an operation key 1119, a CCD (Charge Coupled Devices) camera 1116, a liquid crystal display 1118, a memory unit 1123, a transmitting and receiving circuit unit 1163, an antenna 1114, a microphone 1121, and a speaker 1117.

[0431] When the end call and power keys are turned on by the user, the power supply circuit unit 1151 supplies power from the battery pack to each part, thereby starting up the mobile phone 1100 to an operational state.

[0432] The mobile phone 1100 performs various operations such as sending and receiving voice signals, sending and receiving emails and image data, taking pictures, or recording data in various modes such as voice call mode and data communication mode, based on the control of the main control unit 1150 which consists of a CPU, ROM and RAM, etc.

[0433] For example, in voice call mode, the mobile phone 1100 converts the voice signal collected by the microphone 1121 into digital voice data using the voice codec 1159, modulates and demodulates this data using spread spectrum processing in the modulation / demodulation circuit 1158, and performs digital-to-analog conversion and frequency conversion processing in the transmitting / receiving circuit 1163. The mobile phone 1100 transmits the transmission signal obtained through this conversion process to a base station (not shown) via the antenna 1114. The transmission signal (voice signal) transmitted to the base station is supplied to the other party's mobile phone via the public telephone network.

[0434] Also, for example, in voice call mode, the mobile phone 1100 amplifies the received signal received by the antenna 1114 in the transmitting / receiving circuit 1163, further performs frequency conversion processing and analog-to-digital conversion processing, performs despread spectrum processing in the modulation / demodulation circuit 1158, and converts it into an analog voice signal by the voice codec 1159. The mobile phone 1100 outputs the converted analog voice signal from the speaker 1117.

[0435] Furthermore, for example, when sending an email in data communication mode, the mobile phone 1100 receives the text data of the email entered by operating the operation key 1119 in the operation input control unit 1152. The mobile phone 1100 processes the text data in the main control unit 1150 and displays it as an image on the liquid crystal display 1118 via the LCD control unit 1155.

[0436] The mobile phone 1100 also generates email data in the main control unit 1150 based on text data and user instructions received by the operation input control unit 1152. The mobile phone 1100 performs spread spectrum processing on the email data in the modulation / demodulation circuit unit 1158, and digital-to-analog conversion and frequency conversion processing in the transmitting / receiving circuit unit 1163. The mobile phone 1100 transmits the transmission signal obtained from the conversion process to a base station (not shown) via the antenna 1114. The transmission signal (email) transmitted to the base station is supplied to a predetermined destination via the network and mail server, etc.

[0437] Also, for example, when receiving an email in data communication mode, the mobile phone 1100 receives the signal transmitted from the base station via the antenna 1114 in the transmitting / receiving circuit unit 1163, amplifies it, and then performs frequency conversion processing and analog-to-digital conversion processing. The mobile phone 1100 then performs despread spectrum processing on the received signal in the modulation / demodulation circuit unit 1158 to restore the original email data. The mobile phone 1100 displays the restored email data on the liquid crystal display 1118 via the LCD control unit 1155.

[0438] The mobile phone 1100 can also record (store) received email data in the storage unit 1123 via the recording and playback unit 1162.

[0439] This storage unit 1123 is any rewritable storage medium. The storage unit 1123 may be, for example, a semiconductor memory such as RAM or built-in flash memory, a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a removable media such as a memory card. Of course, it may also be something other than these.

[0440] Furthermore, for example, when transmitting image data in data communication mode, the mobile phone 1100 generates image data with the CCD camera 1116 by imaging. The CCD camera 1116 has optical devices such as lenses and apertures and a CCD as a photoelectric conversion element, and it images a subject, converts the intensity of the received light into an electrical signal, and generates image data of the subject. The CCD camera 1116 encodes the image data with the image encoder 1153 via the camera I / F unit 1154 and converts it into encoded image data.

[0441] The mobile phone 1100 uses the image encoding device 300 described above as an image encoder 1153 that performs such processing. The image encoder 1153 controls the selection of the I_PCM mode in units smaller than LCU, similar to the case of the image encoding device 300. In other words, the image encoder 1153 can further reduce redundant processing during encoding and further reduce redundant information contained in the encoded data. As a result, the image encoder 1153 can improve encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

[0442] At the same time, the mobile phone 1100 converts the sound collected by the microphone 1121 while the CCD camera 1116 is capturing images, using an audio codec 1159, and then encodes it.

[0443] The mobile phone 1100, in the multiplexing / decompression unit 1157, multiplexes encoded image data supplied from the image encoder 1153 and digital audio data supplied from the audio codec 1159 in a predetermined manner. The mobile phone 1100 then performs spread spectrum processing on the resulting multiplexed data in the modulation / demodulation circuit unit 1158, and digital-to-analog conversion and frequency conversion processing in the transmitting / receiving circuit unit 1163. The mobile phone 1100 transmits the transmission signal obtained from the conversion process to a base station (not shown) via the antenna 1114. The transmission signal (image data) transmitted to the base station is supplied to the communication partner via a network or the like.

[0444] If image data is not transmitted, the mobile phone 1100 can also display the image data generated by the CCD camera 1116 on the liquid crystal display 1118 via the LCD control unit 1155, without going through the image encoder 1153.

[0445] Also, for example, in data communication mode, when receiving video data linked to a simple homepage, the mobile phone 1100 receives the signal transmitted from the base station via the antenna 1114 in the transmitting / receiving circuit unit 1163, amplifies it, and then performs frequency conversion processing and analog-to-digital conversion processing. The mobile phone 1100 performs despread spectrum processing on the received signal in the modulation / demodulation circuit unit 1158 to restore the original multiplexed data. The mobile phone 1100 separates the multiplexed data in the multiplexing / decompression unit 1157, dividing it into encoded image data and audio data.

[0446] The mobile phone 1100 generates playable video data by decoding the encoded image data in the image decoder 1156, and displays this on the liquid crystal display 1118 via the LCD control unit 1155. As a result, for example, video data contained in a video file linked to a simple homepage is displayed on the liquid crystal display 1118.

[0447] The mobile phone 1100 uses the image decoding device 500 described above as an image decoder 1156 that performs such processing. In other words, the image decoder 1156 appropriately decodes encoded data in which the selection of the I_PCM mode is controlled in units smaller than LCU, similar to the case of the image decoding device 500. Therefore, the image decoder 1156 can reduce redundant processing during encoding and reduce redundant information contained in the encoded data. As a result, the image decoder 1156 can improve encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

[0448] At the same time, the mobile phone 1100 converts the digital audio data into an analog audio signal using the audio codec 1159 and outputs it through the speaker 1117. This allows, for example, the audio data contained in a video file linked to a simple homepage to be played.

[0449] In addition, as with email, the mobile phone 1100 can also record (store) data linked to a received simple homepage, etc., in the storage unit 1123 via the recording and playback unit 1162.

[0450] In addition, the mobile phone 1100 can analyze the two-dimensional code captured by the CCD camera 1116 in the main control unit 1150 and acquire the information recorded in the two-dimensional code.

[0451] Furthermore, the mobile phone 1100 can communicate with external devices using infrared light via the infrared communication unit 1181.

[0452] The mobile phone 1100 uses the image encoding device 300 as the image encoder 1153, which allows it to improve encoding efficiency while suppressing a reduction in the efficiency of the encoding process when encoding and transmitting, for example, image data generated by the CCD camera 1116.

[0453] Furthermore, by using the image decoding device 500 as the image decoder 1156, the mobile phone 1100 can improve the encoding efficiency of the data while suppressing a reduction in the efficiency of the encoding process when generating data (encoded data) of video files linked to a simple homepage, for example.

[0454] In the above description, the mobile phone 1100 has been described as using a CCD camera 1116, but instead of the CCD camera 1116, an image sensor using CMOS (Complementary Metal Oxide Semiconductor) (CMOS image sensor) may be used. In this case as well, the mobile phone 1100 can capture an image of a subject and generate image data of the subject, just as when using the CCD camera 1116.

[0455] In addition, although the above has been described using the mobile phone 1100 as an example, the image encoding device 300 and the image decoding device 500 can be applied to any device that has imaging and communication functions similar to the mobile phone 1100, such as PDAs (Personal Digital Assistants), smartphones, UMPCs (Ultra Mobile Personal Computers), netbooks, and notebook personal computers.

[0456] <6. Sixth Embodiment> [Hard disk recorder] Figure 24 is a block diagram showing a typical configuration of a hard disk recorder using an image encoding device 300 and an image decoding device 500.

[0457] The hard disk recorder (HDD recorder) 1200 shown in Figure 24 is a device that stores audio and video data of broadcast programs contained in broadcast wave signals (television signals) transmitted from satellite or terrestrial antennas, etc., received by the tuner, on its built-in hard disk, and provides the stored data to the user at a timing according to the user's instructions.

[0458] The hard disk recorder 1200 can, for example, extract audio data and video data from a broadcast wave signal, decode them as appropriate, and store them on its built-in hard disk. The hard disk recorder 1200 can also, for example, acquire audio data and video data from other devices via a network, decode them as appropriate, and store them on its built-in hard disk.

[0459] Furthermore, the hard disk recorder 1200 can, for example, decode audio data and video data recorded on its internal hard disk and supply it to the monitor 1260, display the image on the monitor 1260's screen, and output the sound from the monitor 1260's speaker. The hard disk recorder 1200 can also, for example, decode audio data and video data extracted from a broadcast wave signal acquired via a tuner, or audio data and video data acquired from another device via a network, supply it to the monitor 1260, display the image on the monitor 1260's screen, and output the sound from the monitor 1260's speaker.

[0460] Of course, other actions are also possible.

[0461] As shown in Figure 24, the hard disk recorder 1200 includes a receiving unit 1221, a demodulation unit 1222, a demultiplexer 1223, an audio decoder 1224, a video decoder 1225, and a recorder control unit 1226. The hard disk recorder 1200 further includes an EPG data memory 1227, a program memory 1228, a work memory 1229, a display converter 1230, an OSD (On Screen Display) control unit 1231, a display control unit 1232, a recording and playback unit 1233, a D / A converter 1234, and a communication unit 1235.

[0462] The display converter 1230 also has a video encoder 1241. The recording and playback unit 1233 has an encoder 1251 and a decoder 1252.

[0463] The receiving unit 1221 receives an infrared signal from a remote controller (not shown), converts it into an electrical signal, and outputs it to the recorder control unit 1226. The recorder control unit 1226 is configured, for example, by a microprocessor and performs various processes according to a program stored in the program memory 1228. At this time, the recorder control unit 1226 uses the work memory 1229 as needed.

[0464] The communication unit 1235 is connected to a network and performs communication processing with other devices via the network. For example, the communication unit 1235 is controlled by the recorder control unit 1226 and communicates with a tuner (not shown) and mainly outputs tuning control signals to the tuner.

[0465] The demodulation unit 1222 demodulates the signal supplied from the tuner and outputs it to the demultiplexer 1223. The demultiplexer 1223 separates the data supplied from the demodulation unit 1222 into audio data, video data, and EPG data, and outputs them to the audio decoder 1224, video decoder 1225, or recorder control unit 1226, respectively.

[0466] The audio decoder 1224 decodes the input audio data and outputs it to the recording and playback unit 1233. The video decoder 1225 decodes the input video data and outputs it to the display converter 1230. The recorder control unit 1226 supplies the input EPG data to the EPG data memory 1227 for storage.

[0467] The display converter 1230 encodes the video data supplied from the video decoder 1225 or the recorder control unit 1226 into, for example, NTSC (National Television Standards Committee) video data using the video encoder 1241, and outputs it to the recording and playback unit 1233. The display converter 1230 also converts the screen size of the video data supplied from the video decoder 1225 or the recorder control unit 1226 to a size corresponding to the size of the monitor 1260, converts it into NTSC video data using the video encoder 1241, converts it into an analog signal, and outputs it to the display control unit 1232.

[0468] The display control unit 1232, under the control of the recorder control unit 1226, superimposes the OSD (On Screen Display) signal output by the OSD control unit 1231 onto the video signal input from the display converter 1230, outputs it to the display of the monitor 1260, and displays it.

[0469] The monitor 1260 is also supplied with audio data output by the audio decoder 1224, which has been converted into an analog signal by the D / A converter 1234. The monitor 1260 outputs this audio signal from its built-in speaker.

[0470] The recording and playback unit 1233 has a hard disk as a storage medium for recording video data, audio data, etc.

[0471] The recording and playback unit 1233 encodes, for example, the audio data supplied from the audio decoder 1224 using the encoder 1251. The recording and playback unit 1233 also encodes the video data supplied from the video encoder 1241 of the display converter 1230 using the encoder 1251. The recording and playback unit 1233 combines the encoded audio data and the encoded video data using a multiplexer. The recording and playback unit 1233 amplifies the combined data by channel coding and writes the data to the hard disk via the recording head.

[0472] The recording and playback unit 1233 plays back the data recorded on the hard disk via the playback head, amplifies it, and separates it into audio data and video data using a demultiplexer. The recording and playback unit 1233 decodes the audio data and video data using a decoder 1252. The recording and playback unit 1233 performs D / A conversion on the decoded audio data and outputs it to the speaker of the monitor 1260. The recording and playback unit 1233 also performs D / A conversion on the decoded video data and outputs it to the display of the monitor 1260.

[0473] The recorder control unit 1226 reads the latest EPG data from the EPG data memory 1227 based on user instructions indicated by an infrared signal from the remote controller received via the receiver unit 1221, and supplies it to the OSD control unit 1231. The OSD control unit 1231 generates image data corresponding to the input EPG data and outputs it to the display control unit 1232. The display control unit 1232 outputs the video data input from the OSD control unit 1231 to the display of the monitor 1260 and displays it. As a result, the EPG (Electronic Program Guide) is displayed on the display of the monitor 1260.

[0474] The hard disk recorder 1200 can also acquire various types of data, such as video data, audio data, or EPG data, supplied from other devices via a network such as the Internet.

[0475] The communication unit 1235 is controlled by the recorder control unit 1226 and acquires encoded data such as video data, audio data, and EPG data transmitted from other devices via the network, and supplies it to the recorder control unit 1226. The recorder control unit 1226, for example, supplies the acquired video data and audio data encoded data to the recording and playback unit 1233 and stores it on the hard disk. At this time, the recorder control unit 1226 and the recording and playback unit 1233 may perform processing such as re-encoding as needed.

[0476] The recorder control unit 1226 decodes the encoded data of the acquired video data and audio data and supplies the resulting video data to the display converter 1230. The display converter 1230 processes the video data supplied from the recorder control unit 1226 in the same way as the video data supplied from the video decoder 1225 and supplies it to the monitor 1260 via the display control unit 1232 to display the image.

[0477] In addition, the recorder control unit 1226 may supply the decoded audio data to the monitor 1260 via the D / A converter 1234 in conjunction with the image display, and output the audio from the speaker.

[0478] Furthermore, the recorder control unit 1226 decodes the encoded data of the acquired EPG data and supplies the decoded EPG data to the EPG data memory 1227.

[0479] The hard disk recorder 1200 described above uses an image decoding device 500 as a decoder built into the video decoder 1225, decoder 1252, and recorder control unit 1226. In other words, the decoders built into the video decoder 1225, decoder 1252, and recorder control unit 1226 appropriately decode encoded data in units smaller than LCUs, with the I_PCM mode selection controlled in units smaller than LCUs, similar to the case of the image decoding device 500. Therefore, the decoders built into the video decoder 1225, decoder 1252, and recorder control unit 1226 can reduce redundant processing during encoding and reduce redundant information contained in the encoded data. As a result, the decoders built into the video decoder 1225, decoder 1252, and recorder control unit 1226 can improve encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

[0480] Therefore, the hard disk recorder 1200 can improve the encoding efficiency of the data while suppressing a reduction in the efficiency of the encoding process when generating video data (encoded data) received by the tuner or communication unit 1235, or video data (encoded data) played back by the recording and playback unit 1233.

[0481] The hard disk recorder 1200 also uses the image encoding device 300 as the encoder 1251. Therefore, the encoder 1251 controls the selection of the I_PCM mode in units smaller than LCU, similar to the case of the image encoding device 300. In other words, the encoder 1251 can further reduce redundant processing during encoding and further reduce redundant information contained in the encoded data. As a result, the encoder 1251 can improve encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

[0482] Therefore, the hard disk recorder 1200 can improve encoding efficiency while suppressing a reduction in the efficiency of the encoding process when generating encoded data to be recorded on the hard disk, for example.

[0483] In the above description, a hard disk recorder 1200 that records video data and audio data on a hard disk has been described, but of course, any recording medium can be used. For example, even in a recorder that uses a recording medium other than a hard disk, such as flash memory, optical disc, or videotape, the image encoding device 300 and the image decoding device 500 can be applied in the same way as in the case of the hard disk recorder 1200 described above.

[0484] <7. Seventh Embodiment> [camera] Figure 25 is a block diagram showing a typical configuration of a camera using an image encoding device 300 and an image decoding device 500.

[0485] The camera 1300 shown in Figure 25 captures an image of the subject, displays the image of the subject on the LCD 1316, or records it as image data on the recording medium 1333.

[0486] The lens block 1311 directs light (i.e., the image of the subject) onto the CCD / CMOS 1312. The CCD / CMOS 1312 is an image sensor using a CCD or CMOS, which converts the intensity of the received light into an electrical signal and supplies it to the camera signal processing unit 1313.

[0487] The camera signal processing unit 1313 converts the electrical signals supplied from the CCD / CMOS 1312 into Y, Cr, and Cb color difference signals and supplies them to the image signal processing unit 1314. The image signal processing unit 1314, under the control of the controller 1321, performs predetermined image processing on the image signals supplied from the camera signal processing unit 1313 and encodes the image signals with the encoder 1341. The image signal processing unit 1314 supplies the encoded data generated by encoding the image signals to the decoder 1315. Furthermore, the image signal processing unit 1314 acquires display data generated on the on-screen display (OSD) 1320 and supplies it to the decoder 1315.

[0488] In the above processing, the camera signal processing unit 1313 appropriately utilizes the DRAM (Dynamic Random Access Memory) 1318 connected via the bus 1317 and stores image data and encoded data obtained by encoding that image data in the DRAM 1318 as needed.

[0489] The decoder 1315 decodes the encoded data supplied from the image signal processing unit 1314 and supplies the resulting image data (decoded image data) to the LCD 1316. The decoder 1315 also supplies the display data supplied from the image signal processing unit 1314 to the LCD 1316. The LCD 1316 appropriately combines the image of the decoded image data supplied from the decoder 1315 with the image of the display data and displays the combined image.

[0490] The on-screen display 1320, under the control of the controller 1321, outputs display data such as menu screens and icons consisting of symbols, characters, or shapes to the image signal processing unit 1314 via the bus 1317.

[0491] The controller 1321 executes various processes based on signals indicating the content of commands given by the user using the operation unit 1322, and also controls the image signal processing unit 1314, DRAM 1318, external interface 1319, on-screen display 1320, and media drive 1323, etc. via the bus 1317. The FLASH ROM 1324 stores programs and data necessary for the controller 1321 to execute various processes.

[0492] For example, the controller 1321 can encode image data stored in the DRAM 1318 or decode encoded data stored in the DRAM 1318 on behalf of the image signal processing unit 1314 and the decoder 1315. In this case, the controller 1321 may perform the encoding and decoding process using the same encoding and decoding method as the image signal processing unit 1314 and the decoder 1315, or it may perform the encoding and decoding process using a method not supported by the image signal processing unit 1314 and the decoder 1315.

[0493] Also, for example, if the operation unit 1322 instructs the start of image printing, the controller 1321 reads image data from the DRAM 1318 and supplies it to the printer 1334 connected to the external interface 1319 via the bus 1317 to print.

[0494] Furthermore, for example, if image recording is instructed from the operation unit 1322, the controller 1321 reads encoded data from the DRAM 1318 and supplies it to the recording medium 1333 mounted on the media drive 1323 via the bus 1317 for storage.

[0495] The recording medium 1333 is any read-write removable medium, such as a magnetic disk, magneto-optical disk, optical disk, or semiconductor memory. The type of removable medium for the recording medium 1333 is, of course, arbitrary and may be a tape device, a disk, or a memory card. It may also be a contactless IC card, etc.

[0496] Alternatively, the media drive 1323 and the recording medium 1333 may be integrated and configured using a non-portable storage medium such as an internal hard disk drive or an SSD (Solid State Drive).

[0497] The external interface 1319 is configured, for example, as a USB input / output terminal, and is connected to the printer 1334 when printing images. A drive 1331 is also connected to the external interface 1319 as needed, and removable media 1332 such as magnetic disks, optical disks, or magneto-optical disks are appropriately mounted, and computer programs read from them are installed in the FLASH ROM 1324 as needed.

[0498] Furthermore, the external interface 1319 has a network interface that connects to a predetermined network such as a LAN or the Internet. The controller 1321 can, for example, read encoded data from the DRAM 1318 in accordance with instructions from the operation unit 1322 and supply it to other devices connected via the network through the external interface 1319. The controller 1321 can also acquire encoded data and image data supplied from other devices via the network through the external interface 1319 and store them in the DRAM 1318 or supply them to the image signal processing unit 1314.

[0499] The camera 1300 described above uses the image decoding device 500 as the decoder 1315. In other words, the decoder 1315 appropriately decodes encoded data in units smaller than LCU, with the I_PCM mode selection controlled in units smaller than LCU, similar to the case of the image decoding device 500. Therefore, the decoder 1315 can reduce redundant processing during encoding and reduce redundant information contained in the encoded data. As a result, the decoder 1315 can improve encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

[0500] Therefore, the camera 1300 can improve the encoding efficiency of data while suppressing a reduction in the efficiency of the encoding process when generating, for example, image data generated in the CCD / CMOS 1312, encoded data of video data read from the DRAM 1318 or recording medium 1333, and encoded data of video data acquired via the network.

[0501] The camera 1300 also uses the image encoding device 300 as the encoder 1341. The encoder 1341 controls the selection of the I_PCM mode in units smaller than LCU, similar to the case of the image encoding device 300. In other words, the encoder 1341 can further reduce redundant processing during encoding and further reduce redundant information contained in the encoded data. As a result, the encoder 1341 can improve encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

[0502] Therefore, the camera 1300 can improve encoding efficiency while suppressing a reduction in the efficiency of the encoding process when generating encoded data to be recorded in the DRAM 1318 or the recording medium 1333, or encoded data to be provided to other devices.

[0503] The decoding method of the image decoding device 500 may be applied to the decoding process performed by the controller 1321. Similarly, the encoding method of the image encoding device 300 may be applied to the encoding process performed by the controller 1321.

[0504] The image data captured by the camera 1300 may be a moving image or a still image.

[0505] Of course, the image encoding device 300 and the image decoding device 500 can also be applied to devices and systems other than those described above.

[0506] This technology can be applied to image encoding and decoding devices used when receiving image information (bitstream) compressed by orthogonal transformations such as discrete cosine transform and motion compensation, such as MPEG and H.26x, via network media such as satellite broadcasting, cable TV, the internet, and mobile phones, or when processing it on storage media such as optical, magnetic disk, and flash memory.

[0507] This technology can also be configured as follows: (1) An encoding mode setting unit that sets whether to select an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, as the encoding mode when encoding the image data, for each encoding unit having a hierarchical structure, An encoding unit that encodes the image data for each encoding unit according to the mode set by the encoding mode setting unit. An image processing device equipped with the following features. (2) A shift processing control unit that controls the encoding unit for which the uncompressed mode has been set by the encoding mode setting unit to skip the shift processing that expands the bit precision when encoding or decoding, A shift processing unit that performs the shift processing on the image data is provided for an encoding unit that is controlled by the shift processing control unit to perform the shift processing on the image data. The image processing apparatus according to (1) further comprising the above. (3) A filter processing control unit that controls the encoding unit for which the uncompressed mode has been set by the encoding mode setting unit to skip the filter processing that performs filtering on the locally decoded image, A filter coefficient calculation unit calculates the filter coefficients for the filtering process using image data of an encoded unit controlled by the filter processing control unit to perform the filtering process, A filter processing unit performs the filtering process for each block, which is the unit of the filtering process, using the filter coefficients calculated by the filter coefficient calculation unit. The image processing apparatus according to (1) or (2) above, further comprising the above. (4) The filter processing unit performs the filter processing only on pixels included in the current block that is to be processed, which are controlled by the filter processing control unit to perform the filter processing. The image processing apparatus described in (3) above. (5) The system further comprises a filter identification information generation unit that generates filter identification information, which is flag information indicating whether the filtering process is performed, for each block. The image processing apparatus described in (3) or (4) above. (6) The filter processing unit performs an adaptive loop filter, which is an adaptive filtering process using classification processing, on the locally decoded image. The image processing apparatus described in any of (3) to (5) above. (7) The encoding mode setting unit sets the encoding mode of the current encoding unit to the uncompressed mode if the amount of encoding data obtained from the image data of the current encoding unit is less than or equal to the input data amount which is the amount of data of the image data of the current encoding unit. The image processing apparatus according to any one of (1) to (6) above. (8) The system further comprises an input data amount calculation unit that calculates the amount of input data, The encoding mode setting unit compares the input data amount calculated by the input data amount calculation unit with the code amount, with respect to the current encoding unit. The image processing apparatus described in (7) above. (9) The system further comprises an identification information generation unit that generates identification information for each encoding unit indicating whether the uncompressed mode has been set by the encoding mode setting unit. The image processing apparatus according to any one of (1) to (8) above. (10) An image processing method for an image processing device, The encoding mode setting unit sets whether to select the uncompressed mode, which is the encoding mode that outputs the image data as encoded data, as the encoding mode when encoding the image data, for each encoding unit having a hierarchical structure. The encoding unit encodes the image data for each encoding unit according to the set mode. Image processing methods. (11) An encoding mode determination unit that determines whether an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, is selected as the encoding mode when encoding the image data, for each encoding unit having a hierarchical structure, A decoding unit that decodes the encoded data for each encoding unit according to the mode determined by the encoding mode determination unit. An image processing device equipped with the following features. (12) A shift processing control unit that controls the encoding unit to skip the shift processing that expands the bit precision when encoding or decoding an encoding unit in which the uncompressed mode has been determined to be selected by the encoding mode determination unit, A shift processing unit that performs the shift processing on the image data is provided for an encoding unit that is controlled by the shift processing control unit to perform the shift processing on the image data. The image processing apparatus according to (11) further comprising the above. (13) A filter processing control unit that controls the encoding unit to skip the filtering process that performs filtering on the locally decoded image, for encoding units in which the uncompressed mode has been determined to be selected by the encoding mode determination unit, For each block which is the unit of the filtering process, a filtering processing unit is provided which performs the filtering process on the image data. Furthermore, The filter processing unit performs the filter processing only on pixels included in the current block that are to be processed and which have been controlled by the filter processing control unit to perform the filter processing. The image processing apparatus described in (11) or (12) above. (14) The filter processing unit performs an adaptive loop filter, which is an adaptive filtering process using classification processing, on the locally decoded image. The image processing apparatus described in (13) above. (15) The filter processing unit shall perform the filter processing only if the filter identification information indicating whether or not the filter processing has been performed indicates that the filter processing has been performed on the image data of the current block that is the target of processing, and the filter processing control unit is controlled to perform the filter processing on all pixels included in the current block. The image processing apparatus described in (13) or (14) above. (16) The coding mode determination unit determines whether the uncompressed mode has been selected for each coding unit based on the identification information indicating whether the uncompressed mode has been selected. The image processing apparatus according to any one of (11) to (15) above. (17) An image processing method for an image processing device, The encoding mode determination unit determines, for each encoding unit having a hierarchical structure, whether the uncompressed mode, which is the encoding mode that outputs the image data as encoded data, has been selected as the encoding mode when encoding the image data. The decoding unit decodes the encoded data for each encoding unit according to the determined mode. Image processing methods.

[0508] 300 Image encoding device, 303 Adaptive left shift unit, 307 Reversible encoding unit, 312 Loop filter, 313 Adaptive right shift unit, 315 Adaptive left shift unit, 320 Rate control unit, 321 PCM encoding unit, 331 NAL encoding unit, 332 CU encoding unit, 341 I_PCM_flag generation unit, 342 PCM determination unit, 351 Deblocking filter, 352 Pixel sorting unit, 353 Filter coefficient calculation unit, 354 Filtering unit, 361 Input data amount calculation unit, 362 PCM determination unit, 363 Encoding control unit, 364 Adaptive shift control unit, 365 Filter control unit, 500 Image decoding device, 502 Reversible decoding unit, 506 Loop filter, 507 Adaptive right shift unit, 511 Adaptive left shift unit, 516 PCM decoding unit, 531 NAL decoding unit, 532 CU decoding unit, 541 I_PCM_flag buffer, 542 PCM control unit, 551 deblocking filter, 552 pixel sorting unit, 553 filtering unit

Claims

[1] An encoding mode setting unit that sets whether to select an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, as the encoding mode when encoding the image data, for each encoding unit having a hierarchical structure, An encoding unit that encodes the image data for each encoding unit according to the mode set by the encoding mode setting unit. An image processing device equipped with the following features. [2] A shift processing control unit that controls the encoding unit for which the uncompressed mode has been set by the encoding mode setting unit to skip the shift processing that expands the bit precision when encoding or decoding, A shift processing unit that performs the shift processing on the image data is provided for an encoding unit that is controlled by the shift processing control unit to perform the shift processing on the image data. The image processing apparatus according to claim 1, further comprising: [3] A filter process control unit that controls to skip a filter process of performing filtering on a locally decoded image for an encoding unit in which the non-compression mode is set by the encoding mode setting unit, A filter coefficient calculation unit that calculates a filter coefficient of the filter process using image data of an encoding unit that is controlled to perform the filter process by the filter process control unit, A filter process unit that performs the filter process for each block that is a unit of the filter process using the filter coefficient calculated by the filter coefficient calculation unit The image processing apparatus according to claim 1, further comprising: [4] The filter processing unit performs the filter processing only on pixels included in the current block that are to be processed and which are controlled by the filter processing control unit to perform the filter processing. The image processing apparatus according to claim 3. [5] The system further comprises a filter identification information generation unit that generates filter identification information, which is flag information indicating whether the filtering process is performed, for each block. The image processing apparatus according to claim 3. [6] The filter processing unit performs an adaptive loop filter, which is an adaptive filtering process using classification processing, on the locally decoded image. The image processing apparatus according to claim 3. [7] The encoding mode setting unit sets the encoding mode of the current encoding unit to the uncompressed mode if the amount of encoding of the encoded data obtained from the image data of the current encoding unit is less than or equal to the input data amount which is the amount of data of the image data of the current encoding unit. The image processing apparatus according to claim 1. [8] The system further comprises an input data amount calculation unit that calculates the amount of input data, The encoding mode setting unit compares the input data amount calculated by the input data amount calculation unit with the code amount, with respect to the current encoding unit. The image processing apparatus according to claim 7. [9] The system further comprises an identification information generation unit that generates identification information for each encoding unit indicating whether the uncompressed mode has been set by the encoding mode setting unit. The image processing apparatus according to claim 1. [10] An image processing method for an image processing device, The encoding mode setting unit sets whether to select the uncompressed mode, which is the encoding mode that outputs the image data as encoded data, as the encoding mode when encoding the image data, for each encoding unit having a hierarchical structure. The encoding unit encodes the image data for each encoding unit according to the set mode. Image processing methods. [11] An encoding mode determination unit that determines whether an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, is selected as the encoding mode when encoding the image data, for each encoding unit having a hierarchical structure, A decoding unit that decodes the encoded data for each encoding unit according to the mode determined by the encoding mode determination unit. An image processing device equipped with the following features. [12] A shift processing control unit that controls the encoding unit to skip the shift processing that expands the bit precision when encoding or decoding an encoding unit in which the uncompressed mode has been determined to be selected, A shift processing unit that performs the shift processing on the image data is provided for an encoding unit that is controlled by the shift processing control unit to perform the shift processing on the image data. The image processing apparatus according to claim 11, further comprising: [13] A filter processing control unit that controls the encoding unit to skip the filtering process that performs filtering on the locally decoded image, for encoding units in which the uncompressed mode has been selected by the encoding mode determination unit, For each block which is the unit of the filtering process, a filtering processing unit is provided which performs the filtering process on the image data. Furthermore, The filter processing unit performs the filter processing only on pixels included in the current block that are to be processed and which have been controlled by the filter processing control unit to perform the filter processing. The image processing apparatus according to claim 11. [14] The filter processing unit performs an adaptive loop filter, which is an adaptive filtering process using classification processing, on the locally decoded image. The image processing apparatus according to claim 13. [15] The filter processing unit performs the filter processing only when the filter identification information indicating whether or not the filter processing has been performed indicates that the filter processing has been performed on the image data of the current block to be processed, and the filter processing control unit is controlled to perform the filter processing on all pixels included in the current block. The image processing apparatus according to claim 13. [16] The coding mode determination unit determines whether the uncompressed mode is selected for each coding unit based on identification information indicating whether the uncompressed mode is selected. The image processing apparatus according to claim 11. [17] An image processing method for an image processing device, The encoding mode determination unit determines, for each encoding unit having a hierarchical structure, whether the uncompressed mode, which is the encoding mode that outputs the image data as encoded data, has been selected as the encoding mode when encoding the image data. The decoding unit decodes the encoded data for each encoding unit according to the determined mode. Image processing methods. [1] An encoding mode setting unit that sets whether to select an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, as the encoding mode when encoding the image data, for each encoding unit having a hierarchical structure, An encoding unit that encodes the image data for each encoding unit according to the mode set by the encoding mode setting unit. An image processing device equipped with the following features. [2] A shift processing control unit that controls the encoding unit for which the uncompressed mode has been set by the encoding mode setting unit to skip the shift processing that expands the bit precision when encoding or decoding, A shift processing unit that performs the shift processing on the image data is provided for an encoding unit that is controlled by the shift processing control unit to perform the shift processing on the image data. The image processing apparatus according to claim 1, further comprising: [3] A filter processing control unit that controls the encoding unit for which the uncompressed mode has been set by the encoding mode setting unit to skip the filter processing that performs filtering on the locally decoded image, A filter coefficient calculation unit calculates the filter coefficients for the filtering process using image data of an encoded unit controlled by the filter processing control unit to perform the filtering process, A filter processing unit performs the filtering process for each block, which is the unit of the filtering process, using the filter coefficients calculated by the filter coefficient calculation unit. The image processing apparatus according to claim 1, further comprising: [4] The filter processing unit performs the filter processing only on pixels included in the current block that are to be processed and which are controlled by the filter processing control unit to perform the filter processing. The image processing apparatus according to claim 3. [5] [After correction] The system further comprises a filter identification information generation unit that generates filter identification information for each block indicating whether the filtering process is performed. The image processing apparatus according to claim 3. [6] The filter processing unit performs an adaptive loop filter, which is an adaptive filtering process using classification processing, on the locally decoded image. The image processing apparatus according to claim 3. [7] [After Correction] The encoding mode setting unit sets the encoding mode of the current encoding unit to the uncompressed mode if the input data amount, which is the data amount of the image data of the current encoding unit to be encoded, is less than or equal to the code amount of the encoded data after the image data of the current encoding unit has been encoded. The image processing apparatus according to claim 1. [8] The system further comprises an input data amount calculation unit that calculates the amount of input data, The encoding mode setting unit compares the input data amount calculated by the input data amount calculation unit with the code amount, with respect to the current encoding unit. The image processing apparatus according to claim 7. [9] The system further comprises an identification information generation unit that generates identification information for each encoding unit indicating whether the uncompressed mode has been set by the encoding mode setting unit. The image processing apparatus according to claim 1. [10] An image processing method for an image processing device, The encoding mode setting unit sets whether to select the uncompressed mode, which is the encoding mode that outputs the image data as encoded data, as the encoding mode when encoding the image data, for each encoding unit having a hierarchical structure. The encoding unit encodes the image data for each encoding unit according to the set mode. Image processing methods. [11] An encoding mode determination unit that determines whether an uncompressed mode, which is an encoding mode that outputs the image data as encoded data, is selected as the encoding mode when encoding the image data, for each encoding unit having a hierarchical structure, A decoding unit that decodes the encoded data for each encoding unit according to the mode determined by the encoding mode determination unit. An image processing device equipped with the following features. [12] A shift processing control unit that controls the encoding unit to skip the shift processing that expands the bit precision when encoding or decoding an encoding unit in which the uncompressed mode has been determined to be selected, A shift processing unit that performs the shift processing on the image data is provided for an encoding unit that is controlled by the shift processing control unit to perform the shift processing on the image data. The image processing apparatus according to claim 11, further comprising: [13] A filter processing control unit that controls the encoding unit to skip the filtering process that performs filtering on the locally decoded image, for encoding units in which the uncompressed mode has been selected by the encoding mode determination unit, For each block which is the unit of the filtering process, a filtering processing unit is provided which performs the filtering process on the image data. Furthermore, The filter processing unit performs the filter processing only on pixels included in the current block that are to be processed and which have been controlled by the filter processing control unit to perform the filter processing. The image processing apparatus according to claim 11. [14] The filter processing unit performs an adaptive loop filter, which is an adaptive filtering process using classification processing, on the locally decoded image. The image processing apparatus according to claim 13. [15] The filter processing unit performs the filter processing only when the filter identification information indicating whether or not the filter processing has been performed indicates that the filter processing has been performed on the image data of the current block to be processed, and the filter processing control unit is controlled to perform the filter processing on all pixels included in the current block. The image processing apparatus according to claim 13. [16] The coding mode determination unit determines whether the uncompressed mode is selected for each coding unit based on identification information indicating whether the uncompressed mode is selected. The image processing apparatus according to claim 11. [17] An image processing method for an image processing device, The encoding mode determination unit determines, for each encoding unit having a hierarchical structure, whether the uncompressed mode, which is the encoding mode that outputs the image data as encoded data, has been selected as the encoding mode when encoding the image data. The decoding unit decodes the encoded data for each encoding unit according to the determined mode. Image processing methods.

Images