Filtering methods, decryption methods, and encryption methods.

Abstract

DEPCT69 A filtering method that performs deblocking filter management within the boundaries between... An IPCM (IntraPulse Code Modulation) block and a non-IPCM block that are adjacent to each other, with... The first quantization parameter determination procedure (S401) performs the determination of the quantization parameters. The first-order TICE procedure for the non-IPCM block and the decision step determine the quantization parameters. The second step (S402) makes a decision to determine the second-order quantization parameters, supporting IPCM blocks and steps. At the decision-making stage, the filtering strength level (S403) was determined using the 1st quantization parameter as follows: The aforementioned second-order quantization parameters and the procedures for managing the removable filter are mentioned. Block (deblocking filter)(S404) at the said boundary, with the said filtering strength level. Included in the calculation;

Description

[0001] The present invention relates to a filtering method, a video decoding method, a video encoding method, a video decoding device, a video encoding device, and a video encoding and decoding device.

[0002] An IPCM (Intra Pulse Code Modulation) block is a block containing uncompressed video or image samples, where the signals included in the encoded stream are represented by samples of the luminance and chromaticity of the original image. These are used when the entropy encoding unit generates more bits than the number of bits to be reduced when encoding a block of image samples. In other words, in an IPCM block, the pixel values ​​are not compressed, and the original pixel values ​​of the image are used as they are. This IPCM block is introduced in the H.264 / AVC video compression standard.

[0003] In the H.264 video standard, when IPCM blocks are encoded into an encoded stream, the IPCM blocks are encoded as uncompressed data. Decoding of these blocks is then omitted. However, post-processing (including filtering such as deblocking and filtering) is still performed on block boundaries, which tend to degrade the image quality of these blocks (see, for example, Non-Patent Document 1).

[0004] ISO / IEC 14496-10 “MPEG-4 Part 10 Advanced Video Coding”

[0005] It is desirable to be able to perform more appropriate filtering on the boundary between such IPCM blocks and non-IPCM blocks.

[0006] Therefore, the object of the present invention is to provide a filtering method that can perform appropriate filtering on the boundary between an IPCM block and a non-IPCM block.

[0007] To achieve the above objective, a filtering method according to one embodiment of the present invention is a filtering method that performs deblocking filtering on the boundary between adjacent IPCM (Intra Pulse Code Modulation) blocks and non-IPCM blocks that are not IPCM blocks, which are included in an image, and includes: a first quantization parameter determination step of determining a first quantization parameter for the non-IPCM blocks; a second quantization parameter determination step of determining a second quantization parameter corresponding to the IPCM blocks and for determining the filter intensity using the first quantization parameter; a filter intensity determination step of determining the filter intensity using the first quantization parameter and the second quantization parameter; and a filtering step of performing deblocking filtering on the boundary with the determined filter intensity.

[0008] Based on the above, the present invention can provide a filtering method that can perform appropriate filtering on the boundary between an IPCM block and a non-IPCM block.

[0009] Figure 1 is a diagram showing the method for determining the filter strength at the block boundary between an IPCM block and a non-IPCM block in the H.264 method. Figure 2 is a flowchart of the block boundary filtering process in the H.264 method. Figure 3 is a flowchart of the filter strength determination process in the H.264 method. Figure 4 is a diagram showing the filter strength in the filtering method according to Embodiment 1 of the present invention. Figure 5 is a flowchart of the filtering method according to Embodiment 1 of the present invention. Figure 6 is a block diagram of the video encoding device according to Embodiment 1 of the present invention. Figure 7A is a diagram showing an example of a block boundary according to Embodiment 1 of the present invention. Figure 7B is a diagram showing an example of a block boundary according to Embodiment 1 of the present invention. Figure 8A is a diagram showing the operation of the filter processing unit according to Embodiment 1 of the present invention. Figure 8B is a diagram showing the operation of the filter processing unit according to Embodiment 1 of the present invention. Figure 9 is a block diagram of the video decoding device according to Embodiment 1 of the present invention. Figure 10A is a diagram showing an example of the configuration of the filter processing unit according to Embodiment 1 of the present invention. Figure 10B is a diagram showing an example of the configuration of the filter processing unit according to Embodiment 1 of the present invention. Figure 10C is a diagram showing an example of the configuration of the filter processing unit according to Embodiment 1 of the present invention. Figure 10D is a diagram showing an example configuration of a filter processing unit according to Embodiment 1 of the present invention. Figure 10E is a diagram showing an example configuration of a filter processing unit according to Embodiment 1 of the present invention. Figure 10F is a diagram showing an example configuration of a filter processing unit according to Embodiment 1 of the present invention. Figure 10G is a diagram showing an example configuration of a filter processing unit according to Embodiment 1 of the present invention. Figure 10H is a diagram showing an example configuration of a filter processing unit according to Embodiment 1 of the present invention. Figure 11 is a flowchart of a modified example of the filtering method according to Embodiment 1 of the present invention. Figure 12 is a flowchart of the filter intensity determination process according to Embodiment 1 of the present invention. Figure 13 is a diagram showing the filter intensity and block units according to Embodiment 1 of the present invention. Figure 14A is a diagram showing the scope of application of the filter ON flag according to a comparative example of the present invention. Figure 14B is a diagram showing the scope of application of the filter ON flag according to Embodiment 1 of the present invention. Figure 15 is a flowchart of a video encoding method according to a modified example of Embodiment 1 of the present invention.Figure 16 is a flowchart of a video decoding method according to a modified embodiment of Embodiment 1 of the present invention. Figure 17 is a block diagram of a video encoding device according to Embodiment 2 of the present invention. Figure 18 is a block diagram of a video decoding device according to Embodiment 2 of the present invention. Figure 19 is a flowchart of a filtering method according to Embodiment 2 of the present invention. Figure 20 is a flowchart of a specific example of a filtering method according to Embodiment 2 of the present invention. Figure 21 is a flowchart of a video encoding method according to a modified embodiment of Embodiment 2 of the present invention. Figure 22 is a flowchart of a video decoding method according to a modified embodiment of Embodiment 2 of the present invention. Figure 23 is an overall configuration diagram of a content supply system that realizes a content distribution service. Figure 24 is an overall configuration diagram of a digital broadcasting system. Figure 25 is a block diagram showing an example of a television configuration. Figure 26 is a block diagram showing an example of an information playback / recording unit that reads and writes information to a recording medium which is an optical disc. Figure 27 is a diagram showing an example of the structure of a recording medium which is an optical disc. Figure 28A is a diagram showing an example of a mobile phone. Figure 28B is a block diagram showing an example of a mobile phone configuration. Figure 29 is a diagram showing the configuration of multiplexed data. Figure 30 is a schematic diagram showing how each stream is multiplexed in the multiplexed data. Figure 31 is a diagram showing in more detail how the video stream is stored in the PES packet sequence. Figure 32 is a diagram showing the structure of the TS packet and source packet in the multiplexed data. Figure 33 is a diagram showing the data structure of the PMT. Figure 34 is a diagram showing the internal structure of the multiplexed data information. Figure 35 is a diagram showing the internal structure of the stream attribute information. Figure 36 is a diagram showing the steps for identifying video data. Figure 37 is a block diagram showing an example of the configuration of an integrated circuit that realizes the video encoding method and video decoding method of each embodiment. Figure 38 is a diagram showing a configuration for switching the drive frequency. Figure 39 is a diagram showing the steps for identifying video data and switching the drive frequency. Figure 40 is a diagram showing an example of a lookup table that associates the video data standard with the drive frequency. Figure 41A is a diagram showing an example of a configuration in which the signal processing module is shared. Figure 41B is a diagram showing another example of a configuration in which the signal processing module is shared.

[0010] (Knowledge underlying the present invention) The present inventors have found that "the following problems occur.

[0011] First, before describing the embodiments of the present invention, pixel filtering (deblocking filter) processing at the boundary between an IPCM block and a non - IPCM block in the H.264 encoding and decoding methods will be described.

[0012] FIG. 1 is a diagram for explaining the concept of a method for determining the filter strength of a pixel - level filter at the boundary between an IPCM block (macroblock) and a non - IPCM block (macroblock) in the H.264 encoding and decoding methods.

[0013] FIG. 1 schematically shows the boundary between two macroblocks, one being a non - IPCM macroblock (left side of the figure) and the other being an IPCM macroblock (right side of the figure). The three circles located on the left side of FIG. 1 indicate three pixels (typically referred to as p0, p1, p2 in order from the boundary). These three pixels on the left side belong to the first block (p - block) in the first unit (encoding unit block, hereinafter referred to as a CU block). At the same time, these three pixels belong to the first non - IPCM - type macroblock in the block of the macroblock unit, which is a unit larger than the first unit (hereinafter referred to as an MB block).

[0014] Similarly, the three circles located on the right side of FIG. 1 indicate three pixels (typically referred to as q0, q1, q2 in order from the boundary). These three pixels on the left side belong to the second block (q - block) in the first unit. At the same time, these three pixels belong to the second IPCM - type macroblock in the MB block.

[0015] Hereinafter, a CU block belonging to an IPCM - type macroblock will be referred to as an IPCM block, and a CU block belonging to a non - IPCM - type macroblock will be referred to as a non - IPCM block. That is, a non - IPCM block means a block that is not an IPCM block.

[0016] Hereinafter, a method for determining the filter strength applied to the pixels q0, q1, p0, and p1 at this block boundary (or the boundary of a block unit larger than the encoding unit) will be described.

[0017] In the filtering method of H. 264 (the filtering method described in Section 8.7 of the standard), the filtering strength for the boundary between two blocks is usually determined by the average value of qPp, which is derived from the quantization parameter QPp of the first macroblock, and QPq, which is derived from the quantization parameter QPq of the second macroblock. Specifically, the following (Equation 1), shown in Equation 8-461 of the standard, is used.

[0018] QPav = (QPp+QPq+1) >>1 => (QPp+1) >>1 (Formula 1)

[0019] This (Equation 1) shows the following calculation. The filter strength is designed so that the larger the value of the quantization parameter, the stronger (more smooth) the filter applied, for the purpose of absorbing quantization errors, etc.

[0020] In the figure, the quantization parameter QPp on the left is the quantization parameter encoded for the first macroblock (the p-side block). For convenience, here QP is used in the same sense as the value qP used for the purpose of filtering. The quantization parameter QPq on the right is the quantization parameter that should be applied to the second macroblock (the q-side block).

[0021] Here, as described in Section 8.7.2 of the H. 264 standard, the value of the quantization parameter qPq (QPq in the figure) of the IPCM block is set to 0. That is, "Both sides filtered with weak strength". This means that at the boundary between the two blocks, a filter of the same filter strength is applied to both blocks. This also means that the filter strengths of the two blocks cannot be different. In other words, at the boundary between an IPCM block and a non-IPCM block, filtering of the same filter strength is performed on both blocks.

[0022] Figure 2 is a flowchart illustrating the concept of block boundary filtering as described in Section 8.7 “Deblocking filter process” of the H.264 standard.

[0023] This flowchart explains three main things about the H.264 filter.

[0024] (1) Order of determining the filter strength (bS) in Section 8.7.2.1 Step S101 corresponds to the “Deviation process for the lumina content dependent boundary filtering strength” process described in Section 8.7.2.1. This process determines the filter strength for filtering a single block boundary according to the block type, etc. Here, the filter strength is classified from strong filter strength (bS=4) to no filter (bS=0). This point is explained in Figure 3.

[0025] (2) Process for setting the quantization parameter qPz = 0 for the IPCM block Steps S102 to S107 are the process of setting the value of the quantization parameter qP for determining the filter strength, as explained in Figure 1. For a normal non-IPCM block (No in step S102 or S105), the quantization parameter QP[i] (where i is either 0 or 1) of the macroblock to which the block belongs is set as the quantization parameter qP[i] for determining the filter strength (steps S103 and S106). On the other hand, if the target block is an IPCM block (Yes in S102 or S105), the quantization parameter qP of the IPCM block is set to 0 (steps S104 and S107).

[0026] Next, in step S108, qPav is calculated using the above (formula 1).

[0027] (3) One bS (or filterSampleFlag) is shared by both blocks.

[0028] The following explains how the determined filter strength (or filtering or not filtering flag) is applied in common (the same value) to the two blocks that straddle the boundary.

[0029] First, after step S108, calculations are performed using equations 8-462 to 8-467 in the above standard. Specifically, (1) an index for fine-tuning the filter strength set in step S101 is derived, and (2) a threshold for edge determination is derived.

[0030] Then, the filter strength determined by these processes is set for both blocks (S109). Specifically, regardless of whether the filter strength bS is 1 to 4, the value derived using the same method for deriving bS is applied to both blocks. For example, if the filter strength bS = 4, the value of pixel p in the first block is derived using the formulas (8-486 to 487) in the above standard. Also, the value of pixel q in the second block is derived using the same filter strength used to derive the value of pixel p. Furthermore, in order to handle cases where the block boundary is actually an edge, a determination is made as to whether or not to apply a filter (derivation of the value of filterSamplesFlag (also called the filter execution flag)). Specifically, this determination is made by comparing the two thresholds (two_threats(α,β)) derived in step S109 with the actual pixel values ​​of p and q (see the above standard, equation (8-468)). However, as mentioned above, the values ​​(or whether or not the filter is executed) of both the filter intensity bS and the filter execution flag cannot be changed between the two blocks.

[0031] In other words, H.264 cannot achieve processing suitable for IPCM when viewed solely as a filtering process.

[0032] Figure 3 is a flowchart showing the order of determination (decision order) for the filter intensity (bS) applied to pixels located at the boundary between two macroblocks, as described in Section 8.7.2.1 of the above standard. This flowchart explains the decision order of step S101 shown in Figure 2 and follows the decision flow in Section 8.7.2.1 of the standard.

[0033] First, it is determined whether the boundary formed by the pixel p0 of the first block and the pixel q0 of the second block also corresponds to the boundary of a macroblock (S121). In other words, it is determined whether p0 and q0 are located at the boundary of a macroblock.

[0034] If the block boundary to be processed is not a macroblock boundary (No in S121), the filter strength (bS) is determined to be one of 3, 2, 1, or 0, which is a value less than N (=4) (S124).

[0035] On the other hand, if the block boundary to be processed is a macroblock boundary (Yes in S121), it is determined whether or not one (or both) of p0 and q0 belongs to a macroblock in intra prediction mode (S122).

[0036] If neither block belongs to a macroblock in intra prediction mode (No in S122), the other determination conditions are checked (S125).

[0037] On the other hand, if at least one block belongs to a macroblock of intra prediction mode (Yes in S122), the filter strength is set to bS = 4, which means the strongest strength, without any other determination conditions (always) (S123).

[0038] Thus, in conventional filtering methods, it is not possible to perform different processing (filter strength and whether or not to apply a filter) on two blocks separated by a single boundary during the internal processing of the filtering process. Furthermore, although the standard takes into account the determination of the filter strength focusing on IPCM, it is not possible to control the output of the pixel values ​​of the IPCM block as they are when one block is an IPCM block and the other block is a non-IPCM block.

[0039] An IPCM block is a block containing pixel values ​​that faithfully represent the "original image" without encoding loss. Therefore, it is desirable to be able to control the filtering process at the boundary with the IPCM block on one side, or the application of the filter to the IPCM block, within the filtering process.

[0040] As described above, the filter strength at the boundary between two blocks is usually determined based on the average value qPav of the value qPp derived from the quantization parameter QPp of the first macroblock and the quantization parameter QPq of the second macroblock. Furthermore, the value of the quantization parameter qPq of the IPCM block is set to 0. As a result, the average value qPav used to determine the filter strength at the boundary between the IPCM block and the non-IPCM block is half the value of the quantization parameter QPq of the non-IPCM block. In other words, at the boundary between the IPCM block and the non-IPCM block, the average value qPav becomes smaller than in the normal case (boundary between non-IPCM blocks). Thus, the inventors have found that the filter strength is not set appropriately at the boundary between the IPCM block and the non-IPCM block.

[0041] In contrast, a filtering method according to one embodiment of the present invention is a filtering method that performs deblocking filtering on the boundary between adjacent IPCM (Intra Pulse Code Modulation) blocks and non-IPCM blocks that are not IPCM blocks, which are included in an image, and includes: a first quantization parameter determination step of determining a first quantization parameter for the non-IPCM blocks; a second quantization parameter determination step of determining a second quantization parameter corresponding to the IPCM blocks and for determining the filter intensity using the first quantization parameter; a filter intensity determination step of determining the filter intensity using the first quantization parameter and the second quantization parameter; and a filtering step of performing deblocking filtering on the boundary with the determined filter intensity.

[0042] According to this, a filtering method according to one embodiment of the present invention determines the quantization parameter of an IPCM block using the quantization parameter of a non-IPCM block. As a result, the filtering method can perform more appropriate filtering on the boundary between an IPCM block and a non-IPCM block compared to the case where zero is used as the quantization parameter of the IPCM block.

[0043] In addition, in the second quantization parameter determination step, the value of the second quantization parameter may be determined to be the same value as the first quantization parameter.

[0044] In addition, in the filter strength determination step, the average value of the first quantization parameter and the second quantization parameter may be calculated, and the filter strength may be determined using the calculated average value.

[0045] Furthermore, a video decoding method according to one embodiment of the present invention is a video decoding method for decoding an encoded bitstream, comprising: a difference information acquisition step of obtaining difference information indicating that the difference between the quantization parameter of the immediately preceding block in the processing order and the quantization parameter of the block to be processed is zero by analyzing the encoded bitstream; and a filtering step of executing the filtering method, wherein in the second quantization parameter determination step, the value of the second quantization parameter is determined to be the same as the value of the first quantization parameter according to the difference information.

[0046] According to this, a video decoding method according to one embodiment of the present invention can determine the quantization parameters of an IPCM block according to differential information used for other purposes. Therefore, this video decoding method can appropriately determine the quantization parameters of an IPCM block without adding a function to the video decoding device that performs special processing on the IPCM block.

[0047] In addition, in the second quantization parameter determination step, if the non-IPCM block is located immediately before the IPCM block in the processing order, the value of the second quantization parameter may be determined to be the same as the value of the first quantization parameter according to the difference information.

[0048] The video decoding method further includes a decoding step of generating quantization coefficients by decoding the encoded bitstream, an inverse quantization and inverse transformation step of generating a decoded residual signal by inverse quantization and inverse transformation of the quantization coefficients, and an addition step of generating a decoded image signal by adding a predicted image signal to the decoded residual signal, wherein the IPCM block and the non-IPCM block are included in the decoded image signal, and the video decoding method further includes a prediction step of generating the predicted image signal by performing prediction processing using the image signal after the deblocking filter processing in the filter step.

[0049] The video decoding method may also switch between decoding in accordance with the first standard and decoding in accordance with the second standard depending on an identifier indicating the first standard or the second standard included in the encoded bitstream, and when the ID indicates the first standard, the differential information acquisition step and the filtering step may be performed as decoding in accordance with the first standard.

[0050] Furthermore, a video encoding method according to one embodiment of the present invention is a video encoding method that generates an encoded bitstream by encoding an input image signal, comprising: a filtering step of executing the filtering method; and a bitstream generation step of generating the encoded bitstream which includes, as information indicating that the second quantization parameter has the same value as the first quantization parameter, difference information indicating that the difference between the quantization parameter of the block immediately preceding the processing block and the quantization parameter of the block to be processed is zero.

[0051] According to this, a video encoding method according to one embodiment of the present invention can transmit information to a video decoding device for determining the quantization parameters of an IPCM block using differential information used for other purposes. Therefore, the video decoding device can appropriately determine the quantization parameters of an IPCM block without adding a function to the video decoding device to perform special processing on the IPCM block.

[0052] In addition, the bitstream generation step may generate the difference information if the non-IPCM block is located immediately before the IPCM block in the processing order.

[0053] The video encoding method further includes a subtraction step of generating a residual signal by subtracting a predicted image signal from the input image signal; a conversion and quantization step of generating quantization coefficients by converting and quantizing the residual signal; an encoding step of generating the encoded bitstream by encoding the quantization coefficients; an inverse quantization and inverse conversion step of generating a decoded residual signal by inverse quantization and inverse conversion of the quantization coefficients; and an addition step of generating a decoded image signal by adding the predicted image signal to the decoded residual signal, wherein the IPCM block and the non-IPCM block are included in the decoded image signal, and the video encoding method further includes a prediction step of generating the predicted image signal by performing prediction processing using the image signal after the deblocking filter processing in the filter step.

[0054] Furthermore, a motion image decoding device according to one embodiment of the present invention is a motion image decoding device that performs deblocking filtering on the boundary between adjacent IPCM (Intra Pulse Code Modulation) blocks and non-IPCM blocks that are not IPCM blocks, comprising: a first quantization parameter determination unit that determines a first quantization parameter for the non-IPCM blocks; a second quantization parameter determination unit that uses the first quantization parameter to determine a second quantization parameter corresponding to the IPCM blocks and for determining the filter intensity; a filter intensity determination unit that uses the first quantization parameter and the second quantization parameter to determine the filter intensity; and a filter unit that performs deblocking filtering on the boundary with the determined filter intensity.

[0055] According to this configuration, the motion image decoding device according to one embodiment of the present invention determines the quantization parameters of the IPCM block using the quantization parameters of the non-IPCM block. As a result, the motion image decoding device can perform appropriate filtering on the boundary between the IPCM block and the non-IPCM block compared to the case where zero is used as the quantization parameter of the IPCM block.

[0056] Furthermore, a motion image encoding device according to one embodiment of the present invention is a motion image encoding device that performs deblocking filtering on the boundary between adjacent IPCM (Intra Pulse Code Modulation) blocks and non-IPCM blocks that are not IPCM blocks, comprising: a first quantization parameter determination unit that determines a first quantization parameter for the non-IPCM blocks; a second quantization parameter determination unit that uses the first quantization parameter to determine a second quantization parameter corresponding to the IPCM blocks and for determining the filter intensity; a filter intensity determination unit that uses the first quantization parameter and the second quantization parameter to determine the filter intensity; and a filter unit that performs deblocking filtering on the boundary with the determined filter intensity.

[0057] According to this configuration, the motion image encoding device according to one embodiment of the present invention determines the quantization parameters of the IPCM block using the quantization parameters of the non-IPCM block. As a result, the motion image encoding device can perform more appropriate filtering on the boundary between the IPCM block and the non-IPCM block compared to the case where zero is used as the quantization parameter of the IPCM block.

[0058] Furthermore, a motion image encoding and decoding device according to one embodiment of the present invention comprises the motion image encoding device and the motion image decoding device.

[0059] These general or specific embodiments may be implemented as a system, method, integrated circuit, computer program or recording medium, or as any combination of a system, method, integrated circuit, computer program and recording medium.

[0060] Hereinafter, a video decoding device and a video encoding device according to one aspect of the present invention will be described in detail with reference to the drawings.

[0061] The embodiments described below are all specific examples of the present invention. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, and the order of steps shown in the following embodiments are examples and are not intended to limit the present invention. In addition, among the components in the following embodiments, components that are not described in the independent claim representing the highest-level concept will be described as optional components.

[0062] (Embodiment 1) The following describes a filtering method according to Embodiment 1 of the present invention.

[0063] Figure 4 is a diagram showing the conditions under which the filtering method according to this embodiment is applied and the concept of how to determine the filter strength of the inter-pixel filter. The three circles on the left in the figure indicate pixels included in the first block, as in Figure 1. Note that explanations of other parts are omitted as they are the same as in Figure 1.

[0064] The filtering method according to this embodiment performs filtering on multiple blocks contained in an image. Typically, this filtering method is applied to deblocking filtering performed on the boundaries of adjacent blocks. In the following, an example of applying the present invention to deblocking filtering is described, but the present invention can also be applied to other intra-loop filtering (adaptive loop filter) methods.

[0065] The filtering method according to this embodiment differs from the filtering method described in Figure 1 in the following respects.

[0066] First, the unfiltered pixel values ​​are output as the pixel values ​​of the three pixels on the IPCM block side on the right side of the figure.

[0067] Furthermore, the first block and the second block are controlled by different handling of the filter. For example, a filter is applied to one block (left side) of a boundary in the figure, while no filter is applied to the other block (right side). In this way, control is performed by creating differences in filtering between blocks.

[0068] Next, the filter intensity of the left-hand block to which the filter is applied is derived using only the quantization parameter QPp of the left-hand block. That is, the filter intensity of the left-hand non-IPCM block is derived without using the quantization parameter QPq of the right-hand macroblock, or any other alternative fixed value (0 in the conventional example).

[0069] In Figure 2, the determination of IPCM in H.264 was whether or not it was an IPCM macroblock, but this determination is performed in a variable-size prediction unit (PU unit). In other words, below, an IPCM block is a block belonging to an IPCM type PU block, and a non-IPCM block is a block belonging to a PU block that is not of the IPCM type.

[0070] The following will explain these operations with reference to diagrams.

[0071] Figure 5 is a flowchart showing the processing order of the filtering method according to this embodiment.

[0072] The filtering method according to this embodiment is performed as part of the encoding process or the decoding process. Therefore, this filtering method is performed by a filtering processing unit in the encoding loop or decoding loop within the video encoding device shown in Figure 6 or the video decoding device shown in Figure 9, which will be described later, and a control unit that controls this filter.

[0073] The control unit first determines whether the type of PU block in either of the two blocks constituting the boundary is IPCM (S201). For example, in the example shown in Figure 4, the PU block on the right is an IPCM block, so it is determined that one of them is of IPCM type. Specifically, the control unit performs this determination using attribute parameters of the image data, such as the macroblock type or motion-guaranteed block size.

[0074] If at least one of the two blocks is an IPCM block (Yes in S201), the control unit determines whether the other of the two blocks is an IPCM block (S202). For example, in the diagram shown in Figure 4, the block on the right is an IPCM block. Therefore, the control unit determines whether the other block, the block on the left, is an IPCM block.

[0075] In other words, in steps S201 and S202, the control unit determines whether each of the plurality of blocks is an IPCM block or a non-IPCM block. Specifically, the control unit determines whether (1) both blocks are non-IPCM blocks (No in S201), (2) both blocks are IPCM blocks (Yes in S202), or (3) one is an IPCM block and the other is a non-IPCM block (No in S202).

[0076] If the other block is an IPCM block (Yes in S202), that is, if both blocks are IPCM blocks, then no filtering is performed on pixels p and q of both blocks (both the first block and the second block) (S203).

[0077] On the other hand, if the other block is not an IPCM block (No in S202), that is, if only one block is an IPCM block and the other block is not an IPCM block, the control unit controls the filter processing unit to perform the filtering process in steps S204 and S205.

[0078] First, the filter processing unit performs a filter with a predetermined intensity on the pixels included in the non-IPCM block (for example, the three pixels on the left side of Figure 4), and outputs the filtered pixel values ​​as the pixel values ​​of the non-IPCM block (S204). In addition, this filter processing uses not only the pixel values ​​of the non-IPCM block but also the pixel values ​​of the IPCM block. Specifically, the filter processing unit calculates the filtered pixel values ​​of the non-IPCM block by smoothing the pixel values ​​of the non-IPCM block and the pixel values ​​of the IPCM block.

[0079] The filter processing unit also outputs unfiltered pixel values ​​for the pixels included in the IPCM block (pixels q0, q1, etc. on the q side) (S205). Here, outputting unfiltered pixel values ​​can be assumed to mean the following two cases.

[0080] The first method is to perform filtering on non-IPCM blocks and output the original pixel values ​​without performing filtering on IPCM blocks.

[0081] The second method involves performing a filter on both non-IPCM blocks and IPCM blocks, replacing the IPCM block pixel values ​​with their original values ​​before filtering, and outputting the resulting pixel values. In either case, the output IPCM block pixel values ​​are the original pixel values ​​before filtering.

[0082] The above filtering method can also be understood as controlling the filtering method (filter strength, presence or absence of filtering, or number of pixels to which filtering is applied) between one block and the other block.

[0083] The filtering process in steps S204 and S205 (in particular, the operation of the control unit and the filter processing unit) will be explained later with reference to Figures 6 to 8B.

[0084] Also, if both blocks are non-IPCM blocks in step S201 (No in S201), the control unit performs normal filtering (S206). That is, the control unit performs filtering on both blocks with a predetermined filter strength.

[0085] The following describes a video encoding device using the above filtering method.

[0086] Figure 6 is a functional block diagram of the moving image encoding apparatus 100 according to the present embodiment. The moving image encoding apparatus 100 shown in FIG. 6 generates an encoded bit stream 132 by encoding an input image signal 120. This moving image encoding apparatus 100 includes a subtractor 101, an orthogonal transformation unit 102, a quantization unit 103, an inverse quantization unit 104, an inverse orthogonal transformation unit 105, an adder 106, a filter processing unit 115, a memory 109, a prediction unit 110, a variable length encoding unit 111, a selection unit 112, and a control unit 113.

[0087] The subtractor 101 generates a residual signal 121 by calculating the difference between the input image signal 120 and the predicted image signal 130. The orthogonal transformation unit 102 generates transformation coefficients 122 by performing an orthogonal transformation on the residual signal 121. The quantization unit 103 generates quantized coefficients 123 by quantizing the transformation coefficients 122.

[0088] The inverse quantization unit 104 generates transformation coefficients 124 by inverse quantizing the quantized coefficients 123. The inverse orthogonal transformation unit 105 generates a decoded residual signal 125 by performing an inverse orthogonal transformation on the transformation coefficients 124. The adder 106 generates a decoded image signal 126 by adding the decoded residual signal 125 and the predicted image signal 130.

[0089] The filter processing unit 115 generates an image signal 128 by performing a filter process on the decoded image signal 126, and stores the generated image signal 128 in the memory 109.

[0090] The prediction unit 110 generates a predicted image signal 130 by selectively performing intra prediction processing and inter prediction processing using the image signal 128 stored in the memory 109.

[0091] The variable length encoding unit 111 generates an encoded signal 131 by performing variable length encoding (entropy encoding) on the quantized coefficients 123.

[0092] When the target block is an IPCM block, the selection unit 112 selects the input image signal 120, and when the target block is a non - IPCM block, the selection unit 112 selects the encoded signal 131, and outputs the selected signal as the encoded bit stream 132.

[0093] The control unit 113 controls the filter processing unit 115 and the selection unit 112.

[0094] The orthogonal transformation unit 102 and the quantization unit 103 are examples of transformation and quantization units that generate quantization coefficients by applying transformation and quantization processing to the residual signal. The variable length coding unit 111 is an example of an coding unit that generates an coded signal by coding the quantization coefficients. The inverse quantization unit 104 and the inverse orthogonal transformation unit 105 are examples of inverse quantization and inverse transformation units that generate a decoded residual signal by applying inverse quantization processing and inverse transformation processing to the quantization coefficients.

[0095] In this embodiment of the video encoding device 100, the main components are the control unit 113 and the filter processing unit 115.

[0096] As described above, the filtering method according to this embodiment is performed as part of the encoding and decoding process. Therefore, the filtering unit 115 is located before the memory 109 which holds the reference image, etc. The filtering unit 115 stores the result of performing filtering (or the result of not performing filtering) in the memory 109 within the loop. In this respect, the filtering unit 115 is the same as the filter in H.264 called a Loop filter.

[0097] The filter processing unit 115 also has two input systems. The first input signal is a decoded image signal 126 indicating the pixel values ​​of non-IPCM blocks, and the second input signal is an input image signal 120 indicating the pixel values ​​of IPCM blocks. Here, the decoded image signal 126 is a restored encoded image signal after transformation, quantization, inverse quantization, and inverse transformation processing. The input image signal 120 is the original image signal that has not gone through encoding and decoding processing.

[0098] In accordance with the control of the control unit 113, the filter processing unit 115 outputs the original pixel value without filtering for pixels in the IPCM block, and filters the pixels in the non-IPCM block and outputs the value after filtering.

[0099] The filter processing unit 115 comprises a filter unit 107 and a selection unit 108. The filter unit 107 generates an image signal 127 by applying a filter to the decoded image signal 126. The selection unit 108 selects the image signal 127 if the target block is an IPCM block, and selects the input image signal 120 if the target block is a non-IPCM block, and outputs the selected signal as the image signal 128.

[0100] Figures 7A and 7B illustrate the pixels at the boundary between two blocks. In the example shown in Figure 7A, the two blocks are adjacent horizontally. Here, the block containing the pixels p0 to pn on the left is called the first block. This first block is a non-IPCM block. The other block is called the second block. This second block is an IPCM block. It goes without saying that the filtering process of this embodiment can also be applied when the IPCM block and the non-IPCM block are adjacent vertically, as shown in Figure 7B.

[0101] The following describes a specific example of the operation of the filter processing unit 115.

[0102] Figures 8A and 8B show the operation of the filter processing unit 115 when filtering is applied to pixels p[i] and q[j] included in the two blocks illustrated in Figure 7A. That is, the first block belongs to the non-IPCM block, and the second block belongs to the IPCM block.

[0103] The filter processing unit 115 performs the operations shown in Figures 8A and 8B in response to the control signal from the control unit 113.

[0104] Figure 8A shows the operation of the filter processing unit 115 for non-IPCM blocks. This operation corresponds to step S204 shown in Figure 5. That is, the filter processing unit 115 uses the pixel values ​​of both the first block (p0, p1...) and the second block (q0, q1...) to calculate the output results pf0, pf1... for the pixels corresponding to the first block.

[0105] Figure 8B shows the operation of the filter processing unit 115 for the IPCM block. This operation corresponds to step S205 shown in Figure 5. That is, for the pixels of the second block, the filter processing unit 115 outputs the same value as the input q0, q1, and q2 (unfiltered pixel value).

[0106] The following describes a video decoding device using the above filtering method.

[0107] Figure 9 is a functional block diagram of the video decoding device according to this embodiment.

[0108] The video decoding device 200 shown in Figure 9 generates an output image signal 220 by decoding the encoded bitstream 232. Here, the encoded bitstream 232 is, for example, the encoded bitstream 132 generated by the video encoding device 100.

[0109] The video decoding device 200 comprises an inverse quantization unit 204, an inverse orthogonal transformation unit 205, an adder 206, a filter processing unit 215, a memory 209, a prediction unit 210, a variable length decoding unit 211, a distribution unit 212, and a control unit 213.

[0110] The distribution unit 212 supplies the encoded bitstream 232 to the filter processing unit 215 if the target block is an IPCM block, and supplies the encoded bitstream 232 to the variable-length decoding unit 211 if the target block is a non-IPCM block.

[0111] The variable-length decoding unit 211 generates quantization coefficients 223 by performing variable-length decoding (entropy decoding) on ​​the encoded bitstream 232.

[0112] The inverse quantization unit 204 generates conversion coefficients 224 by inverse quantization of the quantization coefficients 223. The inverse orthogonal transformation unit 205 generates a decoded residual signal 225 by inverse orthogonal transformation of the conversion coefficients 224. The adder 206 generates a decoded image signal 226 by adding the decoded residual signal 225 and the predicted image signal 230.

[0113] The filter processing unit 215 generates an image signal 228 by applying a filter to the decoded image signal 226, and stores the generated image signal 228 in the memory 209.

[0114] This filter processing unit 215 includes a filter unit 207 and a selection unit 208. The filter unit 207 generates an image signal 227 by applying a filter to the decoded image signal 226. The selection unit 208 selects the image signal 227 if the target block is an IPCM block, and selects the encoded bitstream 232 if the target block is a non-IPCM block, and outputs the selected signal as the image signal 228.

[0115] In addition, the image signal 228 stored in the memory 209 is output as the output image signal 220.

[0116] The prediction unit 210 generates a predicted image signal 230 by selectively performing intra-prediction processing and inter-prediction processing using the image signal 228 stored in the memory 209.

[0117] The control unit 213 controls the filter processing unit 215 and the distribution unit 212.

[0118] The variable-length decoding unit 211 is an example of a decoding unit that generates quantization coefficients by decoding an encoded bitstream. The inverse quantization unit 204 and the inverse orthogonal transform unit 205 are examples of inverse quantization and inverse transform units that generate a decoded residual signal by applying inverse quantization processing and inverse transform processing to the quantization coefficients.

[0119] Here, the operation of the filter processing unit 215 is the same as the operation of the filter processing unit 115 of the video encoding device 100. The control unit 213 differs from the control unit 113 of the video encoding device 100 in that it determines whether the type of the PU unit of the first block or the second block is IPCM from the encoded bitstream 232, which is the input code sequence, but other functions are the same.

[0120] The following describes the configuration of modified versions of the filter processing units 115 and 215 described above.

[0121] Figures 10A to 10H show possible configurations for the input-output relationship of the filters of the filter processing units 115 and 215 according to this embodiment.

[0122] As shown in Figure 10A, the filter sections 107 and 207 may include a plurality of filter sections 301 and 302 connected in series. For example, the first filter section 301 and the second filter section 302 may perform different processing. In this case, for example, all filtering is bypassed for the IPCM block.

[0123] As shown in Figure 10B, the filter unit 311 may perform filtering using both input signals. In this case, the selection unit 312 outputs the unfiltered value for IPCM blocks and the filtered value for non-IPCM blocks.

[0124] As shown in Figure 10C, different filtering processes may be applied to the IPCM block and the non-IPCM block. For example, different filtering processes may be filtering processes with different filter strengths. Also, for example, the filter strength for the IPCM block may be weaker than the filter strength for the non-IPCM block.

[0125] Specifically, the distribution unit 321 outputs the input signal to the filter unit 322 if the target block is a non-IPCM block, and outputs the input signal to the filter unit 323 if the target block is an IPCM block. Here, the input signal includes both the decoded image signal 126 and the input image signal 120 described above. The filter unit 322 generates the pixel value of the target block by performing a first filter intensity filter processing using the input signal. The filter unit 323 generates the pixel value of the target block by performing a second filter intensity filter processing which is weaker than the first filter intensity. The selection unit 324 outputs the pixel value of the target block after it has been filtered by the filter unit 322 if the target block is a non-IPCM block, and outputs the pixel value of the target block after it has been filtered by the filter unit 323 if the target block is an IPCM block.

[0126] As shown in Figure 10D, processing of IPCM blocks does not need to be performed at all. Specifically, the distribution unit 331 outputs the input signal to the filter unit 332 if the target block is a non-IPCM block, and outputs the input signal to the selection unit 333 if the target block is an IPCM block. The selection unit 333 outputs the pixel value of the target block after it has been filtered by the filter unit 332 if the target block is a non-IPCM block, and outputs the pixel value of the target block from the signal from the distribution unit 331 if the target block is an IPCM block.

[0127] As shown in Figure 10E, instead of switching the output side of the filter unit, the input side may be switched. Furthermore, the number of stages of the filter unit may differ for IPCM blocks and non-IPCM blocks. Specifically, the distribution unit 341 outputs the input signal to the filter unit 342 when the target block is a non-IPCM block, and outputs the input signal to the filter unit 344 when the target block is an IPCM block. The filter unit 342 performs filtering using the input signal. The filter unit 343 performs filtering using the signal after it has been filtered by the filter unit 342, and outputs the pixel value of the target block after filtering. The filter unit 344 performs filtering using the input signal, and outputs the pixel value of the target block after filtering. The filtering performed by the filter unit 344 may be the same as or different from the filtering performed by the filter unit 342 or the filtering performed by the filter unit 343.

[0128] As shown in Figure 10F, the output side of the filter unit may be switched. Specifically, filter unit 351 performs filtering using the first input signal. Filter unit 352 performs filtering using the signal after it has been filtered by filter unit 351 and outputs the pixel value of the target block after filtering. Filter unit 353 performs filtering using the second input signal and outputs the pixel value of the target block after filtering. If the target block is a non-IPCM block, the selection unit 354 outputs the pixel value of the target block after it has been filtered by filter unit 352, and if the target block is an IPCM block, it outputs the pixel value of the target block after it has been filtered by filter unit 353.

[0129] Outputting the unfiltered value includes replacing the filtered pixel value pf with the original input value p and outputting it.

[0130] As shown in Figure 10G, the filtering of the other channel may be performed using the signal after filtering of one of the two channels. Specifically, the filter unit 361 performs filtering using the second input signal. The filter unit 362 performs filtering using the first input signal and the signal after filtering by the filter unit 361. The selection unit 363 outputs the pixel value of the target block after filtering by the filter unit 362 if the target block is a non-IPCM block, and outputs the pixel value of the target block after filtering by the filter unit 361 if the target block is an IPCM block. Alternatively, the selection unit 363 may output the pixel value of the target block after filtering by the filter unit 362 if the target block is an IPCM block, and output the pixel value of the target block after filtering by the filter unit 361 if the target block is a non-IPCM block.

[0131] As shown in Figure 10H, a value once stored in memory 373 may be used as input. Specifically, the selection unit 371 selects one of the input signal and the signal held in memory 373. The filter unit 372 performs filtering using the signal selected by the selection unit 371.

[0132] These are examples only, and the filter processing unit 115 according to this embodiment only needs to be able to achieve the function of "outputting values ​​that have not been filtered to the pixels of the IPCM block."

[0133] The following describes a modified version of the filtering method according to this embodiment. Figure 11 is a flowchart showing the operation of a modified version of the filtering method according to this embodiment.

[0134] In the above description, in steps S204 and S205 shown in Figure 5, a filter is applied to the non-IPCM block and the unfiltered pixel values ​​are output for the IPCM block. However, this may be achieved by the following steps. In other words, instead of steps S204 and S205 shown in Figure 5, the process shown in Figure 11 may be performed.

[0135] First, the pixel values ​​of the first block (block [0]) and the second block (block y [1]) which are adjacent to each other are obtained (S221). Here, for example, the first block is a non-IPCM block and the second block is an IPCM block.

[0136] Next, the filter strength bS[0] to be applied to the first block and the filter strength bS[1] to be applied to the second block are derived (S222 and S223). Here, the filter strength bS[0] and the filter strength bS[1] represent different strengths. Note that in the conventional technology, only one filter strength was set for each block boundary. For example, in this embodiment, the filter strength for the IPCM block is set to be weaker than the filter strength for the non-IPCM block.

[0137] Next, filter processing is performed on both blocks with filter intensity bS[0] and the pixel values ​​of the first block after filtering are output (S224). Next, filter processing is performed on both blocks with filter intensity bS[1] and the pixel values ​​of the second block after filtering are output (S225).

[0138] Here, the filter strength value can be set to 0 to control whether or not filtering is performed. In other words, a flag (filterSamplesFlag) can be derived for each block to control whether or not filtering is performed.

[0139] As described above, the filtering method according to this embodiment can perform filtering on one block with a first filtering strength while performing filtering on the other block with a second filtering strength different from the first filtering strength. Furthermore, this filtering method can realize such processing within the filtering process.

[0140] Figure 12 is a flowchart of another modification of the filtering method according to this embodiment. The process shown in Figure 12 has step S401 added to the process shown in Figure 3.

[0141] Step S401 is added to provide an appropriate filter strength to IPCM blocks that would otherwise be determined to be intra-predicted blocks. In step S401, it is determined whether at least one of the first block and the second block is an IPCM block. If at least one of the first block and the second block is an IPCM block (Yes in S401), the filter strength (bS) is determined to be one of 3, 2, 1, or 0, which is a value less than N (=4) (S124). If both the first block and the second block are not IPCM blocks (No in S401), the filter strength is set to bS = N (=4), which means the strongest strength (S123).

[0142] In the filtering method shown in Figure 3, if one or both blocks are macroblocks in intra-prediction mode (Yes in S122), the filter strength itself is always set to bS = 4, which represents the strongest strength, without any other determination conditions.

[0143] On the other hand, in the modified embodiment shown in Figure 12, even if one or both blocks are macroblocks in intra prediction mode (Yes in S122), if one of the blocks is an IPCM block (Yes in S401), a weaker filter strength (bS = 0 to 3) is set compared to the filter strength (bS = 4) set in step S123.

[0144] Figure 13 shows the filter strength determined by the filtering method according to this embodiment and the block units that determine the boundary.

[0145] As shown in Figure 13, when macroblock MB[0] is an inter-prediction mode macroblock and macroblock MB[1] is an intra-prediction mode macroblock (Yes in S122), and both the first and second blocks are non-IPCM blocks (No in S401), bS=4 is set for both blocks (S123).

[0146] On the other hand, if PU block [0] is in non-IPCM mode and PU block [1] is in IPCM mode, that is, if CU block [0] is a non-IPCM block and CU block [1] is an IPCM block (Yes in S401), then bS = 0 to 3 is set for CU block [0] and CU block [1]. In this example, bS = 0 is set for CU block [1], which is an IPCM block, and bS = 1 to 3 is set for CU block [0], which is a non-IPCM block.

[0147] Figures 14A and 14B are diagrams illustrating how the scope of application of the filter ON flag is expanded by handling the IPCM block according to this embodiment. Figure 14A shows a comparative example where the method of this embodiment is not applied. Figure 14B shows a case where the method of this embodiment is applied.

[0148] As shown in Figure 14B, the scope of application of the filter ON flag can be expanded by using the filtering method according to this embodiment.

[0149] As described above, in the filtering method according to this embodiment, the filtering processing unit or control unit uses the implicit code interpretation rule for determining whether to filter the IPCM block in the processing of the loop filter. As a result, as shown in Figures 14A and 14B, it is possible to specify whether to enable or disable the filter over a wider range for the code sequence. As a result, the filtering method according to this embodiment can reduce the number of bits.

[0150] In addition, although the above description has described an example of applying this embodiment to a deblocking filter process, the same method can be applied to other processes. For example, instead of a deblocking filter process, the above process may be applied to an adaptive loop filter (ALF) process or an adaptive offset process.

[0151] Deblocking filtering is a filtering process used on reconstructed pixel samples located near block boundaries. By performing this deblocking filtering, noise at block boundaries that occurs due to block-level quantization can be reduced.

[0152] Adaptive loop filtering is a filtering process that reduces noise in a target pixel by using the pixel values ​​of the surrounding pixels.

[0153] Adaptive offset processing is a process in which one offset value is added to or subtracted from multiple pixel values ​​contained in each block.

[0154] The following describes the processing flow of the video encoding device 100 and the video decoding device 200 in this case.

[0155] Figure 15 is a flowchart of a video encoding method according to a modified example of this embodiment.

[0156] First, the video encoding device 100 determines the prediction mode for the block to be processed (S301). This prediction mode is either IPCM mode or non-IPCM mode.

[0157] Next, the video encoding device 100 writes the determined prediction mode to the encoded bitstream 132 (S302). In other words, the variable-length encoding unit 111 generates an encoded bitstream 132 (encoded signal 131) that includes the determined prediction mode.

[0158] Next, the video encoding device 100 determines whether the prediction mode is IPCM mode (S303). If the prediction mode is IPCM mode (Yes in S303), the video encoding device 100 stores the input image signal 120 in the memory 109 as a reference image used for inter or intra prediction (S306).

[0159] On the other hand, if the prediction mode is not IPCM mode (No in S303), the video encoding device 100 generates a decoded image signal 126 by reconstructing blocks of image samples based on the prediction mode (S304). The video encoding device 100 then generates an image signal 128 by processing the decoded image signal 126 (S305). This processing includes at least one of the following: deblocking filter processing, adaptive loop filter processing, and adaptive offset processing. The video encoding device 100 then stores the generated image signal 128 in the memory 109 as a reference image (S306).

[0160] Figure 16 is a flowchart of a video decoding method according to a modified example of this embodiment.

[0161] First, the video decoding device 200 obtains a prediction mode for the blocks to be processed included in the encoded bitstream 232 by analyzing the encoded bitstream 232 (S311). This prediction mode is either an IPCM mode or a non-IPCM mode.

[0162] Next, the video decoding device 200 determines whether the prediction mode is IPCM mode (S312). If the prediction mode is IPCM mode (Yes in S312), the video decoding device 200 stores the image signal of the target block included in the encoded bitstream 232 in the memory 209 as a reference image used for inter or intra prediction (S315).

[0163] On the other hand, if the prediction mode is a non-IPCM mode (No in S312), the video decoding device 200 generates a decoded image signal 226 by reconstructing blocks of image samples based on the prediction mode (S313). The video decoding device 200 then generates an image signal 228 by processing the decoded image signal 226 (S314). This processing includes at least one of deblocking filter processing, adaptive loop filter processing, and adaptive offset processing. The video decoding device 200 then stores the generated image signal 228 in the memory 109 as a reference image (S315).

[0164] (Embodiment 2) The filtering method according to this embodiment determines the quantization parameters of the IPCM block using the quantization parameters of the non-IPCM block in a deblocking filtering process on the boundary between the IPCM block and the non-IPCM block. For example, the filtering method sets the value of the quantization parameters of the IPCM block to the same value as the quantization parameters of the non-IPCM block. This allows the filtering method to perform filtering on the boundary between the IPCM block and the non-IPCM block with an appropriate filtering strength.

[0165] In the following, we will mainly explain the differences from Embodiment 1, and omit any redundant explanations.

[0166] Figure 17 is a block diagram of a video encoding device 400 according to this embodiment. The video encoding device 400 shown in Figure 17 performs deblocking filtering on the boundaries between adjacent IPCM blocks and non-IPCM blocks that are not IPCM blocks, which are included in the image. This video encoding device 400 comprises a first quantization parameter determination unit 401, a second quantization parameter determination unit 402, a filter intensity determination unit 403, and a filter unit 404. The first quantization parameter determination unit 401, the second quantization parameter determination unit 402, the filter intensity determination unit 403, and the filter unit 404 are included, for example, in the filter processing unit 115 or filter unit 107 shown in Figure 6. Furthermore, the video encoding device 400 may further include all or some of the processing units that are included in the video encoding device 100 shown in Figure 6.

[0167] Figure 18 is a block diagram of a video decoding device 500 according to this embodiment. The video decoding device 500 shown in Figure 18 performs deblocking filtering on the boundaries between adjacent IPCM blocks and non-IPCM blocks included in the image. This video decoding device 500 includes a first quantization parameter determination unit 501, a second quantization parameter determination unit 502, a filter intensity determination unit 503, and a filter unit 504. The first quantization parameter determination unit 501, the second quantization parameter determination unit 502, the filter intensity determination unit 503, and the filter unit 504 are included, for example, in the filter processing unit 215 or filter unit 207 shown in Figure 9. Furthermore, the video decoding device 500 may further include all or some of the multiple processing units that are included in the video decoding device 200 shown in Figure 9.

[0168] Since the filtering processes performed by the video encoding device 400 and the video decoding device 500 are the same, the filtering process performed by the video encoding device 400 will be described below as a representative example.

[0169] Figure 19 is a flowchart of the filtering method by the video encoding device 400 according to this embodiment.

[0170] First, the first quantization parameter determination unit 401 determines a first quantization parameter 411 for the non-IPCM block (S301). For example, the first quantization parameter determination unit 401 obtains the quantization parameter of the non-IPCM block used in the quantization unit 103 or the inverse quantization unit 104 as the first quantization parameter 411. Similarly, the first quantization parameter determination unit 401 obtains, for example, the quantization parameter of the non-IPCM block used in the inverse quantization unit 204 as the first quantization parameter 411.

[0171] Next, the second quantization parameter determination unit 402 uses the first quantization parameter 411 to determine a second quantization parameter 412 corresponding to the IPCM block and for determining the filter intensity (S302). For example, the second quantization parameter determination unit 402 determines the second quantization parameter 412 to the same value as the first quantization parameter 411.

[0172] Next, the filter strength determination unit 403 determines the filter strength 413 using the first quantization parameter 411 and the second quantization parameter 412 (S303). For example, the filter strength determination unit 403 calculates the average value of the first quantization parameter 411 and the second quantization parameter 412, and determines the filter strength 413 using the calculated average value.

[0173] Finally, the filter unit 404 performs deblocking filtering at the boundary between the non-IPCM block and the IPCM block with the determined filter strength 413 (S304).

[0174] The following describes a specific example of this filtering process.

[0175] Figure 20 is a flowchart showing an example of filtering according to this embodiment.

[0176] First, the video encoding device 400 sets the parameter i to an initial value of zero (S411). Next, the video encoding device 400 determines whether the parameter i is 1 or greater (S412).

[0177] If parameter i is 1 or less (Yes in S412), the video encoding device 400 determines whether block [i] is an IPCM block (S413). The following processing is performed for i = 0 and 1, that is, for block [0] and block [1]. Here, block [0] and block [1] are two adjacent blocks, and deblocking filtering is performed on the boundary between these two blocks.

[0178] If block [i] is a non-IPCM block (No in S413), the first quantization parameter determination unit 401 calculates the quantization parameter qP[i] using the following (Equation 2) (S414).

[0179] qP[i]=QPy[i] (Formula 2)

[0180] The quantization parameter QPy is the quantization parameter of the luminance component used in the quantization process, and the quantization parameter qP is a parameter for calculating the filter intensity. In other words, the first quantization parameter determination unit 401 sets the quantization parameter used in the quantization process for the luminance component of the non-IPCM block to the quantization parameter qP[i] of the non-IPCM block.

[0181] On the other hand, if block [i] is an IPCM block (Yes in S413), the second quantization parameter determination unit 402 calculates the quantization parameter qP[i] using the following (Equation 3) (S415).

[0182] qP[i]=QPy[(i+1)%2] (Formula 3)

[0183] In this equation (3), when i = 0, qP[0] = QPy[1], and when i = 1, qP[1] = QPy[0]. In other words, the second quantization parameter determination unit 402 sets the quantization parameter used for the quantization process of the luminance component of the non-IPCM block to the quantization parameter qP[i] of the IPCM block.

[0184] Next, the video encoding device 400 adds "1" to the parameter i and performs the processing from step S412 onward. That is, steps S413 to S415 are performed for block [0] and block [1] respectively. As a result, the quantization parameter qP[0] for block [0] and the quantization parameter qP[1] for block [1] are calculated.

[0185] When the above series of processes is completed, in step S416 the parameter i is set to "2". In this case (No in S412), the filter strength determination unit 403 then calculates the parameter qPav for determining the filter strength using the following (Equation 4) (S417).

[0186] qPav=(qP[0]+qP[1]+1)>>1 (Formula 4)

[0187] In other words, the filter intensity determination unit 403 sets the parameter qPav to the average value of qP[0] and qP[1].

[0188] Finally, the filter intensity determination unit 403 determines the filter intensity 413 using the parameter qPav. The method for determining the filter intensity 413 can be, for example, the method described in Embodiment 1.

[0189] Here, we assume that block [0] is a non-IPCM block and block [1] is an IPCM block. In this case, qPav = qPy[0] + qPy[1] + 1 >> 1 = QP[0] + QPy[0] + 1 >> 1 = QPy[0]. In other words, the parameter qPav, i.e., the filter intensity 413, is determined using only the quantization parameters for the luminance component of the non-IPCM block (block [0]).

[0190] As described above, the video encoding device 400 according to this embodiment can avoid setting a weak filter strength at the boundary between IPCM blocks and non-IPCM blocks. In this way, the video encoding device 400 can perform filtering with an appropriate filter strength at the boundary between IPCM blocks and non-IPCM blocks.

[0191] The filtering process by the video decoding device 500 is the same as that of the video encoding device 400. In other words, the video encoding device 400, the first quantization parameter determination unit 401, the second quantization parameter determination unit 402, the filter intensity determination unit 403, the filter unit 404, the first quantization parameter 411, the second quantization parameter 412, and the filter intensity 413 in the above description can be replaced with the video decoding device 500, the first quantization parameter determination unit 501, the second quantization parameter determination unit 502, the filter intensity determination unit 503, the filter unit 504, the first quantization parameter 511, the second quantization parameter 512, and the filter intensity 513, respectively.

[0192] In addition, in the video decoding device 500, the second quantization parameter determination unit 502 may determine the second quantization parameter 512 using the first quantization parameter 511 according to ΔQP. Here, ΔQP is difference information that shows the difference between the quantization parameter of the immediately preceding block and the quantization parameter of the block to be processed in the processing order (encoding order or decoding order). In other words, if ΔQP is zero, the second quantization parameter 412 of the IPCM block is set to the same value as the first quantization parameter 411 of the non-IPCM block.

[0193] The following describes the flow of the video encoding method and video decoding method when using ΔQP.

[0194] Figure 21 is a flowchart of a video encoding method according to a modified example of this embodiment. The process shown in Figure 21 has steps S421 and S422 added to the process shown in Figure 19.

[0195] In step S421, the video encoding device 400 sets ΔQP for the IPCM block to "0". Next, the video encoding device 400 generates an encoded bitstream including ΔQP (S422).

[0196] Figure 22 is a flowchart of a video decoding method according to a modified example of this embodiment. The process shown in Figure 22 has step S431 added to the process shown in Figure 19, and step S402 is changed to step S402A.

[0197] In step S431, the video decoding device 500 obtains ΔQP contained in the encoded bitstream by analyzing the encoded bitstream.

[0198] In step S402A, the second quantization parameter determination unit 502 determines the second quantization parameter 512 using the first quantization parameter 511 according to ΔQP. Here, if the block to be processed is an IPCM block, ΔQP is set to "0". Therefore, according to this ΔQP, the second quantization parameter determination unit 502 sets the second quantization parameter 512 to the same value as the quantization parameter of the block immediately preceding it in the processing order.

[0199] In other words, if the block immediately preceding the processing block is a non-IPCM block, the second quantization parameter of the IPCM block is set to the same value as the first quantization parameter of the non-IPCM block, similar to the processing described above. In other words, at the boundary between an IPCM block and the non-IPCM block adjacent to its left, the quantization parameter of the IPCM block is set to the same value as the quantization parameter of the non-IPCM block. On the other hand, at the upper, right, and lower boundaries of the IPCM block, the quantization parameter of the IPCM block and the quantization parameter of the non-IPCM block are not necessarily set to the same value. However, since the quantization parameter of the IPCM block is usually set to the same value as the quantization parameter of the block to its right, which is not zero, the filter strength becomes stronger compared to the case where the quantization parameter of the IPCM block is fixed at zero. In other words, by setting the ΔQP of the non-IPCM block to "0" in this way, an appropriate filter strength can be set for the boundary between the IPCM block and the non-IPCM block.

[0200] The difference information that indicates ΔQP is "0" and is included in the encoded bitstream is any information that allows the video decoding device 500 to determine that ΔQP is "0". In other words, the difference information may be a parameter that explicitly indicates "ΔQP = 0", or it may be something else. For example, it may be specified that "if the parameter ΔQP is not included in the encoded bitstream, ΔQP is considered to be 0". In this case, the video encoding device 400 generates an encoded bitstream that does not include the parameter ΔQP for the IPCM block. Also, if the encoded bitstream does not include the parameter ΔQP, the video decoding device 500 considers ΔQP to be zero.

[0201] The filtering method, video encoding method, video decoding method, video encoding device, and video decoding device according to embodiments of the present invention have been described above, but the present invention is not limited to these embodiments.

[0202] For example, at least some of the functions of the filtering method, video encoding method, video decoding method, video encoding device, and video decoding device according to the above embodiment, as well as modified versions thereof, may be combined.

[0203] Furthermore, the division of functional blocks in the block diagram is just one example; multiple functional blocks may be implemented as a single functional block, a single functional block may be divided into multiple parts, or some functions may be moved to other functional blocks. In addition, the functions of multiple functional blocks having similar functions may be processed in parallel or time-sharing by a single piece of hardware or software.

[0204] Furthermore, the order in which the steps included in the above filtering method are performed is illustrative for the purpose of specifically illustrating the present invention, and may be in a different order. Also, some of the above steps may be performed simultaneously (in parallel) with the other steps.

[0205] For example, the order of steps S201 and S202 shown in Figure 5 is not limited to this order. In other words, steps S204 and S205 should be executed only when, as a result, "of the two blocks that straddle the boundary, one block is included in the IPCM block and the other block is not included in the IPCM block." Also, the order of steps S204 and S205 is arbitrary.

[0206] Similarly, the order of steps S222 to S225 shown in Figure 11 is not limited to this order. Specifically, as long as step S224 comes after step S222 and step S225 comes after step S223, the order of steps S222 to S225 can be arbitrary.

[0207] (Embodiment 3) By recording a program for realizing the configuration of the video encoding method (image encoding method) or video decoding method (image decoding method) shown in each of the above embodiments onto a storage medium, the processes shown in each of the above embodiments can be easily performed on an independent computer system. The storage medium can be anything that can record a program, such as a magnetic disk, optical disk, magneto-optical disk, IC card, or semiconductor memory.

[0208] Next, we will describe application examples of the video encoding method (image encoding method) and video decoding method (image decoding method) shown in each of the above embodiments, and a system using them. The system is characterized by having an image encoding and decoding device comprising an image encoding device using the image encoding method and an image decoding device using the image decoding method. Other configurations in the system can be appropriately changed as needed.

[0209] Figure 23 shows the overall configuration of the content supply system ex100 that realizes the content distribution service. The area where the communication service is provided is divided into a desired size, and base stations ex106, ex107, ex108, ex109, and ex110, which are fixed radio stations, are installed in each cell.

[0210] This content supply system ex100 connects various devices such as a computer ex111, a PDA (Personal Digital Assistant) ex112, a camera ex113, a mobile phone ex114, and a game console ex115 to the internet ex101, an internet service provider ex102, a telephone network ex104, and a base station ex106 via ex110.

[0211] However, the content supply system ex100 is not limited to the configuration shown in Figure 23, and any combination of elements may be used for connection. Also, each device may be directly connected to the telephone network ex104 from the base station ex106, which is a fixed radio station, without going through ex110. Furthermore, each device may be directly connected to each other via short-range radio or the like.

[0212] Camera ex113 is a device capable of shooting video, such as a digital video camera, and camera ex116 is a device capable of taking still images and shooting video, such as a digital camera. Mobile phone ex114 can be a mobile phone using the GSM (Registered Trademark) (Global System for Mobile Communications) system, CDMA (Code Division Multiple Access) system, W-CDMA (Wideband-Code Division Multiple Access) system, LTE (Long Term Evolution) system, HSPA (High Speed ​​Packet Access) system, or PHS (Personal Handyphone System), etc., and any of these is acceptable.

[0213] In the content supply system ex100, cameras ex113 and the like are connected to the streaming server ex103 via the base station ex109 and the telephone network ex104, enabling live streaming and the like. In live streaming, content captured by the user using the camera ex113 (for example, video of a music concert, etc.) is encoded as described in each of the embodiments above (i.e., it functions as an image encoding device according to one aspect of the present invention) and transmitted to the streaming server ex103. Meanwhile, the streaming server ex103 streams the transmitted content data to the requesting client. Clients include computers ex111, PDAs ex112, cameras ex113, mobile phones ex114, game consoles ex115, etc., which are capable of decoding the encoded data. Each device that receives the distributed data decodes the received data and plays it back (i.e., it functions as an image decoding device according to one aspect of the present invention).

[0214] The encoding process of the captured data may be performed by the camera ex113, by the streaming server ex103 which handles the data transmission, or the tasks may be shared between them. Similarly, the decoding process of the transmitted data may be performed by the client, by the streaming server ex103, or the tasks may be shared between them. In addition, not limited to the camera ex113, still images and / or video data captured by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. In this case, the encoding process may be performed by the camera ex116, the computer ex111, or the streaming server ex103, or the tasks may be shared between them.

[0215] In addition, these encoding and decoding processes are generally performed by the computer ex111 or the LSI ex500 in each device. The LSI ex500 may be a single chip or a configuration consisting of multiple chips. Furthermore, the software for video encoding and decoding may be embedded in some recording medium (CD-ROM, flexible disk, hard disk, etc.) that can be read by the computer ex111, and the encoding and decoding process may be performed using that software. Moreover, if the mobile phone ex114 has a camera, video data acquired by that camera may be transmitted. In this case, the video data is data that has been encoded by the LSI ex500 in the mobile phone ex114.

[0216] The streaming server ex103 may also consist of multiple servers or multiple computers that process, record, and distribute data in a distributed manner.

[0217] In this way, the content supply system ex100 allows the client to receive and play back encoded data. Thus, the content supply system ex100 allows the client to receive, decode, and play back information transmitted by the user in real time, enabling personal broadcasting even for users who do not possess special rights or equipment.

[0218] Not limited to the content supply system ex100, as shown in Figure 24, the digital broadcasting system ex200 can also incorporate at least one of the motion image encoding device (image encoding device) or motion image decoding device (image decoding device) of each of the above embodiments. Specifically, at the broadcasting station ex201, multiplexed data, in which music data etc. is multiplexed with video data, is transmitted via radio waves to the communication or satellite ex202. This video data is data encoded by the motion image encoding method described in each of the above embodiments (i.e., data encoded by an image encoding device according to one aspect of the present invention). The broadcasting satellite ex202 receives this and transmits radio waves for broadcasting, which are received by a household antenna ex204 capable of receiving satellite broadcasts. The received multiplexed data is decoded and reproduced by a device such as a television (receiver) ex300 or a set-top box (STB) ex217 (i.e., it functions as an image decoding device according to one aspect of the present invention).

[0219] Furthermore, the video decoding device or video encoding device described in each of the above embodiments can also be implemented in a reader / recorder ex218 that reads and decodes multiplexed data recorded on a recording medium ex215 such as a DVD or BD, or encodes a video signal onto the recording medium ex215, and, in some cases, multiplexes it with a music signal and writes it. In this case, the reproduced video signal is displayed on a monitor ex219, and the video signal can be reproduced on other devices or systems using the recording medium ex215 on which the multiplexed data is recorded. Alternatively, the video decoding device may be implemented in a set-top box ex217 connected to a cable television cable ex203 or a satellite / terrestrial broadcast antenna ex204, and displayed on the television monitor ex219. In this case, the video decoding device may be built into the television instead of the set-top box.

[0220] Figure 25 shows a television (receiver) ex300 using the video decoding method and video encoding method described in each of the above embodiments. The television ex300 includes a tuner ex301 that acquires or outputs multiplexed data in which audio data is multiplexed with video data via an antenna ex204 or cable ex203, etc. that receives the above broadcast, a modulation / demodulation unit ex302 that demodulates the received multiplexed data or modulates it into multiplexed data to be transmitted externally, and a multiplexing / separation unit ex303 that separates the demodulated multiplexed data into video data and audio data, or multiplexes the video data and audio data encoded by the signal processing unit ex306.

[0221] The television ex300 also includes a signal processing unit ex306 having an audio signal processing unit ex304 that decodes audio data and video data, respectively, or encodes the respective information, and a video signal processing unit ex305 (which functions as an image encoding device or image decoding device according to one aspect of the present invention), and an output unit ex309 having a speaker ex307 that outputs the decoded audio signal and a display unit ex308 such as a display that shows the decoded video signal. Furthermore, the television ex300 has an interface unit ex317 having an operation input unit ex312 that receives user operation input. Furthermore, the television ex300 has a control unit ex310 that comprehensively controls each unit and a power supply circuit unit ex311 that supplies power to each unit. The interface unit ex317 may include, in addition to the operation input unit ex312, a bridge ex313 for connecting to external devices such as a reader / recorder ex218, a slot unit ex314 for inserting recording media such as an SD card ex216, a driver ex315 for connecting to external recording media such as a hard disk, and a modem ex316 for connecting to a telephone network. The recording media ex216 enables the electrical recording of information using non-volatile / volatile semiconductor memory elements that it stores. The various parts of the television ex300 are connected to each other via a synchronization bus.

[0222] First, a configuration in which the television ex300 decodes and plays back multiplexed data acquired from an external source by an antenna ex204, etc. will be described. The television ex300 receives user operation from a remote controller ex220, etc., and based on the control of a control unit ex310 having a CPU, etc., the multiplexed data demodulated by the modulation / demodulation unit ex302 is separated by the multiplexing / separation unit ex303. Furthermore, the television ex300 decodes the separated audio data with an audio signal processing unit ex304, and decodes the separated video data with a video signal processing unit ex305 using the decoding method described in each of the above embodiments. The decoded audio signal and video signal are output to the outside from the output unit ex309, respectively. When outputting, it is preferable to temporarily store these signals in buffers ex318, ex319, etc., so that the audio signal and video signal are played back in sync. The television ex300 may also read multiplexed data from recording media ex215, ex216 such as magnetic / optical disks or SD cards, rather than from broadcasts, etc. Next, a configuration in which the TV ex300 encodes audio and video signals and transmits them externally or writes them to a recording medium will be described. The TV ex300 receives user operations from a remote controller ex220 or the like, and based on the control of the control unit ex310, encodes audio signals in the audio signal processing unit ex304 and encodes video signals in the video signal processing unit ex305 using the encoding method described in each of the above embodiments. The encoded audio and video signals are multiplexed in the multiplexing / decompression unit ex303 and output externally. When multiplexing, it is advisable to temporarily store these signals in buffers ex320, ex321, etc., so that the audio and video signals are synchronized. Note that there may be multiple buffers ex318, ex319, ex320, and ex321 as shown in the figure, or one or more buffers may be shared. Furthermore, in addition to what is shown in the figure, data may also be stored in buffers as a buffer to avoid system overflow and underflow, for example, between the modulation / demodulation unit ex302 and the multiplexing / decompression unit ex303.

[0223] In addition to acquiring audio and video data from broadcasts and recording media, the TV ex300 may also be configured to accept AV inputs from microphones and cameras, and may perform encoding processing on the data acquired from them. Here, the TV ex300 has been described as having a configuration that can perform the above-mentioned encoding processing, multiplexing, and external output, but it may also be configured to only be able to perform the above-mentioned reception, decoding processing, and external output, without being able to perform these processes.

[0224] Furthermore, when the reader / recorder ex218 reads or writes multiplexed data from the recording medium, the above decoding or encoding process may be performed by either the television ex300 or the reader / recorder ex218, or the television ex300 and the reader / recorder ex218 may share the task.

[0225] As an example, Figure 26 shows the configuration of the information playback / recording unit ex400 when reading or writing data from an optical disc. The information playback / recording unit ex400 comprises the elements ex401, ex402, ex403, ex404, ex405, ex406, and ex407, which are described below. The optical head ex401 writes information by irradiating the recording surface of the recording medium ex215, which is an optical disc, with a laser spot, and reads the information by detecting the reflected light from the recording surface of the recording medium ex215. The modulation recording unit ex402 electrically drives the semiconductor laser built into the optical head ex401 and modulates the laser light according to the recorded data. The playback / demodulation unit ex403 amplifies the playback signal electrically detected by a photodetector built into the optical head ex401 that detects the reflected light from the recording surface, separates and demodulates the signal components recorded on the recording medium ex215, and plays back the necessary information. The buffer ex404 temporarily holds information to be recorded on the recording medium ex215 and information reproduced from the recording medium ex215. The disk motor ex405 rotates the recording medium ex215. The servo control unit ex406 controls the rotational drive of the disk motor ex405 and moves the optical head ex401 to a predetermined information track, performing laser spot tracking. The system control unit ex407 controls the entire information reproduction / recording unit ex400. The above reading and writing processes are realized by the system control unit ex407 using various information held in the buffer ex404, generating and adding new information as needed, and performing information recording and reproduction through the optical head ex401 while coordinating the operation of the modulation recording unit ex402, the reproduction / demodulation unit ex403, and the servo control unit ex406. The system control unit ex407 is composed of, for example, a microprocessor and executes these processes by running read / write programs.

[0226] In the above description, the optical head ex401 was described as irradiating a laser spot, but a configuration that uses near-field light to perform higher-density recording is also possible.

[0227] Figure 27 shows a schematic diagram of a recording medium ex215, which is an optical disc. Guide grooves are formed in a spiral shape on the recording surface of the recording medium ex215, and address information indicating the absolute position on the disc is recorded in advance on the information track ex230 by changes in the shape of the grooves. This address information includes information for identifying the position of the recording block ex231, which is the unit in which data is recorded, and the recording block can be identified by playing back the information track ex230 and reading the address information in a recording or playback device. The recording medium ex215 also includes a data recording area ex233, an inner circumference area ex232, and an outer circumference area ex234. The area used for recording user data is the data recording area ex233, and the inner circumference area ex232 and outer circumference area ex234, which are located inward or outward from the data recording area ex233, are used for specific purposes other than recording user data. The information playback / recording unit ex400 reads and writes encoded audio data, video data, or multiplexed data obtained by multiplexing such data to the data recording area ex233 of the recording medium ex215.

[0228] The above explanation uses single-layer optical discs such as DVDs and Blu-rays as examples, but is not limited to these; optical discs with a multilayer structure that can record on surfaces other than the surface may also be used. Furthermore, optical discs with a multidimensional recording / playback structure may be used, such as recording information using light of various different wavelengths of color in the same location on the disc, or recording different layers of information from various angles.

[0229] In addition, in the digital broadcasting system ex200, it is also possible to receive data from satellite ex202 etc. in a car ex210 equipped with antenna ex205 and play video on a display device such as a car navigation system ex211 in the car ex210. The configuration of the car navigation system ex211 can be, for example, one of the configurations shown in Figure 25 with the addition of a GPS receiver, and similar things can be considered for the computer ex111 and the mobile phone ex114, etc.

[0230] Figure 28A shows a mobile phone ex114 using the video decoding method and video encoding method described in the above embodiment. The mobile phone ex114 includes an antenna ex350 for transmitting and receiving radio waves with a base station ex110, a camera unit ex365 capable of taking video and still images, and a display unit ex358 such as a liquid crystal display that displays data decoded from video captured by the camera unit ex365, video received by the antenna ex350, etc. The mobile phone ex114 further includes a main unit having an operation key unit ex366, an audio output unit ex357 such as a speaker for outputting sound, an audio input unit ex356 such as a microphone for inputting sound, a memory unit ex367 for storing encoded or decoded data such as captured video, still images, recorded audio, or received video, still images, email, etc., or a slot unit ex364 which is an interface unit with a recording medium for storing data.

[0231] Furthermore, an example of the configuration of the mobile phone ex114 will be explained using Figure 28B. In the mobile phone ex114, the main control unit ex360 comprehensively controls each part of the main body which has a display unit ex358 and an operation key unit ex366, and the power supply circuit unit ex361, operation input control unit ex362, video signal processing unit ex355, camera interface unit ex363, LCD (Liquid Crystal Display) control unit ex359, modulation / demodulation unit ex352, multiplexing / separation unit ex353, audio signal processing unit ex354, slot unit ex364, and memory unit ex367 are all connected to each other via the bus ex370.

[0232] When the end call and power keys are turned on by the user, the power supply circuit unit ex361 supplies power from the battery pack to each part, thereby starting up the mobile phone ex114 to an operational state.

[0233] The mobile phone ex114, based on the control of the main control unit ex360 which has a CPU, ROM, RAM, etc., converts the audio signal picked up by the audio input unit ex356 into a digital audio signal in the audio signal processing unit ex354 when in voice call mode, modulates / demodulates this into a spread spectrum, performs digital-to-analog conversion and frequency conversion processing in the transmit / receive unit ex351 and then transmits it via the antenna ex350. The mobile phone ex114 also amplifies the received data received via the antenna ex350 when in voice call mode, performs frequency conversion and analog-to-digital conversion processing, performs despread spectrum processing in the modulate / demodulate unit ex352, converts it into an analog audio signal in the audio signal processing unit ex354 and then outputs it from the audio output unit ex357.

[0234] Furthermore, when sending an email in data communication mode, the text data of the email entered by operating the operation key unit ex366 on the main unit is sent to the main control unit ex360 via the operation input control unit ex362. The main control unit ex360 performs spread spectrum processing on the text data in the modulation / demodulation unit ex352, and after performing digital-to-analog conversion and frequency conversion processing in the transmission / reception unit ex351, it is transmitted to the base station ex110 via the antenna ex350. When receiving an email, the received data is processed in almost the reverse order and output to the display unit ex358.

[0235] When transmitting video, still images, or video and audio in data communication mode, the video signal processing unit ex355 compresses and encodes the video signal supplied from the camera unit ex365 using the video encoding method shown in each of the above embodiments (i.e., it functions as an image encoding device according to one aspect of the present invention), and sends the encoded video data to the multiplexing / separation unit ex353. The audio signal processing unit ex354 encodes the audio signal picked up by the audio input unit ex356 while the camera unit ex365 is capturing video, still images, etc., and sends the encoded audio data to the multiplexing / separation unit ex353.

[0236] The multiplexing / decompression unit ex353 multiplexes the encoded video data supplied from the video signal processing unit ex355 and the encoded audio data supplied from the audio signal processing unit ex354 in a predetermined manner, performs spread spectrum processing on the resulting multiplexed data in the modulation / demodulation unit (modulation / demodulation circuit unit) ex352, performs digital-to-analog conversion processing and frequency conversion processing in the transmission / reception unit ex351, and then transmits it via the antenna ex350.

[0237] When receiving video data linked to a homepage or the like in data communication mode, or when receiving an email with video and / or audio attached, the multiplexing / decomposition unit ex353 separates the multiplexed data received via antenna ex350 into a bitstream of video data and a bitstream of audio data, and supplies the encoded video data to the video signal processing unit ex355 and the encoded audio data to the audio signal processing unit ex354 via the synchronization bus ex370. The video signal processing unit ex355 decodes the video signal by decoding it using a video decoding method corresponding to the video encoding method shown in each of the above embodiments (i.e., it functions as an image decoding device according to one aspect of the present invention), and the video and still images contained in the video file linked to a homepage are displayed on the display unit ex358 via the LCD control unit ex359. The audio signal processing unit ex354 decodes the audio signal, and the audio is output from the audio output unit ex357.

[0238] Furthermore, terminals such as the above-mentioned mobile phone ex114 can be implemented in three ways, similar to the television ex300: a transceiver-type terminal with both an encoder and a decoder, a transmitting terminal with only an encoder, and a receiving terminal with only a decoder. In addition, although the digital broadcasting system ex200 was described as receiving and transmitting multiplexed data in which music data etc. is multiplexed with video data, it may also be data in which text data related to the video is multiplexed in addition to audio data, or it may be video data itself instead of multiplexed data.

[0239] Thus, the video encoding method or video decoding method shown in each of the above embodiments can be used in any of the above-described devices and systems, thereby obtaining the effects described in each of the above embodiments.

[0240] Furthermore, the present invention is not limited to the above embodiments, and various modifications or alterations are possible without departing from the scope of the present invention.

[0241] (Embodiment 4) It is also possible to generate video data by appropriately switching between the video encoding method or apparatus shown in each of the above embodiments and video encoding methods or apparatus conforming to different standards such as MPEG-2, MPEG4-AVC, and VC-1, as needed.

[0242] When multiple video data sets conforming to different standards are generated, it is necessary to select a decoding method corresponding to each standard when decoding. However, since it is not possible to identify which standard the video data to be decoded conforms to, a problem arises in that it is not possible to select the appropriate decoding method.

[0243] To solve this problem, the multiplexed data, which is multiplexed with audio data etc., video data, is configured to include identification information that indicates which standard the video data conforms to. The specific configuration of the multiplexed data including video data generated by the video encoding method or apparatus shown in each of the above embodiments will be described below. The multiplexed data is a digital stream in MPEG-2 transport stream format.

[0244] Figure 29 is a diagram showing the configuration of multiplexed data. As shown in Figure 29, multiplexed data is obtained by multiplexing one or more of the following: video stream, audio stream, presentation graphics stream (PG), and interactive graphics stream. The video stream represents the main and secondary images of a film, the audio stream (IG) represents the main audio portion of a film and the secondary audio mixed with the main audio, and the presentation graphics stream represents the subtitles of a film. Here, the main image refers to the normal image displayed on the screen, and the secondary image refers to the image displayed on a small screen within the main image. The interactive graphics stream represents an interactive screen created by placing GUI components on the screen. The video stream is encoded by the video encoding method or apparatus shown in each of the above embodiments, or by a video encoding method or apparatus conforming to conventional standards such as MPEG-2, MPEG4-AVC, and VC-1. The audio stream is encoded using methods such as Dolby AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD, or Linear PCM.

[0245] Each stream included in the multiplexed data is identified by a PID. For example, the video stream used for movie footage is assigned 0x1011, the audio streams are assigned 0x1100 to 0x111F, the presentation graphics are assigned 0x1200 to 0x121F, the interactive graphics streams are assigned 0x1400 to 0x141F, the video streams used for secondary footage in movies are assigned 0x1B00 to 0x1B1F, and the audio streams used for secondary audio mixed with the main audio are assigned 0x1A00 to 0x1A1F.

[0246] Figure 30 schematically shows how multiplexed data is multiplexed. First, a video stream ex235 consisting of multiple video frames and an audio stream ex238 consisting of multiple audio frames are converted into PES packet sequences ex236 and ex239, respectively, and then into TS packets ex237 and ex240. Similarly, the data from the presentation graphics stream ex241 and interactive graphics ex244 are converted into PES packet sequences ex242 and ex245, respectively, and then further into TS packets ex243 and ex246. The multiplexed data ex247 is constructed by multiplexing these TS packets into a single stream.

[0247] Figure 31 shows in more detail how a video stream is stored in a PES packet sequence. The first row in Figure 31 shows a sequence of video frames of the video stream. The second row shows a sequence of PES packets. As indicated by the arrows yy1, yy2, yy3, and yy4 in Figure 31, the multiple Video Presentation Units in the video stream, namely I-picture, B-picture, and P-picture, are separated picture by picture and stored in the payload of a PES packet. Each PES packet has a PES header, which contains the PTS (Presentation Time-Stamp), the time the picture was displayed, and the DTS (Decoding Time-Stamp), the time the picture was decoded.

[0248] Figure 32 shows the format of the TS packet that is ultimately written to the multiplexed data. The TS packet is a fixed-length 188-byte packet consisting of a 4-byte TS header containing information such as the PID that identifies the stream, and a 184-byte TS payload that stores the data. The PES packet is divided and stored in the TS payload. In the case of BD-ROM, a 4-byte TP_Extra_Header is attached to the TS packet, forming a 192-byte source packet that is written to the multiplexed data. The TP_Extra_Header contains information such as ATS (Arrival_Time_Stamp). The ATS indicates the start time of forwarding the TS packet to the PID filter of the decoder. As shown in the lower part of Figure 32, the source packets are arranged in the multiplexed data, and the number that is incremented from the beginning of the multiplexed data is called the SPN (Source Packet Number).

[0249] In addition to the individual streams such as video, audio, and subtitles, the TS packets included in the multiplexed data also contain PAT (Program Association Table), PMT (Program Map Table), PCR (Program Clock Reference), etc. The PAT indicates what the PID of the PMT used in the multiplexed data is, and the PID of the PAT itself is registered as 0. The PMT has the PIDs of each stream such as video, audio, and subtitles included in the multiplexed data, as well as attribute information of the stream corresponding to each PID, and also has various descriptors related to the multiplexed data. Descriptors include copy control information that instructs whether to allow or deny copying of the multiplexed data. PCR packets contain information about the STC time corresponding to the ATS time when they are forwarded to the decoder, in order to synchronize the ATC (Arrival Time Clock), which is the time axis of the ATS, with the STC (System Time Clock), which is the time axis of the PTS / DTS.

[0250] Figure 33 is a diagram illustrating the data structure of a PMT in detail. At the beginning of a PMT is a PMT header that indicates the length of the data contained in the PMT. Following that are multiple descriptors related to the multiplexed data. The copy control information mentioned above is written as a descriptor. After the descriptors are multiple stream information entries for each stream contained in the multiplexed data. The stream information consists of stream descriptors that describe the stream type, the stream's PID, and the stream's attribute information (frame rate, aspect ratio, etc.) to identify the compression codec of the stream. There are as many stream descriptors as there are streams in the multiplexed data.

[0251] When recording on a recording medium or the like, the above multiplexed data is recorded together with the multiplexed data information file.

[0252] The multiplexed data information file, as shown in Figure 34, is management information for the multiplexed data, has a one-to-one correspondence with the multiplexed data, and consists of multiplexed data information, stream attribute information, and an entry map.

[0253] The multiplexed data information consists of the system rate, playback start time, and playback end time, as shown in Figure 34. The system rate indicates the maximum transfer rate of the multiplexed data to the PID filter of the system target decoder, which will be described later. The interval of the ATS included in the multiplexed data is set to be less than or equal to the system rate. The playback start time is the PTS of the first video frame of the multiplexed data, and the playback end time is set to the PTS of the last video frame of the multiplexed data plus the playback interval of one frame.

[0254] As shown in Figure 35, attribute information for each stream included in the multiplexed data is registered for each PID. The attribute information differs for each video stream, audio stream, presentation graphics stream, and interactive graphics stream. Video stream attribute information includes information such as what compression codec was used to compress the video stream, the resolution of the individual picture data that make up the video stream, the aspect ratio, and the frame rate. Audio stream attribute information includes information such as what compression codec was used to compress the audio stream, the number of channels included in the audio stream, the languages ​​it supports, and the sampling frequency. This information is used for initializing the decoder before the player plays the audio.

[0255] In this embodiment, the stream type included in the PMT is used from the multiplexed data. Also, if the multiplexed data is recorded on the recording medium, the video stream attribute information included in the multiplexed data information is used. Specifically, in the video encoding method or apparatus shown in each of the embodiments above, a step or means is provided to set unique information indicating that the video data was generated by the video encoding method or apparatus shown in each of the embodiments above, for the stream type included in the PMT or the video stream attribute information. This configuration makes it possible to distinguish between video data generated by the video encoding method or apparatus shown in each of the embodiments above and video data conforming to other standards.

[0256] Figure 36 shows the steps of the video decoding method in this embodiment. In step exS100, the stream type included in the PMT or the video stream attribute information included in the multiplexed data information is obtained from the multiplexed data. Next, in step exS101, it is determined whether the stream type or video stream attribute information indicates that the multiplexed data was generated by the video encoding method or device shown in each of the above embodiments. If it is determined that the stream type or video stream attribute information was generated by the video encoding method or device shown in each of the above embodiments, in step exS102, decoding is performed using the video decoding method shown in each of the above embodiments. If the stream type or video stream attribute information indicates that it conforms to conventional standards such as MPEG-2, MPEG4-AVC, or VC-1, in step exS103, decoding is performed using a video decoding method conforming to the conventional standard.

[0257] In this way, by setting new unique values ​​for the stream type or video stream attribute information, it is possible to determine whether the video can be decoded using the video decoding method or apparatus shown in each of the above embodiments when decoding. Therefore, even if multiplexed data conforming to different standards is input, an appropriate decoding method or apparatus can be selected, making it possible to decode without errors. Furthermore, the video encoding method or apparatus, or video decoding method or apparatus, shown in this embodiment can be used with any of the above-mentioned devices or systems.

[0258] (Embodiment 5) The video encoding method and apparatus, and video decoding method and apparatus described in each of the above embodiments are typically implemented as an integrated circuit (LSI). As an example, Figure 37 shows the configuration of a single-chip LSI ex500. The LSI ex500 comprises elements ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508, and ex509, which are described below, and each element is connected via a bus ex510. The power supply circuit ex505 starts up to an operational state by supplying power to each part when the power supply is turned on.

[0259] For example, when performing encoding processing, the LSI ex500 receives AV signals from a microphone ex117, camera ex113, etc. via AV I / O ex509, based on the control of the control unit ex501, which has a CPU ex502, memory controller ex503, stream controller ex504, drive frequency control unit ex512, etc. The input AV signals are temporarily stored in an external memory ex511 such as SDRAM. Based on the control of the control unit ex501, the stored data is divided into multiple parts as appropriate depending on the amount of processing and processing speed and sent to the signal processing unit ex507, where the audio signal and / or video signal encoding is performed. Here, the video signal encoding process is the encoding process described in each embodiment above. The signal processing unit ex507 further performs processing such as multiplexing the encoded audio data and encoded video data, and outputs it to the outside via stream I / O ex506. This output multiplexed data is transmitted to the base station ex107 or written to the recording medium ex215. When multiplexing, it is recommended to temporarily store the data in buffer ex508 to ensure synchronization.

[0260] In the above description, memory ex511 was described as an external component of LSIex500, but it may also be an internal component of LSIex500. Buffer ex508 is not limited to one, but may have multiple buffers. Furthermore, LSIex500 may be a single chip or multiple chips.

[0261] In addition, although the above states that the control unit ex 501 has a CPU ex 502, a memory controller ex 503, a stream controller ex 504, a drive frequency control unit ex 512, etc., the configuration of the control unit ex 501 is not limited to this configuration. For example, the signal processing unit ex 507 may also have a CPU. By providing a CPU inside the signal processing unit ex 507, it is possible to further improve the processing speed. Another example is that the CPU ex 502 may have a signal processing unit ex 507, or a part of the signal processing unit ex 507, such as an audio signal processing unit. In such a case, the control unit ex 501 will have a CPU ex 502 that has a signal processing unit ex 507, or a part thereof.

[0262] Although the term LSI is used here, depending on the degree of integration, they may also be called IC, system LSI, super LSI, or ultra LSI.

[0263] Furthermore, the method of integrated circuit implementation is not limited to LSIs, and may also be implemented using dedicated circuits or general-purpose processors. After LSI manufacturing, FPGAs (Field Programmable Gate Arrays) that can be programmed or reconfigurable processors that allow for the reconfiguration of the connections and settings of circuit cells inside the LSI may also be used.

[0264] Furthermore, if advances in semiconductor technology or other derived technologies lead to the emergence of integrated circuit technologies that can replace LSIs, then naturally, functional blocks may be integrated using those technologies. The application of biotechnology is one possible possibility.

[0265] (Embodiment 6) When decoding video data generated by the video encoding methods or devices described in each of the above embodiments, the processing load is likely to increase compared to decoding video data conforming to conventional standards such as MPEG-2, MPEG-4-AVC, and VC-1. Therefore, in the LSIex500, it is necessary to set the drive frequency of the CPUex502 to a higher frequency than when decoding video data conforming to conventional standards. However, increasing the drive frequency leads to the problem of increased power consumption.

[0266] To solve this problem, video decoding devices such as the TV ex300 and LSI ex500 are configured to identify which standard the video data conforms to and to switch the drive frequency according to the standard. Figure 38 shows the configuration ex800 in this embodiment. The drive frequency switching unit ex803 sets a high drive frequency when the video data is generated by the video encoding method or device shown in each of the above embodiments. It then instructs the decoding processing unit ex801, which executes the video decoding method shown in each of the above embodiments, to decode the video data. On the other hand, when the video data is video data conforming to a conventional standard, the drive frequency is set lower than when the video data is generated by the video encoding method or device shown in each of the above embodiments. It then instructs the decoding processing unit ex802, which conforms to a conventional standard, to decode the video data.

[0267] More specifically, the drive frequency switching unit ex803 consists of the CPU ex502 and the drive frequency control unit ex512 shown in Figure 37. The decoding processing unit ex801, which executes the video decoding method shown in each of the above embodiments, and the decoding processing unit ex802, which conforms to the conventional standard, correspond to the signal processing unit ex507 shown in Figure 37. The CPU ex502 identifies which standard the video data conforms to. Based on the signal from the CPU ex502, the drive frequency control unit ex512 sets the drive frequency. Based on the signal from the CPU ex502, the signal processing unit ex507 decodes the video data. Here, for example, the identification information described in Embodiment 4 can be used to identify the video data. The identification information is not limited to that described in Embodiment 4, and any information that can identify which standard the video data conforms to is acceptable. For example, if it is possible to identify which standard the video data conforms to based on an external signal that identifies whether the video data is to be used on a television or on a disk, then identification may be made based on such an external signal. Furthermore, the selection of the drive frequency in CPUex502 can be performed based on a lookup table that associates the video data standard with the drive frequency, as shown in Figure 40. The lookup table can be stored in buffer ex508 or the internal memory of the LSI, and CPUex502 can select the drive frequency by referring to this lookup table.

[0268] Figure 39 shows the steps for implementing the method of this embodiment. First, in step exS200, the signal processing unit ex507 acquires identification information from the multiplexed data. Next, in step exS201, the CPU ex502 identifies whether the video data was generated by the encoding method or device shown in each of the above embodiments based on the identification information. If the video data was generated by the encoding method or device shown in each of the above embodiments, in step exS202, the CPU ex502 sends a signal to the drive frequency control unit ex512 to set a higher drive frequency. The drive frequency control unit ex512 then sets a higher drive frequency. On the other hand, if the video data is compliant with conventional standards such as MPEG-2, MPEG4-AVC, or VC-1, in step exS203, the CPU ex502 sends a signal to the drive frequency control unit ex512 to set a lower drive frequency. Then, in the drive frequency control unit ex512, a lower drive frequency is set compared to the case where the video data is generated by the encoding method or apparatus shown in each of the above embodiments.

[0269] Furthermore, the power saving effect can be further enhanced by changing the voltage supplied to the LSIex500 or the device including the LSIex500 in conjunction with the switching of the drive frequency. For example, when the drive frequency is set low, it is possible to set the voltage supplied to the LSIex500 or the device including the LSIex500 lower compared to when the drive frequency is set high.

[0270] Furthermore, the method for setting the drive frequency is not limited to the above-described method; a higher drive frequency is set when the processing load during decoding is large, and a lower drive frequency is set when the processing load during decoding is small. For example, if the processing load for decoding video data conforming to the MPEG4-AVC standard is greater than the processing load for decoding video data generated by the video encoding method or apparatus shown in each of the above embodiments, the drive frequency can be set in the opposite way to the above-described method.

[0271] Furthermore, the method for setting the drive frequency is not limited to a configuration that lowers the drive frequency. For example, if the identification information indicates that the video data is generated by the video encoding method or device shown in each of the above embodiments, the voltage supplied to the LSIex500 or the device including the LSIex500 may be set to a high value, and if the identification information indicates that the video data conforms to conventional standards such as MPEG-2, MPEG4-AVC, and VC-1, the voltage supplied to the LSIex500 or the device including the LSIex500 may be set to a low value. Another example is that if the identification information indicates that the video data is generated by the video encoding method or device shown in each of the above embodiments, the CPUex502 may not be driven, and if the identification information indicates that the video data conforms to conventional standards such as MPEG-2, MPEG4-AVC, and VC-1, the CPUex502 may be temporarily driven as there is processing capacity. Even if the identification information indicates that the video data was generated by the video encoding method or apparatus described in each of the above embodiments, if there is sufficient processing capacity, it is possible to temporarily suspend the operation of the CPUex 502. In this case, it is possible to set a shorter suspension time compared to when the data is compliant with conventional standards such as MPEG-2, MPEG4-AVC, or VC-1.

[0272] In this way, power saving can be achieved by switching the drive frequency according to the standard to which the video data conforms. In addition, when the LSIex500 or a device including the LSIex500 is powered by a battery, the battery life can be extended as a result of the power saving.

[0273] (Embodiment 7) Televisions, mobile phones, and other devices and systems mentioned above may receive multiple video data streams conforming to different standards. To enable decoding even when multiple video data streams conforming to different standards are received, the signal processing unit ex507 of the LSIex500 needs to support multiple standards. However, using separate signal processing units ex507 for each standard would increase the circuit size of the LSIex500 and raise costs.

[0274] To solve this problem, a configuration is adopted in which the decoding processing unit for executing the video decoding method shown in each of the above embodiments is partially shared with a decoding processing unit that conforms to conventional standards such as MPEG-2, MPEG4-AVC, and VC-1. An example of this configuration is shown in ex900 of Figure 41A. For example, the video decoding method shown in each of the above embodiments and the video decoding method conforming to the MPEG4-AVC standard have some common processing content in processes such as entropy coding, inverse quantization, deblocking filter, and motion compensation. For the common processing content, a decoding processing unit ex902 corresponding to the MPEG4-AVC standard is shared, and for other processing content specific to one aspect of the present invention that does not correspond to the MPEG4-AVC standard, a dedicated decoding processing unit ex901 is used. In particular, since one aspect of the present invention is characterized by a deblocking filter, for example, a dedicated decoding unit ex901 may be used for the deblocking filter, while the decoding unit may be shared for any or all of the other processes, such as inverse quantization, entropy decoding, and motion compensation. Regarding the sharing of the decoding unit, a configuration may be used in which the decoding unit for executing the video decoding method shown in each of the above embodiments is shared for common processing content, and a dedicated decoding unit is used for processing content specific to the MPEG4-AVC standard.

[0275] Another example of partially sharing processing is shown in Figure 41B, ex1000. In this example, a dedicated decoding processing unit ex1001 corresponds to processing content specific to one aspect of the present invention, a dedicated decoding processing unit ex1002 corresponds to processing content specific to other conventional standards, and a shared decoding processing unit ex1003 corresponds to processing content common to the video decoding method according to one aspect of the present invention and the video decoding method of other conventional standards is used. Here, the dedicated decoding processing units ex1001 and ex1002 are not necessarily specialized for processing content specific to one aspect of the present invention or other conventional standards, but may be capable of executing other general-purpose processing. Furthermore, the configuration of this embodiment can also be implemented in LSI ex500.

[0276] Thus, by sharing the decoding processing unit for processing content common to one aspect of the present invention and a conventional standard video decoding method, it is possible to reduce the circuit size of the LSI and lower costs.

[0277] The present invention can be applied to filtering methods, motion image encoding devices, and motion image decoding devices. For example, the present invention can be used in high-resolution information display devices or imaging devices such as televisions, digital video recorders, car navigation systems, mobile phones, digital cameras, and digital video cameras.

[0278] 100, 400 video encoding device 101 Subtractor 102 Orthogonal Transformation Unit 103 Quantization section 104, 204 Inverse quantization unit 105, 205 Inverse orthogonal transformation section 106, 206 Adder 107, 207, 301, 302, 311, 322, 323, 332, 342, 343, 344, 351, 352, 353, 361, 362, 372, 404, 504 Filter section 108, 112, 208, 312, 324, 333, 354, 363, 371 Selection section 109,209,373 memory 110, 210 Prediction section 111 Variable-length coding unit 113, 213 Control Unit 115, 215 Filtering section 120 Input image signal 121. Residual signal 122, 124, 224 conversion coefficients 123, 223 quantization coefficients 125, 225 Decoded residual signal 126,226 decoded image signals 127, 128, 227, 228 image signals 130, 230 Predicted image signals 131 Coded signal 132,232 encoded bitstream 200, 500 video decoding device 211 Variable-length decoding unit 212, 321, 331, 341 distribution section 220 Output image signal 401, 501 First quantization parameter determination unit 402, 502 Second quantization parameter determination unit 403, 503 Filter strength determination unit 411, 511 First quantization parameter 412, 512 Second quantization parameter 413, 513 filter strength

Claims

DEPCT691. An encoding method that encodes an image containing IntraPulse Code Modulation (IPCM) blocks into a bitstream. This encoding method includes a filtering step that performs block-by-block filtering between the IPCM blocks and adjacent non-IPCM blocks included in the image, and an encoding step that encodes quantize information into the bitstream to determine the first quantize parameter for the quantize of the non-IPCM blocks. In this filtering step, the second quantize parameter for the IPCM blocks is determined using a predefined zero-differential data set and the first quantize parameter. The filter strength of the block-by-block filtering is determined using the first and second quantize parameters.The block-removal filtering is performed on a per-bounds basis using the determined filtering strength.

2. The encoder unit encodes the image containing the IPCM (IntraPulse Code Modulation) block into the bitstream. This encoder unit consists of: a filter section that performs block-removal filtering on a per-bounds basis between the IPCM block and the adjacent non-IPCM block included in the image; and an encoder section that encodes the quantization information into the bitstream to determine the first quantization parameter for the quantization of the non-IPCM block. The filter section then determines the second quantization parameter for the IPCM block using a predefined zero-differential data. The filtering strength of the block-removal filtering is determined using the first and second quantization parameters.The aforementioned block-removal filtering is performed against the specified boundary using the determined filtering strength level.

3. The encoder set that encodes the image containing the IPCM (IntraPulse Code Modulation) blocks into the bitstream is an encoder set consisting of a processor and a memory unit accessible from the processor, where the processor encodes the IPCM blocks and non-IPCM blocks by performing operations in the read phase, which reads the adjacent IPCM and non-IPCM blocks included in the image stored in the memory unit, and the filtering phase, which performs block-removal filtering against the boundary between the IPCM and non-IPCM blocks.The encoding procedure encodes the quantized information into the bitstream to determine the first quantization parameter for quantizing the non-IPCM block. The filtering procedure then determines the second quantization parameter for the IPCM block using a predefined zero-differential information and the first quantization parameter. The filtering strength of the deblock filtering is then determined using the first and second quantization parameters, and the deblock filtering is performed beyond that bound using the determined filtering strength.