Image decoding apparatus and method, program and storage medium

Abstract

Maintain the randomness of the noise in the image generated by decoding. [Solution] A decoding device that generates image data by decoding encoded data obtained by encoding image data for each block composed of a predetermined number of pixels, the decoding device comprises a decoding means that decodes the encoded data for each block and outputs decoded data for each pixel, and an inverse quantization means that inversely quantizes the decoded data to generate image data, wherein the inverse quantization means sets an adjustment value for each pixel based on predetermined conditions and adjusts the inverse quantized data obtained by adding 1 / 2 of the quantization step to the product obtained by multiplying the decoded data by the respective quantization step, using the adjustment value.

Description

Technical Field

[0001] The present invention relates to an image decoding apparatus and method, a program, and a storage medium, and particularly relates to image encoding and decoding technologies.

Background Art

[0002] In recent years, in imaging apparatuses such as digital video cameras, the amount of image data per unit time handled by the system has increased significantly along with higher resolution and higher frame rates. For this reason, high-speed operation of the image memory and bus interface circuit in the apparatus is required. In response to this, performing compression encoding of images before and after the image memory and the bus interface has been considered. That is, by suppressing the data amount per unit time on the bus interface, the required level of high-speed operation of the circuit is mitigated.

[0003] In this case, an encoding scheme for compression encoding is desired to have a small circuit scale and a small encoding delay, and conventional DCT-based encoding schemes typified by JPEG and MPEG2 are not suitable. Accordingly, compression encoding using a predictive encoding scheme based on DPCM (Differential Pulse Code Modulation) has been proposed (Patent Literatures 1 and 2).

Prior Art Document

Patent Literature

[0004]

Patent Literature 1

Patent Literature 2

Summary of Invention

Problem to be Solved by the Invention

[0005] In this case, for images where random noise appears, such as dark images containing dark current noise, it is desirable that the noise be reproduced randomly even after encoding and decoding. However, sometimes the randomness of the noise is lost after encoding and decoding, resulting in an unnatural, striped appearance.

[0006] This invention was made in view of the above-mentioned problems, and aims to maintain the randomness of noise in images generated by decoding. [Means for solving the problem]

[0007] In order to achieve the above objective, To achieve the above objective, the decoding device of the present invention generates image data by decoding the encoded data obtained by encoding image data for each block composed of a predetermined number of pixels, decoding each block, comprises a decoding means that decodes the encoded data for each block and outputs decoded data for each pixel, and an inverse quantization means that inversely quantizes the decoded data to generate image data, wherein the inverse quantization means sets an adjustment value for each pixel based on predetermined conditions, and generates the image data by adjusting the inverse quantized data obtained by adding 1 / 2 of the quantization step to the product obtained by multiplying each of the decoded data by the quantization step, using the adjustment value. [Effects of the Invention]

[0008] According to the present invention, the randomness of noise in the image generated by decoding can be maintained. [Brief explanation of the drawing]

[0009] [Figure 1] A block diagram showing the schematic configuration of an image processing device in an embodiment of the present invention. [Figure 2] A block diagram showing the configuration of the image encoding unit in the first embodiment. [Figure 3] A block diagram showing the detailed configuration of the encoding unit in the first embodiment. [Figure 4]A block diagram showing the configuration of the image decoding unit in the first embodiment. [Figure 5] A figure showing an example of the format of the image data to be encoded in the first embodiment. [Figure 6] A diagram illustrating the quantized data and loss bits in the first embodiment. [Figure 7] A flowchart of the process for determining the applicable coding mode and applicable QP in the first embodiment. [Figure 8] A flowchart showing the details of the process in S704 in Figure 7. [Figure 9] A diagram illustrating a specific example of the relationship between block code length and degradation level. [Figure 10] This figure shows an example of a priority pattern output by the loss bit priority determination unit in the first embodiment. [Figure 11A] This figure shows an example configuration of a pseudo-random value generation circuit used in the loss bit priority determination unit in the first embodiment. [Figure 11B] This figure shows an example configuration of a pseudo-random value generation circuit used in the loss bit priority determination unit in the first embodiment. [Figure 11C] This figure shows an example configuration 3 of a pseudo-random value generation circuit used in the loss bit priority determination unit in the first embodiment. [Figure 12] A diagram showing the format of encoded data in DPCM mode in the first embodiment. [Figure 13] A diagram illustrating the method for storing loss bits in the first embodiment. [Figure 14] A diagram showing the format of encoded data in PCM mode in the first embodiment. [Figure 15] A flowchart illustrating the method for determining the number of lost bits stored by the multiplexing unit in the first embodiment. [Figure 16] A flowchart illustrating the inverse quantization process in the second embodiment. [Figure 17] A flowchart showing the details of the inverse quantization adjustment value generation process in S1603, as shown in Figure 16. [Figure 18]Diagram for explaining differences in inverse quantization results due to differences in inverse quantization correction values. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. A plurality of features are described in the embodiments, not all of the plurality of features are necessarily essential to the invention, and the plurality of features may be arbitrarily combined. Furthermore, in the accompanying drawings, the same reference numerals are assigned to the same or similar configurations, and duplicate descriptions are omitted.

[0011] <First Embodiment> ●Configuration of Image Processing Apparatus Figure 1 is a block diagram showing the configuration of an image processing apparatus 100 having functions of image encoding and image decoding in the present embodiment. The image processing apparatus 100 includes an acquisition unit 10, an image encoding unit 20, a memory 30, and an image decoding unit 40.

[0012] The functions of each block of the image processing apparatus 100 may be implemented by either software or hardware, excluding physical devices such as the storage element of the memory 30. For example, the function of each block may be implemented by hardware such as a dedicated device, a logic circuit, and a memory, or may be implemented by a computer such as a memory, a processing program stored in the memory, and a CPU that executes the processing program. The image processing apparatus 100 can be implemented as, for example, a digital camera, but may also be implemented as other devices. For example, the image processing apparatus 100 can be implemented as any information processing terminal or electronic device such as a personal computer, a mobile phone, a smartphone, a PDA, a tablet terminal, a digital video camera, a drone, a robot, or a drive recorder.

[0013] The acquisition unit 10 has the function of acquiring image data from an image source. The image source may be, for example, an imaging unit equipped with an image sensor, or a communication unit that receives image data from an external source via a transmission line. Alternatively, the acquisition unit 10 may acquire image data via an interface that reads image data from a recording medium or the like. The acquired image data may be still image data or video data. If the image data acquired by the acquisition unit 10 is video data, multiple frames may be acquired sequentially. The acquisition unit 10 supplies the acquired image data to the image encoding unit 20.

[0014] The image encoding unit 20 encodes the image data supplied from the acquisition unit 10 according to the encoding method described later, and outputs encoded data with compressed information. The output encoded data is stored in the memory 30. This memory 30 functions as a buffer memory and has the storage capacity necessary to store the encoded data output from the image encoding unit 20.

[0015] The image decoding unit 40 reads the encoded data stored in the memory 30, decodes it into image data using a decoding method described later, and outputs it to a subsequent processing unit (not shown).

[0016] In Figure 1, the acquisition unit 10, image encoding unit 20, memory 30, and image decoding unit 40 are shown separately, but some or all of these blocks may be integrated into a single chip.

[0017] ●Image coding unit The image encoding unit 20 of this embodiment encodes each pixel included in a block of interest, which consists of a predetermined number of pixels, generates encoded data with a code amount less than or equal to a preset target code amount (fixed length), and outputs the data after adding the missing bits (loss bits, described later) necessary to achieve the target code amount.

[0018] Figure 2 is a block diagram showing the configuration of the image encoding unit 20 in this embodiment. Its configuration and operation will be described below with reference to Figure 2. The image coding unit 20 includes a provisional coding system 210 and a main coding system 220. The provisional coding system 210 includes coding units 211A to 211D and a QP determination unit 215, and operates to determine the coding mode and quantization parameters (QP) to be used by the main coding system 220 when performing main coding. The main coding system 220 includes a delay unit 221, a coding unit 211E, a multiplexing unit 223, and a loss bit priority determination unit 226. The main coding system 220 then operates to perform main coding, including quantization processing, according to the coding mode and QP determined by the provisional coding system 210. In the following description, coding units 211A to 211E have the same internal configuration, so they may be collectively referred to as the coding unit 211.

[0019] The image encoding unit 20 may be configured as an integrated hardware unit including a dedicated device, logic circuit, and memory, or it may be configured as a distributed unit across multiple devices. Alternatively, the image encoding unit 20 may be configured as a memory, a processing program stored in the memory, and a computer such as a CPU that executes the processing program.

[0020] The image encoding unit 20 receives the image data to be encoded, acquired by the acquisition unit 10, via the input terminal 201. In the following description, the RGB format shown in Figure 5(a) is used as an example of the image data format. The image data is input in raster scan order, and the pixel data for each color element, R (red), G (green), and B (blue), is time-division multiplexed and input sequentially. In this embodiment, the bit depth of each pixel is assumed to be 10 bits.

[0021] Note that the image data format and bit depth are not limited to the RGB format shown in Figure 5(a). For example, the input image data may be in the YCbCr4:2:2 format shown in Figure 5(b), or in the Bayer array format shown in Figure 5(c). Furthermore, the image data bit depth may be 8 bits, 12 bits, or other values.

[0022] ● Coded Block In this embodiment, the image encoding process performed by the image encoding unit 20 divides the image data to be encoded into blocks (encoding blocks) having a predetermined size (number of pixels), and encoding is performed on a unit encoding block basis. In the following description, as shown in Figure 5(a), each encoding block is assumed to consist of 24 data points: 8 horizontal pixels × 1 vertical pixel × 3 color components. Also, as shown in Figure 5(a), each encoding block consists of a total of 8 pixels, each having R, G, and B color components, and is arranged in the order R0, G0, B0, R1, G1, B1, ... in raster scan order. R0, G0, and B0 represent the color components that make up one pixel, but in this embodiment, each color component is conveniently referred to as a "pixel" for explanation. Therefore, in Figure 5(a), each of the 24 data points is assigned a pixel number from 0 to 23. Encoding is performed in order from the smallest pixel number within the encoding block.

[0023] Note that the configuration of the encoded block is not limited to the configuration shown in Figure 5(a). For example, in the case of image data in the YCbCr4:2:2 format shown in Figure 5(b), the encoded block may consist of a total of 32 data points: 16 Y, 8 Cb, and 8 Cr. In the case of Figure 5(b), the pixel numbers assigned would be 0 to 31.

[0024] Alternatively, in the case of image data in the Bayer array format shown in Figure 5(c), the encoded block may consist of a total of 32 data points: 8 Gr, 8 Gb, 8 R, and 8 B. In the case of Figure 5(c), the pixel numbers assigned would be 0 to 31. Furthermore, in the case of the Bayer array, the encoded block may have a two-dimensional structure such as 24 horizontal pixels and 2 vertical pixels.

[0025] In this embodiment, the compression ratio for the encoding block shown in Figure 5(a) will be described as 6 / 10. Since the amount of information in the image data before encoding is 24 pixels × 10 bits = 240 bits per encoding block, the target code amount for each encoding block is 144 bits. Note that the target code amount of 144 bits is merely an example for ease of understanding and is not limited to this. The image encoding unit 20 also selects the encoding mode and QP for each encoding block so as not to exceed the target code amount. The encoding modes include the PCM mode, which uses quantized image data as code data, and the DPCM mode, which encodes the difference between the predicted value and the quantized image data.

[0026] ●Encoding section 211 Figure 3 is a block diagram showing the detailed configuration of the coding unit 211, and as described above, in this embodiment, coding units 211A to 211E have a common configuration. As shown in Figure 3, the coding unit 211 includes a quantization unit 301, a prediction unit 302, a subtractor 303, a variable-length coding unit 304, a code length calculation unit 305, a selector 307, and a selector 308.

[0027] The encoding unit 211 is provided with image data in encoding block units, a quantization parameter QP, and, although not shown in the provisional encoding system 210 in Figure 2, an encoding mode flag. In the provisional encoding system 210, QP is pre-assigned as a fixed value to each of the encoding units 211A to 211D. In this embodiment, as shown in Figure 2, 0, 1, 2, and 3 are assigned as QP to the encoding units 211A to 211D in that order. In the provisional encoding system 210, each of the encoding units 211A to 211D may also hold a pre-existing QP value. In the provisional encoding system 210, a fixed value indicating DPCM mode is input as the encoding mode flag. The encoding mode flag and QP determined by the QP determination unit 215 of the provisional encoding system 210 are supplied to the encoding unit 211E of the main encoding system 220.

[0028] The specific configuration and operation of the encoding unit 211 will be described in detail below. First, the image data of the encoded block input to the encoding unit 211 (here, described as 24 pixel data as shown in Figure 5(a)) is input to the quantization unit 301. The quantization unit 301 quantizes each pixel data of the input image data according to the given QP, and generates quantized data and bit data lost due to quantization (loss bits). The quantization unit 301 then outputs the generated quantized data to the prediction unit 302, subtractor 303, selector 307, and outside the encoding unit 211, respectively. The quantization unit 301 also outputs the generated loss bits outside the encoding unit 211.

[0029] In this embodiment, the value of QP is described as an integer value with a minimum value of 0, and can be changed within the range of 0 to 3. However, a larger value of QP may be set, and quantization may be performed using a larger quantization step.

[0030] In this embodiment, the quantization unit 301 makes the quantization steps smaller (finer) as QP decreases, and makes the quantization steps larger (coarser) as QP increases. Specifically, the quantized data is generated by the operation represented by equation (1), and the loss bits are generated by the operation represented by equation (2).

[0031] Quant = Data >> QP …(1) Loss = Data - (Quant << QP) …(2)

[0032] Here, Quant is the quantized data, Loss is the loss bits, Data is the pixel data of the input image data, and QP is the quantization parameter. Also, >> indicates a right shift calculation, and << indicates a left shift calculation. Expressed in other words, the quantized data Quant is the quotient when the pixel data value is divided by 2QP, indicating that the number of effective bits is reduced from 10 bits to 10-QP bits. Also, the loss bits Loss represent the remainder that is truncated in that division, and the number of bits is QP.

[0033] Here, we will explain quantized data and loss bits with reference to Figure 6. Figure 6 is a diagram that shows the concept of all bits that make up the image data contained in each encoded block, representing an arrangement of 24 pixel data horizontally and a bit depth of 10 bits (bit0 to bit9) vertically.

[0034] Figure 6(a) shows the quantized data corresponding to QP=0 (output of the quantization unit 301 of the encoding unit 211A), indicating that the input image data is output as is without quantization. In this case, there are no loss bits.

[0035] Figure 6(b) shows the quantized data and loss bits (output of the quantization unit 301 of the encoding unit 211B) corresponding to QP=1, indicating that the upper 9 bits are output as quantized data and the lower 1 bit is output as a loss bit.

[0036] Figure 6(c) shows the quantized data and loss bits (output of the quantization unit 301 of the encoding unit 211C) corresponding to QP=2, indicating that the upper 8 bits are output as quantized data and the lower 2 bits are output as loss bits.

[0037] Figure 6(d) shows the quantized data and loss bits (output of the quantization unit 301 of the encoding unit 211D) corresponding to QP=3, indicating that the upper 7 bits are output as quantized data and the lower 3 bits are output as loss bits.

[0038] In this embodiment, for example, increasing QP from "A" to "A+1" (where A is 0, 1, or 2 in this embodiment) is expressed as raising the quantization step by one level, making it larger, or making it coarser. Conversely, decreasing QP from "B" to "B-1" (where B is 1, 2, or 3 in this embodiment) is expressed as lowering the quantization step by one level, making it smaller, or making it finer.

[0039] The code length calculation unit 305 determines the code length of the quantized data per pixel output from the quantization unit 301 using equation (3) based on the bit depth of the image data (10 bits in this embodiment) and QP.

[0040] Quantized data code length = bit depth of image data - QP …(3)

[0041] In this embodiment, the code length of the quantized data decreases by 1 bit for each increase of the QP value. Therefore, with QP=0 as the initial value, the code length of the quantized data decreases by 1 bit from 10 bits for each increase of QP. Here, in the provisional coding system 210, the value of QP is fixedly assigned to each of the coding units 211A to 211D, so the quantized data code length is also a fixed value. Therefore, the code length calculation unit 305 may be configured to hold and output a fixed value of the quantized data code length based on the assigned QP value, rather than calculating the quantized data code length by the calculation in equation (3). The code length calculation unit 305 outputs the determined quantized data code length to the selector 308.

[0042] Next, the operation of the prediction unit 302 will be explained. The prediction unit 302 generates prediction data for the current quantized data (predicted value of the quantized value of the current pixel) using surrounding quantized data. A simple method is to use the previous pixel data of the same color as the prediction data. For example, in this embodiment, as shown in Figure 5(a), image data for each RGB color element is input sequentially, and after encoding image data G0, image data B0 and R1 are encoded before encoding G1. Therefore, the prediction unit 302 outputs the prediction data with a delay of 3 pixels. The prediction unit 302 may also perform calculations such as linear prediction using multiple previous pixel data of the same color. Alternatively, if the encoding block has a two-dimensional structure, it may perform calculations such as planar prediction using surrounding pixel data of the same color, or MED prediction as used in JPEG-LS. Note that for the first three pixels of the encoding block (R0, G0, B0), there is no previous pixel data. Therefore, the prediction unit 302 outputs a predetermined value (for example, 0) as prediction data to the subtractor 303 for the pixel data of the first three pixels of the encoded block.

[0043] The subtractor 303 outputs the difference between the quantized data from the quantization unit 301 and the predicted data from the prediction unit 302 as predicted difference data to the variable-length coding unit 304. The predicted difference data has positive and negative values, and in flat areas of the image data with little variation, the value is close to 0, while in edge areas with large variation, the value is large. The predicted difference data generally has the characteristics of a Laplace distribution centered on 0.

[0044] The variable-length coding unit 304 encodes the input predicted difference data using a predetermined variable-length coding scheme and outputs the generated variable-length coded data to the selector 307 and the code length of the variable-length coded data to the selector 308. Examples of variable-length coding schemes that can be used include Huffman coding and Golomb coding. In the variable-length coding scheme executed by the variable-length coding unit 304, the shortest code length coded data is assigned when the input value is 0, and the code length of the coded data increases as the absolute value of the input value increases.

[0045] The selector 307 receives quantized data and variable-length coded data as input, selects one of them according to the coding mode flag, and outputs the selected data as coded data to the outside of the coding unit 211. In this embodiment, the selector 307 selects variable-length coded data when the coding mode flag is 0, and selects quantized data when the coding mode flag is 1.

[0046] The selector 308 receives input from the quantized data code length and the variable-length code data code length, selects one of them according to the encoding mode flag, and outputs the selected code length to the outside of the encoding unit 211. In this embodiment, the selector 308 selects the variable-length code data code length if the value of the encoding mode flag is 0, and selects the quantized data code length if the value of the encoding mode flag is 1.

[0047] Here, we will explain the encoding modes. The encoding unit 211 switches between DPCM mode and PCM mode depending on the encoding mode flag that indicates the encoding mode. In DPCM mode, the subtractor 303 generates predicted difference data using the quantized data quantized by the quantization unit 301 and the predicted data generated by the prediction unit 302, and the variable-length encoding unit 304 performs variable-length encoding. In DPCM mode, variable-length coded data is generated from image data and QP. In DPCM mode, the encoding mode flag is "0". On the other hand, in PCM mode, the input image data is quantized directly by the quantization unit 301 and output as quantized data. In PCM mode, fixed-length coded data determined by QP is generated. In PCM mode, the encoding mode flag is "1".

[0048] In PCM mode, during the multiplexing of coded data described later, it is necessary to multiplex a 1-bit flag indicating that the coded data was generated in PCM mode. Therefore, one predetermined pixel in the coded block is quantized by increasing the QP by one step, and it operates to output coded data and a coded length that is 1 bit less than that of the other pixels. The predetermined pixel is, for example, the pixel located at the left end (first) of the coded block.

[0049] ●Explanation of the mock coding system Now, referring again to Figure 2, we will explain the processing in the provisional encoding system 210. Image data input to the provisional encoding system 210 in Figure 2 is provisionally encoded by multiple encoding units 211A to 211D with the encoding mode set to DPCM mode and the QP set to 0 to 3. The encoding units 211A to 211D then output the encoding result to the QP determination unit 215. As shown in Figure 3, the signals output from the encoding units 211A to 211D include the code data, code length, quantization data, and loss bit output signals, but the QP determination unit 215 uses only the code length. In the provisional encoding system 210, the encoding mode flag is a fixed value (0) indicating DPCM mode, so the code length of the variable-length code data is effectively used by the QP determination unit 215.

[0050] In this embodiment, the range of QP used for encoding is set to 0 to 3, so the provisional encoding system 210 has four encoding units 211A to 211D. However, this is for the purpose of illustrating and simplifying the explanation, and the number of encoding units 211 can be changed depending on the range of QP used for encoding.

[0051] Although not shown in the diagram, the QP determination unit 215 is given the target code amount and the number of pixels in the encoded block. The target code amount can be input as the target code amount itself, or the amount of image data in the encoded block and the compression ratio can be input, and the QP determination unit 215 can determine the target code amount.

[0052] The QP determination unit 215 determines the encoding mode and QP (hereinafter referred to as "applicable encoding mode" and "applicable QP," respectively) for the encoding block to be processed in the encoding system 220, based on the code length information for one encoding block for each QP input from the encoding units 211A to 211D. The QP determination unit 215 also performs a free area determination process to calculate the number of free bits (free_num[qp] described later) from the target code amount and the code length of the block. The free area determination process will be described later. In this embodiment, the QP determination unit 215 is assumed to have a built-in buffer for temporarily storing the data input from each of the encoding units 211A to 211D. The details of how the QP determination unit 215 determines the applicable encoding mode and applicable QP will be described below.

[0053] Figure 7 is a flowchart illustrating the details of the process for determining the applicable coding mode and applicable QP in the QP determination unit 215. First, in S701, the QP determination unit 215 acquires information on the code length calculated for each QP assigned to each coding unit from the coding units 211A to 211D, on a pixel-by-pixel basis.

[0054] In S702, the QP determination unit 215 sums the code lengths of the acquired pixels and calculates the code length of the entire encoded block (hereinafter referred to as the "block code length") for each QP. In this embodiment, the provisional encoding system 210 has four encoding units 211A to 211D, so the QP determination unit 215 calculates four block code lengths. When calculating the block code length, it is necessary to take into account the code length of the header information that is multiplexed on the encoded data. Header information is information for each encoded block that is necessary during decoding. In this embodiment, the code length of the header information is a total of 3 bits: 2 bits for representing QP (0 to 3) and 1 bit for the encoding mode flag. Therefore, in S702, the QP determination unit 215 calculates the block code length bl_size[qp] by adding the code length of the header information hd_size (=3 bits) and the sum of the code lengths of the pixels. Here, [qp] is the value corresponding to QP. The number of bits used to represent QP can be changed depending on the range of QP used for encoding.

[0055] In S703, the QP determination unit 215 calculates the number of free bits free_num[qp] and the number of lost bits loss_num[qp] for each QP using the target code size target_size (144 bits in this embodiment) and the block code length bl_size[qp] for each QP. The QP determination unit 215 also calculates the degradation degree bl_dist[qp] for each QP using the number of free bits free_num[qp] and the number of lost bits loss_num[qp]. The number of free bits free_num[qp] is calculated by equation (4).

[0056] free_num[qp] = target_size - bl_size[qp] …(4) The number of free bits is the value obtained by subtracting the block code length from the target code amount, as shown in equation (4). A positive value for the number of free bits indicates that the block code length in the corresponding QP is smaller than the target code amount. A negative value for the number of free bits indicates that the block code length in the corresponding QP is larger than the target code amount.

[0057] The number of lost bits, loss_num[qp], is calculated by equation (5).

[0058] loss_num[qp] = QP × pix_num …(5) Here, pix_num is the number of pixels in the encoding block (in this embodiment, "24").

[0059] The number of lost bits, loss_num[qp], indicates the number of bits lost due to quantization and is a value uniquely determined by QP and the number of pixels in the coded block. Here, since the number of pixels in the coded block in this embodiment is pix_num=24, for example, when QP=2, the number of lost bits, loss_num[2], is 48.

[0060] The degradation level bl_dist[qp] is calculated using equation (6).

[0061] bl_dist[qp] = loss_num[qp] - free_num[qp] …(6) The degradation level bl_dist[qp] indicates the degree of encoding degradation. In this embodiment, when the block code length is smaller than the target code amount, encoding is performed in such a way that the bit data lost due to quantization (lost bits) is stored in the empty bit area as much as possible. During decoding, the degradation due to quantization is reduced by restoring the lost bits stored in the empty bit area. Therefore, the final degree of degradation is the value obtained by subtracting the number of empty bits from the number of lost bits.

[0062] In S704, the QP determination unit 215 determines the coding mode and the applicable QP using the bl_size[qp] calculated in S702 and the bl_dist[qp] calculated in S703.

[0063] Now, referring to Figure 8, we will explain the details of the S704 process. In S801, the QP determination unit 215 initializes the variables. Specifically, the QP determination unit 215 initializes qp (the value of QP) to 0 and min_dist (the value of the minimum degradation) to the pre-set MAX_DIST. The QP determination unit 215 also initializes sel_qp (the applicable QP) to the value of MAX_QP+1. Here, MAX_DIST is the maximum possible value for the degradation of the encoded block, and is the value obtained by subtracting the target code amount from the amount of image data of the encoded block before encoding. In this embodiment, MAX_DIST = 240 - 144 = 96. Also, MAX_QP is the maximum value of the QP used for encoding in the encoding units 211A to 211D. In this embodiment, since MAX_QP is 3, sel_qp is initialized to 4. The initial value of sel_qp is initialized to be the QP that is applied when the conditions for selecting the applicable QP, which will be described below, are not met by any of the QPs. Additionally, the initial value of the applied encoding mode, enc_mode, is set to PCM mode.

[0064] In S802, the QP determination unit 215 determines whether the following conditions are met: the block code length bl_size[qp] calculated in S702 for the current QP (variable qp) is less than or equal to the target code amount target_size, and the encoding block degradation degree bl_dist[qp] calculated in S703 is less than or equal to min_dist. If these conditions are met, the QP determination unit 215 proceeds to S803; otherwise, it proceeds to S804.

[0065] In S803, the QP determination unit 215 updates sel_qp to the current qp as a candidate for the QP to be selected. The QP determination unit 215 also updates min_dist to the current degradation level bl_dist[qp]. At this point, it is decided to use DPCM mode as the encoding mode, so the QP determination unit 215 changes enc_mode to DPCM mode.

[0066] In S804, the QP determination unit 215 determines whether or not the value of the current QP (variable qp) is smaller than the maximum value MAX_QP. If qp < MAX_QP, the process proceeds to S805; otherwise, the processing of this flowchart ends.

[0067] In S805, the QP determination unit 215 adds 1 to qp. Thereafter, the same processing is performed again from S802 for the next QP value.

[0068] Note that if no "YES" determination is ever made in S802 and a "NO" determination is made in S804, this means that in the DPCM mode, there is no QP having a block code length smaller than the target code amount, and the block code length cannot be kept within the target code amount. In this case, the processing of Fig. 8 ends with enc_mode and sel_qp remaining respectively at the PCM mode and 4 (=MAX_QP+1) initialized in S801.

[0069] As a result of the above-described processing, if any of the block code lengths bl_size[qp] calculated in S702 is equal to or less than the target code amount, the DPCM mode is selected as the applicable coding mode, and the QP that minimizes the degradation degree bl_dist[qp] of the coded block calculated in S703 is selected as the applicable QP. If all of the block code lengths bl_size[qp] exceed the target code amount, the PCM mode is selected as the applicable coding mode, and the qp value "4" obtained by adding 1 to the maximum value (MAX_QP) of the QP values is used as the applicable QP.

[0070] Then, the QP determination unit 215 outputs the coding mode flag of the applicable coding mode obtained by the above-described processing, the applicable QP, and the number of remaining bits for the applicable QP.

[0071] Figure 9 shows specific examples of block code length bl_size[qp] and degradation level bl_dist[qp]. In the example in Figure 9(a), when QP=3, the block code length bl_size[3]=92 bits is less than or equal to the target code amount of 144 bits, and the degradation level bl_dist[3]=20 is the minimum. Therefore, in the case of Figure 9(a), QP=3 is selected as the applicable QP and DPCM mode is selected as the applicable coding mode by the process shown in Figure 8. In the example in Figure 9(b), the block code length exceeds the target code amount for all QPs. Therefore, in the case of Figure 9(b), "4" is selected as the applicable QP and PCM mode is selected as the applicable coding mode by the process shown in Figure 8.

[0072] As explained with reference to equation (6), the degradation degree bl_dist[qp] is the value obtained by subtracting the number of free bits free_num[qp] from the number of lost bits loss_num[qp]. Substituting the right-hand side of equation (4) for the number of free bits free_num[qp] in equation (6), we obtain the following equation (7).

[0073] bl_dist[qp] = loss_num[qp] - (target_size - bl_size[qp]) = loss_num[qp] + bl_size[qp] - target_size …(7) In equation (7), the target code size is constant regardless of the value of QP, and the block code length corresponding to the header information within the block code length bl_size[qp] is also constant regardless of the value of QP. Therefore, the relationship between the degrees of degradation between QPs is determined by the sum of the code lengths (block code length) output by the encoding unit 211 and the number of lost bits loss_num[qp]. Accordingly, the QP determination unit 215 can determine the applicable QP and the applicable encoding mode based on the block code length and the number of lost bits without calculating the degree of degradation. In the example in Figure 9(a), among the QPs where the block code length is smaller than the target code size, the sum of the block code length and the number of lost bits is smallest when QP=3, so DPCM mode is determined as the applicable encoding mode and QP=3 is determined as the applicable QP. In the example in Figure 9(b), since all of the block code lengths corresponding to each QP exceed the target code size, PCM mode is determined as the applicable encoding mode and QP=4 is determined as the applicable QP so that the code size of the encoded block is less than or equal to the target code size.

[0074] ● Encoding process in this encoding system 220 The encoding system 220 receives the same image data as the encoding block input to the temporary encoding system 210. However, the encoding process of the encoding system 220 cannot begin until the QP determination unit 215 of the temporary encoding system 210 determines and outputs the applicable encoding mode and applicable QP. Therefore, the input image data of the encoding block is first input to the delay unit 221 and delayed for the number of processing cycles required for the temporary encoding system 210 to determine the applicable encoding mode and applicable QP. The delayed image data of the encoding block is then input from the delay unit 221 to the encoding unit 211E. As a result, the encoding unit 211E can perform encoding using the applicable encoding mode and applicable QP determined by the temporary encoding system 210.

[0075] The encoding unit 211E is supplied with the delayed encoded block image data and the encoding mode flag and applied QP from the data output from the QP determination unit 215. The encoding unit 211E then performs the encoding process on the delayed image data according to the input encoding mode flag and applied QP. This generates encoded data with the same code length as the block code length determined by the QP determination unit 215. The encoding unit 211E outputs the generated encoded data, along with the code length and loss bits, to the multiplexing unit 223. The encoding unit 211E also outputs the quantized data, which has been quantized according to the applied QP, to the loss bit priority determination unit 226.

[0076] The loss bit priority determination unit 226 determines the priority for determining the number of loss bits to be stored for each pixel in the multiplexing unit 223, which will be described later, and outputs it to the multiplexing unit 223. This priority is a unique value determined for each pixel in the encoding block. For example, if the encoding block has 24 pixels, there are 24 values ​​corresponding to pixel numbers 0 to 23, and this set is called the priority pattern. The storage of loss bits will now be explained, and how the priority works in that process.

[0077] Loss bits are stored sequentially from the MSB side of each pixel's loss bits, as long as they can be stored in the available bits, which is free_num[sel_qp] (where sel_qp is the applied QP).

[0078] When storing the loss bits for each pixel from the MSB side, the number of loss bits that can be stored often differs depending on the pixel. For example, if an encoded block has 24 pixels and 38 bits remaining, 14 of the 24 pixels can store 2 of the loss bits, and the remaining 10 pixels can store 1 of the loss bits. Therefore, priority is used to determine which pixel stores one extra loss bit. Pixels with higher priority will be treated to store one extra loss bit.

[0079] For example, if the priority of each pixel is set sequentially from left to right in the encoded block, pixels that can store one more bit tend to be consecutive on the left side of the block. As a result, the quantization error will, on average, be smaller on the left side of the encoded block and larger on the right side. When the image obtained by decoding such an encoded block is viewed across the entire screen, the quantization error becomes more easily visible. In addition, if the order of pixels in the encoded block is set from right to left, or from the left and right edges to the center, so that high-priority pixel positions and low-priority pixel positions are consecutive, the regions with small and large quantization errors within the encoded block will be unevenly distributed, leading to a similar problem.

[0080] Therefore, in this embodiment, a random priority is generated for the pixel numbers so that pixels that can store one extra bit of loss bits and pixels that cannot store one extra bit do not appear consecutively within the encoding block.

[0081] Figures 10(a) and 10(b) show an example of a priority pattern for storing loss bits for a given pixel number in a coding block. Figure 10(a) shows the priority pattern when the coding block has 24 pixels, and Figure 10(b) shows an example of a priority pattern when the coding block has 32 pixels. In Figure 10(a), each pixel number is assigned a priority value from 0 to 23 in a random order, and this can be determined arbitrarily. A smaller priority number assigned to a pixel number means that the loss bits are allocated with higher priority. In the same figure, when the pixel numbers are arranged by priority, they are 16, 12, 3... and the last one is 11.

[0082] The specific configuration of the loss bit priority determination unit 226 may be a ROM or RAM device that stores priority patterns in advance as a table, or it may be a pseudo-random value generation circuit that generates priority patterns.

[0083] Furthermore, while the above example of priority patterns shows an example where the same priority pattern is output for each coding block, the priority pattern may also be changed according to the pixel values ​​of the input image data so that the priority pattern changes for each coding block. By doing so, the pixel position where quantization errors occur changes for each coding block, making the coding errors less visible.

[0084] As a method for changing the priority pattern for each encoded block, the coordinate position information of the encoded block may be used, or the quantized data from the encoding unit 211E may be used.

[0085] Figure 11A shows an example of the configuration of a pseudo-random value generation circuit used in the loss bit priority determination unit 226. Figure 11A is configured to output a different priority pattern for each encoded block by using quantized data as the seed value of the pseudo-random value generation circuit. In the figure, the pseudo-random value generation circuit consists of a seed value generation unit 1101, a replacement unit 1102, a selector 1103, a register 1104, a random value generation unit 1105, and a selector 1106.

[0086] Next, we will explain the operation of the pseudo-random value generation circuit. Here, we will explain how to generate random values ​​from 0 to 31 when the number of pixels in the coding block is 32. First, the seed value generation unit 1101 generates a seed value for generating random values ​​using the quantized data input from the coding unit 211E. The number of bits in the seed value must be the same as the number of bits in the random value to be generated, so in this example it will be 5 bits. If the number of bits in the input quantized data is greater than this, the required number of bits is adjusted using a logical operation such as exclusive OR (XOR). For example, if the quantized data is 10 bits, the upper 5 bits and the lower 5 bits can be XORed to get 5 bits. Even with other numbers of bits, any combination can be used to generate a seed value with the same number of bits as the random value.

[0087] The generated seed value is input to the substitution unit 1102, and if the seed value is 0, it is replaced with a predetermined non-zero value. This is because, in the random value generation unit 1105 described later, if the seed value is 0, no random value is generated (it remains 0). The seed value output from the substitution unit 1102 is input to the register 1104 via the selector 1103. The selector 1103 operates to select the seed value from the seed value generation unit 1101 in the first cycle of the encoding block. The register 1104 is a multi-bit register that holds the seed value and random value, and the data input by the clock signal (not shown) is stored there. The value stored in the register 1104 is input to the random value generation unit 1105.

[0088] The random value generation unit 1105 uses the principle of random generation with a general linear feedback shift register, but is designed to execute multiple bits simultaneously. As an example, equations (8) to (12) show logical formulas for generating 5-bit output data out[4:0] from 5-bit input data in[4:0]. The symbol ^ in the formulas represents the XOR operation.

[0089] out[0]=in[4]^in[1] …(8) out[1]=in[3]^in[0] …(9) out[2]=in[2]^in[4]^in[1] …(10) out[3]=in[1]^in[3]^in[0] …(11) out[4]=in[0]^in[2]^in[4]^in[1] …(12) The output of the random value generation unit 1105 is connected to register 1104 via selector 1103. As a result, register 1104 holds the seed value from the seed value generation unit 1101 in the first cycle of the coding block, and holds the output of the random value generation unit 1105 in the other cycles. The value held in register 1104 is input to the random value generation unit 1105, which then generates random values ​​sequentially. Therefore, in each cycle in which the clock is input, values ​​from 1 to 31 are generated in a random order.

[0090] Note that the output from register 1104 is the seed value in the first cycle of the coding block, and a random value between 1 and 31 is output in the following 31 cycles. In other words, since 0 does not appear, selector 1106 is used to replace the seed value output at the beginning of the coding block with 0.

[0091] Furthermore, in this case, the value at the beginning of the encoded block will always be 0, and the highest priority will be fixed to the first pixel of the block. Therefore, as shown in Figure 11B, a bit inversion unit 1107 may be provided after the selector 1106 to invert some or all of the bits of the random value. This will shift the value with priority 0 to a different pixel position, thus avoiding the case where the first pixel of the encoded block always has priority 0.

[0092] Alternatively, as shown in Figure 11C, an adder 1108 may be provided after the selector 1106 to add a seed value to the priority data (with the carry bit truncated). This improves the randomness of the priority of the leading pixel of the coded block.

[0093] ●Explanation of the multiplexing unit 223 in this encoding system 220 Referring again to Figure 2, the multiplexing unit 223 in this encoding system 220 will be explained. The multiplexing unit 223 receives the coded data, coded length, and loss bits from the coding unit 211E, as well as the coding mode flag, applied QP, and number of free bits output from the QP determination unit 215. Furthermore, the multiplexing unit 223 receives the priority pattern from the loss bit priority determination unit 226. The multiplexing unit 223 then performs multiplexing of this data in a predetermined format for each coded block.

[0094] Now, referring to Figure 12, the encoding format in DPCM mode will be explained. As mentioned above, if the number of pixels in the encoding block is 24, the bit depth of the pixel data is 10 bits, and the compression ratio is 6 / 10, the size of the encoding block 1201 shown in Figure 12 is equal to the target code amount of 144 bits.

[0095] The encoded block 1201 includes a header section 1202 (3 bits) and a pixel data section 1203 (141 bits). The header section 1202 includes an encoded mode flag 1204 (1 bit) indicating the encoded mode and a QP value section 1205 (2 bits) indicating the QP. The pixel data section 1203 includes a variable-length coded data section 1206, a loss bit section 1207, and a stuffing section 1208. Note that the stuffing section 1208 may not be present.

[0096] The coding mode flag and QP output from the QP determination unit 215 are stored in the coding mode flag 1204 (1 bit) and the QP value unit 1205 (2 bits) of the header unit 1202, respectively. In this embodiment, the value of the coding mode flag representing DPCM mode is assumed to be 0. The variable-length coded data (for 24 pixels) from the coding unit 211E is sequentially packed bit by bit using the coded length for each pixel and stored in the variable-length coded data unit 1206.

[0097] The loss bits output from the encoding unit 211E are sequentially stored in the loss bit section 1207 of the encoding block 1201 based on a priority pattern. The loss bit section 1207 is the area immediately following the variable-length coded data section 1206. Details of how the loss bits are stored will be described later.

[0098] Furthermore, if the variable-length code data amount in the variable-length code data section 1206 is the same size as the pixel data section 1203, there will be no area left in the coding block 1201 to store the loss bits. In this case, the loss bits will not be stored in the coding block 1201. Also, if the combined size of the header section 1202, the variable-length code data section 1206, and the loss bit section 1207 is less than the target code amount, the remaining area will be filled with stuffing section 1208 (dummy bits).

[0099] Here, with reference to Figure 13, we will explain how to store loss bits in DPCM mode. For clarity, Figure 13 shows all bits of the image data of an encoded block when QP=3 is selected, representing a horizontal arrangement of 24 pixel data points and a vertical bit depth of 10 bits.

[0100] Figure 13(a) shows all the bits of the encoded block before the multiplexing unit 223 stores the loss bits, with the upper 7 bits of each pixel being the quantized data and the lower 3 bits being the loss bits. The multiplexing unit 223 determines the number of loss bits to store for each pixel according to the priority for multiplexing the loss bits, and stores the loss bits of the loss bit region shown in Figure 13(a) in the loss bit unit 1207 shown in Figure 12.

[0101] Here, we will explain how to determine the number of loss bits to store for each pixel, referring to Figure 15. Figure 15 is a flowchart showing how to determine the number of loss bits to store for each pixel. In this figure, free_num[sel_qp] is the number of free bits in the applied QP, pix_num is the number of pixels in the encoded block, less_bits is the number of loss bits that can be stored per pixel, more_bits is the number of loss bits that can be stored per pixel, with the number of pixels that can store one more bit, more_pix_num is the number of pixels that can store one more bit of loss bits, pix is ​​an internal variable for loop processing, priority[pix] is the priority for storing loss bits for each pixel, and store_bits[pix] represents the number of loss bits to store per pixel. The arithmetic symbols " / " represent division (rounding down the decimal part) and "%" represent the remainder.

[0102] In S1501, the number of loss bits that can be stored per pixel, less_bits, is calculated. This is the quotient (integer) obtained by dividing the number of free bits, free_num[sel_qp], by the number of pixels in the coding block, pix_num. Next, the number of loss bits that can be stored per pixel, more_bits, is calculated for pixels that can store one more loss bit. This is calculated by adding 1 to less_bits. Next, the number of pixels that can store one more loss bit, more_pix_num, is calculated. This is the remainder when the number of free bits, free_num[sel_qp], is divided by the number of pixels in the coding block, pix_num.

[0103] In S1501, the internal variable pix is ​​reset to 0 to loop through S1502 to S1505 for the number of pixels.

[0104] In S1502, the priority[pix] for storing the loss bits for each pixel, determined by the loss bit priority determination unit 226, is compared with the number of pixels that can store one more loss bit, more_pix_num. If priority[pix] is smaller, the process proceeds to S1503, and the number of loss bits that can be stored by pixels, more_bits, is assigned to the number of loss bits to be stored for each pixel, store_bits[pix]. If priority[pix] is greater than or equal to the number of pixels that can store one more loss bit, more_pix_num, the process proceeds to S1504, and the number of loss bits that can be stored for each pixel, less_bits, is assigned to the number of loss bits to be stored for each pixel, store_bits[pix].

[0105] In S1505, the internal variable pix is ​​compared with the number of pixels in the coding block, pix_num. If the internal variable pix is ​​smaller, 1 is added to pix in S1506 to determine the number of loss bits to store for the next pixel, and the process proceeds to S1502, repeating the above steps. When the internal variable pix becomes equal to or greater than the number of pixels in the coding block, pix_num, it is determined that the determination of the number of loss bits to store for all pixels is complete, and the process terminates. In this way, the number of loss bits to store for each pixel, store_bits[pix], is determined.

[0106] Figure 13(b) shows all the bits of the encoded block after the multiplexing unit 223 has stored the loss bits, and also shows the priority for storing the loss bits for each pixel and the number of loss bits stored for each pixel.

[0107] The figure shows an example where the number of free bits, free_num[sel_qp], is 38. In this case, the number of pixels that can store one extra bit of loss, more_pix_num, is 14. Therefore, it is shown that pixels with a priority[pix] of less than 14 can store one extra bit of loss.

[0108] Next, with reference to Figure 14, the encoding format for PCM mode will be described. In this embodiment, the size of the encoding block 1401 is 144 bits, the target code amount, similar to the case of DPCM mode.

[0109] The encoding block 1401 includes an encoding mode flag 1402 (1 bit) indicating the encoding mode and a quantization data section 1403 (143 bits). The encoding mode flag from the QP determination unit 215 is stored in the encoding mode flag 1402 (1 bit). In this embodiment, the value of the encoding mode flag representing PCM mode is assumed to be 1. The quantization data (for 24 pixels) from the encoding unit 211E is stored in the quantization data section 1403, which is sequentially packed bit by bit using the code length for each pixel. In this embodiment, the QP in PCM mode is 4, but as mentioned above, one pixel of the pixels in the encoding block is quantized with a QP of 5, which is one step higher. Therefore, 23 out of 24 pixels are quantized to 6 bits per pixel, and one pixel is quantized to 5 bits. Thus, a total of 23 × 6 + 5 = 143 bits of quantized data are stored in the quantization data section 1403.

[0110] In this way, the encoded data, which is multiplexed according to the encoding mode for each encoding block, is output as stream data to the output terminal 202 and input to the memory 30.

[0111] As described above, in this embodiment, if the code amount obtained by encoding a pixel-level portion of one encoding block is less than the target code amount, the image encoding unit 20 stores the bits lost due to quantization in an empty area based on a randomly set priority. As a result, the quantization error is distributed throughout the encoding block without being biased towards a specific location, and the unnatural appearance of encoding errors as stripes in the decoded image can be reduced.

[0112] ●Image decoding unit 40 Next, referring to Figure 4, the configuration and operation of the image decoding unit 40, which decodes the encoded data generated by the image encoding unit 20, will be described. Figure 4 is a block diagram showing the detailed configuration of the image decoding unit 40. The image decoding unit 40 can decode the encoded data held in the memory 30.

[0113] The image decoding unit 40 shown in Figure 4 includes a separation unit 403, a variable-length decoding unit 404, an adder 405, a selector 406, an inverse quantization unit 407, a prediction unit 408, a loss bit priority determination unit 409, and a loss bit addition unit 410. The image decoding unit 40 may be configured as an integrated hardware including a dedicated device, logic circuit, and memory, or it may be configured as a distributed system using multiple devices. Alternatively, the image decoding unit 40 may be implemented by a computer such as a CPU that executes a memory, a processing program stored in the memory, and the processing program.

[0114] The encoded data generated by the image encoding unit 20 and temporarily stored in the memory 30 is input to the separation unit 403 via the input terminal 401 for each encoding block. Hereinafter, the encoded data for each encoding block will be referred to as "stream data".

[0115] The separation unit 403 separates the encoding mode flag, QP, coded data, and loss bits at the beginning of the input stream data and outputs them sequentially after each processing cycle.

[0116] The separation unit 403 outputs the coded data (quantized data in the case of PCM mode) to the variable-length decoding unit 404 and the selector 406. In the case of DPCM mode, the variable-length decoding unit 404 performs variable-length decoding of the input coded data, outputs the decoded data to the adder 405, and outputs the coded length of the coded data at that time to the separation unit 403. The separation unit 403 accumulates and adds the coded lengths until the variable-length decoding unit 404 has decoded the coded data for 24 pixels contained in one coding block using the variable-length decoding unit 404. Then, the separation unit 403 determines the storage position of the loss bit portion in the fixed-length stream data (more precisely, the total size of the loss bit portion 1207 + stuffing portion 1208) by subtracting the accumulated value from the fixed length, and outputs the data including the loss bit portion to the loss bit addition unit 410.

[0117] The adder 405 adds the predicted data from the prediction unit 408 with the decoded data to obtain a decoded value (decoded data), which is then output to the selector 406 and the prediction unit 408.

[0118] The prediction unit 408 generates prediction data for the current image data using the decoded value from the adder 405. The method for generating the prediction data is the same as the method for generating prediction data in the prediction unit 302 of the encoding unit 211 described above.

[0119] The selector 406 selects coded data from the decoupling unit 403 in PCM mode and decoded data output from the adder 405 in DPCM mode, according to the coding mode flag, and outputs it as quantized data to the dequantization unit 407 and the loss bit priority determination unit 409.

[0120] The inverse quantization unit 407 generates inverse quantized data by inverse quantizing the quantized data from the selector 406 using the QP input from the separation unit 403, and outputs it to the loss bit addition unit 410. When the encoding mode is PCM mode, the separation unit 403 outputs QP as "4".

[0121] The loss bit priority determination unit 409 has the same configuration as the loss bit priority determination unit 226 of the encoding system 220, and determines the priority for determining the number of loss bits stored for each pixel in the same manner. Therefore, the priority determined here is the same as the priority for each encoding block determined by the loss bit priority determination unit 226 of the encoding system 220.

[0122] The loss bit addition unit 410 has a buffer memory that stores the inverse quantized data and loss bits for the encoded block. The loss bit addition unit 410 adds the loss bits output from the separation unit 403 to the inverse quantized data based on priority to generate decoded image data. The method for adding the loss bits is the same as that of the multiplexing unit 223 of the encoding system 220, and in this embodiment, the non-lossy bit portion shown in Figure 13(b) is added.

[0123] Here, a specific example of the loss bit addition process of the loss bit addition unit 410 will be explained. Assume that the pixel to be encoded here is the first pixel in Figure 13(b). The upper 7 bits of the 10 bits have been obtained by inverse quantization. Based on the priority from the loss bit priority determination unit 409, the loss bit addition unit 410 can determine to add 2 bits to the corresponding pixel in the same way as the multiplexing unit 223 of this encoding system 220. The loss bit addition unit 410 reads the corresponding 2 bits from the buffer and replenishes the dominant upper 2 bits (bit2, bit1) for the loss bit count "3" of the pixel of interest. The loss bit addition unit 410 also adds a pre-set value (for example, "0") to bit0, the least significant bit that cannot be replenished. As a result, the loss bit addition unit 410 can decode the 10-bit pixel data of the pixel of interest. The loss bit addition unit 410 outputs the 10-bit data representing the replenished pixel of interest to the output terminal 402.

[0124] The image encoding unit 20 may perform the main encoding using the main encoding system 220 without performing the provisional encoding using the provisional encoding system 210. In this case, the encoding unit 211E of the main encoding system 220 may perform quantization according to, for example, predetermined quantization parameters. If quantization is performed according to predetermined quantization parameters, the image decoding unit 40 may also perform inverse quantization according to the same quantization parameters.

[0125] As described above, according to the first embodiment, by randomly assigning the number of loss bits to be stored to the pixels in the coding block, noise in low-luminance portions of an image caused by image coding and decoding can be made less noticeable.

[0126] Note that in the present embodiment, the temporary coding system 210 has four coding units 211A to 211D, each of which performs coding in a DPCM mode with QP=0, 1, 2, 3. If the target code amount is exceeded even when QP=3, the main coding system 220 performs coding in the PCM mode with QP=4. However, coding by the PCM mode is not necessarily essential. For example, the bit depth of one pixel is defined as M, and the target code amount of one coding block is defined as Ctarget. It is also assumed that the value of QP for ensuring that the total code amount per coding block in the DPCM mode is less than or equal to the target code amount even in the worst case is "N" (N<M-1). If the coded data in the DPCM mode with QP=N is allowed as final data, the temporary coding system 210 is configured to include N+1 coding units with QP=0 to N. Then, the QP determination unit 215 determines the code amount of each of the N+1 coding units, and determines the maximum QP at which the total code amount of one coding block is equal to or less than the target code amount Ctarget as the applied QP. The main coding system 220 may perform coding according to the DPCM mode in accordance with the determined applied QP.

[0127] Further, in the present invention, the number of coding units 211 may be changed according to a desired compression ratio. For example, when the bit depth of each pixel data is M and it is desired to set the compression ratio to D / M, (M-D) coding units may be provided, and 0 to (M-D-1) may be set as QP for each of them. Also, the QP does not need to be consecutive values, and for example, discrete values equal to or less than (M-D-1) such as 0, 2, 4... may be used, and the configuration may be made with a number of coding units smaller than (M-D).

[0128] Conversely, if the compression ratio is variable and the number of provided encoding units 211 is fixed, and the number of encoding units 211 is greater than the number of encoding units 211 required to change the QP value by 1 at the desired compression ratio, then it is sufficient to control the system to use only a portion of the provided encoding units 211. Also, if the number of encoding units 211 is less than the number of encoding units 211 required to change the QP value by 1 at the desired compression ratio, then the QP value may be set discretely.

[0129] Furthermore, this embodiment may be applied to imaging devices such as digital cameras and video cameras. In that case, for example, the imaging unit is equipped with a control circuit including an image encoding unit 20, and the side that receives the imaging results and performs various image processing is equipped with a control circuit including an image decoding unit 40. As a result, even if the imaging resolution is high, normal communication can be performed without straining the communication bandwidth between the two control circuits.

[0130] <Second Embodiment> Next, a second embodiment of the present invention will be described. In the second embodiment, the inverse quantization process performed by the inverse quantization unit 407 of the image decoding unit 40 shown in Figure 4 will be described. Note that the processing other than the processing in the inverse quantization unit 407 and the configuration of the image processing apparatus 100 are the same as those described in the first embodiment, so their description will be omitted.

[0131] Figures 16 and 17 are flowcharts showing the inverse quantization process performed in the inverse quantization unit 407 in the second embodiment. First, in S1601, the inverse quantization unit 407 obtains the first quantized data Quant in the encoded block, which is obtained by decoding the stream data.

[0132] Next, in S1602, the inverse quantization unit 407 generates unadjusted inverse quantized data by the operation represented by equation (13).

[0133] XQuant=(Quant< <QP)+QS / 2 …(13) In equation (13), << is a left shift calculation, XQuant is the unadjusted inverse quantized data, and QS is the quantization step value. In this embodiment, 2QP It is equal to . Also, the QS / 2 (1 / 2 of QS) part in equation (13) is specifically called the "inverse quantization correction value". In addition, although equation (13) shows an example in which the quantized data is left-shifted by QP when generating the unadjusted inverse quantized data, it is also possible to use a configuration in which the quantized data is multiplied by the quantization step value QS.

[0134] Here, we will explain the inverse quantization correction value using Figure 18. Figure 18 shows the difference in the inverse quantization result due to the difference in the inverse quantization correction value, using the case where QS=4 (i.e., QP=2) as an example.

[0135] In the diagram, "input image data" is a list of the values ​​of the image data before quantization. Here, for example, in the case of 10 bits, it lists the range from 0 to 1023 that can be represented by 4 bits (0 to 15).

[0136] "Quantized data" refers to the values ​​obtained by applying quantization (log2(QS) bit right shift, i.e., a 2-bit right shift) to the input image data using QS=4 during encoding. In other words, through quantization, input image data values ​​0-3 become 0, 4-7 become 1, 8-11 become 2, and 12-15 become 3.

[0137] The "pre-corrected inverse quantized data" represents the value obtained by applying pre-corrected inverse quantization processing (log2(QS) bit left shift, i.e., a 2-bit left shift) to the quantized data using QS=4. Therefore, the pre-corrected inverse quantized data corresponding to input image data 0-3 is 0, for 4-7 it is 4, for 8-11 it is 8, and for 12-15 it is 12, all of which are the minimum values ​​among the corresponding input image data.

[0138] The "ideal value of inverse quantized data" represents the value obtained when inverse quantization is performed on quantized data, by taking the average of the input image data corresponding to the quantized data. That is, when QS=4, the ideal value of inverse quantized data corresponding to input image data 0-3 is (0+1+2+3) / 4=1.5. Similarly, the ideal values ​​of inverse quantized data corresponding to input image data 4-7 are 5.5, 8-11 are 9.5, and 12-15 are 13.5, none of which are integer values. In general, each value in image data takes an integer value, so these ideal values ​​cannot be used directly.

[0139] On the other hand, a method has been known for some time in which a representative value is shifted by adding an inverse quantization correction value to such uncorrected inverse quantization data, thereby generating inverse quantization data. "Inverse Quantization Data_A" to "Inverse Quantization Data_C" show examples of this conventional method.

[0140] The inverse quantized data_A represents the value obtained by setting the inverse quantization correction value added to the uncorrected inverse quantized data to 0 (no correction). That is, for input image data 0-3, it becomes 0; for 4-7, it becomes 4; for 8-11, it becomes 8; and for 12-15, it becomes 12.

[0141] The inverse quantization data_B shows the value obtained by adding an inverse quantization correction value of QS / 2 (2 in this case) to the uncorrected inverse quantization data. That is, for input image data 0-3, the value is 2; for 4-7, it is 6; for 8-11, it is 10; and for 12-15, it is 14.

[0142] The inverse quantized data_C represents the value obtained by adding an inverse quantization correction value of QS / 2-1 (1 in this case) to the uncorrected inverse quantized data. That is, for input image data 0-3, it becomes 1; for 4-7, it becomes 5; for 8-11, it becomes 9; and for 12-15, it becomes 13.

[0143] However, the inverse quantized data A, B, and C always differ from the ideal inverse quantized data by -1.5, +0.5, and -0.5, respectively. In other words, a bias is always present in one direction (either positive or negative).

[0144] Therefore, in this embodiment, in order to suppress the occurrence of such biased values, the inverse quantization unit 407 generates an inverse quantization adjustment value by a calculation described later, and adds this to the pre-adjusted inverse quantization data to generate inverse quantization data.

[0145] In S1603, the inverse quantization unit 407 generates an inverse quantization adjustment value. Figure 17 is a flowchart showing the inverse quantization adjustment value generation process performed in S1603.

[0146] In S1701, the inverse quantization unit 407 determines whether the pixel to be processed is the pixel of the first color component of the encoded block (for example, R0, G0, B0 in Figure 5(a)). If it is the pixel of the first color component, the process proceeds to S1703; otherwise, it proceeds to S1702.

[0147] In S1702, the inverse quantization unit 407 generates an inverse quantization adjustment value by the operations shown in equations (14) and (15).

[0148] AdjVal = -1 (when Adjcnt ≤ XQuant) ... (14) AdjVal=0 (when Adjcnt>XQuant) …(15) In equations (14) and (15), AdjVal is the inverse quantization adjustment value, and Adjcnt is the adjacent pixel data. As shown in equations (14) and (15), the inverse quantization adjustment value takes either a value of 0 or -1.

[0149] The adjacent pixel data is the unadjusted inverse quantized data (XQuant) corresponding to the pixel of the same color adjacent to the pixel being processed. The adjacent pixel data is generated by delaying the unadjusted inverse quantized data and appropriately selecting it as the unadjusted inverse quantized data corresponding to the previous pixel (i.e., the pixel to the left) of the same color component.

[0150] In other words, the inverse quantization unit 407 generates an inverse quantization adjustment value of -1 if the quantization data of the pixel to be inversely quantized is greater than or equal to the pre-adjusted inverse quantization data of the previous pixel with the same color component, and an inverse quantization adjustment value of 0 if it is smaller.

[0151] Furthermore, in S1703, the inverse quantization unit 407 generates an inverse quantization adjustment value by the operations shown in equations (16) and (17).

[0152] AdjVal = -1 (when Adjcnt ≥ XQuant) ... (16) AdjVal=0(Adjcnt <XQuantのとき) …(17) For the first three pixels of the encoding block for each color component where no preceding pixel exists, the unadjusted inverse quantized data corresponding to the pixel to the right of each color component (for example, R1, G1, and B1 in Figure 5(a)) may be used as the adjacent pixel data.

[0153] Next, in S1604, the inverse quantization unit 407 generates inverse quantization data by adding the inverse quantization adjustment value to the pre-adjusted inverse quantization data.

[0154] IQuant = XQuant + AdjVal …(18) In equation (18), IQuant is the inverse quantized data. By adding the inverse quantization adjustment value to the unadjusted inverse quantized data, the inverse quantization correction value effectively takes on either the value of QS / 2 or QS / 2-1.

[0155] Next, in S1604, the inverse quantization unit 407 determines whether processing is complete for all quantized data within the encoded block. If it is complete, it terminates the process; otherwise, it returns to S1601 to obtain the next quantized data within the encoded block.

[0156] Subsequently, the processes S1601 to S1605 are repeated until they have been performed for all the quantized data within the encoding block.

[0157] As described above, according to the second embodiment, when decoding encoded data, the inverse quantization adjustment value is switched and selectively used according to the relative magnitude of adjacent pixel data, thereby suppressing the occurrence of a bias that is always skewed in one direction. Furthermore, for the inverse quantization adjustment values ​​corresponding to the first three pixels of the encoded block for each color component, the relative magnitude of adjacent pixel data is reversed compared to the other pixels. As a result, the selection result is the same as that of the pixel to the right of each component, making the block boundary less noticeable.

[0158] In the example described above, the case where the inverse quantization adjustment value is changed according to the relative magnitude of adjacent pixel data was explained, but the present invention is not limited to this. For example, 0 and -1 may be assigned alternately depending on the pixel position. For example, 0 may be selected for even-numbered pixels and -1 for odd-numbered pixels. This simplifies the circuit configuration and is effective in cases where the circuit cost requirements of the application are strict. Alternatively, 0 and -1 may be assigned randomly.

[0159] Furthermore, although the above description described the case in which coded data encoded by the image coding unit 20 of the first embodiment is decoded, the invention is not limited to this. The image decoding unit 40 described in the second embodiment can perform decoding on coded data encoded in PCM mode or DPDM mode. In that case, the loss bit priority determination unit 409 and the loss bit addition unit 410 can be omitted, and the output from the inverse quantization unit 407 can be output directly as image data.

[0160] <Other Embodiments> Furthermore, the present invention may be applied to a system consisting of multiple devices or to a device consisting of a single device.

[0161] Furthermore, the present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0162] <Summary> This embodiment includes the following configuration.

[0163] (Item 1) A decoding device that generates image data by encoding image data for each block composed of a predetermined number of pixels, and then decoding the encoded data obtained for each block, A decoding means that decodes the encoded data for each block and outputs the decoded data for each pixel, The system includes an inverse quantization means that inversely quantizes the decoded data to generate image data, The aforementioned inverse quantization means is Based on predetermined conditions, set the adjustment value for each pixel. The de-quantized data obtained by multiplying the decoded data by the respective quantization steps and adding half of the quantization step to the product is then adjusted using the adjustment value. An image decoding device characterized by generating the aforementioned image data by doing so. (Item 2) The image decoding device according to item 1, characterized in that the adjustment value is 0 or -1 and is added to the inverse quantization data. (Item 3) The image decoding apparatus according to item 2, characterized in that the inverse quantization means compares the inverse quantization data with the inverse quantization data of adjacent pixels of the same color that were previously decoded, sets 0 if the inverse quantization data of the adjacent pixels of the same color is larger, and sets -1 if it is not larger. (Item 4) The image decoding apparatus according to item 3, characterized in that the inverse quantization means sets the same adjustment value as the inverse quantization data of adjacent pixels of the same color that were decoded later, when there is no inverse quantization data for adjacent pixels of the same color that were decoded earlier. (Item 5) The image decoding apparatus according to item 3, characterized in that, when there is no inverse quantization data for adjacent pixels of the same color that have been decoded earlier, the inverse quantization means compares the inverse quantization data with the inverse quantization data of adjacent pixels of the same color that have been decoded later, sets -1 if the inverse quantization data of the adjacent pixels of the same color is larger, and sets 0 if it is not larger. (Item 6) The image decoding apparatus according to item 2, characterized in that the inverse quantization means alternately sets the adjustment value to 0 and -1 according to the pixel position. (Item 7) The image decoding apparatus according to item 2, characterized in that the inverse quantization means randomly sets 0 and -1 as the adjustment value. (Item 8) The image decoding apparatus according to any one of items 1 to 7, characterized in that the quantization step is a power of 2. (Item 9) A decoding method for generating image data by encoding image data for each block composed of a predetermined number of pixels, and then decoding the encoded data obtained for each block, wherein the decoding method generates image data. The decoding means includes a decoding step of decoding the encoded data for each block and outputting the decoded data for each pixel, The inverse quantization means includes an inverse quantization step of inverse quantizing the decoded data to generate image data, In the aforementioned inverse quantization process, Based on predetermined conditions, set the adjustment value for each pixel. The de-quantized data obtained by multiplying the decoded data by the respective quantization steps and adding half of the quantization step to the product is then adjusted using the adjustment value. An image decoding method characterized by generating the aforementioned image data by doing so. (Item 10) A program for causing a computer to function as one of the means of an image decoding apparatus described in any one of items 1 through 8. (Item 11) A computer-readable storage medium containing the program described in item 10.

[0164] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0165] 10...Acquisition unit, 20...Image encoding unit, 30...Memory, 40...Image decoding unit, 210...Temporary encoding system, 220...Main encoding system, 211A~211E...Encoding unit, 215...QP determination unit, 221...Delay unit, 223...Multiplexing unit, 226...Loss bit priority determination unit, 301...Quantization unit, 302...Prediction unit, 303...Subtractor, 304...Variable length encoding unit, 305...Code length calculation unit, 307, 308...Selector, 403...Separation unit, 404...Variable length decoding unit, 405...Adder, 406...Selector, 407...Inverse quantization unit, 408...Prediction unit, 409...Loss bit priority determination unit, 410...Loss bit addition unit

Claims

1. A decoding device that generates image data by encoding image data for each block composed of a predetermined number of pixels, and then decoding the encoded data obtained for each block, A decoding means that decodes the encoded data for each block and outputs the decoded data for each pixel, The system includes an inverse quantization means that inversely quantizes the decoded data to generate image data, The aforementioned inverse quantization means is Based on predetermined conditions, set the adjustment value for each pixel. The de-quantized data obtained by multiplying the decoded data by the respective quantization steps and adding half of the quantization step to the product is then adjusted using the adjustment value. An image decoding device characterized by generating the aforementioned image data by doing so.

2. The image decoding apparatus according to claim 1, characterized in that the adjustment value is 0 or -1 and is added to the inverse quantization data.

3. The image decoding apparatus according to claim 2, characterized in that the inverse quantization means compares the inverse quantization data with the inverse quantization data of adjacent pixels of the same color that were previously decoded, sets 0 if the inverse quantization data of the adjacent pixels of the same color is larger, and sets -1 if it is not larger.

4. The image decoding apparatus according to claim 3, characterized in that the inverse quantization means sets the same adjustment value as the inverse quantization data of adjacent pixels of the same color that were decoded later, when there is no inverse quantization data for adjacent pixels of the same color that were decoded earlier.

5. The image decoding apparatus according to claim 3, characterized in that the inverse quantization means, when there is no inverse quantization data for adjacent pixels of the same color that have been decoded earlier, compares the inverse quantization data with the inverse quantization data of adjacent pixels of the same color that have been decoded later, sets -1 if the inverse quantization data of the adjacent pixels of the same color is larger, and sets 0 if it is not larger.

6. The image decoding apparatus according to claim 2, characterized in that the inverse quantization means alternately sets the adjustment value to 0 and -1 according to the pixel position.

7. The image decoding apparatus according to claim 2, characterized in that the inverse quantization means randomly sets 0 and -1 as the adjustment value.

8. The image decoding apparatus according to any one of claims 1 to 7, characterized in that the quantization step is a power of 2.

9. A decoding method for generating image data by encoding image data for each block composed of a predetermined number of pixels, and then decoding the encoded data obtained for each block, wherein the decoding method generates image data. The decoding means includes a decoding step of decoding the encoded data for each block and outputting the decoded data for each pixel, The inverse quantization means includes an inverse quantization step of inverse quantizing the decoded data to generate image data, In the aforementioned inverse quantization process, Based on predetermined conditions, set the adjustment value for each pixel. The de-quantized data obtained by multiplying the decoded data by the respective quantization steps and adding half of the quantization step to the product is then adjusted using the adjustment value. An image decoding method characterized by generating the aforementioned image data by doing so.

10. A program for causing a computer to function as one of the means of an image decoding apparatus according to any one of claims 1 to 8.

11. A computer-readable storage medium storing the program described in claim 10.

Images