Coding device and program

Abstract

To improve a coding efficiency and an image quality in the case of performing a coding control by a 2-path encoding.SOLUTION: A coding device 1 that outputs a bit stream by performing a coding processing using a quantization parameter (QP) to an input video image, comprises: first coding means 1A of conducting a correspondence relation with first QP and a consumption bit amount in each frame by performing a coding trial by using the first QP as a temporary QP; and second coding means 1B of performing the coding processing by using a second QP as the QP that is optimized on the basis of the correspondence relation to be conducted. The first coding means corrects a QP off-set adopted to the frame of a coding object in accordance with a coding result of a plurality of coded frames in the coding trial, and calculates the first QP by adding the corrected QP off-set to a base QP.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an encoding device and a program. [Background technology]

[0002] In video transmission services such as broadcasting, the available bandwidth (bit rate) is limited. In order to maximize image quality under this constraint, coding control technology plays an important role. General coding control consists of two processes: bit rate allocation and mode selection.

[0003] In the bit allocation process, the encoding device allocates bits to the elements that make up the video (GOP (Group Of Pictures), frames, blocks, etc.) so as not to exceed the target bit rate. In the mode selection process, the encoding device selects the encoding mode (number of block divisions, prediction mode, etc.) so as not to exceed the allocated bit rate. If these processes are not performed appropriately, image quality will deteriorate significantly even if the bit rate constraint is met.

[0004] To solve these problems, research is being conducted into methods to optimize bit allocation. VVenC, one of the latest software encoders, implements a coding control called the R-QP model, which has significantly improved coding efficiency under bitrate constraints compared to VTM (VVC Test Model). VVenC is a software encoder that is an accelerated version of VTM, the reference software used in the international standardization of VVC (Versatile Video Coding), the latest coding method. In the R-QP model, the quantization parameter (QP) is given as a function of the bitrate (R) (the R-QP formula). This formulation makes it possible to calculate a QP that does not exceed the target bitrate R.

[0005] In the R-QP model implemented in VVenC, two-pass encoding is performed (see, for example, Non-Patent Document 1). Hereinafter, the first and second pass encoding processes are also referred to as the first encoding and the second encoding, respectively. The result of the first encoding is used as prior information for optimizing the second encoding.

[0006] Specifically, in the first encoding, encoding is performed in a fixed QP mode, and a correspondence relationship between the consumed bit amount and the quantization parameter is derived for each frame. In the fixed QP mode, the encoding device performs a provisional encoding process (trial encoding) using a provisional QP (provisional QP, hereinafter also referred to as "first QP") obtained by adding a QP offset fixedly set for each frame to a base QP. Since the QP offset performs the same function as the bit amount allocation for each frame in encoding control, adding the QP offset can achieve higher image quality than when the QP offset is not added.

[0007] In the second encoding, the encoding device allocates a bit amount to each frame based on the results of the first encoding, and performs normal encoding processing using the optimized final QP (hereinafter also referred to as the "second QP"). [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] CR Helmrich, I. Zupancic, J. Brandenburg, V. George, A. Wieckowski and B. Bross, "Visually Optimized Two-Pass Rate Control for Video Coding Using the Low-Complexity XPSNR Model," 2021 International Conference on Visual Communications and Image Processing (VCIP), 2021, pp. 1-5, doi: 10.1109 / VCIP53242.2021.9675364. Summary of the Invention [Problem to be solved by the invention]

[0009] In a coding device that performs coding control by two-pass coding, such as coding control of VVenC, the first coding is performed in a fixed QP mode, so that the QP offset is constant (fixed) regardless of the input video.

[0010] However, since the optimal QP offset depends on the input image, the information on the consumed bit amount obtained from the first encoding may not be optimal as prior information for the second encoding. As a result, the bit amount allocation process in the second encoding is not performed appropriately, and the encoding efficiency decreases depending on the image. For the same reason, if a large difference in the allocated bit amount occurs between frames due to inappropriate bit amount allocation, the image quality may vary greatly between frames, and the subjective image quality may deteriorate.

[0011] Therefore, an object of the present invention is to provide a coding device and a program capable of improving coding efficiency and image quality when coding control is performed by two-pass coding. [Means for solving the problem]

[0012] The encoding device according to a first aspect is a device that performs encoding processing on an input video using a quantization parameter (QP) and outputs a bitstream. The encoding device includes a first encoding means that performs encoding trials using a first QP that is a provisional QP, thereby deriving a correspondence relationship between the first QP and a consumed bit amount for each frame, and a second encoding means that performs the encoding processing using a second QP that is the QP optimized based on the derived correspondence relationship. In the encoding trials, the first encoding means modifies a QP offset to be applied to a frame to be encoded according to encoding results of a plurality of encoded frames, and calculates the first QP by adding the modified QP offset to a base QP.

[0013] A program according to the second aspect causes a computer to function as the encoding device according to the first aspect. Effect of the Invention

[0014] According to the present invention, it is possible to provide a coding device and a program capable of improving coding efficiency and image quality when coding control is performed by two-pass coding. [Brief description of the drawings]

[0015] [Figure 1] 1 is a diagram illustrating an example of a video transmission system including an encoding device according to an embodiment. [Diagram 2] FIG. 1 is a diagram illustrating a configuration of an encoding device according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a configuration of a first encoding control unit according to the embodiment. [Figure 4] 4 is a diagram for explaining the operation of a first encoding control unit according to the embodiment. FIG. [Diagram 5] FIG. 1 is a diagram showing a QP offset for each frame in a GOP according to an embodiment. [Figure 6] 11 is a diagram for explaining the operation of a modified QP offset determination unit according to the embodiment. FIG. [Figure 7] FIG. 2 is a diagram illustrating a configuration of a second encoding control unit according to the embodiment. [Figure 8] 4 is a flow chart showing an example of the operation of a first encoding control unit in the encoding device according to the embodiment. FIG. [Figure 9] FIG. 11 is a diagram for explaining a second modified example according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] An encoding device according to an embodiment will be described with reference to the drawings. The encoding device encodes video (i.e., moving images) as represented by MPEG. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0017] (1) Video transmission system First, an example of a video transmission system including an encoding device 1 according to the present embodiment will be described with reference to Fig. 1. Note that, in the present embodiment, the encoding method used in the encoding process is VVC, but the encoding method used in the encoding process may be another encoding method, for example, HEVC (High Efficiency Video Coding), etc.

[0018] The video transmission system shown in FIG.

[0019] The encoding device 1 is a device that performs encoding processing using a quantization parameter (QP) on an input video and outputs a bit stream. The encoding device 1 according to this embodiment performs encoding control by two-pass encoding. In the encoding control, the encoding device 1 performs control so that the bit rate after the encoding processing does not exceed a bandwidth (target bit rate) according to the transmission path.

[0020] The bit rate varies depending on the QP used for quantization in the encoding process. Therefore, in the encoding control, the encoding device 1 selects a QP (optimum QP) that maximizes image quality from among QPs that do not exceed the target bit rate. The optimal QP is estimated from the QP of the encoded frame or block and the amount of consumed bits.

[0021] The encoding device 1 has a first encoding means 1A that performs a first-pass encoding process in two-pass encoding (trial encoding, also referred to as "first encoding"), and a second encoding means 1B that performs a second-pass encoding process in two-pass encoding (regular encoding, also referred to as "second encoding"). The result of the first encoding is used as prior information for optimizing the second encoding. That is, in two-pass encoding, trial encoding (tentative encoding) is performed prior to the regular encoding process, and an optimal QP for converging to a target bit rate is determined from the result of the trial encoding, and the optimal QP is used in the regular encoding process.

[0022] Specifically, the first encoding means 1A performs an encoding trial (first encoding) using a first QP that is a provisional QP, thereby deriving a correspondence relationship between the first QP and the amount of consumed bits for each frame. Then, the second encoding means 1B performs a regular encoding process (second encoding) using a second QP that is an optimized QP based on the derived correspondence relationship.

[0023] In this embodiment, the first encoding means 1A modifies the QP offset to be applied to the frame to be encoded in the first encoding according to the encoding results of the multiple encoded frames, and calculates the first QP by adding the modified QP offset to the base QP. That is, the first encoding means 1A modifies the QP offset according to the fluctuation in image quality after encoding of the encoded frames.

[0024] By correcting the QP offset according to the input video in this way, it is possible to use an optimal QP offset according to the input video. Therefore, the information on the consumed bit amount obtained from the first encoding becomes optimal as prior information for the second encoding. As a result, the bit amount allocation process in the second encoding is performed appropriately, and the encoding efficiency is improved. In addition, it is possible to suppress the deterioration of subjective image quality caused by a large fluctuation in image quality between frames.

[0025] The bit stream output by the encoding device 1 is transmitted to the decoding device 2 on the receiving side via a transmission path. The bit stream output by the encoding device 1 may be stored in a video server and then transmitted to the decoding device 2 via the transmission path. The decoding device 2 decodes the original video from the bit stream and outputs the decoded video. The transmission path may be an optical transmission path or a wireless transmission path. Although not particularly limited, the transmission path may be a broadcast transmission path or a communication transmission path. As the transmission method, RTP (Realtime Transport Protocol) or MMT (MPEG Media Transport) may be adopted.

[0026] (2) Configuration of the encoding device Next, the configuration of the encoding device 1 according to this embodiment will be described with reference to Fig. 2. As shown in Fig. 2, the encoding device 1 has an encoding section 10 and a control section 20.

[0027] The encoding unit 10 performs encoding processing using the QP on the input video to generate a bit stream, and outputs the generated bit stream. A specific configuration example of the encoding unit 10 will be described later.

[0028] The control unit 20 controls the encoding unit 10. In this embodiment, the control unit 20 performs encoding control by two-pass encoding. The control unit 20 has a first-encoding control unit 21 that controls the first encoding in the two-pass encoding, and a second-encoding control unit 22 that controls the second encoding in the two-pass encoding. Specific configuration examples of the first-encoding control unit 21 and the second-encoding control unit 22 will be described later.

[0029] The encoding unit 10 and the first encoding control unit 21 constitute a first encoding means 1A. The encoding unit 10 and the second encoding control unit 22 constitute a second encoding means 1B. However, a coding unit 10 for second encoding may be provided separately from the coding unit 10 for first encoding. In that case, these two coding units 10 may be operated in parallel while delaying the input video to the coding unit 10 for second encoding by a predetermined time.

[0030] (2.1) Encoding section The configuration of the encoding unit 10 itself is the same as a conventional configuration compatible with, for example, VVC, but an outline of the configuration will be described here. The encoding unit 10 generates a bit stream by encoding each image (frame) constituting the input video, and outputs the generated bit stream. Note that a frame is also called a picture.

[0031] As shown in FIG. 2, the encoding unit 10 includes a block division unit 100, a subtraction unit 110, a transform / quantization unit 120, an entropy encoding unit 130, an inverse quantization / inverse transform unit 140, a synthesis unit 150, a memory 160, and a prediction unit 170.

[0032] The block division unit 100 divides an original image in units of frames (pictures) constituting an input video (moving image) into a plurality of image blocks, and outputs the image blocks obtained by division to the subtraction unit 110. The size of the image block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the image block is not limited to a square, but may be a rectangle (non-square). Such an image block may be called a coding block (CU).

[0033] The subtraction unit 110 calculates a prediction residual representing a difference (error) between the coded block output by the block division unit 100 and a predicted block obtained by predicting the coded block by the prediction unit 170. Specifically, the subtraction unit 110 calculates the prediction residual by subtracting each pixel value of the predicted block from each pixel value of the block, and outputs the calculated prediction residual to the transformation / quantization unit 120.

[0034] The transform / quantization unit 120 performs transform processing and quantization processing on a block-by-block basis. The transform / quantization unit 120 includes a transform unit 121 and a quantization unit 122.

[0035] The transform unit 121 performs a transform process on the prediction residual output by the subtraction unit 110 to calculate a transform coefficient, and outputs the calculated transform coefficient to the quantization unit 122. The transform refers to, for example, a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen Loeve transform (KLT), etc. The transform process includes a transform skip in which no transform process is performed. The transform skip also includes a transform in which a transform process is applied only horizontally, or a transform in which a transform process is applied only vertically. The transform unit 121 may also perform a secondary transform process in which a further transform process is applied to the transform coefficient obtained by the transform process. The secondary transform process may be applied only to a partial area of ​​the transform coefficient.

[0036] The quantization unit 122 quantizes the transform coefficients output by the transform unit 121 using a quantization parameter (QP) and a quantization matrix, and outputs the quantized transform coefficients, which are the quantized transform coefficients, to the entropy coding unit 130 and the inverse quantization and inverse transform unit 140. QP is a parameter commonly applied to each transform coefficient in a block, and is a parameter that determines the coarseness of quantization. The smaller the value of QP, the finer the quantization and the more bits consumed. On the other hand, the larger the value of QP, the coarser the quantization and the less bits consumed. The quantization matrix is ​​a matrix whose elements are quantization values ​​used when quantizing each transform coefficient.

[0037] The entropy coding unit 130 performs entropy coding on the quantized transform coefficients output by the quantization unit 122, compresses the data, generates a bit stream, and outputs the bit stream to the outside of the coding device 1. For the entropy coding, Huffman codes, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. may be used. Note that the entropy coding unit 130 receives input of prediction-related information (flags and indexes) from the prediction unit 170, and also codes the input information and outputs the bit stream.

[0038] The inverse quantization and inverse transform unit 140 performs inverse quantization processing and inverse transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142.

[0039] The inverse quantization unit 141 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122. Specifically, the inverse quantization unit 141 reconstructs the transform coefficients by inverse quantizing the quantized transform coefficients output by the quantization unit 122 using a quantization parameter and a quantization matrix, and outputs the reconstructed transform coefficients to the inverse transform unit 142.

[0040] The inverse transform unit 142 performs an inverse transform process corresponding to the transform process performed by the transform unit 121. For example, when the transform unit 121 performs a discrete cosine transform, the inverse transform unit 142 performs an inverse discrete cosine transform. The inverse transform unit 142 performs an inverse transform process on the transform coefficients output by the inverse quantization unit 141 to reconstruct a prediction residual, and outputs the reconstructed prediction residual, which is the reconstructed prediction residual, to the synthesis unit 150.

[0041] The synthesis unit 150 synthesizes the reconstructed prediction residual output by the inverse transform unit 142 by adding it to the predicted block output by the prediction unit 170 on a pixel-by-pixel basis. The synthesis unit 150 adds each pixel value of the reconstructed prediction residual to each pixel value of the predicted block to decode (reconstruct) the block, and outputs the reconstructed block to the memory 160.

[0042] The memory 160 stores the reconstructed blocks output by the synthesis unit 150, and accumulates the reconstructed blocks as decoded images (reconstructed images) on a frame-by-frame basis. The memory 160 outputs the stored reconstructed blocks or decoded images to the prediction unit 170. Note that a loop filter may be provided between the synthesis unit 150 and the memory 160.

[0043] The prediction unit 170 performs prediction on a block-by-block basis. The prediction unit 170 includes an inter prediction unit 171, an intra prediction unit 172, and a switching unit 173.

[0044] The inter prediction unit 171 uses the decoded image stored in the memory 160 as a reference image, calculates a motion vector by a method such as block matching, predicts a coding block to generate an inter prediction block, and outputs the generated inter prediction block to the switching unit 173. Here, the inter prediction unit 171 selects an optimal inter prediction method from inter prediction using multiple reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 171 outputs information related to the inter prediction (motion vector, etc.) to the entropy coding unit 130.

[0045] The intra prediction unit 172 generates an intra prediction block by referring to decoded pixels around a block among the decoded images stored in the memory 160, and outputs the generated intra prediction block to the switching unit 173. In general, the intra prediction unit 172 selects an intra prediction mode to be applied to a predictive coding block of intra prediction from among a plurality of intra prediction modes, and predicts the coding block of intra prediction using the selected intra prediction mode. The intra prediction unit 172 outputs information related to the selected intra prediction mode to the entropy coding unit 130.

[0046] The switching unit 173 switches between the inter predicted block output by the inter prediction unit 171 and the intra predicted block output by the intra prediction unit 172, and outputs either the predicted block to the subtraction unit 110 and the synthesis unit 150.

[0047] (2.2) First encoding control unit Next, the first encoding control unit 21 according to this embodiment will be described with reference to Fig. 3. In this embodiment, the encoding structure is random access, and the GOP size and intra period are both 32. For simplicity, the frames to which the QP offset correction is applied are limited to I frames. Other conditions will be described in modified examples.

[0048] As shown in FIG. 3, the first encoding control unit 21 includes a base QP determination unit 211, a QP offset control unit 212, and an addition unit 213.

[0049] The base QP determination unit 211 determines a base QP according to at least one of the resolution, frame rate, number of frames, and target bit rate of the input video, and outputs the base QP to the addition unit 213. The base QP is a QP that is commonly applied to each frame. An existing method may be used as the process of determining the base QP.

[0050] The QP offset control unit 212 modifies the QP offset to be applied to the frame to be coded, depending on the coding results of the multiple coded frames, and outputs the modified QP offset to the addition unit 213.

[0051] In this embodiment, the plurality of coded frames are a group of coded frames (coded GOP) immediately before a current group of frames (current GOP) to be coded, as shown in Fig. 4. Fig. 4 shows an example in which each GOP is composed of an I frame (also called "I picture") located at the beginning and a plurality of B frames (also called "B pictures") following the I frame.

[0052] An I-frame is an independent frame that does not refer to other frames, and can be encoded using only the data of this frame. A B-frame is a frame that encodes the difference between the changes in the previous and next frames. Since an I-frame is an independent frame, it can be decoded by itself, but a B-frame cannot be decoded unless the other frames it references are decoded.

[0053] The QP offset control unit 212 includes an index difference calculation unit 212a and a modified QP offset determination unit 212b.

[0054] The index difference calculation unit 212a calculates the image quality index of the reconstructed image for two or more predetermined frames in the coded GOP, and calculates the difference Δ between the image quality indexes for the two or more frames. In this embodiment, the index difference calculation unit 212a calculates the difference Δ between the image quality index of the I frame included in the coded GOP and the image quality index of the frame with the largest QP offset (in this embodiment, the B frame with the largest QP offset) among the frames other than the I frame included in the coded GOP. The difference Δ is calculated by the calculation formula of "image quality index of I frame" - "image quality index of the frame with the largest QP offset that is other than the I frame". In addition, the configuration of the GOPs in FIG. 4 shows an example in which each GOP does not include a P frame (P picture), but each GOP may include a P frame. Even in this case, the "frame with the largest QP offset" is generally a B frame in random access. However, the difference Δ may be calculated using a P frame instead of a B frame.

[0055] The image quality index is an index indicating image quality calculated based on a reconstructed image and its corresponding original image. In this embodiment, a peak signal-to-noise ratio (PSNR) is used as the image quality index. However, the image quality index is not limited to PSNR, and other image quality indexes (e.g., MSE, SSIM, VMAF, etc.) may be used. Note that, when there are multiple B frames with the maximum QP offset among the B frames included in the encoded frame group, the index difference calculation unit 212a calculates the average value of the PSNRs of the multiple B frames as the image quality index.

[0056] 5 is a diagram showing the QP offset of each frame in a GOP according to this embodiment. The QP offset shown in the figure is the QP offset used in VVenC (fixed QP mode), and is the value (i.e., the default value) before correction by the QP offset control unit 212. The QP offset shown in the figure is an example when the base QP is 32, but is designed so that the QP offset also changes when the base QP is changed.

[0057] Specifically, in the fixed QP mode, a QP offset (default QP offset) that is preset for each frame is added to the base QP. The default QP offset is set so as to be determined according to the coding structure such as the GOP size and the intra period.

[0058] In the example of Fig. 5, the QP offset of the I frame is -3. By using such a QP offset, the value of the first QP used in the first encoding is reduced. Therefore, the quantization of the I frame becomes finer, which may improve the image quality, but the amount of consumed bits increases.

[0059] On the other hand, in the example of Fig. 5, the QP offset of each of the B frames with the largest QP offset is +9. By using such a QP offset, the value of the first QP used in the first encoding becomes large. Therefore, the quantization of the B frame becomes coarse, so the image quality may decrease, but the amount of consumed bits is reduced.

[0060] In this way, the default QP offset can allocate a large amount of bits to the I frame and a small amount of bits to the B frame, so that the image quality of the I frame can be improved while suppressing an increase in the bit rate. Therefore, the image quality of the B frame that directly or indirectly references the I frame can also be improved. If this assumption is true, the image quality (image quality index) of the I frame and the B frame can be equivalent.

[0061] However, when the input video has a high coding difficulty, the image quality of the B frames that directly or indirectly reference the I frames may not improve significantly even if the image quality of the I frames is improved. In such cases, a discrepancy occurs between the image quality (image quality index) of the I frames and the image quality of the B frames. This causes a large variation in image quality between frames, resulting in a deterioration of the subjective image quality.

[0062] In this embodiment, the modified QP offset determination unit 212b determines that the greater the difference Δ between the image quality (image quality index) of the I frame and the B frame, the higher the encoding difficulty of the input video, and modifies the QP offset of the I frame of the subsequent GOP to be greater than a specified value. This prevents the image quality (image quality index) of the I frame and the B frame of the subsequent GOP from diverging, improving the subjective image quality. In addition, by making the QP offset of the I frame of the subsequent GOP greater than a specified value, the amount of bits allocated to the I frame can be reduced, and the reduced amount of bits can be allocated to other frames, improving the overall image quality (encoding efficiency).

[0063] In addition, since two temporally consecutive GOPs can be said to have similar characteristics, if the encoding difficulty of an encoded GOP is high, the encoding difficulty of the current GOP immediately following it can also be said to be high, and if the encoding difficulty of an encoded GOP is low, the encoding difficulty of the current GOP immediately following it can also be said to be low. Therefore, in this embodiment, the modified QP offset determination unit 212b modifies the QP offset to be applied to the I frame included in the current GOP immediately following the encoded GOP according to the difference Δ for the encoded GOP.

[0064] For example, the modified QP offset determination unit 212b calculates the modified QP offset from the difference Δ using the following sigmoid type modified QP offset calculation formula: However, the modified QP offset calculation formula may be changed depending on the encoding conditions (e.g., random access, low delay, intra period, GOP size, base Q, etc.).

[0065]

number

[0066] Here, the floor function is a function that rounds down a numerical value to a multiple of a specified base value, and is used to obtain an integer.

[0067] By using such a modified QP offset calculation formula, modified QP offset determination unit 212b obtains an integer from -5 to 5 as a modified QP offset from the difference Δ, as shown in Fig. 6. Note that, instead of using the modified QP offset calculation formula, modified QP offset determination unit 212b may define the correspondence relationship between the difference Δ and the modified QP offset shown in Fig. 6 by a table, and convert the difference Δ into a modified QP offset using the table.

[0068] In this embodiment, the modified QP offset determination unit 212b outputs a modified QP offset corresponding to the difference Δ to the adder 213 for the I frame of the current GOP to be encoded, and outputs a corresponding default QP offset to the adder 213 for each B frame.

[0069] The adder 213 obtains a first QP by adding the default QP offset or the modified QP offset output by the modified QP offset determination unit 212b to the base QP output by the base QP determination unit 211. In this embodiment, the adder 213 adds the modified QP offset to the base QP for the I frame of the current GOP to be coded, and adds the corresponding default QP offset to the base QP for each B frame, and outputs the addition result to the coding unit 10 (quantization unit 122) as the first QP.

[0070] In the first encoding, the encoding unit 10 (quantization unit 122) switches the first QP output by the first encoding control unit 21 on a frame-by-frame basis and applies it to quantization. As a result of the first encoding, a correspondence relationship between the first QP and the amount of consumed bits is derived for each frame. The second encoding control unit 22 acquires information on such a correspondence relationship (information on the amount of consumed bits) as prior information for the second encoding.

[0071] (2.3) Second encoding control section Next, the second encoding control unit 22 according to this embodiment will be described with reference to Fig. 7. The second encoding control unit 22 has a bit amount allocation unit 221 and a second QP determination unit 222.

[0072] The bit amount allocation unit 221 allocates a bit amount for each frame based on at least one of the amount of bits consumed in the first encoding, the vertical and horizontal resolutions of the input video, and the frame rate of the input video. The second QP determination unit 222 determines a QP (second QP) to be used in the second encoding for each frame from the R-QP formula based on the allocated bit amount for each frame obtained by the bit amount allocation unit 221 and the first QP.

[0073] Here, the following method can be used to derive the amount of allocated bits from the amount of consumed bits. For example, if the GOP size is 4, and the total amount of bits for these 4 frames is R GOP It is considered to allocate the amount of bits so that

[0074] Assuming that the consumed bit amounts of each frame in the first encoding are r0, r1, r2, and r3, the bit amount allocation unit 221 calculates R GOP The value obtained by dividing the number of bits by the ratio r0:r1:r2:r3 is assigned to each frame. In this case, it is not possible to encode each frame with exactly the assigned bit amount. Therefore, an additional correction process is performed on the assigned bit amount based on the actual bit amount consumed so that the total bit amount is not exceeded.

[0075] The second QP determination unit 222 calculates an appropriate QP (second QP) from the amount of allocated bits obtained by the bit amount allocation unit 221 and the R-QP formula obtained in the first encoding. The second QP is calculated as follows.

[0076] Assign bits to r' f Then, the quantization parameter QP' for the second encoding is f is obtained from the following R-QP formula:

[0077]

number

[0078] where c lowis an experimentally obtained constant, e.g., 0.82. Furthermore, QP' f If the value of is small (<24), the following corrections are made to this value:

[0079]

number

[0080] Here, QP start 24, c high is obtained by using the vertical resolution H as follows:

[0081]

number

[0082] In the second encoding, the encoding unit 10 (quantization unit 122) switches the second QP output by the second encoding control unit 22 on a frame-by-frame basis and applies it to quantization.

[0083] (3) Example of operation flow Next, an example of the operation of the first encoding control unit 21 in the encoding device 1 according to this embodiment will be described with reference to FIG.

[0084] In step S1, the base QP determination unit 211 determines a base QP in accordance with at least one of the resolution, frame rate, number of frames, and target bit rate of an input video, and outputs the base QP to the addition unit 213.

[0085] In step S2, the index difference calculation unit 212a calculates the image quality index (PSNR I ) and the image quality index (PSNR) of the B frame with the largest QP offset among the B frames in the coded GOP immediately preceding the current GOP. B ) and calculate.

[0086] In step S3, the index difference calculation unit 212a calculates the PSNRI and PSNR B The difference Δ between them is calculated.

[0087] In step S4, the modified QP offset determination unit 212b modifies the QP offset to be applied to the I frame included in the current GOP using the above-mentioned modified QP offset calculation formula or table according to the difference Δ calculated in step S3, and outputs the modified QP offset to the addition unit 213. In addition, the modified QP offset determination unit 212b outputs the default QP offset to the addition unit 213 for each frame other than the I frame included in the current GOP.

[0088] In step S5, the addition unit 213 adds the default QP offset or the modified QP offset output by the modified QP offset determination unit 212b in step S1 to the base QP output by the base QP determination unit 211 in step S1 to obtain a first QP.

[0089] (4) Example of changes Next, a modification of the above embodiment will be described.

[0090] (4.1) First Change Example In the above embodiment, an example in which the modified QP offset is applied to an I frame has been described, but the modified QP offset may be applied to a frame other than an I frame.

[0091] For example, in this modification, a QP offset calculation formula (or table) may be created separately for frames other than I frames (eg, B frames), and the QP offset may be corrected for frames other than I frames.

[0092] (4.2) Second Change Example In the above embodiment, the modified QP offset is applied only to the I-frame, but the QP offsets of all the frames in the GOP may be modified collectively according to the value of the modified QP offset.

[0093] In the above embodiment, 11 integers from -5 to 5 are obtained as modified QP offsets. Therefore, in this modified example, a QP offset table corresponding to each integer of the modified QP offset is created in advance, and the QP offset table is stored in the modified QP offset determination unit 212b. In the QP offset table, modified QP offsets of all frames constituting a GOP are determined in advance, as shown in FIG.

[0094] In the example of Fig. 9, the GOP includes 32 frames, and the modified QP offset for each of the 32 frames is defined in the QP offset table. The modified QP offset determination unit 212b outputs the modified QP offset for each frame of the GOP to the addition unit 213 according to, for example, the difference Δ.

[0095] Alternatively, the modified QP offset of each frame defined in the QP offset table may be a difference value (offset value) with respect to the default QP offset. In this case, the modified QP offset determination unit 212b outputs the result of adding the difference value (offset value) to the default QP offset for each frame constituting the GOP to the addition unit 213 as the modified QP offset.

[0096] (5) Other embodiments In the above embodiment, the index difference calculation unit 212a calculates the difference Δ between the image quality index of the I frame included in the coded GOP and the image quality index of a frame other than the I frame in the coded GOP that has the largest QP offset. However, the frame does not necessarily have to have the largest QP offset. Instead of the frame with the largest QP offset, any frame other than the I frame may be used. Even if it is any frame other than the I frame, an improvement effect can be expected.

[0097] In the above embodiment, an example has been described in which each frame group to be coded is one GOP, and the QP offset can be modified for each GOP. However, each frame group to be coded may be a GOP set consisting of two or more GOPs, and the QP offset can be modified for each GOP set.

[0098] Alternatively, each group of frames to be coded may be a "chunk." A chunk is a unit that groups multiple frames together, and is a unit that is divided into bit streams of a certain length when coding video. By grouping a certain number of frames to form a bit stream, the stream can be transmitted and played back more efficiently. Therefore, the GOP in the above-mentioned embodiment may be read as a chunk.

[0099] A program may be provided that causes a computer to execute the processing performed by the encoding device 1. The program may be recorded in a computer-readable medium. The computer-readable medium can be used to install the program in a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM. The circuit that executes the processing performed by the encoding device 1 may be integrated, and the encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC).

[0100] The terms "based on" and "depending on / in response to" used in this disclosure do not mean "based only on" or "depending only on", unless otherwise specified. The term "based on" means both "based only on" and "based at least in part on". Similarly, the term "depending on" means both "based only on" and "depending at least in part on". The terms "include", "comprise", and variations thereof do not mean including only the recited items, but may include only the recited items or may include additional items in addition to the recited items. Additionally, the term "or" used in this disclosure is not intended to be an exclusive or. Additionally, any reference to elements using designations such as "first", "second", etc., used in this disclosure is not intended to generally limit the quantity or order of those elements. These designations may be used herein as a convenient way to distinguish between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed therein or that the first element must precede the second element in some manner. In this disclosure, where articles are added by translation, such as, for example, a, an, and the in English, these articles are intended to include the plural unless the context clearly indicates otherwise.

[0101] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes, etc. are possible without departing from the spirit of the invention.

[0102] (6) Additional Notes The following additional features relate to the above-described embodiment.

[0103] (Appendix 1) An encoding device (1) that performs an encoding process using a quantization parameter (QP) on an input video and outputs a bitstream, a first encoding means (10, 21) for performing encoding trials using a first QP that is a provisional QP, thereby deriving a correspondence relationship between the first QP and a consumed bit amount for each frame; a second encoding means (10, 22) for performing the encoding process using a second QP that is the QP optimized based on the derived correspondence relationship; The first encoding means, in the encoding trial, modifies a QP offset to be applied to the frame to be encoded according to encoding results of a plurality of encoded frames, and calculates the first QP by adding the modified QP offset to a base QP. Encoding device.

[0104] (Appendix 2) The first encoding means a calculation means (212a) for calculating an image quality index of a reconstructed image for two or more predetermined frames in a group of encoded frames including the plurality of encoded frames, and for calculating a difference between the image quality indexes for the two or more frames; and determining means (212b) for determining the modified QP offset to be applied to at least one frame included in the current group of frames to be encoded in response to the difference. 2. An encoding device according to claim 1.

[0105] (Appendix 3) The calculation means calculates the difference between the image quality index of an I frame included in the group of encoded frames and the image quality index of a predetermined frame among frames other than the I frame included in the group of encoded frames. 3. An encoding device according to claim 2.

[0106] (Appendix 4) The predetermined frame is a frame other than the I frame included in the group of encoded frames, the frame having the largest QP offset. 4. An encoding device according to claim 2 or 3.

[0107] (Appendix 5) The determining means determines the modified QP offset to be applied to at least an I-frame of the current group of frames. 5. An encoding device according to any one of claims 2 to 4.

[0108] (Appendix 6) A computer is caused to function as the encoding device according to any one of claims 1 to 5. program. [Explanation of symbols]

[0109] 1: Encoding device 1A: First encoding means 1B: Second encoding means 2: Decryption device 10: Encoding section 20: Control section 21: First encoding control unit 22: Second encoding control unit 100: Block division section 110: Subtraction section 120: Transformation and quantization section 121: Conversion section 122: Quantization section 130: Entropy coding unit 140: Inverse quantization and inverse transformation unit 141: Inverse quantization section 142: Inverse conversion section 150: Composite section 160:Memory 170: Prediction Department 171: Inter prediction section 172: Intra prediction unit 173: Switching section 211: Base QP determination unit 212: QP offset control section 212a: Index difference calculation unit 212b: Modified QP offset determination unit 213: Addition section 221: Bit quantity allocation section 222: 2nd QP decision section

Claims

1. An encoding device that performs an encoding process using a quantization parameter (QP) on an input video and outputs a bitstream, a first encoding means for performing an encoding trial using a first QP that is a provisional QP, and deriving a correspondence relationship between the first QP and a consumed bit amount for each frame; a second encoding means for performing the encoding process using a second QP that is the QP optimized based on the derived correspondence relationship; The first encoding means, in the encoding trial, modifies a QP offset to be applied to a frame to be encoded according to encoding results of a plurality of encoded frames, and calculates the first QP by adding the modified QP offset to a base QP. Encoding device.

2. The first encoding means a calculation means for calculating an image quality index of a reconstructed image for two or more predetermined frames in a group of encoded frames including the plurality of encoded frames, and calculating a difference between the image quality indexes for the two or more frames; and determining means for determining, in response to the difference, the modified QP offset to be applied to at least one frame in a current group of frames to be encoded. The encoding device according to claim 1 .

3. The calculation means calculates the difference between the image quality index of an I frame included in the group of encoded frames and the image quality index of a predetermined frame among frames other than the I frame included in the group of encoded frames. The encoding device according to claim 2.

4. The predetermined frame is a frame other than the I frame included in the group of encoded frames, the frame having the largest QP offset. The encoding device according to claim 3.

5. The determining means determines the modified QP offset to apply to at least an I frame of the current group of frames. An encoding device according to any one of claims 2 to 4.

6. A computer is caused to function as the encoding device according to claim 1. program.

Images