Video encoding device, video decoding device, video encoding method, video decoding method, and video system

Abstract

A video encoding device 10 includes a prediction mode determination unit 11 that determines a prediction mode for intra prediction on the basis of an amount of code and encoding distortion. The prediction mode determination unit 11 includes a code amount estimation unit 12 that estimates the amount of code generated for an encoding target block when the intra prediction is performed without using encoding information of a block adjacent to the encoding target block.

Description

[Technical Field]

[0001] The present invention relates to a video encoding device, a video decoding device, a video encoding method, a video decoding method, and a video system for encoding or decoding a video signal. [Background technology]

[0002] Non-Patent Document 1 discloses a video coding method called VVC (Versatile Video Coding).

[0003] In the video coding method described in Non-Patent Document 1, each frame of a digitized video is divided into coding tree units (CTUs), and each CTU is coded in raster scan order.

[0004] Each CTU is divided into coding units (CUs) using a quad-tree (QT) structure or a multi-type tree (MMT) structure and then coded. In division using a quad-tree structure, a block is divided into equal parts horizontally and vertically. In division using a multi-type tree structure, a block is divided into two or three parts horizontally or vertically.

[0005] Each CU is divided into prediction units (PUs) and predictively coded. Predictive coding includes intra prediction and motion compensation prediction (hereinafter referred to as inter prediction). The prediction error of each PU is transform coded based on frequency transformation.

[0006] Intra prediction is a prediction that generates a predicted image from a reconstructed image that has the same display time as the frame to be coded. Non-Patent Document 1 defines 65 types of angular intra prediction as shown in FIG. 14. In FIG. 14, the numbers indicate prediction mode numbers. In angular intra prediction, an intra prediction signal is generated by extrapolating reconstructed pixels around the block to be coded in one of 65 directions. In addition to angular prediction, Non-Patent Document 1 also defines DC prediction, which averages reconstructed pixels around the block to be coded, and planar prediction, which linearly interpolates reconstructed pixels around the block to be coded. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] "Versatile video coding", Recommendation ITU-T H.266 (08 / 2020) Summary of the Invention [Problem to be solved by the invention]

[0008] 15 is an explanatory diagram illustrating the dependency of intra prediction of a current block on the prediction mode of an adjacent block above the current block and the prediction mode of an adjacent block to the left of the current block. In FIG. 15, arrows indicate the dependency.

[0009] Due to the above-described dependency, it is not possible to perform the calculation of the coding cost in units of CUs in parallel, and as a result, it is not possible to perform the intra prediction in units of CUs in parallel.

[0010] Note that if the prediction direction selection unit 205 does not calculate the amount of coding, i.e., if the amount of coding calculation unit 211 is removed, the calculation of the coding cost can be performed in parallel. However, if the prediction direction selection unit 205 is configured in this way, the coding efficiency is calculated based only on the coding distortion. As a result, the accuracy of selecting a prediction mode for the block to be coded decreases. In other words, the coding efficiency by intra prediction decreases.

[0011] The present invention aims to provide a video encoding device, a video decoding device, a video encoding method, a video decoding method, and a video system that are capable of performing intra prediction in parallel on a CU-by-CU basis while preventing a decrease in encoding efficiency. [Means for solving the problem]

[0012] A video encoding device according to the present invention includes a prediction mode determination means for determining a prediction mode for intra prediction based on code amount and encoding distortion, and the prediction mode determination means includes code amount estimation means for estimating the code amount generated for a block to be encoded when intra prediction is performed without using encoding information of blocks adjacent to the block to be encoded.

[0013] The video decoding device according to the present invention includes a correspondence storage means in which a relationship between a prediction mode determined based on the amount of code generated for a block to be coded when intra prediction is performed, the amount of code generated being estimated without using coding information for blocks adjacent to the block to be coded, and a syntax element related to intra prediction is set, and a prediction mode identification means for identifying a prediction mode based on the relationship set in the correspondence storage means.

[0014] The video encoding method according to the present invention determines the prediction mode for intra prediction based on the code amount and encoding distortion, and when determining the prediction mode, estimates the code amount generated for a block to be encoded when intra prediction is performed without using encoding information for blocks adjacent to the block to be encoded.

[0015] The video decoding method according to the present invention identifies a prediction mode based on a relationship set in a correspondence storage means in which a relationship between a prediction mode determined based on a generated code amount of a block to be coded when intra prediction is performed, the generated code amount being estimated without using coding information of blocks adjacent to the block to be coded, and a syntax element related to intra prediction is set.

[0016] A video system according to the present invention includes the above video encoding device and the above video decoding device. [Effects of the Invention]

[0017] According to the present invention, it is possible to prevent a decrease in coding efficiency and perform intra prediction in units of CUs in parallel. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a block diagram illustrating an example of the configuration of a video encoding device. [Figure 2] FIG. 10 is a block diagram illustrating an example configuration of a prediction direction selection unit that selects a prediction direction. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of a prediction direction selection unit in the first embodiment. [Figure 4] FIG. 2 is an explanatory diagram showing the relationship between a prediction mode and an estimated code amount in the first embodiment. [Figure 5] 10 is a flowchart illustrating an operation of a prediction direction selection unit. [Figure 6] FIG. 10 is an explanatory diagram showing the relationship between a prediction mode and an estimated code amount in the second embodiment. [Figure 7] FIG. 11 is a block diagram showing an example of the configuration of a prediction direction selection unit in the third embodiment. [Figure 8] 10 is a flowchart showing the operations of a prediction direction storage unit and a prediction direction update unit. [Figure 9] FIG. 1 is a block diagram illustrating an example of the configuration of a video decoding device. [Figure 10] FIG. 1 is a block diagram illustrating an example of a video system. [Figure 11] FIG. 1 is a block diagram illustrating an example of a computer having a CPU. [Figure 12] 1 is a block diagram showing the main parts of a video encoding device; [Figure 13] FIG. 2 is a block diagram showing the main parts of a video decoding device. [Figure 14] FIG. 10 is an explanatory diagram showing an example of 65 types of angular intra prediction. [Figure 15] FIG. 10 is an explanatory diagram showing the dependency relationship of intra-prediction of a current block to be coded. DETAILED DESCRIPTION OF THE INVENTION

[0019] Fig. 1 is a block diagram showing an example configuration of a video encoding device capable of performing the video encoding described in Non-Patent Document 1. The video encoding device 100 shown in Fig. 1 includes a subtractor 101, a transformer / quantizer 102, a dequantizer / inverse transformer 103, an entropy encoder 104, an adder 105, a block memory 106, an in-loop filter 107, a frame memory 108, and a prediction unit 110. The prediction unit 110 includes an intra predictor 111 and an inter predictor 112.

[0020] Note that the video encoding device shown in FIG. 1 performs encoding processing based on the VVC standard, but may also perform encoding processing based on other standards, such as the HEVC standard, the H.264 / AVC standard, or the MPEG (Moving Picture Experts Group)-2 standard.

[0021] The subtractor 101 subtracts a prediction signal from an input signal (input pixel values) for each block (e.g., PU) to generate a prediction error signal. The prediction error signal is also called a prediction residual or a prediction residual signal. Note that the size of a PU is often set to be equal to the size of a CU, so hereinafter, it is assumed that a block is a CU.

[0022] The transformer / quantizer 102 frequency-transforms the prediction error signal to obtain transform coefficients. The transformer / quantizer 102 further quantizes the frequency-transformed prediction error signal (transform coefficients). Hereinafter, the quantized transform coefficients are referred to as transformed and quantized values.

[0023] The entropy encoder 104 entropy-encodes prediction parameters (encoding information) and the transformed and quantized values. The prediction parameters are information related to the prediction of CTUs (Coding Tree Units) and blocks, such as the prediction mode (intra prediction, inter prediction), intra prediction block size, intra prediction direction, inter prediction block size, and motion vector determined by the prediction unit 110. The entropy-encoded data supplied from the entropy encoder 104 is multiplexed with other data and output as a bitstream.

[0024] The inverse quantization / inverse transformer 103 inversely quantizes the transformed and quantized values ​​to restore the transform coefficients. Furthermore, the inverse quantization / inverse transformer 103 performs inverse frequency transform on the dequantized transform coefficients to restore a prediction error signal. The adder 105 adds the restored prediction error signal and the prediction signal to generate a reconstructed signal (reconstructed image). The block memory 106 and the in-loop filter 107 receive the reconstructed signal.

[0025] The block memory 106 is a buffer memory for storing reference blocks in the picture to be coded.

[0026] The in-loop filter 107 performs appropriate filtering, for example, by using a deblocking filter, a sample adaptive offset filter, and an adaptive loop filter. The reconstructed signal filtered by the in-loop filter 107 is stored in a frame memory 108.

[0027] The intra predictor 111 performs intra prediction on the current block to be coded by referring to a reference block in the frame memory 108 that is different from the current block to be coded, and generates a prediction signal (in this case, an intra prediction signal).

[0028] The intra predictor 111 determines the prediction direction for the block to be coded. When the VVC method is used, the intra predictor 111 selects the prediction direction with the best coding efficiency from among 67 types of prediction directions.

[0029] 2 is a block diagram showing a typical configuration example of a prediction direction selection unit 205 that selects a prediction direction. In the example shown in FIG. 2, the prediction direction selection unit 205 includes a code amount calculation unit 211, a distortion calculation unit 212, and a prediction direction determination unit 213.

[0030] The code amount calculation unit 211 calculates code amount costs (hereinafter referred to as code amounts) corresponding to the respective generated code amounts when the current block is coded using each of a plurality of prediction directions. The distortion calculation unit 212 calculates the respective coding distortions when the current block is coded using each of a plurality of prediction directions. The prediction direction determination unit 213 uses each code amount R and each coding distortion D to calculate an RD (Rate-Distortion) cost expressed by, for example, the following equation (1). In the following equation (1), λ is a Lagrange multiplier. The prediction direction determination unit 213 then determines the prediction direction corresponding to the smallest RD cost as the prediction direction that maximizes coding efficiency.

[0031] Cost=D+λR (1)

[0032] The distortion calculation unit 212 calculates the distortion based on, for example, the difference between the input signal (input image) and the reconstructed image.

[0033] As a method for calculating the code amount, for example, a method of calculating the code amount based on syntax is conceivable. In the VVC method, the following syntax elements are used as syntax elements related to intra prediction. Note that the VVC method may use MPM (Most Probable Mode). MPM is a mechanism for allocating a small code amount to a prediction mode (prediction direction) that is considered to have a high probability of being used in the coding target block.

[0034] intra_luma_mpm_flag[x0][y0] ·intra_luma_not_planar_flag[x0][y0] intra_luma_mpm_idx[x0][y0] ·intra_luma_mpm_remainder[x0][y0]

[0035] intra_luma_mpm_flag[x0][y0] is a flag indicating whether the prediction mode corresponds to one of the prediction modes included in the MPM list used by the MPM. Multiple prediction modes are set in the MPM list. The prediction mode set in the MPM list is set based on the prediction mode of the block adjacent to the upper side of the block to be coded and the prediction mode of the block adjacent to the left side. When intra_luma_mpm_flag[x0][y0] is "1", it indicates that the prediction mode is a prediction mode set in the MPM list.

[0036] Intra_luma_not_planar_flag[x0][y0] is a flag indicating whether the prediction mode is planar. When intra_luma_not_planar_flag[x0][y0] is "1", this indicates that the prediction mode is not planar.

[0037] intra_luma_mpm_idx[x0][y0] is an index indicating a candidate number in the MPM list when the prediction mode is MPM.

[0038] Intra_luma_mpm_remainder[x0][y0] indicates the prediction mode when MPM is not used, that is, the prediction mode not set in the MPM list, which prediction mode number is the prediction mode. Hereinafter, the prediction mode when MPM is not used (the prediction mode not set in the MPM list) may be referred to as the non-MPM prediction mode.

[0039] When the coding amount is expressed in bits, the number of bits of intra_luma_mpm_flag[x0][y0] and intra_luma_not_planar_flag[x0][y0] is a constant value (specifically, 1 bit). Intra_luma_mpm_idx[x0][y0] and intra_luma_mpm_remainder[x0][y0] are variable values ​​according to the dependency relationship. Intra prediction of the current block to be coded uses decoded reconstructed images of blocks located above and to the left of the current block as reference signals. Therefore, the intra prediction of the current block to be coded is dependent on the prediction modes of the blocks adjacent to the current block above and to the left.

[0040] Therefore, the prediction modes of the blocks located above and to the left of the current block are also taken into consideration when determining the amount of code generated for the current block.

[0041] The prediction modes of the blocks located above and to the left of the block to be coded are expressed as follows: dir_(x0-1,y0) dir_(x0,y0-1)

[0042] dir_(x0-1, y0) represents the prediction mode (prediction direction) of the adjacent block on the left side of the current block to be coded. dir_(x0, y0-1) represents the prediction mode (prediction direction) of the adjacent block on the above side of the current block to be coded.

[0043] Based on the above considerations, the code amount Code(dir) (dir: 0 to 66) related to the prediction mode can be expressed as in the following equation (2): F1 denotes a function with the parameters in parentheses as variables.

[0044] Code(dir) = F1(A(x0,y0), B(x0,y0), C(x0,y0), D(x0,y0), dir_(x0-1,y0), dir_(x0,y0-1)) ···(2)

[0045] In equation (2), A(x0,y0) is intra_luma_mpm_flag[x0][y0]. B(x0,y0) is intra_luma_not_planar_flag[x0][y0]. C(x0,y0) is intra_luma_mpm_idx[x0][y0]. D(x0,y0) is intra_luma_mpm_remainder[x0][y0]. Hereinafter, A(x0,y0), B(x0,y0), C(x0,y0), and D(x0,y0) may be written as A, B, C, and D. Therefore, the correspondence between A, B, C, and D and syntax elements is as follows:

[0046] ·A:intra_luma_mpm_flag[x0][y0] ·B:intra_luma_not_planar_flag[x0][y0] ·C:intra_luma_mpm_idx[x0][y0] ·D:intra_luma_mpm_remainder[x0][y0]

[0047] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0048] Embodiment 1. 3 is a block diagram showing an example of the configuration of a prediction direction selection unit that selects a prediction direction in the first embodiment. Note that the overall configuration of the video encoding device is the same as the configuration shown in FIG.

[0049] 3 , the prediction direction selecting unit 200 includes a code amount estimating unit 210, a distortion calculating unit 212, and a prediction direction determining unit 213. The prediction direction selecting unit 200 is provided, for example, in the intra predictor 111. The prediction direction selecting unit 200 may also be provided outside the intra predictor 111.

[0050] The code amount estimation unit 210 estimates the amount of code generated when a current block is coded using each of a plurality of prediction directions. The distortion calculation unit 212 calculates the coding distortion when a current block is coded using each of a plurality of prediction directions, similar to the example shown in Fig. 2. The prediction direction determination unit 213 calculates the RD cost expressed by, for example, the above formula (1) using each code amount and each coding distortion, similar to the example shown in Fig. 2.

[0051] The code amount estimation unit 210 does not consider the dependency of prediction modes when estimating the code amount for each prediction mode. Therefore, the code amount Code(dir) for a prediction mode can be determined, for example, as shown in the following equation (3). F2 denotes a function with the parameters in parentheses as variables. Note that the code amount represented by F2 corresponds to the estimated code amount (estimated code amount). As shown in equation (3), when determining the prediction mode to be used in intra prediction, prediction parameters (e.g., prediction direction) of blocks adjacent to the block to be coded are not used.

[0052] Code(dir) = F2(A(x0,y0), B(x0,y0), C(x0,y0), D(x0,y0)) ···(3)

[0053] When the code amount Code(dir) is expressed as the number of bits that can represent the syntax elements (A, B, C, D) corresponding to each prediction mode, it can be expressed as follows: When determining the code amount Code(dir), both syntax elements related to MPM and syntax elements related to non-MPM are taken into consideration.

[0054] ·Code(0)=A+B=2 Code(1)=A+B+C=3 or A+D=2 ·Code(dir)(dir: other than 0, 1)=A+D=1+ceil(log2(dir+1)) Ceil means round up to the nearest integer.

[0055] If the number of bits that can represent a syntax element in MPM differs from the number of bits that can represent a syntax element in non-MPM, the larger number of bits is used. That is, the code amount estimation unit 210 uses the larger of multiple values ​​that can represent the code amount (the expected maximum value). In the above example, for example, 3 is used for Code(1). Note that the code amount estimation unit 210 may use any value instead of using the expected maximum value.

[0056] Fig. 4 is an explanatory diagram showing the relationship between prediction modes and estimated code amounts based on the above-mentioned concept. The code amounts shown in Fig. 4 are stored in advance as a code amount table corresponding to prediction modes in the prediction direction selection unit 200, for example. The code amount estimation unit 210 estimates the generated code amount by obtaining the code amount corresponding to each prediction mode from the code amount table.

[0057] FIG. 5 is a flowchart showing the operation of the prediction direction selecting unit 200.

[0058] In the prediction direction selection unit 200, the prediction direction determination unit 213 sets an initial value in an RD cost buffer (step S101). Furthermore, the prediction direction determination unit 213 sets a variable i to 0 (step S102). The RD cost buffer is provided in the prediction direction selection unit 200. The initial value is, for example, the maximum value that can be set in the RD cost buffer.

[0059] The code amount estimation unit 210 obtains the code amount corresponding to dir(i) from the code amount table (step S103). The distortion calculation unit 212 calculates the coding distortion (step S104). Note that the distortion calculation unit 212 may use any known method to calculate the coding distortion.

[0060] The prediction direction determination unit 213 calculates the RD cost using the above formula (1) (step S105). The prediction direction determination unit 213 compares the calculated RD cost with the RD cost set in the RD cost buffer (step S105). If the calculated RD cost is equal to or greater than the RD cost set in the RD cost buffer, the process proceeds to step S108 (step S106).

[0061] If the calculated RD cost is smaller than the RD cost set in the RD cost buffer, the prediction direction determination unit 213 updates the RD cost set in the RD cost buffer with the RD cost calculated in the process of step S105 (step S107). In addition, the prediction direction determination unit 213 stores the value of the variable i at that time in, for example, the RD cost buffer.

[0062] Next, the prediction direction determining unit 213 checks whether the value of the variable i has reached 66 (step S108). If the value of the variable i has not reached 66, the prediction direction determining unit 213 increments the value of the variable i by 1 (step S109). Then, the process returns to step S103.

[0063] If the value of the variable i has reached 66, the prediction direction determining unit 213 sets the prediction mode indicated by the value of the variable i stored in the RD cost buffer as the prediction mode to be used in intra prediction (step S110).

[0064] Through the above process, the prediction direction determination unit 213 can determine the prediction mode to be used in intra prediction performed in the encoding process. When determining the prediction mode, the prediction direction selection unit 200 does not use dependency on the prediction modes of adjacent blocks. Therefore, the prediction direction selection unit 200 can perform coding cost calculations in parallel on a block-by-block basis. As a result, the intra predictor 111 can perform intra predictions in parallel on a CU-by-CU basis. Furthermore, the prediction direction selection unit 200 does not consider dir_(x0-1, y0) and dir_(x0, y0-1), but estimates the code amount Code(dir) based on the syntax elements (A, B, C, D) corresponding to each prediction mode, so that the encoding efficiency does not decrease significantly.

[0065] Embodiment 2. The overall configuration of the video encoding device of the second embodiment is the same as the configuration shown in Fig. 1. The configuration of the prediction direction selection unit in the second embodiment is the same as the configuration shown in Fig. 3.

[0066] In the second embodiment, the prediction direction selection unit 200 also basically adopts the larger number of bits (expected maximum value) of multiple values ​​that can represent the number of bits that can represent syntax elements for each prediction mode. However, in the second embodiment, the prediction direction selection unit 200 reduces the amount of code (number of bits) for prediction modes that frequently appear in general intra prediction (prediction modes that are frequently used in actually performed intra prediction) compared to the amount of code in the first embodiment. Note that the "actually performed intra prediction" refers to intra prediction performed using the determined optimal prediction mode.

[0067] Examples of frequently occurring prediction modes include the following prediction modes of dir (prediction mode number):

[0068] dir=0(Planar) dir=1(DC) ·dir=50 (vertical direction) ·dir=18 (horizontal direction) dir=46 dir=54

[0069] When the code amount Code(dir) is expressed as the number of bits that can represent the syntax elements (A, B, C, D) corresponding to each prediction mode, it can be expressed as follows:

[0070] ·Code(0)=A+B=2 ·Code(1)=A+B+C=3 ·Code(50)=A+B+C=4 ·Code(18)=A+B+C=5 ·Code(46)=A+B+C=6 ·Code(54)=A+B+C=7 Code(dir) (dir: other than the above) = A + D = 1 + ceil(log2(dir+1))

[0071] Fig. 6 is an explanatory diagram showing the relationship between prediction modes and estimated code amounts in the second embodiment. As in the first embodiment, the code amounts shown in Fig. 6 are stored in advance in a code amount table in association with prediction modes in the prediction direction selection unit 200. The code amount estimation unit 210 obtains the code amount corresponding to each prediction mode from the code amount table.

[0072] Comparing the relationship shown in Fig. 6 with the relationship in the first embodiment shown in Fig. 4, the amount of code corresponding to dir=50, dir=18 (horizontal direction), and dir=46 is particularly small. Note that the MPM is used to allocate the number of bits to frequently occurring prediction modes. That is, prediction modes other than dir=0 (dir=1, dir=50, dir=18, dir=46, and dir=54) are specified by C.

[0073] In the second embodiment, the prediction direction selection unit 200 also performs the process exemplified in the flowchart of FIG. 5. However, in the second embodiment, the amount of code (estimated amount of code) for a prediction mode that frequently appears in intra prediction is smaller than in the first embodiment. Therefore, compared to the first embodiment, a prediction mode that frequently appears in general intra prediction is more likely to be selected. As a result, it is possible to improve the coding efficiency that may be slightly reduced by determining a prediction mode using the estimated amount of code, i.e., the coding efficiency in the first embodiment.

[0074] It should be noted that the frequently occurring prediction modes are not limited to the above examples, and a set of other prediction modes may be set as the frequently occurring prediction modes.

[0075] Embodiment 3. In the third embodiment, the relationship between the prediction mode and the estimated code amount is determined using the distribution of prediction directions from adjacent blocks in the intra prediction of the current block to be coded that was previously performed.

[0076] 7 is a block diagram showing an example of the configuration of a prediction direction selection unit that selects a prediction direction in the third embodiment. Note that the overall configuration of the video encoding device is the same as the configuration shown in FIG.

[0077] Compared to the prediction direction selection unit 200 shown in FIG. 3, the prediction direction selection unit 201 shown in FIG. 7 additionally includes a prediction direction storage unit 214 and a prediction direction update unit 215.

[0078] The operations of the prediction direction storage unit 214 and the prediction direction update unit 215 will be described with reference to the flowchart of Fig. 8. Note that the prediction direction selection unit 201 determines the optimal prediction mode through processing according to the flowchart illustrated in Fig. 5.

[0079] In the following description, it is assumed that the prediction direction selection unit 201 determines a frequently occurring prediction mode during actual operation of the video encoding device. However, the prediction direction selection unit 201 may determine a frequently occurring prediction mode before the video encoding device is put into operation. That is, before the video encoding device is put into operation, the prediction direction selection unit 201 may experimentally execute the process shown in FIG. 8.

[0080] The prediction direction storage unit 214 stores the number of times that a prediction mode determined as an optimal prediction mode (the prediction mode determined in the processing of step S110) has been determined. That is, the prediction direction storage unit 214 stores the number of times that a prediction mode used in intra prediction has been used across multiple intra predictions that are actually performed. Specifically, when intra prediction is performed, the prediction direction storage unit 214 increments, for example, the value of a counter corresponding to the prediction mode used in the intra prediction by 1 (step S201). The number of counters provided corresponds to the number of prediction modes. When a predetermined number of prediction modes have been stored in the prediction direction storage unit 214, that is, when a predetermined number of intra predictions have been performed, the prediction direction update unit 215 extracts multiple (e.g., six) prediction modes from the prediction direction storage unit 214 in descending order of frequency of use (steps S202 and S203).

[0081] Assume that the prediction direction selecting unit 201 stores the code amount table illustrated in FIG. 6. In this case, the prediction direction updating unit 215 updates the content of the code amount table with the extracted multiple prediction modes (step S204). Specifically, the prediction direction updating unit 215 assigns a code amount to each prediction mode extracted from the prediction direction storage unit 214 in the process of step S203. The prediction direction updating unit 215 assigns a code amount to a prediction mode based on, for example, the same concept as the way of setting the code amount set in the code amount table illustrated in FIG. 6. That is, the prediction direction updating unit 215 reuses, for example, MPM. Then, the prediction direction updating unit 215 sets the code amount together with the prediction mode in the code amount table. Note that the previous code amount and prediction mode in the code amount table are deleted. Furthermore, a smaller number of bits is set as the code amount to be newly set in the code amount table, similar to the frequently occurring prediction mode in the second embodiment.

[0082] When the process shown in Fig. 8 is performed, the intra predictor 111 may perform intra prediction of a general VVC system on the current block to be coded. That is, the intra predictor 111 may perform intra prediction in an optimal prediction mode determined using the function of the code amount calculation unit 211 shown in Fig. 2. In other words, when the process of steps S201 to S203 is performed, the optimal prediction mode may be determined using a general method that takes into account the dependency of the prediction modes. In this case, the prediction direction selection unit 201 also has the function of the code amount calculation unit 211 shown in Fig. 2.

[0083] After the process of step S204 is performed, the prediction direction selection unit 201 performs the same process as in the second embodiment. That is, the prediction direction selection unit 201 estimates the code amount using a code amount table without using the dependency relationship between prediction modes.

[0084] In the third embodiment, too, a frequently occurring prediction mode is more likely to be selected. As a result, it is possible to improve the coding efficiency that may be slightly reduced by determining a prediction mode using an estimated code amount, that is, the coding efficiency in the first embodiment. Furthermore, in the third embodiment, a frequently occurring prediction mode is more likely to be selected, so that it is expected that the coding efficiency will be further improved.

[0085] In the above example, the prediction direction selection unit 201 stores the code amount table illustrated in Fig. 6 in advance. However, the code amount table illustrated in Fig. 4 may be stored instead. Furthermore, the prediction direction selection unit 201 may not store the code amount table before the process according to the flowchart illustrated in Fig. 8 is started. In this case, the prediction direction update unit 215 does not update the code amount table, but creates the code amount table.

[0086] Furthermore, after the prediction direction selection unit 201 updates or creates the code amount table, the process shown in FIG. 8 may be performed again after a predetermined period of time has elapsed or after a predetermined number of intra predictions have been performed.

[0087] Furthermore, the code amount estimation unit 210 may estimate the generated code amount based on a pre-analysis of each image constituting the video to be coded. The pre-analysis is a process of analyzing, before the video coding device is put into operation, for example, which prediction mode number is likely to be determined as the prediction mode for the block to be coded in the video signal to be coded. Note that "pre-analysis" refers to an analysis that is performed before the video coding device is actually put into operation.

[0088] Embodiment 4. Fig. 9 is a block diagram showing an example configuration of a video decoding device. The video decoding device 300 shown in Fig. 9 includes an entropy decoder 301, an inverse quantization / inverse transformer 302, an adder 303, a prediction unit 310, and a buffer 304. The prediction unit 310 includes an intra predictor 311, an inter predictor 312, and a corresponding storage unit 313. The video decoding device shown in Fig. 9 can receive a bitstream from, for example, the video encoding device according to any one of the first to third embodiments.

[0089] The entropy decoder 301 entropy-decodes the entropy-coded data. The entropy-coded data includes data in which prediction parameters are entropy-coded and data in which transformed and quantized values ​​are entropy-coded. The entropy decoder 301 supplies the entropy-decoded transformed and quantized values ​​to the inverse quantization / inverse transformer 302. The entropy decoder 301 also supplies the entropy-decoded prediction parameters to the prediction unit 310.

[0090] The inverse quantization / inverse transformer 302 inversely quantizes the transformed quantized values, and further inversely frequency transforms the inversely quantized frequency transform coefficients.

[0091] In the prediction unit 310, the correspondence storage unit 313 stores a relationship between a prediction mode determined based on the amount of code generated for the block to be coded when intra prediction is performed, the amount of code being estimated without using coding information for blocks adjacent to the block to be coded, and syntax elements related to intra prediction. That is, the relationship between [A+B], [A+B+C], or [A+D] (see FIGS. 4 and 6) used to calculate the amount of code in the first to third embodiments and a prediction mode number is stored in advance in the correspondence storage unit 313.

[0092] The prediction unit 304 generates a prediction signal for each block based on the prediction parameters. If the prediction parameters indicate intra prediction, the intra predictor 311 generates the prediction signal. If the prediction parameters indicate inter prediction, the inter predictor 312 generates the prediction signal.

[0093] The intra predictor 311 identifies a prediction mode based on the prediction parameters (specifically, syntax related to intra prediction) and the relationship set in the correspondence storage unit 313.

[0094] The prediction error signal that has been inverse frequency transformed by the inverse quantization / inverse transformer 302 is added by an adder 303 to a prediction signal supplied from a prediction unit 310, and the resultant signal is supplied as a reconstructed image to a buffer 304. The buffer 304 stores the reconstructed images. The reconstructed images stored in the buffer 304 are output as decoded video in display order.

[0095] Since the video decoding device 300 is configured and operates as described above, interoperability between the video encoding devices and the video decoding device according to the first to third embodiments is ensured.

[0096] Embodiment 5. Fig. 10 is a block diagram showing an example of a video system, in which the video encoding device according to any one of the first to third embodiments and the video decoding device 300 shown in Fig. 9 are connected via a wireless or wired transmission path 400.

[0097] The configuration and operation of video encoding device 100 are the same as those of the video encoding device shown in Fig. 1. The configuration and operation of video decoding device 300 are the same as those of video decoding device 300 shown in Fig. 9.

[0098] In the above embodiment, a video encoding device that performs encoding processing based on the VVC standard and a video decoding device that performs decoding processing based on the VVC standard are used as examples, but the video encoding device and the video decoding device may also perform encoding processing and decoding processing based on other standards such as the HEVC standard or the H.264 / AVC standard.

[0099] Furthermore, each of the above embodiments can be configured by hardware, but can also be realized by a computer program.

[0100] Fig. 11 is a block diagram showing an example of a computer having a CPU (Central Processing Unit). The computer can be implemented in both the video encoding device 100 shown in Fig. 1 and the video decoding device 300 shown in Fig. 9. The CPU 1000 executes processing in accordance with a program (a video encoding program or a video decoding program) stored in a storage device 1001, thereby realizing the functions of each of the above-described embodiments.

[0101] That is, when the computer is implemented in the video encoding device 100 shown in FIG. 1, the CPU 1000 realizes the functions of the subtractor 101, transformer / quantizer 102, inverse quantization / inverse transformer 103, entropy encoder 104, adder 105, in-loop filter 107, and the intra predictor 111 and inter predictor 112 in the prediction unit 110 in the video encoding device 100.

[0102] When the computer is implemented in the video decoding device 300 shown in FIG. 9, the CPU 1000 realizes the functions of the entropy decoder 301, the inverse quantization / inverse transformer 302, the adder 303, and the intra predictor 311 and the inter predictor 312 in the prediction unit 310 in the video decoding device 300.

[0103] The storage device 1001 is, for example, a non-transitory computer-readable medium. The non-transitory computer-readable medium includes various types of tangible storage media. Specific examples of non-transitory computer-readable media include magnetic recording media (e.g., hard disks), CD-ROMs (Compact Disc-Read Only Memory), CD-Rs (Compact Disc-Recordable), CD-R / Ws (Compact Disc-Rewritable), and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), and flash ROMs).

[0104] The program may also be stored in various types of transitory computer-readable media, to which the program is supplied, for example, via a wired or wireless communication path, i.e., via an electrical signal, an optical signal, or an electromagnetic wave.

[0105] The memory 1002 is realized by, for example, a RAM (Random Access Memory). The memory 1002 is a storage means for temporarily storing data when the CPU 1000 executes processing. A configuration is also conceivable in which a program held in the storage device 1001 or a temporary computer-readable medium is transferred to the memory 1002, and the CPU 1000 executes processing based on the program in the memory 1002.

[0106] The memory 1002 can also be used as the block memory 106 and frame memory 108 shown in Fig. 1. The memory 1002 can also be used as the prediction direction storage unit 214 shown in Fig. 7. The memory 1002 can also be used as the buffer 304 and correspondence storage unit 313 shown in Fig. 9.

[0107] Fig. 12 is a block diagram showing the main components of a video encoding device. The video encoding device 10 shown in Fig. 12 includes a prediction mode determination unit 11 (prediction mode determination means: in the embodiment, this is realized by prediction direction selection units 200 and 201) that determines a prediction mode for intra prediction based on the code amount and encoding distortion, and the prediction mode determination unit 11 includes a code amount estimation unit 12 (code amount estimation means: in the embodiment, this is realized by the code amount estimation unit 210) that estimates the code amount generated for a block to be encoded when intra prediction is performed without using encoding information for blocks adjacent to the block to be encoded.

[0108] The prediction mode determination unit 11 may include a frequent prediction mode identification unit (frequent prediction mode identification means: in an embodiment, realized by the prediction direction memory unit 214 and the prediction direction update unit 215) that identifies a frequent prediction mode from among the prediction modes used for blocks encoded by each of multiple past intra predictions.

[0109] Fig. 13 is a block diagram showing main components of a video decoding device. The video decoding device 20 shown in Fig. 13 includes a correspondence storage unit 21 (correspondence storage means: in the embodiment, implemented by the correspondence storage unit 313) in which a relationship between a prediction mode determined based on the amount of code generated for a block to be coded when intra prediction is performed, which is estimated without using coding information of blocks adjacent to the block to be coded, and syntax elements related to intra prediction is set, and a prediction mode identification unit 22 (prediction mode identification means: in the embodiment, implemented by the intra predictor 311) that identifies a prediction mode based on the relationship set in the correspondence storage unit 21.

[0110] Although part or all of the above-described embodiments can be described as follows, the present invention is not limited to the following configurations.

[0111] (Supplementary Note 1) A video encoding device having at least an intra-prediction function, a prediction mode determination means for determining a prediction mode of intra prediction based on an amount of code and coding distortion; The prediction mode determination means includes code amount estimation means for estimating the amount of code generated for a block to be coded when intra prediction is performed without using coding information of blocks adjacent to the block to be coded. Video encoding device.

[0112] (Note 2) The code amount estimation means estimates the generated code amount based on the number of bits that can express a syntax element related to a prediction mode. 2. A video encoding device according to claim 1.

[0113] (Appendix 3) A video encoding device that performs intra prediction using the VCC method, The code amount estimation means estimates the generated code amount based on the number of bits that can represent syntax elements intra_luma_mpm_flag, intra_luma_not_planar_flag, intra_luma_mpm_idx, and intra_luma_mpm_remainder. 3. A video encoding device according to claim 2.

[0114] (Note 4) The code amount estimation means reduces the number of bits that can represent syntax elements related to frequently occurring prediction modes. 4. A video encoding device according to claim 2 or 3.

[0115] (Supplementary Note 5) The prediction mode determination means includes a frequent prediction mode identification means for identifying a frequently occurring prediction mode from prediction modes used for blocks coded by each of a plurality of past intra predictions. 5. A video encoding device according to claim 4.

[0116] (Supplementary Note 6) The prediction mode determination means includes a frequent prediction mode identification means for analyzing in advance which prediction mode number is likely to be determined as a prediction mode for a block to be coded in a video signal to be coded, and identifying a frequently occurring prediction mode based on the analysis result. 5. A video encoding device according to claim 4.

[0117] (Supplementary Note 7) A video decoding device having at least an intra-prediction function, a correspondence storage means in which a relationship between a prediction mode determined based on a generated code amount of a block to be coded when intra prediction is performed, the generated code amount being estimated without using coding information of blocks adjacent to the block to be coded, and syntax elements related to intra prediction is set; a prediction mode specifying means for specifying a prediction mode based on the relationship set in the correspondence storage means; A video decoding device comprising:

[0118] (Appendix 8) A video coding method that performs at least intra prediction, determining a prediction mode for intra prediction based on the code amount and coding distortion; When determining the prediction mode, the amount of code generated for the block to be coded when intra prediction is performed is estimated without using coding information of blocks adjacent to the block to be coded. Video coding method.

[0119] (Appendix 9) The amount of generated code is estimated based on the number of bits that can represent the syntax element related to the prediction mode. 9. A video encoding method according to claim 8.

[0120] (Supplementary Note 10) A video coding method that performs intra prediction in a VCC system, The generated code amount is estimated based on the number of bits that can represent the syntax elements intra_luma_mpm_flag, intra_luma_not_planar_flag, intra_luma_mpm_idx, and intra_luma_mpm_remainder. 10. A video encoding method according to claim 9.

[0121] (Appendix 11) Reducing the number of bits that can represent frequently occurring syntax elements related to prediction modes 11. A video encoding method according to claim 9 or 10.

[0122] (Supplementary Note 12) Identifying a frequently occurring prediction mode from prediction modes used for blocks coded by each of multiple past intra predictions 12. A video encoding method according to claim 11.

[0123] (Supplementary Note 13) A video decoding method that performs at least intra prediction, The prediction mode is determined based on the amount of generated code of the block to be coded when intra prediction is performed, which is estimated without using coding information of blocks adjacent to the block to be coded, and the prediction mode is specified based on the relationship set in a correspondence storage means in which the relationship between the syntax elements related to intra prediction is set. Video decoding method.

[0124] (Appendix 14) To the computer, determining a prediction mode of intra prediction based on the code amount and coding distortion; When the prediction mode is determined, the amount of code generated for the block to be coded when intra prediction is performed is estimated without using coding information of blocks adjacent to the block to be coded. A video encoding program for

[0125] (Appendix 15) To the computer, The prediction mode is determined based on the amount of generated code of the block to be coded when intra prediction is executed, which is estimated without using coding information of blocks adjacent to the block to be coded, and the prediction mode is specified based on the relationship set in a correspondence storage means in which the relationship between the prediction mode and syntax elements related to intra prediction is set. Video decoding program for.

[0126] (Appendix 16) When executed on a computer, determining a prediction mode for intra prediction based on the code amount and coding distortion; When the prediction mode is determined, the amount of code generated for the block to be coded when intra prediction is performed is estimated without using coding information of blocks adjacent to the block to be coded. A non-transitory computer-readable recording medium on which a video encoding program is stored.

[0127] (Appendix 17) When executed on a computer, The prediction mode is determined based on the amount of generated code of the block to be coded when intra prediction is performed, which is estimated without using coding information of blocks adjacent to the block to be coded, and the prediction mode is specified based on the relationship set in a correspondence storage means in which the relationship between the syntax elements related to intra prediction is set. A non-transitory computer-readable recording medium storing a video decoding program.

[0128] (Supplementary Note 18) A video encoding device according to any one of Supplementary Note 1 to Supplementary Note 6; A video decoding device according to Supplementary Note 7; A video system comprising:

[0129] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0130] This application claims priority based on Japanese Patent Application No. 2021-172893, filed on October 22, 2021, the entire disclosure of which is incorporated herein by reference. [Explanation of symbols]

[0131] 10 Video Encoding Device 11 Prediction mode determination unit 12 Code amount estimator 20 Video decoder 21 Corresponding memory unit 22 Prediction mode identification unit 100 Video encoding device 101 Subtractor 102 Transform / Quantizer 103 Inverse Quantization / Inverse Transformer 104 Entropy Encoder 105 Adder 106 Block Memory 107 In-loop filter 108 Frame Memory 110 Prediction Department 111 Intra Predictor 112 Inter Predictor 200, 201 Prediction direction selection unit 210 Code amount estimation unit 211 Code amount estimator 212 Distortion Calculation Unit 213 Prediction direction determination unit 214 Prediction direction memory unit 215 Prediction direction update unit 300 Video decoder 301 Entropy Decoder 302 Inverse Quantization / Inverse Transformer 303 Adder 304 Buffer 310 Prediction Department 311 Intra Predictor 312 Inter Predictor 313 Corresponding memory unit 1000 CPU 1001 Storage device 1002 memory

Claims

1. A video encoding device having at least an intra prediction function, a prediction mode determination means for determining a prediction mode of intra prediction based on an amount of code and coding distortion; The prediction mode determination means includes code amount estimation means for estimating the amount of code generated for a block to be coded when intra prediction is performed without using coding information of blocks adjacent to the block to be coded. Video encoding device.

2. The code amount estimation means estimates the generated code amount based on the number of bits that can express a syntax element related to a prediction mode. The video encoding device according to claim 1 .

3. A video encoding device that performs intra prediction using a VVC method, The code amount estimation means estimates the generated code amount based on the number of bits that can represent syntax elements intra_luma_mpm_flag, intra_luma_not_planar_flag, intra_luma_mpm_idx, and intra_luma_mpm_remainder. The video encoding device according to claim 2 .

4. The code amount estimation means reduces the number of bits that can represent syntax elements related to frequently occurring prediction modes.

4. The video encoding device according to claim 2 or 3.

5. The prediction mode determination means includes a frequent prediction mode identification means for identifying a frequently occurring prediction mode from prediction modes used for blocks coded by each of a plurality of past intra predictions.

5. The video encoding device according to claim 4.

6. A video decoding device having at least an intra prediction function, a correspondence storage means in which a relationship between a prediction mode determined based on a generated code amount of a block to be coded when intra prediction is performed, the generated code amount being estimated without using coding information of blocks adjacent to the block to be coded, and syntax elements related to intra prediction is set; a prediction mode specifying means for specifying a prediction mode based on the relationship set in the correspondence storage means; A video decoding device comprising:

7. 1. A video coding method performing at least intra prediction, comprising: determining a prediction mode for intra prediction based on the code amount and coding distortion; When determining the prediction mode, the amount of code generated for the block to be coded when intra prediction is performed is estimated without using coding information of blocks adjacent to the block to be coded. Video coding method.

8. The amount of generated code is estimated based on the number of bits that can represent syntax elements related to the prediction mode. The video encoding method according to claim 7.

9. 1. A video decoding method that performs at least intra prediction, comprising: The prediction mode is determined based on the amount of generated code of the block to be coded when intra prediction is performed, which is estimated without using coding information of blocks adjacent to the block to be coded, and the prediction mode is specified based on the relationship set in a correspondence storage means in which the relationship between the syntax elements related to intra prediction is set. Video decoding method.

10. A video encoding device according to any one of claims 1 to 5; The video decoding device according to claim 6 , A video system comprising:

Images