Video encoding device, video decoding device, video encoding method, and video decoding method

Abstract

To avoid a situation where a probability distribution cannot be estimated.SOLUTION: A video encoding device 50 comprises: reduction means 51 that generates tensors of a plurality of reduction areas from a tensor to be encoded; a plurality of probability distribution estimation means 52 that receive input of the tensors of the plurality of reduction areas to estimate the probability distributions of the tensors; entropy encoding means 53 that subjects the tensor to be encoded and the tensors of the plurality of reduction areas to entropy encoding by using the probability distributions, and generates a bit stream; and multiplexing means 54 that multiplexes the bit stream. The entropy encoding means 53 includes reduction control means 55 that uses the probability distribution estimated from the tensor to be encoded or the tensor of the reduction area having a smaller size than the reduction area, and when the size of the tensor to be encoded is smaller than a predetermined size, controls reduction processing performed by the reduction means 51 so that one or more elements of the tensor of the reduction area are input to each of the plurality of probability distribution estimation means 52.SELECTED DRAWING: Figure 16

Description

Technical Field

[0001] The present disclosure relates to a video encoding device, a video decoding device, a video encoding method, and a video decoding method that utilize a neural network.

Background Art

[0002] A new video encoding technique that combines an auto-encoder, which is one type of neural network, quantization, and entropy encoding is described in Non-Patent Document 1.

[0003] An auto-encoder compresses input data into a low-dimensional feature tensor so as to include only important features. Then, the auto-encoder generates reconstructed data by reconstructing the low-dimensional feature tensor to the original dimension. The process of dropping into the low-dimensional feature tensor (the first half) is called encoding. The process of generating the reconstructed data (the second half) is called decoding.

[0004] The learning of the auto-encoder is advanced so as to minimize the reconstruction error (the difference between the input data and the reconstructed data). The auto-encoder is designed to impose constraints on the structure of the encoding or add a regularization term to the loss function of the network so as to obtain meaningful feature amounts.

[0005] Non-Patent Document 2 describes a method of inputting a predetermined tensor and entropy encoding the input tensor using a probability model (i.e., a probability distribution of prediction). In this method, first, the input tensor is downsampled to obtain a tensor with a scale smaller than the scale of the input tensor. A probability distribution is estimated from the obtained tensor, and the tensor is entropy encoded. Then, the input tensor is entropy encoded using the estimated probability distribution. Such a method is called MSPSM (Multi-Scale Progressive Statistical Model) entropy encoding. MSPSM entropy encoding is simply denoted as MSPSM.

[0006] Hereinafter, as a predetermined tensor, a feature tensor obtained by applying quantization to the tensor output by the encoder in the autoencoder will be taken as an example.

Prior Art Documents

Non-Patent Documents

[0007]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0008] FIG. 1 is a block diagram showing an implementation example of MSPSM. The configuration shown in FIG. 1 is taken as an MSPSM system.

[0009] The MSPSM system downsamples the input feature tensor from the encoder. The MSPSM system further downsamples the feature tensor obtained by the downsampling. The MSPSM system repeats the downsampling of the feature tensor obtained by the downsampling. The feature tensor obtained by the i-th downsampling is called the feature tensor of scale i. i is called the number of scales.

[0010] The feature tensor of scale i is y imay be expressed as. Let p be the probability distribution estimated from the feature tensor at scale i i and let z i be the context information when estimating p i .

[0011] Figure 1 shows a configuration example of the MSPSM system for the case of i = 4. The MSPSM system illustrated in Figure 1 includes downsamplers 11 to 14 corresponding to each of scales i (i = 1 to 4), probability models (probability model estimators 21 to 24) for estimating a probability distribution using the outputs of the downsamplers 11 to 14, and entropy encoders 30 to 34 for entropy encoding a feature tensor using the probability distribution estimated by the probability model estimators 21 to 24.

[0012] The downsamplers 11 to 14 downsample the input feature tensor at a predetermined interval. For example, the downsamplers 11 to 14 downsample the input feature tensor by a factor of 1 / 2. That is, the number of samples is reduced to 1 / 2 both vertically and horizontally. Specifically, for example, when the downsamplers 11 to 14 downsample by a factor of 1 / 2, they extract the lower right element of a 2×2 element (feature quantity).

[0013] The probability model estimators 21 to 23 estimate the probability model p i+2 from the feature tensor at scale (i + 1) and the context information z i+1 . For example, the probability model p i+1 is obtained by inputting the feature tensor and the context information into a convolutional neural network. Note that the probability model estimator 24 estimates the probability model p 4 from the context information with a predetermined initial value set and the feature tensor at scale (i + 1). Here, the predetermined initial value is, for example, a zero value.

[0014] The entropy encoders 31 to 33 encode the feature tensor at scale i using the probability model p i+1Entropy encoding is performed using. Note that the entropy encoder 30 uses the input feature tensor from the encoder as the probability model p 1 Entropy encoding is performed using. Also, the entropy encoder 34 performs entropy encoding on the feature tensor of scale 4 using a predetermined probability distribution.

[0015] The bitstream generator 40 uses the outputs of the entropy encoders 30 to 34 as the bitstream.

[0016] FIG. 2 is an explanatory diagram showing an example of the state of downsampling. In FIG. 2, as an example, the state in which a feature tensor of 32×32 elements is reduced is illustrated. The small rectangles in FIG. 2 correspond to the elements. The markings inside the small rectangles are made to facilitate understanding of the correspondence relationship of the elements between the scales.

[0017] The y illustrated in FIG. 2 1 is obtained by downsampling y 0 (input feature tensor). y 2 is obtained by downsampling y 1 is obtained by downsampling y 3 is obtained by downsampling y 2 is obtained by downsampling y 4 is obtained by downsampling y 3 is obtained by downsampling y

[0018] As shown in FIG. 3, each element of the feature tensor y i of scale i is entropy encoded using the probability model estimated from the feature tensor of scale (i + 1). That is, in the MSPSM, when the entropy encoders 31 to 33 encode the elements of the feature tensor of a certain scale, they use the probability distribution estimated from the feature tensor of a size smaller than that scale (for example, one size smaller). Note that the entropy encoder 34 uses the feature tensor y 4For the (minimum feature tensor), entropy coding is performed using a predetermined probability distribution. The predetermined probability distribution may be referred to as a fixed probability distribution. The fixed probability distribution is shared between the video encoding device and the video decoding device. The entropy encoder 30 uses the probability model estimated from the feature tensor y 1 to entropy code each element of the input feature tensor y 0 .

[0019] Each element of the feature tensor y i at scale i is entropy coded using the probability model estimated from the feature tensor at scale (i + 1). Therefore, in the MSPSM system, the entropy encoder 33 performs entropy coding after the processing of the probability model estimator 24 is completed. The entropy encoder 32 performs entropy coding after the processing of the probability model estimator 23 is completed. The entropy encoder 31 performs entropy coding after the processing of the probability model estimator 22 is completed. The entropy encoder 30 performs entropy coding after the processing of the probability model estimator 21 is completed.

[0020] Note that the entropy encoders 31 to 34 do not perform entropy coding on the elements that have been entropy coded at a scale (a scale with a larger value of i for scale i) that was the target of entropy coding previously.

[0021] The entropy encoders 30 to 34 may manipulate the value of each element of the feature tensor input to the entropy encoder before the entropy coding process. For example, there may be rounding to an integer value. In such a case, the probability model estimators 21 to 24 apply the process for aligning with the value to be entropy decoded to the feature tensor at scale (i + 1), and then execute the estimation process using the feature tensor after the application.

[0022] As described above, the feature tensor y iEach element is entropy-coded using a probability model estimated from the feature tensor at scale (i + 1). Therefore, when entropy-coding the elements of the feature tensor y at scale i i when entropy-coding the elements of i , the probability model used reflects not only the previous elements (the upper and left elements at each scale in FIG. 2) in raster scan order with respect to the element to be coded at scale (i + 1), but also the subsequent elements (the right and lower elements at each scale in FIG. 2) in raster scan order.

[0023] Hereinafter, when estimating the probability model, considering the subsequent elements in raster scan order may be expressed as look-ahead of elements, or look-ahead processing of elements.

[0024] Note that the probability model is updated as appropriate. For example, the probability model is updated each time entropy-coding of an element is performed.

[0025] Since the probability model that reflects the subsequent elements in raster scan order is used, the probability distribution used for entropy-coding becomes closer to the probability distribution of the entire screen. As a result, even when the probability distribution varies depending on the region on the screen, the coding performance is improved. From a different perspective on look-ahead, when entropy-coding each element of the feature tensor y at scale i i a probability model based on elements discretely arranged within the screen is used, so it can be said that the probability distribution used for entropy-coding becomes closer to the probability distribution of the entire screen.

[0026] To maintain the effect of look-ahead, each scale needs to include one or more elements among the elements (the small rectangles marked with horizontal lines in FIG. 2) included in the feature tensor of the smallest scale (scale 4 in the example shown in FIG. 2 (number of scales = 4)). In the example shown in FIG. 2, the minimum size of the feature tensor under such a constraint is 16 × 16 elements.

[0027] FIG. 4 is an explanatory diagram for explaining the minimum size. In FIG. 4(A), a feature tensor y of scale 0 composed of 16×16 elements is shown. 0 In FIG. 4(B), a feature tensor y of scale 0 composed of 16×13 elements is shown. 0 is shown.

[0028] For example, assuming that the scale number i = 4 and the downsamplers 11 to 14 are configured to extract the lower right element of a 2×2 element (feature amount) and downsample it by 1 / 2, for the feature tensor y of 16×16 elements 0 Regarding the feature tensor y 1 ~y 4 each contains one or more elements. However, regarding the feature tensor y of 16×13 elements 0 Regarding the feature tensor y 4 the number of elements is 0. Since there are no elements at the minimum scale, the probability distribution cannot be estimated from the feature tensor y 4 As a result, it is difficult to implement MSPSM. Therefore, it can be said that the minimum size of the feature tensor is 16×16 elements.

[0029] An object of the present invention is to provide a video encoding device, a video decoding device, a video encoding method, and a video decoding method that can avoid a situation where the probability distribution cannot be estimated when performing entropy encoding using the probability distribution and utilizing the division of the feature tensor.

Means for Solving the Problems

[0030] The video encoding device according to the present disclosure includes a reduction means for generating tensors of a plurality of reduced regions from a tensor to be encoded, a plurality of probability distribution estimation means for inputting tensors of the plurality of reduced regions and estimating respective probability distributions, an entropy encoding means for performing entropy encoding on the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution to generate a bitstream, and a multiplexing means for multiplexing the bitstream. The entropy encoding means uses a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region, and when the size of the tensor to be encoded is smaller than a predetermined size, includes a reduction control means for controlling the reduction process of the reduction means so that one or more elements of the tensor of the reduced region are input to each of the plurality of probability distribution estimation means.

[0031] The video decoding device according to the present disclosure includes a demultiplexing means for demultiplexing a multiplexed bitstream in which a bitstream generated by entropy encoding the tensor to be encoded and tensors of a plurality of reduced regions generated from the tensor to be encoded using a probability distribution is multiplexed, an entropy decoding means for performing entropy decoding on the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution, and a synthesizing means for synthesizing a plurality of tensors obtained by the entropy decoding. At the time of video encoding, it is guaranteed that one or more elements of the tensor of the reduced region are input to each of the means for estimating the probability distribution used in the entropy encoding. The entropy decoding means uses a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be decoded or the reduced region.

[0032] The video encoding method according to the present disclosure inputs tensors of a plurality of reduced regions, estimates each probability distribution, and entropy-encodes the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution to generate a bitstream, multiplexes the bitstream, and when performing entropy encoding, uses a probability distribution estimated from a tensor of a reduced region having a size smaller than that of the tensor to be encoded or the reduced region. When the size of the tensor to be encoded is smaller than a predetermined size, the reduction process is controlled so that one or more elements of the tensor of the reduced region can be used when estimating a plurality of probability distributions.

[0033] The video decoding method according to the present disclosure demultiplexes a multiplexed bitstream in which a bitstream generated by entropy-encoding a tensor to be encoded and tensors of a plurality of reduced regions generated from the tensor to be encoded using a probability distribution is multiplexed, performs entropy decoding on the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution, synthesizes a plurality of tensors obtained by the entropy decoding, and at the time of video encoding, it is guaranteed that one or more elements of the tensor of the reduced region are input to each of the means for estimating the probability distribution used in the entropy encoding. When performing entropy decoding, a probability distribution estimated from a tensor of a reduced region having a size smaller than that of the tensor to be decoded or the reduced region is used.

[0034] The video encoding program according to the present disclosure causes a computer to perform a reduction process of generating tensors of a plurality of reduced regions from a tensor to be encoded, a process of inputting tensors of the plurality of reduced regions and estimating respective probability distributions, a process of performing entropy encoding on the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distributions to generate a bitstream, and a process of multiplexing the bitstream. When performing entropy encoding, a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region is used. When the size of the tensor to be encoded is smaller than a predetermined size, the reduction process is controlled so that one or more elements of the tensor of the reduced region can be used when estimating a plurality of probability distributions.

[0035] The video decoding program according to the present disclosure causes a computer to perform a process of demultiplexing a multiplexed bitstream in which a bitstream generated by entropy encoding the tensor to be encoded and tensors of a plurality of reduced regions generated from the tensor to be encoded using a probability distribution is multiplexed, a process of performing entropy decoding on the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution, and a process of synthesizing a plurality of tensors obtained by the entropy decoding. At the time of video encoding, it is guaranteed that one or more elements of the tensor of the reduced region are input to each of the means for estimating the probability distribution used in the entropy encoding. When performing entropy decoding, the computer is caused to use a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be decoded or the reduced region.

Advantages of the Invention

[0036] According to the present invention, when performing entropy encoding using a probability distribution and utilizing the division of a feature tensor, a situation where the estimation of the probability distribution becomes impossible can be avoided.

Brief Description of the Drawings

[0037]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying Out the Invention

[0038] Hereinafter, embodiments will be described with reference to the drawings. In the embodiments described below, as an example of a tensor, a feature tensor output by an encoder in an autoencoder will be used.

[0039] Embodiment 1. The video encoding device according to the first embodiment encodes an input feature tensor based on the MSPSM. As an example of a downsampler for reducing the feature tensor, the downsamplers 11 to 14 illustrated in FIG. 1 will be taken as an example. Also, as an example of a probability model for estimating a probability distribution, the probability model estimators 21 to 24 illustrated in FIG. 1 will be taken as an example.

[0040] The video encoding device determines the number of scales so that one or more elements (elements of the feature tensor) can be input to all the probability model estimators 21 to 24.

[0041] An example of how to determine the number of scales will be described. When the downsamplers 11 to 14 perform downsampling, an example of extracting the lower right element of a 2×2 element (feature amount) will be taken.

[0042] When realizing downsampling by extracting the lower right element of a 2×2 element (feature amount), the number of scales is determined as follows.

[0043] - The number of elements on the short side of the encoding target area (in this embodiment, the input feature tensor) ≥ 16: The number of scales = 4 - 16 > The number of elements on the short side of the encoding target area ≥ 8: The number of scales = 3 - 8 > The number of elements on the short side of the encoding target area ≥ 4: The number of scales = 2 - 4 > The number of elements on the short side of the encoding target area ≥ 2: The number of scales = 1 - 2 > The number of elements on the short side of the encoding target area ≥ 1: The number of scales = 0

[0044] Note that the number of scales actually used is arbitrary. That is, in the above example, for instance, when the number of elements on the short side of the area to be encoded is 16 or more, it is not essential to set the number of scales to 4. Instead, the number of scales may be selected from 1 to 3, or a value of 5 or more may be specified as long as one or more elements can be ensured as the input to the probability model estimator. The number of scales is specified, for example, from outside the video encoding device.

[0045] Generally speaking, when the sampling interval of downsampling is 2, the number of elements on the short side of the area to be encoded is m, and the number of scales is n, the relationship between the number of scales n and the number of elements m on the short side of the area to be encoded is expressed as follows. m ≧ 2 n (1)

[0046] When the sampling interval of downsampling is s, the number of elements on the short side of the area to be encoded is m, and the number of scales is n, the relationship between the number of scales n and the number of elements m on the short side of the area to be encoded is expressed as follows. m ≧ s n (2)

[0047] A specific example will be described using an input feature tensor of 16×13 elements as an example. FIG. 5 is an explanatory diagram showing an example of reduction for an input feature tensor of 16×13 elements.

[0048] In the example shown in FIG. 5, the number of elements on the short side of the area to be encoded is 13. Therefore, the number of scales is determined to be 3. For all of the feature tensors y 1 , y 2 , y 3 , the number of elements is 1 or more. Therefore, 1 or more is ensured as the number of elements input to all the probability model estimators.

[0049] FIG. 6 is a block diagram showing an application example of the MSPSM to an input feature tensor of 16×13 elements. As described above, since the number of scales is determined to be 3, three downsamplers 11 to 13 downsample the input feature tensor at predetermined intervals. Also, three probability model estimators 21 to 23 estimate a probability distribution using the outputs of the downsamplers 11 to 13.

[0050] FIG. 7 is an explanatory diagram showing an example of the probability distribution used at each scale. Referring to FIG. 7, when entropy encoders 30 to 32 encode elements of the input feature tensor or a feature tensor of a certain scale, they use a probability distribution estimated from a feature tensor of a size smaller (for example, one size smaller) than the scale. Note that the entropy encoder 33 performs entropy encoding on the feature tensor y 3 (the smallest feature tensor) using a predetermined probability distribution.

[0051] FIG. 8 is a block diagram showing a configuration example of the video encoding apparatus and the video decoding apparatus according to the first embodiment.

[0052] The video encoding apparatus 100 shown in FIG. 8 includes a scale number control unit 101, a downsampling unit 102, a probability model estimator 103, an entropy encoder 104, and a multiplexer 105.

[0053] The video decoding apparatus 200 shown in FIG. 8 includes a tensor reconstruction unit 201, a scale number estimation unit 202, a probability model estimator 203, an entropy decoder 204, and a demultiplexer 205.

[0054] [Description of Video Encoding Apparatus] In the video encoding apparatus 100, the scale number control unit 101 determines the number of scales based on the size of the tensor to be encoded (tensor to be encoded). Hereinafter, as an example, the tensor to be encoded is assumed to be a feature tensor obtained by applying quantization to a tensor output by an encoder (not shown) in an autoencoder.

[0055] The downsampling unit 102 generates one or more reduced regions from the tensor to be encoded based on the determined number of scales. For example, the downsampling unit 102 downsamples the tensor to be encoded at a predetermined interval. The downsampling unit 102 can be composed of, for example, the downsamplers 11 to 14 shown in FIG. 1.

[0056] The probability model estimator 103 can be composed of, for example, the probability model estimators 21 to 24 shown in FIG. 1. Similar to the probability model estimators 21 to 24, the probability model estimator 103 estimates a probability distribution.

[0057] The entropy encoder 104 can be composed of, for example, the entropy encoders 30 to 34 shown in FIG. 1. The entropy encoder 104 entropy-encodes each of the plurality of feature tensors using the probability distribution estimated by the probability model estimator 103. Specifically, the entropy encoder 104 entropy-encodes each element of the feature tensor. Then, the entropy encoder 104 outputs the entropy-encoded data for the input feature tensor as a bit stream.

[0058] The multiplexer 105 multiplexes the generated bit stream and other information to generate a multiplexed bit stream. As the other information, for example, there is the number of scales determined by the scale number control unit 101.

[0059] Next, an example of the specific operation of the video encoding device 100 will be described with reference to the flowchart of FIG. 9.

[0060] The scale number control unit 101 determines the number of scales based on the size of the tensor to be encoded (input feature tensor) (step S101). As described above, the scale number control unit 101 determines the number of scales so that one or more elements (elements of the feature tensor) can be input to all parts that calculate the probability distribution for each scale in the probability model estimator 103.

[0061] The downsampling unit 102 downsamples the feature tensor at a predetermined interval (step S104). By the downsampling, feature tensors of a plurality of scales as exemplified in FIG. 2 are obtained as an example.

[0062] The probability model estimator 103 estimates a probability distribution from the feature tensor of each scale (step S105).

[0063] The entropy encoder 104 entropy-encodes the feature tensor to generate a bit stream (step S106).

[0064] When encoding an element of the feature tensor of a certain scale, the entropy encoder 104 uses the probability distribution estimated from the feature tensor of a size smaller (for example, one size smaller) than the scale.

[0065] Specifically, taking the example shown in FIG. 2, the probability model estimator 103 calculates a probability distribution from the feature tensor y 4 generated by the downsampling unit 102. The entropy encoder 104 entropy-encodes the feature tensor y 3 using the probability distribution. The probability model estimator 103 calculates a probability distribution from the feature tensor y 3 generated by the downsampling unit 102. The entropy encoder 104 entropy-encodes the feature tensor y 2 using the probability distribution. The probability model estimator 103 calculates a probability distribution from the feature tensor y 2 generated by the downsampling unit 102. The entropy encoder 104 entropy-encodes the feature tensor y 1 using the probability distribution. The probability model estimator 103 calculates a probability distribution from the feature tensor y 1 generated by the downsampling unit 102. The entropy encoder 104 entropy-encodes the feature tensor y 0Perform entropy encoding. The entropy encoder 104 outputs the encoded data of the feature tensors of each scale to the multiplexer 105.

[0066] Note that the entropy encoder 104 uses a fixed probability distribution to perform entropy encoding on the feature tensor y 4 Perform entropy encoding.

[0067] The multiplexer 105 multiplexes the bit stream generated in the process of step S106 and other information (including at least the size of the input feature tensor) to generate a multiplexed bit stream (step S107).

[0068] [Modification Example 1] In the above embodiment, the scale number control unit 101 supplies the size of the input feature tensor to the multiplexer 105. When information on the size of the screen before generating the feature tensor is available, it is possible to estimate the size of the input feature tensor from the size of the screen. Therefore, that information may be used.

[0069] In that case, the multiplexer 105 estimates the size of the input feature tensor from the information on the size of the screen input from outside the video encoding device 100. The multiplexer 105 multiplexes the estimated size of the input feature tensor into the bit stream generated in the process of step S106.

[0070] [Modification Example 2] In the above embodiment, the scale number control unit 101 controls the downsampling so that one or more elements of the feature tensor of the corresponding scale are input to a plurality of probability distribution estimation means (for example, the probability model estimators 21 to 24 when the probability model estimator 103 is configured to include the probability model estimators 21 to 24). Specifically, the scale number control unit 101 determines the number of scales. That is, the scale number control unit 101 executes control to ensure that one or more elements of the feature tensor of each scale are input to the corresponding probability model estimators 21 to 24.

[0071] However, instead of the scale number control unit 101 determining the scale number, when the input feature tensor is smaller than the minimum size, if there is a probability model estimator for the probability that the elements of the feature tensor are not input, the entropy encoder 104 may be configured to perform entropy encoding using a fixed probability distribution.

[0072] For example, the entropy encoder 104 can use a probability distribution estimated based on the encoded elements within the screen. Also, the entropy encoder 104 may use, as the fixed probability distribution, a uniform distribution in which the appearance probabilities of all values are equal.

[0073] [Description of Video Decoding Device] In the video decoding device 210, the demultiplexer 205 demultiplexes the multiplexing of the multiplexed bit stream from the video encoding device 100 to obtain a bit stream regarding the feature tensor input to the video encoding device 100. In the present embodiment, the demultiplexer 205 also obtains the size of the input feature tensor by demultiplexing.

[0074] The scale number estimation unit 202 estimates the scale number. In the present embodiment, the scale number estimation unit 202 estimates the scale number from the size of the input feature tensor acquired from the demultiplexer 205. The scale number estimation unit 202 can estimate the scale number in the same way as the way of thinking when the scale number control unit 101 in the video encoding device 100 determines the scale number. For example, the estimated scale number is supplied to at least the entropy decoder 204.

[0075] The entropy decoder 204 entropy-decodes the elements of the feature tensor for each scale with reference to the probability distribution from the probability model estimator 203. Note that the entropy decoder 204 can also refer to the scale number when performing entropy decoding.

[0076] The probability model estimator 203 operates in the same manner as the probability model estimator 103. That is, the probability model estimator 203 can be composed of, for example, the probability model estimators 21 to 24 shown in FIG. 1. The probability model estimator 203 estimates a probability distribution in the same way as the probability model estimators 21 to 24.

[0077] The tensor reconstruction unit 201 restores the feature tensor input to the video encoding device 100. Specifically, the tensor reconstruction unit 201 uses the feature tensor output by the entropy decoder 204 as the feature tensor input to the video encoding device 100.

[0078] Next, an example of the specific operation of the video decoding device 210 will be described with reference to the flowchart of FIG. 10.

[0079] The demultiplexer 205 demultiplexes the multiplexing of the multiplexed bitstream from the video encoding device 100 to obtain the bitstream of the encoded data sequence and other information (including at least the size of the input feature tensor.) (step S201).

[0080] The scale number estimator 202 estimates the scale number (step S202). In the present embodiment, the scale number estimator 202 estimates the scale number based on the size of the input feature tensor obtained from the demultiplexer 205. The scale number estimator 202 supplies the scale number to the entropy decoder 204.

[0081] The probability model estimator 203 estimates the probability distribution from the feature tensors of each scale (step S204).

[0082] The entropy decoder 204 entropy-decodes the bitstream to obtain the feature tensors of each scale (step S205).

[0083] Specifically, taking the example shown in FIG. 2, when the entropy decoder 204 outputs the feature tensor y 4 the probability model estimator 203 uses the feature tensor y 4Calculate the probability distribution therefrom. The entropy decoder 204 uses the probability distribution to entropy-decode the feature tensor y 3 and outputs the feature tensor y 3 . When the probability model estimator 203 outputs the feature tensor y 3 , the probability model estimator 203 calculates the probability distribution from the feature tensor y 3 . The entropy decoder 204 uses the probability distribution to entropy-decode the feature tensor y 2 and outputs the feature tensor y 2 . When the probability model estimator 203 outputs the feature tensor y 2 , the probability model estimator 203 calculates the probability distribution from the feature tensor y 2 . The entropy decoder 204 uses the probability distribution to entropy-decode the feature tensor y 1 and outputs the feature tensor y 1 . When the probability model estimator 203 outputs the feature tensor y 1 , the probability model estimator 203 calculates the probability distribution from the feature tensor y 1 . The entropy decoder 204 uses the probability distribution to entropy-decode the feature tensor y 0 and outputs the feature tensor y 0 .

[0084] The feature tensor y 0 is supplied to the tensor reconstruction unit 201.

[0085] Note that the entropy decoder 204 entropy-decodes the feature tensor y 4 using a fixed probability distribution.

[0086] When the above processing is executed, the tensor reconstruction unit 201 can obtain the feature tensor input to the video encoding device 100. That is, the tensor reconstruction unit 201 can restore the feature tensor input to the video encoding device 100 (step S206).

[0087] Embodiment 2. In the second embodiment, when a feature tensor smaller than the minimum size is input, the video encoding device performs padding to add dummy data to the input feature tensor to create a feature tensor of the minimum size. In this embodiment, padding is a process of adding dummy data to the input feature tensor that is greater than or equal to the difference between the size of the input feature tensor and the minimum size.

[0088] FIG. 11 is an explanatory diagram for explaining padding. FIG. 11 shows an example in which a feature tensor of 16×13 elements is input when the minimum size is 16×16 elements. The video encoding device adds dummy data of three rows (3×16 elements) with diagonal lines to the input feature tensor to create a feature tensor of the minimum size.

[0089] The video encoding device may add a number of dummy data to the input feature tensor that is more than three rows.

[0090] As described above, the feature tensor of the minimum size is a tensor that includes one or more elements at the minimum scale.

[0091] Zero padding may be performed as padding, but the values of the dummy data may be selected based on a fixed probability distribution or an estimated probability distribution. The estimated probability distribution indicates the probability distribution of the feature tensors entropy-encoded in the past. For example, when encoding a moving image, the probability distribution regarding the screen at a time earlier than the time of the screen to be encoded can be used. Also, when the screen is divided and encoding is performed for each divided region, the probability distribution regarding the other encoded regions can be used when encoding a certain divided region.

[0092] As an example, when encoding based on a probability distribution where the probability distribution regarding other regions is P(0)=8 / 16, P(1)=6 / 16, P(2)=2 / 16, the video encoding device pads with 8 dummy data of 0, 6 dummy data of 1, and 2 dummy data of 2. Note that P(x) is the occurrence probability of x.

[0093] FIG. 12 is a block diagram showing a configuration example of a video encoding device and a video decoding device according to a second embodiment.

[0094] The video encoding device 110 shown in FIG. 12 includes a downsampling unit 102, a probability model estimator 103, an entropy encoder 104, a multiplexer 105, and a padding unit 106.

[0095] The video decoding device 210 shown in FIG. 12 includes a tensor reconstruction unit 201, a probability model estimator 203, an entropy decoder 204, a demultiplexer 205, and a padding control unit 206.

[0096] [Description of Video Encoding Device] In the video encoding device 110, the configurations and operations of the downsampling unit 102, the probability model estimator 103, the entropy encoder 104, and the multiplexer 105 are the same as those in the first embodiment.

[0097] However, in the second embodiment, the multiplexer 105 does not multiplex the size of the input feature tensor in the first embodiment.

[0098] When a feature tensor smaller than the minimum size is input, the padding unit 106 performs padding to add dummy data to the input feature tensor and supplies the size of the input tensor to the multiplexing unit 110. When the size of the input feature tensor is equal to or greater than the minimum size, the padding unit 106 does not perform padding.

[0099] Next, an example of a specific operation of the video encoding device 110 will be described with reference to the flowchart of FIG. 13.

[0100] The padding unit 106 checks whether the input feature tensor is smaller than the minimum size (step S102). If the input feature tensor is smaller than the minimum size, the padding unit 106 performs padding (step S103).

[0101] As described above, padding is a process of adding dummy data to the input feature tensor so that one or more elements are included at the minimum scale.

[0102] The processes of steps S104 to S107 are the same as those in the first embodiment.

[0103] Note that the entropy encoder 104 does not target dummy data for encoding. Therefore, the encoded data of the dummy data is not transmitted to the video decoder 210.

[0104] [Description of Video Decoder] In the video decoder 210, the configurations and operations of the tensor reconstruction unit 201, the probability model estimator 203, the entropy decoder 204, and the demultiplexer 205 are the same as those in the first embodiment.

[0105] However, the entropy decoder 204 and the tensor reconstruction unit 201 also execute processes that are weighted on the processes of the first embodiment as follows.

[0106] The padding control unit 206 checks the size of the tensor to be decoded (the feature tensor to be decoded) from the bitstream obtained by the demultiplexer 205 through demultiplexing. The padding control unit 206 can determine whether padding has been performed (whether padding control has been performed in the video encoder 110) from the check result. When it is determined that padding control has been performed, the padding control unit 206 restores the elements in the padding area (for example, the hatched area in FIG. 11). The padding control unit 206 supplies the restored elements (dummy data) to the entropy decoder 204. Note that the padding method is assumed to be determined in advance with the video encoder.

[0107] The entropy decoder 204 may assign the restored dummy data to the feature tensor at any scale.

[0108] The entropy decoder 204 obtains the feature tensor of each scale with the restored dummy data added by entropy decoding.

[0109] The tensor reconstruction unit 201 removes the dummy data from the feature tensor of scale y 0 supplied from the entropy decoder 204, and restores the feature tensor input to the video encoding device 110.

[0110] Next, an example of the specific operation of the video decoding device 210 will be described with reference to the flowchart of FIG. 14.

[0111] The demultiplexer 205 demultiplexes the multiplexing of the multiplexed bit stream from the video encoding device 100 to obtain the bit stream of the encoded data sequence (step S201).

[0112] The padding control unit 206 performs padding control (step S203). The padding control is the confirmation of whether the above-described padding has been performed, the restoration of the elements in the padding area, and the supply of the dummy data to the entropy decoder 204.

[0113] The probability model estimator 203 estimates the probability distribution from the feature tensor of each scale in the same manner as in the first embodiment (step S204).

[0114] The entropy decoder 204 entropy-decodes the bit stream in the same manner as in the first embodiment to obtain the feature tensor of each scale (step S205). In the present embodiment, as described above, the entropy decoder 204 also performs processing related to the dummy data from the padding control unit 206.

[0115] The feature tensor y 0 is supplied to the tensor reconstruction unit 201.

[0116] Note that the entropy decoder 204 uses a fixed probability distribution for the feature tensor y 4Perform entropy decoding.

[0117] When the above processing is executed, the tensor reconstruction unit 201 can obtain the feature tensor input to the video encoding device 100. That is, the tensor reconstruction unit 201 can restore the feature tensor input to the video encoding device 100 (step S206). In the present embodiment, as described above, the tensor reconstruction unit 201 also performs processing related to dummy data.

[0118] Each of the above embodiments can be configured by hardware, but can also be realized by a computer program.

[0119] The information processing system shown in FIG. 15 includes a processor 701 such as a CPU (Central Processing Unit), a program memory 702, a storage medium 703 for storing temporary data, and a storage medium 704 for storing a bitstream. The storage medium 703 and the storage medium 704 may be separate storage media or may be storage areas consisting of the same storage medium. As the storage medium, a magnetic storage device such as a hard disk can be used. As the storage medium, SRAM (Static Random Access Memory) or flash ROM (Read Only Memory) may be used.

[0120] Examples of the temporary data include the feature tensors used by the video encoding devices 100 and 110 and the video decoding devices 200 and 210.

[0121] In the information processing system, the program memory 702 stores a program (video encoding program or video decoding program) for realizing the functions of the respective blocks shown in the above embodiments.

[0122] The processor 701 realizes each function in the above embodiment by executing processing according to the program (software element: code) stored in the program memory 702.

[0123] That is, the processor 701 realizes the functions of the video encoding apparatuses 100 and 110 and the video decoding apparatuses 200 and 210 shown in each embodiment by executing processing according to the programs stored in the program memory 702.

[0124] For example, the function of the video encoding apparatuses 100 and 110 is realized when the processor 701 executes processing according to a video encoding program for realizing the functions of the respective blocks in the video encoding apparatuses 100 and 110 shown in FIGS. 8 and 12.

[0125] Also, for example, the function of the video decoding apparatuses 200 and 210 is realized when the processor 701 executes processing according to a video decoding program for realizing the functions of the respective blocks in the video decoding apparatuses 200 and 210 shown in FIGS. 8 and 12.

[0126] Note that at least the program memory 702 is a non-transitory computer readable medium. The non-transitory computer readable medium includes various types of tangible storage media. Specific examples of the non-transitory computer readable medium include a magnetic recording medium (e.g., hard disk), a magneto-optical recording medium (e.g., magneto-optical disk), a CD-ROM (Compact Disc-Read Only Memory), a CD-R (Compact Disc-Recordable), a CD-R / W (Compact Disc-ReWritable), and a semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM). However, the program may be stored in various types of transitory computer readable media.

[0127] A temporary computer-readable medium is supplied with a program, for example, via a wired communication path or a wireless communication path, that is, via an electrical signal, an optical signal, or an electromagnetic wave.

[0128] FIG. 16 is a block diagram showing a main part of a video encoding device. The video encoding device 50 shown in FIG. 15 (which is realized by the video encoding device 100 in the embodiment) includes a reduction means 51 (which is realized by the downsampling unit 102 in the embodiment) that generates tensors of a plurality of reduced regions (for example, regions by a feature tensor of scale i) from an encoding target tensor, and a plurality of probability distribution estimation means 52 (which are realized by probability model estimators 21 to 24 included in the probability model estimator 103 in the embodiment) that input tensors of the plurality of reduced regions and estimate respective probability distributions, and an entropy encoding means 53 (which is realized by the entropy encoder 104 in the embodiment) that performs entropy encoding on the encoding target tensor and the tensors of the plurality of reduced regions using the probability distribution to generate a bit stream, and a multiplexing means 54 (which is realized by the multiplexer 105 in the embodiment) that multiplexes the bit stream. The entropy encoding means 53 uses a probability distribution estimated from a tensor of a reduced region (for example, a region by a feature tensor of scale (i + 1) with respect to a region by a feature tensor of scale i) having a size smaller than the encoding target tensor or the reduced region, and when the size of the encoding target tensor is smaller than a predetermined size, a reduction control means (which is realized by the scale number control unit 101 in the embodiment) that controls the reduction process of the reduction means 51 so that one or more elements of the tensor of the reduced region are input to each of the plurality of probability distribution estimation means 52.

[0129] FIG. 17 is a block diagram showing the main part of the video decoding apparatus. The video decoding apparatus 60 shown in FIG. 12 (in the embodiment, realized by the video decoding apparatus 200) is such that a bit stream generated by entropy encoding a tensor to be encoded and tensors of a plurality of reduced regions generated from the tensor to be encoded using a probability distribution is demultiplexed by a demultiplexing means 61 (in the embodiment, realized by the demultiplexer 205); entropy decoding means 62 (in the embodiment, realized by the entropy decoder 204) that performs entropy decoding on the tensor to be encoded and the tensors of the plurality of reduced regions using a probability distribution; and combining means 63 (in the embodiment, realized by the tensor reconstruction unit 201) that combines the plurality of tensors obtained by the entropy decoding. At the time of video encoding, it is guaranteed that one or more elements of the tensor of the reduced region are input to each of means for estimating the probability distribution used in the entropy encoding (for example, the probability model estimators 21 to 24) (for example, realized by the video encoding apparatus 100). The entropy decoding means 62 uses a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region to be decoded.

[0130] Some or all of the above embodiments may be described as follows in the appended claims, but are not limited thereto.

[0131] (Appended Claim 1) Reduction means for generating tensors of a plurality of reduced regions from a tensor to be encoded; A plurality of probability distribution estimation means for inputting the tensors of the plurality of reduced regions and estimating each probability distribution; Entropy encoding means for entropy encoding the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution to generate a bit stream; Multiplexing means for multiplexing the bit stream; The entropy encoding means uses a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region; Reduction control means for controlling the reduction process of the reduction means so that when the size of the tensor to be encoded is smaller than a predetermined size, one or more elements of the tensor in the reduced region are input to each of the plurality of probability distribution estimation means. Video encoding device.

[0132] (Appendix 2) The reduction control means determines the number of reduced regions according to the size of the tensor to be encoded. The video encoding device of Appendix 1.

[0133] (Appendix 3) The entropy encoding means uses a probability distribution estimated from the tensor of the reduced region having the size closest to the size of the tensor to be encoded or the reduced region. The video encoding device of Appendix 1 or Appendix 2.

[0134] (Appendix 4) The entropy encoding means performs entropy encoding on the tensor of the smallest reduced region using a predetermined fixed probability distribution. The video encoding device of Appendix 3.

[0135] (Appendix 5) Demultiplexing means for demultiplexing a multiplexed bitstream in which a bitstream generated by entropy encoding the tensor to be encoded and tensors of a plurality of reduced regions generated from the tensor to be encoded using a probability distribution are multiplexed, Entropy decoding means for performing entropy decoding on the tensor to be encoded and the tensors of the plurality of reduced regions using a probability distribution, Composing means for composing a plurality of tensors obtained by the entropy decoding, During video encoding, it is guaranteed that one or more elements of the tensor in the reduced region are input to each of the means for estimating the probability distribution used in the entropy encoding, The entropy decoding means uses a probability distribution estimated from the tensor to be encoded or the tensor of the reduced region smaller than the reduced region for the decoding target. Video decoding device.

[0136] (Appendix 6) Reduction region number estimation means for estimating the number of reduction regions based on the size of the tensor to be encoded (in the embodiment, realized by the scale number estimation unit 202). The video decoding device of Appendix 5.

[0137] (Appendix 7) Execute a reduction process to generate tensors of a plurality of reduction regions from the tensor to be encoded, Input the tensors of the plurality of reduction regions, estimate each probability distribution, Entropy-encode the tensor to be encoded and the tensors of the plurality of reduction regions using the probability distribution to generate a bit stream, Multiplex the bit stream, When performing the entropy encoding, use the probability distribution estimated from the tensor of the reduction region having a size smaller than the tensor to be encoded or the reduction region, When the size of the tensor to be encoded is smaller than a predetermined size, control the reduction process so that one or more elements of the tensor of the reduction region can be used when estimating the plurality of probability distributions. Video encoding method.

[0138] (Appendix 8) Demultiplex the multiplexed bit stream in which the tensor to be encoded and the bit stream generated by entropy-encoding the tensors of a plurality of reduction regions generated from the tensor to be encoded using the probability distribution are multiplexed, Perform entropy decoding on the tensor to be encoded and the tensors of the plurality of reduction regions using the probability distribution, Synthesize the plurality of tensors obtained by the entropy decoding, At the time of video encoding, it is guaranteed that one or more elements of the tensor of the reduction region are input to each of the means for estimating the probability distribution used in the entropy encoding. When performing the entropy decoding, use a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region to be decoded. Video decoding method.

[0139] (Appendix 9) Cause a computer to perform a reduction process of generating tensors of a plurality of reduced regions from a tensor to be encoded, perform a process of inputting the tensors of the plurality of reduced regions and estimating respective probability distributions, perform a process of performing entropy encoding on the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution to generate a bit stream, perform a process of multiplexing the bit stream, when performing the entropy encoding, use a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region, when the size of the tensor to be encoded is smaller than a predetermined size, control the reduction process so that one or more elements of the tensor of the reduced region can be used when estimating the plurality of probability distributions. Video encoding program therefor.

[0140] (Appendix 10) Cause a computer to perform a process of demultiplexing a multiplexed bit stream in which a bit stream generated by entropy encoding the tensor to be encoded and tensors of a plurality of reduced regions generated from the tensor to be encoded is multiplexed, perform a process of performing entropy decoding on the tensor to be encoded and the tensors of the plurality of reduced regions using a probability distribution, perform a process of synthesizing a plurality of tensors obtained by the entropy decoding, At the time of video encoding, it is guaranteed that one or more elements of the tensor of the reduced region are input to each of means for estimating the probability distribution used in the entropy encoding, To cause the computer to use a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region when performing the entropy decoding A video decoding program therefor.

[0141] (Appendix 11) Reduction means for generating tensors of a plurality of reduced regions from a tensor to be encoded, A plurality of probability distribution estimation means for inputting the tensors of the plurality of reduced regions and estimating respective probability distributions, Entropy encoding means for performing entropy encoding on the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution to generate a bit stream, Multiplexing means for multiplexing the bit stream, The entropy encoding means uses a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region, Padding means for adding dummy data equal to or more than the difference between the size and the predetermined size to the tensor to be encoded when the size of the tensor to be encoded is smaller than a predetermined size A video encoding device.

[0142] (Appendix 12) The entropy encoding means does not target dummy data for encoding The video encoding device of Appendix 11.

[0143] (Appendix 13) The entropy encoding means uses a probability distribution estimated from a tensor of a reduced region having a size closest to the size of the tensor to be encoded or the reduced region The video encoding device of Appendix 11 or Appendix 12.

[0144] (Appendix 14) The entropy encoding means performs entropy encoding on the tensor of the smallest reduced region using a predetermined fixed probability distribution The video encoding device of Appendix 13.

[0145] (Appendix 15) Demultiplexing means for demultiplexing a multiplexed bitstream in which a bitstream generated by entropy encoding a tensor to be encoded and tensors of a plurality of reduced regions generated from the tensor to be encoded using a probability distribution are multiplexed; Entropy decoding means for performing entropy decoding on the tensor to be encoded and the tensors of the plurality of reduced regions using a probability distribution; Combining means for combining a plurality of tensors obtained by the entropy decoding; During video encoding, when the size of the tensor to be encoded is smaller than a predetermined size, padding control is performed such that dummy data equal to or greater than the difference between the size and the predetermined size is added to the tensor to be encoded. The entropy decoding means uses a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region that is the decoding target. Video decoding apparatus.

[0146] (Appendix 16) Determination means for determining whether padding control has been performed based on the size of the tensor to be decoded. The video decoding apparatus of Appendix 15.

[0147] (Appendix 17) Execute a reduction process for generating tensors of a plurality of reduced regions from a tensor to be encoded. Input the tensors of the plurality of reduced regions, estimate each probability distribution. Entropy encode the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution to generate a bitstream. Multiplex the bitstream. When performing the entropy encoding, use a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region. When the size of the tensor to be encoded is smaller than a predetermined size, add dummy data equal to or greater than the difference between the size and the predetermined size to the tensor to be encoded. Video encoding method.

[0148] (Appendix 18) Demultiplex the multiplexed bitstream in which the bitstream generated by entropy encoding the tensor to be encoded and the tensors of a plurality of reduced regions generated from the tensor to be encoded using a probability distribution is multiplexed, Perform entropy decoding on the tensor to be encoded and the tensors of the plurality of reduced regions using a probability distribution, Combine the plurality of tensors obtained by the entropy decoding, During video encoding, when the size of the tensor to be encoded is smaller than a predetermined size, padding control is performed to add dummy data equal to or greater than the difference between the size and the predetermined size to the tensor to be encoded, When performing the entropy decoding, use the probability distribution estimated from the tensor of the reduced region having a size smaller than the tensor to be encoded or the reduced region to be decoded, Video decoding method.

[0149] (Appendix 19) Cause a computer to Perform a reduction process of generating tensors of a plurality of reduced regions from a tensor to be encoded, Input the tensors of the plurality of reduced regions and perform a process of estimating each probability distribution, Perform a process of entropy encoding the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution to generate a bitstream, Perform a process of multiplexing the bitstream, When performing the entropy encoding, use the probability distribution estimated from the tensor of the reduced region having a size smaller than the tensor to be encoded or the reduced region, When the size of the tensor to be encoded is smaller than a predetermined size, cause dummy data equal to or greater than the difference between the size and the predetermined size to be added to the tensor to be encoded, Video encoding program therefor.

[0150] (Appendix 20) causing a computer to perform a process of demultiplexing a multiplexed bitstream in which a bitstream generated by entropy encoding a tensor to be encoded and tensors of a plurality of reduced regions generated from the tensor to be encoded using a probability distribution are multiplexed, perform a process of entropy decoding using a probability distribution on the tensor to be encoded and the tensors of the plurality of reduced regions, perform a process of synthesizing a plurality of tensors obtained by the entropy decoding, at the time of video encoding, when the size of the tensor to be encoded is smaller than a predetermined size, perform padding control in which dummy data equal to or greater than the difference between the size and the predetermined size is added to the tensor to be encoded, cause the computer to use a probability distribution estimated from a reduced-region tensor having a size smaller than the tensor to be encoded or the reduced region to be decoded when performing the entropy decoding for a video decoding program.

Explanation of Signs

[0151] 11, 12, 13, 14 Downsampler 21, 22, 23, 24 Probability Model Estimator 30, 31, 32, 33, 34 Entropy Encoder 40 Bitstream Generator 50 Video Encoding Device 51 Reduction Means 52 Probability Distribution Estimation Means 53 Entropy Encoding Means 54 Multiplexing Means 55 Reduction Control Means 60 Video Decoding Device 61 Demultiplexing Means 62 Entropy Decoding Means 63 Synthesis Means 100, 110 Video Encoding Device 101 Scale Number Control Unit 102 Downsampling Unit 103 Probability Model Estimator 104 Entropy Encoder 105 Multiplexer 106 Padding Unit 200, 210 Video Decoder 201 Tensor Reconstruction Unit 202 Scale Number Estimation Unit 203 Probability Model Estimator 204 Entropy Decoder 205 Demultiplexer 206 Padding Control Unit 701 Processor 702 Program Memory 703, 704 Storage Medium

Claims

1. Reduction means for generating tensors of a plurality of reduced regions from a tensor to be encoded, A plurality of probability distribution estimation means for inputting the tensors of the plurality of reduced regions and estimating each probability distribution, Entropy encoding means for performing entropy encoding on the tensor to be encoded and the tensors of the plurality of reduced regions using the probability distribution to generate a bit stream, Multiplexing means for multiplexing the bit stream, The entropy encoding means uses a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region, Reduction control means for controlling the reduction process of the reduction means so that when the size of the tensor to be encoded is smaller than a predetermined size, one or more elements of the tensor of the reduced region are input to each of the plurality of probability distribution estimation means Video encoding apparatus.

2. The reduction control means determines the number of the reduced regions according to the size of the tensor to be encoded The video encoding apparatus according to claim 1.

3. The entropy encoding means uses a probability distribution estimated from a tensor of a reduced region having a size closest to the size of the tensor to be encoded or the reduced region The video encoding apparatus according to claim 1 or claim 2.

4. The entropy encoding means performs entropy encoding on the tensor of the smallest reduced region using a predetermined fixed probability distribution The video encoding apparatus according to claim 3.

5. Demultiplexing means for demultiplexing a multiplexed bit stream in which a bit stream generated by entropy encoding a tensor to be encoded and tensors of a plurality of reduced regions generated from the tensor to be encoded using a probability distribution is multiplexed, Entropy decoding means for performing entropy decoding on the tensor to be encoded and the tensors of the plurality of reduced regions using a probability distribution, Combining means for combining a plurality of tensors obtained by the entropy decoding, At the time of video encoding, it is guaranteed that one or more elements of the tensor of the reduced region are input to each of the means for estimating the probability distribution used in the entropy encoding, The entropy decoding means uses a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be decoded or the reduced region Video decoding device.

6. Comprising a reduction region number estimation means for estimating the number of the reduction regions based on the size of the tensor to be encoded The video decoding device according to claim 5.

7. Performing a reduction process for generating tensors of a plurality of reduction regions from the tensor to be encoded, Inputting the tensors of the plurality of reduction regions, estimating respective probability distributions, Entropy-encoding the tensor to be encoded and the tensors of the plurality of reduction regions using the probability distributions to generate a bit stream, Multiplexing the bit stream, When performing the entropy encoding, using a probability distribution estimated from a tensor of a reduction region having a size smaller than the tensor to be encoded or the reduction region, When the size of the tensor to be encoded is smaller than a predetermined size, controlling the reduction process so that one or more elements of the tensor of the reduction region can be used when estimating the plurality of probability distributions Video encoding method.

8. Demultiplexing a multiplexed bit stream in which a bit stream generated by entropy-encoding a tensor to be encoded and tensors of a plurality of reduction regions generated from the tensor to be encoded using probability distributions is multiplexed, Performing entropy decoding on the tensor to be encoded and the tensors of the plurality of reduction regions using probability distributions, Combining the plurality of tensors obtained by the entropy decoding, At the time of video encoding, it is guaranteed that one or more elements of the tensor of the reduction region are input to each of the means for estimating the probability distribution used in the entropy encoding, When performing the entropy decoding, using a probability distribution estimated from the tensor to be encoded or a tensor of a reduction region having a size smaller than the reduction region that is the decoding target Video decoding method.

9. Causing a computer to perform a reduction process for generating tensors of a plurality of reduction regions from a tensor to be encoded, input the tensors of the plurality of reduction regions and perform a process for estimating respective probability distributions, perform a process for entropy-encoding the tensor to be encoded and the tensors of the plurality of reduction regions using the probability distributions to generate a bit stream, and perform a process for multiplexing the bit stream When performing the entropy encoding, use a probability distribution estimated from a tensor of a reduced region having a size smaller than the tensor to be encoded or the reduced region. When the size of the tensor to be encoded is smaller than a predetermined size, control the reduction process so that one or more elements of the tensor of the reduced region can be used when estimating the plurality of probability distributions. A video encoding program for this purpose.

10. On a computer, A process of demultiplexing a multiplexed bitstream in which a bitstream generated by entropy encoding a tensor to be encoded and tensors of a plurality of reduced regions generated from the tensor to be encoded using a probability distribution is multiplexed; A process of performing entropy decoding on the tensor to be encoded and the tensors of the plurality of reduced regions using a probability distribution; Causing the computer to execute a process of synthesizing a plurality of tensors obtained by the entropy decoding. At the time of video encoding, it is guaranteed that one or more elements of the tensor of the reduced region are input to each of the means for estimating the probability distribution used in the entropy encoding. On the computer, when performing the entropy decoding, use a probability distribution estimated from the tensor to be decoded or a tensor of a reduced region having a size smaller than the reduced region. A video decoding program for this purpose.

Images