Decryption device

Abstract

To improve coding efficiency in hierarchical coding.SOLUTION: An encoding device is a device that performs hierarchical encoding on an input video, and includes a region extraction unit that extracts a main video region included in the input video, a region division unit that divides a remaining video region of the input video other than the main video region into a plurality of sub-pictures, a base hierarchical encoding unit that encodes the main video region output by the region extraction unit and outputs a base hierarchical encoding stream, and an extension hierarchical encoding unit that encodes each of the plurality of sub-pictures output by the region division unit and outputs an extension hierarchical encoding stream.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an encoding device, a decoding device, and a program. [Background technology]

[0002] There are high expectations for services that use large-screen, high-definition video, but high-definition video requires a huge amount of data, so services that involve transmission require a wideband transmission path.When radio waves are used for transmission, the frequency band is limited, so technology is used to compress and encode the video data according to the transmission band.

[0003] However, compression encoding causes degradation of the encoding, and the higher the compression rate, the greater the degradation. For this reason, there is a limit to the compression rate that can be used to obtain quality suitable for a service. For example, if you try to broadcast 8K over terrestrial waves (6 MHz transmission bandwidth), you will not be able to obtain sufficient image quality, and the service cannot be provided as is.

[0004] In conditions where sufficient transmission bandwidth cannot be secured, encoding degradation can be suppressed by lowering the video resolution, but this will result in a decrease in the quality of the service itself. For example, if 8K video is downconverted to 4K and broadcast, the service will naturally have 4K quality.

[0005] Therefore, it is considered that high-quality video can be obtained by separately transmitting complementary information using additional transmission means and using the data sent over these transmission paths together. In other words, it is considered that 4K service is provided over terrestrial waves with limited transmission bandwidth, and additional information is transmitted over a separate transmission path, and this information is combined with the 4K data to obtain 8K video. It is considered that hierarchical coding can be used as a means to realize such a service form (see, for example, Patent Document 1). Note that hierarchical coding is also called scalable coding.

[0006] In hierarchical coding, high-quality video can be reproduced by combining the base layer and the enhancement layer, and video can be reproduced using the base layer alone, although the quality will be lower. Several techniques have been proposed to achieve hierarchical coding. One method is called spatial hierarchical coding, which is a scalable coding method for video with different pixel counts (resolutions), such as the aforementioned 4K and 8K.

[0007] There is also known a technique for hierarchically encoding a video signal from coarse information to fine information (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-135522 [Patent Document 2] Japanese Patent Application Laid-Open No. 2011-244403 Summary of the Invention [Problem to be solved by the invention]

[0009] In spatial scalable coding, for example, low-resolution 4K video is upconverted to generate 8K video, and additional information for generating 8K video from 4K video is transmitted as an extension layer. This leads to poor prediction accuracy and low coding efficiency, which is a problem. This tendency is particularly noticeable when high compression rates are required, such as in terrestrial broadcasting.

[0010] For this reason, to obtain 8K video of sufficient quality using spatial hierarchical coding within a limited transmission bandwidth, the additional information required to obtain 8K video increases compared to 4K. Therefore, it is expected that the transmission capacity required to obtain the same quality will be almost the same compared to transmitting 4K and 8K video as is without hierarchical coding, and the benefits of adopting hierarchical coding are considered to be small.

[0011] Therefore, although conventional hierarchical coding is useful, there is a problem that coding efficiency is poor depending on the conditions.

[0012] Therefore, an object of the present invention is to provide a coding device, a decoding device, and a program that improve coding efficiency in hierarchical coding. [Means for solving the problem]

[0013] A coding device according to a first aspect is a device that clips an input video into a plurality of videos and hierarchically codes the clipped videos. The coding device includes: a clipping position designation unit that outputs clipping control information that designates a position and size of a main video area included in the input video, a region clipping unit that clips the main video area included in the input video based on the clipping control information, a region dividing unit that divides a video area other than the main video area that constitutes the input video into a plurality of sub-pictures, a base hierarchical coding unit that encodes the main video area output by the region clipping unit and outputs a base hierarchical coding stream containing only the coded main video, and an extension hierarchical coding unit that encodes each of the plurality of sub-pictures output by the region dividing unit and outputs an extension hierarchical coding stream that can be used in combination with the base hierarchical coding stream.

[0014] A decoding device according to a second aspect is a device that decodes an encoded stream obtained by hierarchical encoding. The decoding device includes a base layer decoding unit that decodes a base layer encoded stream and outputs a primary video region included in an input video, an extension layer decoding unit that decodes an extension layer encoded stream that can be used in combination with the base layer encoded stream and outputs a plurality of sub-pictures that constitute a video region other than the primary video region of the input video, a region combining unit that combines the sub-pictures output by the extension layer decoding unit and outputs the video region other than the primary video region, and a layer combining unit that combines the primary video region output by the base layer decoding unit with the video region other than the primary video region output by the region combining unit to restore the input video.

[0015] A program according to a third aspect causes a computer to function as the encoding device according to the first aspect.

[0016] A program according to a fourth aspect causes a computer to function as the decryption device according to the second aspect. [Effects of the Invention]

[0017] According to the present invention, it is possible to provide a coding device, a decoding device, and a program that improve coding efficiency in hierarchical coding. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a diagram illustrating a configuration of a video transmission system according to an embodiment. [Figure 2] FIG. 1 is a diagram illustrating a configuration of an encoding device according to an embodiment. [Figure 3] 10A and 10B are diagrams illustrating a method for specifying a cutout position according to the embodiment. [Figure 4] FIG. 4 is a diagram showing a first example of division in the region dividing section according to the embodiment. [Figure 5] FIG. 10 is a diagram illustrating a second example of division in the region dividing unit according to the embodiment. [Figure 6] FIG. 10 is a diagram illustrating a third example of division in the region dividing unit according to the embodiment. [Figure 7] FIG. 2 is a diagram illustrating a configuration of an extension hierarchical encoding unit according to the embodiment. [Figure 8] FIG. 1 is a diagram illustrating a configuration of a decoding device according to an embodiment. [Figure 9] FIG. 10 is a diagram illustrating a configuration of an extension layer decoding unit according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] An encoding device and a decoding device according to an embodiment will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0020] (Video Transmission System) First, a video transmission system according to an embodiment will be described. Fig. 1 is a diagram showing the configuration of a video transmission system 100 according to an embodiment.

[0021] As shown in Fig. 1, a video transmission system 100 includes a transmitting device 10, a receiving device 20A, and a receiving device 20B. The transmitting device 10 transmits video to the receiving devices 20A and 20B via a transmission path. The transmitting device 10 includes an encoding device 1 that performs hierarchical encoding on input video. In the following, an example will be given in which the video to be transmitted is 8K video, but it is not necessarily limited to 8K.

[0022] The encoding device 1 outputs a base hierarchically encoded stream and an extended hierarchically encoded stream through hierarchical encoding. In the example shown in Fig. 1, a transmitting device 10 transmits the base hierarchically encoded stream to receiving devices 20A and 20B via a broadcast transmission path 30. For example, the broadcast transmission path 30 is a transmission path using terrestrial waves. The transmitting device 10 provides a 4K video transmission service via terrestrial waves, which have a limited transmission band.

[0023] The transmitting device 10 also transmits the enhanced hierarchically encoded stream to the receiving device 20B via a communication transmission path 40. For example, the communication transmission path 40 is a wired communication line such as the Internet. The transmitting device 10 separately transmits the enhanced hierarchically encoded stream as complementary information using additional transmission means.

[0024] The receiving device 20A receives the base layer encoded stream via the broadcast transmission path 30. The receiving device 20A has a decoding device 2A that decodes 4K video from the base layer encoded stream. The receiving device 20A is able to play back video from the base layer encoded stream, although the quality will be reduced.

[0025] The receiving device 20B receives the base layer encoded stream via a broadcast transmission path 30, and also receives the extension layer encoded stream via a communication transmission path 40. The receiving device 20B has a decoding device 2B that combines the base layer encoded stream and the extension layer encoded stream to decode 8K video. The receiving device 20B can play back high-quality video by combining the base layer and the extension layer.

[0026] In this way, the video transmission system 100 transmits 4K and 8K video using hierarchical coding. Here, in the case of a large-screen, high-definition video service such as 8K, it is considered that viewers do not keep their eyes evenly fixed on the entire screen, but rather typically watch while concentrating their attention on a certain part of the screen.

[0027] Taking this into consideration, when generating 4K video from 8K, there is no need to reduce the entire screen; by cropping the 4K video (without reducing it) to include the focal part where important information is concentrated, it is possible to convey sufficient information.

[0028] The encoding device 1 of the transmission device 10 according to the embodiment generates 4K video by extracting a main video area (hereinafter referred to as a "base layer area") from 8K video, uses the extracted 4K base layer area as a base layer in hierarchical encoding, and transmits the remaining area (hereinafter referred to as a "base layer area") as an enhancement layer. This makes it possible to provide 8K video services without reducing encoding efficiency while providing 4K video services.

[0029] Here, the position where the 4K base layer region is cut out may be fixed or may be changed for each frame. Also, although an example of a base layer region where 4K video is cut out from 8K video is shown, the size of the base layer region may be changed to suit the performance of the device that displays the base layer.

[0030] The decoding device 2A of the receiving device 20A decodes the base layer decoded stream to obtain 4K base layer video. Meanwhile, the decoding device 2B of the receiving device 20B decodes the base layer region from the base layer stream and the enhancement layer region from the enhancement layer stream, and combines the base layer region and the enhancement layer region to obtain 8K video. In this way, 4K video and 8K video can be obtained.

[0031] Therefore, the video transmission system 100 according to the embodiment is expected to realize diverse, high-quality services by enabling both 4K (base layer) and 8K (base layer + enhancement layer) video without reducing encoding efficiency. This can be used to realize hierarchical services of 4K and 8K by broadcasting 4K on narrow-band terrestrial waves and transmitting additional information via wired communication such as the Internet.

[0032] (encoding device) Next, a coding device according to an embodiment will be described. Fig. 2 is a diagram showing the configuration of a coding device 1 according to an embodiment.

[0033] As shown in FIG. 2, the encoding device 1 includes an extraction position designation unit 11, a region extraction unit 12, a base hierarchical encoding unit 13, a region division unit 14, and an enhancement hierarchical encoding unit 15.

[0034] The cut-out position designation unit 11 outputs cut-out control information that designates the position and size of the base layer region to the region cut-out unit 12, the region division unit 14, and the extended layer coding unit 15. When determining the cut-out position, it is desirable to take care during program production so that important information is included in the base layer region. Alternatively, it is also possible to designate the cut-out position using a separate means for determining the region to be cut out as important information from the 8K video.

[0035] 3 is a diagram showing a method for specifying a cut-out position according to an embodiment. As shown in FIG. 3, the cut-out position specification unit 11 specifies the position and size of a 4K basic layer region to be cut out from an 8K input video as cut-out control information. For example, the cut-out position specification unit 11 outputs the coordinate values ​​(x, y) of the top left corner of the basic layer region and the number of horizontal and vertical pixels (w, h) as the cut-out control information.

[0036] Here, the cut-out position may be fixed or may vary depending on the content of the video content. Coordinate values ​​may be specified by inputting values ​​determined during content creation, or by automatically detecting and specifying the coordinates from the input video. For example, the cut-out position designation unit 11 may designate a basic layer area to be cut out from the 8K video as important information by performing face recognition processing on the input video. The size of the basic layer area to be cut out may also vary depending on the content of the video content.

[0037] In addition, although an example is shown in which the base layer area is arranged so that the long side of the 4K base layer area is parallel to the long side of the 8K input video, this does not necessarily have to be parallel, and the base layer area may be tilted relative to the horizontal direction. However, if the base layer area is not parallel, the division method and encoding process of the enhancement layer, which will be described later, will become more complicated.

[0038] The area cutout unit 12 cuts out a base layer area from the input video based on the cutout control information output by the cutout position designation unit 11, i.e., the coordinate values ​​(x, y) and the area size (w, h), and outputs the cutout base layer area to the base layer encoding unit 13 and the area division unit 14.

[0039] The base layer coding unit 13 codes the base layer region output by the region extraction unit 12 and outputs a base layer coded stream that can be used independently. The coding method used by the base layer coding unit 13 may be any method.

[0040] Based on the cut-out control information output by the cut-out position specification unit 11, the region division unit 14 divides the extension layer region of the input video other than the base layer region into regions of shapes that can be coded by the extension layer coding unit 15. Each of these divided regions is called a "sub-picture." Considering that a typical coding method involves coding rectangular regions as input, the shape of each sub-picture is assumed to be rectangular. The division method may be determined in advance, or may be changed depending on the content, etc., and the division method may be communicated to the decoding side each time by multiplexing it onto the stream, for example.

[0041] FIG. 4 is a diagram showing a first example of division in the region dividing unit 14 according to the embodiment.

[0042] As shown in Figure 4, in the first division example, the extended layer area other than the base layer area is divided into four sub-pictures by extending the vertical sides of the base layer area. The divided rectangular areas, or sub-pictures, are called A, B, C, and D. The shape (number of horizontal and vertical pixels) of each of sub-pictures A, B, C, and D is uniquely determined once the values ​​(x, y, w, h) specified as the crop control information are determined. In other words, if the division method "divide by extending the vertical sides" is specified, the original video can be restored using only the order of sub-pictures A, B, C, and D and the (x, y, w, h) information.

[0043] FIG. 5 is a diagram showing a second example of division in the region dividing unit 14 according to the embodiment.

[0044] 5, in the second division example, the area other than the base layer area is divided into four sub-pictures by extending the top edge of the base layer area to the left, the left edge of the base layer area to the bottom, the bottom edge of the base layer area to the right, and the right edge of the base layer area to the top. In this case, as in the first division example, if the division method is specified in advance, the image can be restored using only the values ​​(x, y, w, h) specified as the clipping control information.

[0045] FIG. 6 is a diagram showing a third example of division in the region dividing unit 14 according to the embodiment.

[0046] As shown in Figure 6, in the third division example, the enhancement layer region is further divided into smaller regions to obtain multiple sub-pictures. For example, if the enhancement layer region can be divided into small regions of the same shape (same size), there is an advantage that all sub-encoders in the enhancement layer coding process described below can be the same. However, it is necessary to limit (x, y, w, h) to values ​​that allow the enhancement layer region to be divided into an integer number of identical small regions both horizontally and vertically, without excess or deficiency.

[0047] The enhancement hierarchical coding unit 15 encodes each of the plurality of sub-pictures output by the region dividing unit 14, and outputs an enhancement hierarchical coded stream. Fig. 7 is a diagram showing the configuration of the enhancement hierarchical coding unit 15 according to an embodiment. As shown in Fig. 7, the enhancement hierarchical coding unit 15 has a plurality of sub-encoders 151a, 151b, 151c, etc., and a stream combining unit 152.

[0048] The multiple sub-encoders 151a, 151b, 151c, etc. encode the multiple sub-pictures output by the region dividing unit 14, respectively, and output the multiple encoded streams to the stream combining unit 152. In the examples of Figures 4 and 5, there are four sub-pictures, A, B, C, and D, so it is sufficient to use four systems of sub-encoders.

[0049] The stream synthesis unit 152 synthesizes the multiple coded streams output by the multiple sub-encoders 151a, 151b, 151c, etc. with the cut-out control information output by the cut-out position designation unit 11, and outputs an extended hierarchical coded stream. Any synthesis method may be used, but it is necessary to be able to determine the cut-out control information and which sub-picture the stream corresponds to and separate them.

[0050] As described above, the encoding device 1 according to the embodiment is a device that performs hierarchical encoding on input video, and includes an area extraction unit 12 that extracts a main video area (base layer area) included in the input video, an area division unit 14 that divides the remaining video area (extension layer area) other than the main video area of ​​the input video into multiple sub-pictures, a base layer encoding unit 13 that encodes the main video area output by the area extraction unit 12 and outputs a base layer encoded stream that can be used independently, and an extension layer encoding unit 15 that encodes each of the multiple sub-pictures output by the area division unit 14 and outputs an extension layer encoded stream that can be used in combination with the base layer encoded stream. This makes it possible to encode both 4K (base layer) and 8K (base layer + extension layer) video without reducing encoding efficiency, which is expected to enable the realization of diverse, high-quality services.

[0051] (Decoding device) Next, a decoding device according to an embodiment will be described. Fig. 8 is a diagram showing the configuration of a decoding device 2 according to an embodiment. Here, the decoding device 2 is the configuration of the decoding device 2B shown in Fig. 1.

[0052] As shown in FIG. 8, the decoding device 2 includes a base layer decoding unit 21, an enhancement layer decoding unit 22, a region synthesis unit 23, and a layer synthesis unit 24.

[0053] The base layer decoding unit 21 decodes the base layer coded stream and outputs the base layer region. If the decoding device 2 is the decoding device 2A shown in Fig. 1, only the base layer region, which is a 4K video, will be reproduced.

[0054] The enhancement layer decoding unit 22 decodes the enhancement layer coded stream and outputs a plurality of sub-pictures constituting the enhancement layer area and clipping control information. Fig. 9 is a diagram showing the configuration of the enhancement layer decoding unit 22 according to an embodiment. As shown in Fig. 9, the enhancement layer decoding unit 22 has a stream separating unit 221 and a plurality of sub-decoders 222a, 222b, 222c, etc.

[0055] The stream separation unit 221 separates the enhanced hierarchical coded stream into a plurality of coded streams corresponding to a plurality of sub-pictures and cut-out control information, and outputs the plurality of coded streams to a plurality of sub-decoders 222a, 222b, 222c, etc., and outputs the cut-out control information to the region synthesis unit 23 and the hierarchical synthesis unit 24. Here, the method of separating the streams and outputting them to the corresponding sub-decoders corresponds to the synthesis method used by the stream synthesis unit 152 of the coding device 1.

[0056] The plurality of sub-decoders 222a, 222b, 222c, etc. decode the plurality of sub-pictures from the plurality of coded streams output by the stream separation unit 221, and output the decoded sub-pictures to the region synthesis unit .

[0057] The region synthesis unit 23 synthesizes multiple sub-pictures output by the extension layer decoding unit 22 (multiple sub-decoders 222a, 222b, 222c, etc.) and outputs an extension layer region. The synthesis method corresponds to the division method used by the region division unit 14 of the encoding device 1. As described above, the division method used by the encoding device 1 may be determined in advance, or may be changed depending on the content, etc., and communicated each time by multiplexing the method into the stream, for example. The synthesis method is determined by the sub-picture division method and the clipping control information, so the region synthesis unit 23 synthesizes multiple sub-pictures accordingly and outputs an extension layer region.

[0058] Based on the cut-out control information output by the extension layer decoding unit 22, the layer synthesis unit 24 synthesizes the base layer region output by the base layer decoding unit 21 and the extension layer region output by the region synthesis unit 23, and restores and outputs the 8K video.

[0059] As described above, the decoding device 2 according to the embodiment is a device for decoding an encoded stream obtained by hierarchical encoding, and includes a base layer decoding unit 21 that decodes a base layer encoded stream that can be used alone and outputs a main video area (base layer area) included in the input video, an extension layer decoding unit 22 that decodes an extension layer encoded stream that can be used in combination with the base layer encoded stream and outputs a plurality of sub-pictures that constitute the remaining video area (extension layer area) other than the main video area of ​​the input video, a region combining unit 23 that combines the plurality of sub-pictures output by the extension layer decoding unit 22 and outputs the remaining video area, and a layer combining unit 24 that combines the main video area output by the base layer decoding unit 21 with the remaining video area output by the region combining unit 23 to restore the input video. This makes it possible to support both 4K (base layer) and 8K (base layer + extension layer) video without reducing encoding efficiency, which is expected to enable the realization of diverse, high-quality services.

[0060] (Other embodiments) In the embodiment, an example of configuring hierarchical coding as two layers using a base layer and an extension layer has been described, but it is also possible to use the method according to the embodiment recursively to configure three or more layers, or to combine it with other hierarchical coding methods.

[0061] Information about the area of ​​interest to the viewer may be fed back from the receiving device 20 (decoding device 2) to the transmitting device 10 (encoding device 1) using the communication transmission path 40. In this case, the cut-out position designation unit 11 may determine the base layer area to be cut out as important information based on the fed-back information.

[0062] In the embodiment, an example has been described in which one base layer region is extracted from the input video, but the region extraction unit 12 may extract two or more base layer regions from the input video. The receiving device 20 may select one base layer region to decode from the two or more base layer regions, and decode only the selected base layer region.

[0063] A program may be provided that causes a computer to execute each process performed by the encoding device 1. Also, a program may be provided that causes a computer to execute each process performed by the decoding device 2. The program may be recorded on a computer-readable medium. Using a computer-readable medium makes it possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

[0064] The circuits that execute the processes performed by the encoding device 1 may be integrated, and the encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC). The circuits that execute the processes performed by the decoding device 2 may be integrated, and the decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC).

[0065] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope that does not deviate from the gist of the invention. [Explanation of symbols]

[0066] 1: Encoding device 2: Decryption device 2A: Decryption device 2B: Decryption device 10: Transmitting device 11: Cut-out position specification section 12: Region extraction section 13: Basic layer encoding section 14: Area division part 15: Extended hierarchical coding unit 20: Receiving device 20A: Receiver 20B: Receiving device 21: Base layer decoding unit 22: Enhanced layer decoding unit 23: Area synthesis section 24: Layer synthesis section 30: Broadcast transmission channel 40: Communication transmission line 100: Video transmission system 151a: Sub-encoder 151b: Sub-encoder 151c: Sub-encoder 152: Stream synthesis unit 221: Stream separation unit 222a: Sub-decoder 222b: Sub-decoder 222c: Sub-decoder

Claims

1. a first decoding unit that decodes the coded stream and outputs a first video region included in the input video; a second decoding unit that decodes an encoded stream that can be used in combination with the encoded stream and outputs one or more sub-pictures that constitute a video area other than the first video area of ​​the input video; an acquisition unit that acquires control information that specifies the position and size of the first video area; a synthesis unit that synthesizes the first video area output by the first decoding unit and the one or more sub-pictures output by the second decoding unit based on the control information to restore the input video, A decoding device, characterized in that the control information specifying the size is limited in accordance with division into sub-pictures of the same size on the encoding side.

2. a first decoding unit that decodes the coded stream and outputs a first video region included in the input video; a second decoding unit that decodes an encoded stream that can be used in combination with the encoded stream and outputs one or more sub-pictures that constitute a video area other than the first video area of ​​the input video; an acquisition unit that acquires control information that specifies the position and size of the first video area; a synthesis unit that synthesizes the first video area output by the first decoding unit and the one or more sub-pictures output by the second decoding unit based on the control information to restore the input video, A decoding device characterized in that, in response to division into sub-pictures of the same size on the encoding side, the size of the sub-picture indicated by the control information is limited so that the input video can be divided into an integer number of identical sub-pictures without excess or deficiency.

Images