Reception apparatus and reception method

By generating and transmitting basic and extended video streams with format-specific identification, the technology addresses the challenge of transmitting high-quality image data alongside basic data, enabling efficient decoding and display adaptation.

JP2024161200AActive Publication Date: 2024-11-15SONY GROUP CORP
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2024151962
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2014-07-31
Filing Date
2024-09-04
Publication Date
2024-11-15
Estimated Expiration
2035-07-09

AI Technical Summary

Technical Problem

Existing technologies face challenges in successfully transmitting a predetermined number of high-quality format image data together with basic format image data, particularly in scenarios where selective use of high-quality data is required based on display capabilities.

Method used

An image encoding unit generates basic and extended video streams by encoding basic and high-quality format image data, with a transmitting unit sending these streams in a container format and an identification information insertion unit inserting format-specific information into the container or video stream layers, enabling selective decoding on the receiving side.

Benefits of technology

This approach allows for efficient transmission and decoding of image data according to display capabilities, ensuring high-quality data is appropriately processed and displayed based on receiver capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024161200000001_ABST
    Figure 2024161200000001_ABST
Patent Text Reader

Abstract

To successfully transmit basic format image data and a predetermined number of pieces of high-quality format image data.SOLUTION: A basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding respectively the predetermined number of pieces of high-quality format image data are generated. A container in a predetermined format including each of the video streams is transmitted. Identification information in a high-quality format corresponding to each of the predetermined number of extended video streams is inserted into a layer of the container and / or the video stream.SELECTED DRAWING: Figure 10
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present technology relates to a transmitting device, a transmitting method, a receiving device and a receiving method, and more particularly to a transmitting device etc. that transmits a predetermined number of high-quality format image data together with basic format image data. [Background technology]

[0002] Conventionally, it is known that high-quality format image data is transmitted together with basic format image data, and the receiving side selectively uses the basic format image data or the high-quality format image data. For example, Patent Document 1 describes that media coding is performed in a scalable manner to generate a base layer stream for a low-resolution video service and an enhancement layer stream for a high-resolution video service, and a broadcast signal including these is transmitted. In addition to high resolution, the high-quality format also includes high frame frequency, high dynamic range, wide color gamut, and high bit length. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2008-543142 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present technology is to successfully transmit a predetermined number of high-quality format image data together with basic format image data. [Means for solving the problem]

[0005] The concept of this technology is as follows: an image encoding unit for generating a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data; a transmitting unit that transmits a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated by the image encoding unit; an identification information inserting unit that inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into the container and / or a layer of the video stream; Located in the transmitting device.

[0006] In the present technology, a basic video stream and a predetermined number of extended video streams are generated by an image encoding unit, where the basic video stream is obtained by encoding basic format image data, and each of the predetermined number of extended video streams is obtained by encoding a predetermined number of high quality format image data.

[0007] For example, the image encoding unit may be configured to, with respect to basic format image data, perform predictive encoding processing within the basic format image data to generate a basic video stream, and with respect to high-quality format image data, selectively perform predictive encoding processing within the high-quality format image data or predictive encoding processing between the basic format image data or other high-quality format image data to generate an extended video stream.

[0008] The transmitting unit transmits a container in a predetermined format including the basic video stream generated by the image encoding unit and a predetermined number of extended video streams. For example, the container may be a transport stream (MPEG-2 TS) adopted in the digital broadcasting standard. Also, for example, the container may be MP4 used for Internet distribution or a container in another format.

[0009] The identification information inserting unit inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into a container or a layer of a video stream. For example, the container may be MPEG2-TS, and when the identification information inserting unit inserts the identification information into a layer of the container, the identification information may be inserted into each video elementary stream loop (video ES loop) corresponding to the predetermined number of extended video streams existing under a program map table (PMT). Also, for example, the video stream may have a NAL (Network Abstraction Layer) unit structure, and the identification information inserting unit may insert the identification information into a header of the NAL unit.

[0010] In this way, in the present technology, identification information of high-quality formats corresponding to a predetermined number of extended video streams is inserted into a container or a layer of a video stream and transmitted, so that the receiving side can easily obtain image data according to the display capability by selectively performing a decoding process on a predetermined video stream based on the identification information.

[0011] In addition, in the present technology, for example, information indicating whether each of the predetermined number of extended video streams is generated by performing a predictive coding process with basic format image data or a predictive coding process with high quality format image data may be added to the identification information inserted into the layer of the container. In this case, the receiving side can easily recognize whether the basic format image data or other high quality format image data was referenced in the predictive coding process when generating each of the predetermined number of extended video streams.

[0012] In addition, in the present technology, for example, information indicating a video stream corresponding to image data referenced in a predictive coding process between basic format image data or other high quality format image data performed when generating each of the predetermined number of extended video streams may be added to the identification information inserted into the layer of the container. In this case, the receiving side can easily recognize which video stream corresponds to the image data referenced in the predictive coding process when generating each of the predetermined number of extended video streams.

[0013] Another concept of the present technology is a receiving unit for receiving a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into the container layer and / or the video stream layer; The video stream receiving device further includes a processor configured to process each of the video streams included in the received container based on the identification information. in the receiving device.

[0014] In the present technology, a receiving unit receives a container including a basic video stream and a predetermined number of extended video streams. Here, the basic video stream is obtained by encoding basic format image data. Also, each of the predetermined number of extended video streams is obtained by encoding a predetermined number of high quality format image data. Identification information of the high quality format to which each of the predetermined number of extended video streams corresponds is inserted into the container and the video stream layers.

[0015] For example, the basic video stream may be generated by performing a predictive coding process within basic format image data on the basic format image data, and the extended video stream may be generated by selectively performing a predictive coding process within the high quality format image data or a predictive coding process between the high quality format image data and basic format image data or other high quality format image data on the high quality format image data.

[0016] The processing unit processes each video stream included in the received container based on the identification information. For example, the processing unit may perform a decoding process on the basic video stream and a predetermined extended video stream based on the identification information and the display capability information to obtain image data corresponding to the display capability.

[0017] In this way, in this technology, a predetermined number of extended video streams inserted into a container or a video stream layer are processed based on the identification information of the corresponding high-quality format, making it easy to selectively decode a predetermined video stream to obtain image data according to the receiving capability. Effect of the Invention

[0018] According to the present technology, a predetermined number of high-quality format image data can be successfully transmitted together with basic format image data. Note that the effects described herein are not necessarily limited to those described herein, and may be any of the effects described in the present disclosure. [Brief description of the drawings]

[0019] [Figure 1] 1 is a block diagram showing an example of the configuration of a transmission / reception system according to an embodiment; [Diagram 2] FIG. 2 is a block diagram showing a configuration example of a transmission device. [Diagram 3]11 is a block diagram showing an example of the configuration of an image data generating unit that generates basic format image data Vb and three high quality format image data Vh1, Vh2, and Vh3. FIG. [Figure 4] 2 is a block diagram showing an example of the configuration of a main part of an encoding unit. FIG. [Diagram 5] A diagram showing an example of the structure of a NAL unit header and the contents of main parameters in the example structure. [Figure 6] FIG. 2 is a diagram showing an example of the configuration of a basic video stream STb and extended video streams STe1, STe2, and STe3. [Figure 7] FIG. 13 is a diagram showing an example of the structure of a scalable extension descriptor. [Figure 8] FIG. 13 is a diagram showing the contents of main information in an example of the structure of a scalable extension descriptor. [Figure 9] 1 is a diagram showing the correspondence between the value of the "type of enhancement" field in the scalable extension descriptor and the value of the "nuh_layer_id" field in the NAL unit header. [Figure 10] FIG. 2 is a diagram illustrating an example of the configuration of a transport stream TS. [Figure 11] FIG. 2 is a block diagram showing a configuration example of a receiving device. [Figure 12] 4 is a block diagram showing an example of the configuration of a main part of a decoding unit; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, modes for carrying out the invention (hereinafter, referred to as "embodiments") will be described in the following order. 1. Embodiment 2. Variations

[0021] <1. Preferred embodiment> [Transmitting and receiving system] 1 shows an example of the configuration of a transmission / reception system 10 according to an embodiment. The transmission / reception system 10 includes a transmission device 100 and a reception device 200.

[0022] The transmitting device 100 transmits the transport stream TS as a container via broadcast waves or network packets. This transport stream TS includes a basic video stream and a predetermined number of extended video streams.

[0023] The basic video stream is generated by encoding basic format image data with, for example, H.264 / AVC, H.265 / HEVC, etc. Here, predictive encoding processing is performed within the basic format image data to generate the basic video stream.

[0024] The predetermined number of extended video streams are generated by encoding a predetermined number of high-quality image data with, for example, H.264 / AVC, H.265 / HEVC, etc. Here, with respect to the high-quality format image data, a predictive encoding process within this high-quality format image data or a predictive encoding process between basic format image data or other high-quality format image data is selectively performed to generate the extended video stream.

[0025] Identification information of the high-quality formats corresponding to the predetermined number of extended video streams is inserted into the layer of the container. The identification information allows the receiving side to easily grasp the high-quality formats corresponding to the predetermined number of extended video streams in the layer of the container. In this embodiment, the identification information is inserted into each video elementary stream loop corresponding to the predetermined number of extended video streams existing under the program map table.

[0026] To this identification information, information indicating whether each of the predetermined number of extended video streams is generated by performing predictive coding processing with basic format image data or predictive coding processing with high quality format image data is added. This information makes it possible for the receiving side to easily recognize in the container layer whether the basic format image data or other high quality format image data was referenced in the predictive coding processing when each of the predetermined number of extended video streams is generated.

[0027] In addition, information indicating a video stream corresponding to image data referenced in a predictive coding process between basic format image data or other high quality format image data performed when generating each of the predetermined number of extended video streams is added to this identification information. This information makes it possible for the receiving side to easily recognize, in the container layer, which video stream corresponds to the image data referenced in the predictive coding process when generating each of the predetermined number of extended video streams.

[0028] Identification information of the high-quality formats to which the predetermined number of extended video streams respectively correspond is inserted into a layer of the video stream. The identification information allows the receiving side to easily grasp the high-quality formats to which the predetermined number of extended video streams respectively correspond. In this embodiment, the identification information is inserted into the header of the NAL unit.

[0029] The receiving device 200 receives the above-mentioned transport stream TS transmitted by the transmitting device 100 on broadcast waves or network packets. As described above, identification information of high-quality formats corresponding to a predetermined number of extended video streams included in the transport stream TS is inserted in the container or video stream layer. Based on this identification information, the receiving device 200 processes each video stream included in the transport stream TS and obtains image data according to its display capability.

[0030] "Transmitter Configuration" 2 shows an example of the configuration of the transmitting device 100. This transmitting device 100 handles basic format image data Vb and three high-quality format image data Vh1, Vh2, and Vh3 as transmission image data. Here, the basic format image data Vb is LDR (Low Dynamic Range) image data with a frame frequency of 50 Hz. The high-quality format image data Vh1 is LDR image data with a frame frequency of 100 Hz. The LDR image data has a luminance range of 0% to 100% with respect to the brightness of the white peak of a conventional LDR image.

[0031] The high-quality format image data Vh2 is HDR (High Dynamic Range) image data with a frame frequency of 50 Hz. The high-quality format image data Vh3 is HDR image data with a frame frequency of 100 Hz. If the brightness of the white peak of a conventional LDR image is 100%, this HDR image data has a brightness range of 0 to 100%*N, for example, 0 to 400% or 0 to 800%.

[0032] 3 shows an example of the configuration of an image data generation unit 150 that generates basic format image data Vb and three high-quality format image data Vh1, Vh2, and Vh3. The image data generation unit 150 includes an HDR camera 151, a frame rate conversion unit 152, a dynamic range conversion unit 153, and a frame rate conversion unit 154.

[0033] The HDR camera 151 captures an image of a subject and outputs HDR image data having a frame frequency of 100 Hz, i.e., high-quality format image data Vh3. The frame rate conversion unit 152 converts the frame frequency of the high-quality format image data Vh3 output from the HDR camera 151 from 100 Hz to 50 Hz, and outputs HDR image data having a frame frequency of 50 Hz, i.e., high-quality format image data Vh2.

[0034] The dynamic range conversion unit 153 performs HDR to LDR conversion processing on the high-quality format image data Vh3 output from the HDR camera 151, and outputs LDR image data with a frame frequency of 100 Hz, i.e., high-quality format image data Vh1. The frame rate conversion unit 154 performs a process of converting the frame frequency of the high-quality format image data Vh1 output from the dynamic range conversion unit 153 from 100 Hz to 50 Hz, and outputs LDR image data with a frame frequency of 50 Hz, i.e., basic format image data Vb.

[0035] Returning to Fig. 2, the transmitting device 100 has a control unit 101, LDR photoelectric conversion units 102 and 103, HDR photoelectric conversion units 104 and 105, a video encoder 106, a system encoder 107, and a transmitting unit 108. The control unit 101 is configured with a CPU (Central Processing Unit), and controls the operation of each unit of the transmitting device 100 based on a control program.

[0036] The LDR photoelectric conversion unit 102 applies photoelectric conversion characteristics for LDR images (LDR OETF curve) to the basic format image data Vb to obtain basic format image data Vb' for transmission. The LDR photoelectric conversion unit 103 applies photoelectric conversion characteristics for LDR images to the high quality format image data Vh1 to obtain high quality format image data Vh1' for transmission.

[0037] The HDR photoelectric conversion unit 104 applies photoelectric conversion characteristics for HDR images (HDR OETF curve) to the high-quality format image data Vh2 to obtain high-quality format image data Vh2' for transmission. The HDR photoelectric conversion unit 105 applies photoelectric conversion characteristics for HDR images to the high-quality format image data Vh3 to obtain high-quality format image data Vh3' for transmission.

[0038] The video encoder 106 has four encoding units 106-0, 106-1, 106-2, and 106-3. The encoding unit 106-0 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on basic format image data Vb' for transmission to generate a basic video stream STb. In this case, the encoding unit 106-0 performs prediction within the image data Vb'.

[0039] The encoding unit 106-1 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh1' for transmission to generate an extended video stream STe1. In this case, the encoding unit 106-1 selectively performs prediction within the image data Vh1' or prediction between the image data Vb' for each coding block in order to reduce a prediction residual.

[0040] The encoding unit 106-2 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh2' for transmission to generate an extended video stream STe2. In this case, the encoding unit 106-2 selectively performs prediction within the image data Vh2' or prediction between the image data Vb' for each coding block in order to reduce a prediction residual.

[0041] The encoding unit 106-3 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh3' for transmission to generate an extended video stream STe3. In this case, the encoding unit 106-3 selectively performs prediction within the image data Vh3' or prediction between the image data Vh2' for each coding block in order to reduce a prediction residual.

[0042] 4 shows a configuration example of the main part of the encoding unit 160. This encoding unit 160 can be applied to the encoding units 106-1, 106-2, and 106-3. This encoding unit 160 has an intra-layer prediction unit 161, an inter-layer prediction unit 162, a prediction adjustment unit 163, a selection unit 164, and an encoding function unit 165.

[0043] The intra-layer prediction unit 161 performs prediction (intra-layer prediction) on the image data V1 to be coded to obtain prediction residual data. The inter-layer prediction unit 162 performs prediction (inter-layer prediction) on the image data V1 to be coded with reference image data V2 to obtain prediction residual data.

[0044] In order to efficiently perform inter-layer prediction in the inter-layer prediction unit 162, the prediction adjustment unit 163 performs the following processing according to the type of scalable extension of the image data V1 to the image data V2. In the case of dynamic range extension, level adjustment is performed to convert from LDR to HDR. In the case of spatial scalable extension, blocks are enlarged to a predetermined size. In the case of frame rate extension, bypass is performed. In the case of color gamut extension, mapping is performed for each of luminance and chrominance. In the case of bit length extension, conversion is performed to align the MSB of pixels.

[0045] For example, in the case of the encoding unit 106-1, the image data V1 is high-quality format image data Vh1' (100 Hz, LDR), the image data V2 is basic format image data Vb' (50 Hz, LDR), and the type of scalable extension corresponds to frame rate extension. Therefore, in the prediction adjustment unit 163, the image data Vb' is bypassed as it is.

[0046] For example, in the case of the encoding unit 106-2, the image data V1 is high-quality format image data Vh2' (50 Hz, HDR), the image data V2 is basic format image data Vb' (50 Hz, LDR), and the type of scalable extension corresponds to dynamic range extension. Therefore, the prediction adjustment unit 163 performs level adjustment on the image data Vb' to convert it from LDR to HDR.

[0047] Also, for example, in the case of the encoding unit 106-3, the image data V1 is high-quality format image data Vh3' (100 Hz, HDR), the image data V2 is high-quality format image data Vh2' (50 Hz, HDR), and the type of scalable extension corresponds to frame rate extension. Therefore, in the prediction adjustment unit 163, the image data Vb' is bypassed as it is.

[0048] The selection unit 164 selectively extracts the prediction residual data obtained by the intra-layer prediction unit 161 or the prediction residual data obtained by the inter-layer prediction unit 162 for each coding block, and sends it to the encoding function unit 165. In this case, the selection unit 164 extracts, for example, the prediction residual having a smaller value. The encoding function unit 165 performs encoding processes such as transform coding, quantization, and entropy coding on the prediction residual data extracted from the selection unit 164 to obtain a video stream ST.

[0049] 2, the video encoder 106 inserts identification information of the corresponding high-quality formats into the layers of the extended video streams STe1, STe2, and STe3. The video encoder 106 inserts this identification information into, for example, the header of the NAL unit.

[0050] Figure 5(a) shows an example structure (Syntax) of a NAL unit header, and Figure 5(b) shows the contents (Semantics) of the main parameters in that example structure. The 1-bit field "Forbidden_zero_bit" must be 0. The 6-bit field "nal_unit_type" indicates the NAL unit type. The 6-bit field "Nuh_layer_id" is an ID indicating the layer extension type of the stream. The 3-bit field "nuh_temporal_id_plus1" indicates temporal_id (0-6) and takes on a value of 1 plus 1 (1-7).

[0051] In this embodiment, the 6-bit field of "nuh_layer_id" indicates identification information of the high quality format that each extended video stream corresponds to (extension category information of the stream). For example, "0" indicates a base stream. "1-4" indicate spatial extension streams. "5-8" indicate frame rate extension streams. "9-12" indicate dynamic range extension streams. "13-16" indicate color gamut extension streams. "17-20" indicate bit length extension streams. "21-24" indicate spatial extension and frame rate extension. "25-28" indicate frame rate extension and dynamic range extension.

[0052] For example, the basic video stream STb corresponds to a base stream, and therefore "nuh_layer_id" in the header of the NAL unit constituting this basic video stream STb is set to "0." Also, for example, the extended video stream STe1 corresponds to a frame rate extended stream, and therefore "nuh_layer_id" in the header of the NAL unit constituting this extended video stream STe1 is set to any value in the range of "5 to 8."

[0053] For example, the extended video stream STe2 corresponds to a dynamic range extended stream, and therefore the "nuh_layer_id" in the header of the NAL unit constituting this extended video stream STe2 is anywhere in the range of "9 to 12." For example, the extended video stream STe3 corresponds to a frame rate extended and dynamic range extended stream, and therefore the "nuh_layer_id" in the header of the NAL unit constituting this extended video stream STe3 is anywhere in the range of "25 to 28."

[0054] 6 shows an example of the configuration of a basic video stream STb and extended video streams STe1, STe2, and STe3. The horizontal axis indicates the picture order of composition (POC), with the left side indicating earlier display times and the right side indicating later display times. Each rectangular frame indicates a picture, and the solid arrows indicate the reference relationship of pictures in predictive coding.

[0055] The basic video stream STb is composed of coded image data for pictures "00", "01", .... The extended video stream STe1 is composed of coded image data for pictures "10", "11", ..., which are located between each picture in the basic video stream STb. The extended video stream STe2 is composed of coded image data for pictures "20", "21", ..., which are located in the same positions as each picture in the basic video stream STb. And the extended video stream STe3 is composed of coded image data for pictures "30", "31", ..., which are located between each picture in the extended video stream STe2.

[0056] 2, the system encoder 107 generates a transport stream TS including the basic video stream STb and the extended video streams STe1, STe2, and STe3 generated by the video encoder 106. Then, the transmitting unit 108 transmits this transport stream TS to the receiving device 200 via broadcast waves or net packets.

[0057] At this time, the system encoder 107 inserts identification information of the high quality formats corresponding to each of the extended video streams STe1, STe2, and STe3 into the layer of the container (transport stream). In this embodiment, for example, a scalable extension descriptor including the identification information is inserted into a video elementary stream loop corresponding to each extended video stream existing under a PMT (Program Map Table).

[0058] Figure 7 shows an example of the structure (Syntax) of this scalable extension descriptor. Figure 8 shows the main information contents (Semantics) in the example structure shown in Figure 7. The 8-bit field "descriptor_tag" indicates the descriptor type, which indicates that it is a scalable extension descriptor in this case. The 8-bit field "descriptor_length" indicates the length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor.

[0059] The 4-bit field of "type of enhancement" indicates the identification information of the high quality format that each enhanced video stream corresponds to (extension category information of the stream). For example, "1" indicates spatial scalable enhancement; "2" indicates frame rate scalable enhancement; "3" indicates dynamic range scalable enhancement; "4" indicates color gamut scalable enhancement; "5" indicates bit depth scalable enhancement; "6" indicates spatial / frame rate scalable enhancement; and "7" indicates frame rate / dynamic range scalable enhancement.

[0060] For example, the extended video stream STe1 corresponds to the frame rate scalable extension, and therefore the "type of enhancement" of the scalable extension descriptor corresponding to this extended video stream STe1 is set to "2".

[0061] Also, for example, the extended video stream STe2 corresponds to the dynamic range scalable extension, and therefore the "type of enhancement" of the scalable extension descriptor corresponding to this extended video stream STe2 is set to "3".

[0062] For example, the extended video stream STe3 corresponds to the frame rate / dynamic range scalable extension, and therefore the "type of enhancement" of the scalable extension descriptor corresponding to this extended video stream STe3 is set to "7".

[0063] Figure 9 shows the correspondence between the value of the "type of enhancement" field and the value of the "nuh_layer_id" field in the NAL unit header. As shown above, it is possible to grasp the identification information of the high quality format (extension category information of the stream) corresponding to each enhanced video stream in the same way from either field.

[0064] Returning to Fig. 7, the 4-bit field of "scalable_priority" indicates the priority of each extended video stream within the same extended category. That is, this field indicates whether each extended video stream is generated by performing predictive coding processing with basic format image data or with high quality format image data.

[0065] For example, "0" indicates that it is a first priority stream that references the basic stream, i.e., that it is generated by performing predictive coding processing with basic format image data, while "1" indicates that it is a second priority stream that references the first priority stream, i.e., that it is generated by performing predictive coding processing with high quality format image data.

[0066] For example, the extended video stream STe1 is related to the encoding of high-quality format image data Vh1' and is generated by performing predictive encoding processing with basic format image data Vb'. Therefore, the "scalable_priority" of the scalable extension descriptor corresponding to this extended video stream STe1 is set to "0".

[0067] Also, for example, the extended video stream STe2 is related to the encoding of high-quality format image data Vh2' and is generated by performing predictive encoding processing with basic format image data Vb'. Therefore, the "scalable_priority" of the scalable extension descriptor corresponding to this extended video stream STe2 is set to "0".

[0068] Also, for example, the extended video stream STe3 is related to the encoding of high-quality format image data Vh3' and is generated by performing predictive encoding processing with high-quality format image data Vh2'. Therefore, the "scalable_priority" of the scalable extension descriptor corresponding to this extended video stream STe3 is set to "1".

[0069] The 32-bit field of "enhancement reference PID" indicates the PID value of the reference stream. That is, this field indicates the PID value of the video stream corresponding to the image data referenced in the predictive coding process between the basic format image data or other high quality format image data performed when generating each enhanced video stream.

[0070] For example, the extended video stream STe1 is related to the encoding of high-quality format image data Vh1' and is generated by performing predictive encoding processing with basic format image data Vb'. Therefore, the "enhancement reference PID" of the scalable extension descriptor corresponding to this extended video stream STe1 indicates the PID value of the basic video stream STb.

[0071] Also, for example, the extended video stream STe2 is related to the encoding of the high-quality format image data Vh2' and is generated by performing predictive encoding processing with the basic format image data Vb'. Therefore, the "enhancement reference PID" of the scalable extension descriptor corresponding to this extended video stream STe2 indicates the PID value of the basic video stream STb.

[0072] Also, for example, the extended video stream STe3 is related to the encoding of the high-quality format image data Vh3' and is generated by performing predictive encoding processing with the high-quality format image data Vh2'. Therefore, the "enhancement reference PID" of the scalable extension descriptor corresponding to this extended video stream STe3 indicates the PID value of the extended video stream STe2.

[0073] [Transport Stream TS Structure] Fig. 10 shows an example of the structure of a transport stream TS. This transport stream TS includes four video streams: a basic video stream STb and extended video streams STe1, STe2, and STe3. In this example, a PES packet "video PES" exists for each video stream.

[0074] The packet identifier (PID) of the basic video stream STb is, for example, PID1. The coded image data of each picture of this video stream includes NAL units such as AUD, VPS, SPS, PPS, PSEI, SLICE, SSEI, and EOS. The "nuh_layer_id" in the headers of these NAL units is set to "0", indicating that they are basic video streams (see FIG. 9).

[0075] The packet identifier (PID) of the extended video stream STe1 is, for example, PID2. The coded image data of each picture in this video stream includes NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS. The "nuh_layer_id" in the headers of these NAL units is, for example, "5", indicating that this is a frame rate extended stream (see FIG. 9).

[0076] The packet identifier (PID) of the extended video stream STe2 is, for example, PID3. The coded image data of each picture in this video stream includes NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS. The "nuh_layer_id" in the headers of these NAL units is, for example, "9", indicating that the stream is an extended dynamic range stream (see FIG. 9).

[0077] Furthermore, the packet identifier (PID) of the extended video stream STe3 is, for example, PID4. The coded image data of each picture in this video stream includes NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS. The "nuh_layer_id" in the headers of these NAL units is, for example, "25", indicating that this is a frame rate extended and dynamic range extended stream (see FIG. 9).

[0078] The transport stream TS also includes a program map table (PMT) as program specific information (PSI). This PSI is information that indicates which program each elementary stream included in the transport stream belongs to.

[0079] The PMT contains a program loop that describes information related to the entire program. The PMT also contains elementary stream loops that contain information related to each elementary stream. In this configuration example, there are four video elementary stream loops (video ES loops) corresponding to four video streams: basic video stream STb and extended video streams STe1, STe2, and STe3. The video elementary stream loop corresponding to basic video stream STb contains information such as the stream type (ST0) and packet identifier (PID1).

[0080] In addition, in the video elementary stream loop corresponding to the extended video stream STe1, information such as a stream type (ST1) and a packet identifier (PID2) is placed, and a descriptor describing information related to this extended video stream STe1 is also placed. As one of these descriptors, the above-mentioned scalable extension descriptor is inserted.

[0081] The "type of enhancement" in this descriptor is set to "2", indicating that it is a frame rate enhancement stream (frame rate scalable enhancement) (see FIG. 9). The "scalable_priority" in this descriptor is set to "0", indicating that it is a first priority stream that references the basic stream. The "enhancement reference PID" in this descriptor is set to "PID1", indicating that it references the basic video stream STb.

[0082] In addition, in the video elementary stream loop corresponding to the extended video stream STe2, information such as the stream type (ST2) and packet identifier (PID3) is placed, and a descriptor describing information related to this extended video stream STe2 is also placed. The above-mentioned scalable extension descriptor is inserted as one of these descriptors.

[0083] The "type of enhancement" in this descriptor is set to "3", indicating that it is a dynamic range enhancement stream (dynamic range scalable enhancement) (see FIG. 9). Also, the "scalable_priority" in this descriptor is set to "0", indicating that it is a first priority stream that references the basic stream. Also, the "enhancement reference PID" in this descriptor is set to "PID1", indicating that it references the basic video stream STb.

[0084] In addition, in the video elementary stream loop corresponding to the extended video stream STe3, information such as the stream type (ST3) and packet identifier (PID4) is placed, and a descriptor describing information related to this extended video stream STe3 is also placed. The above-mentioned scalable extension descriptor is inserted as one of these descriptors.

[0085] The "type of enhancement" in this descriptor is set to "7", indicating that it is a frame rate and dynamic range extended stream (frame rate / dynamic range scalable extension) (see FIG. 9). The "scalable_priority" in this descriptor is set to "1", indicating that it is a second priority stream that references the first priority stream. The "enhancement reference PID" in this descriptor is set to "PID3", indicating that it references the enhanced video stream STe2.

[0086] The operation of the transmitting device 100 shown in Fig. 2 will be briefly described. Basic format image data Vb, which is LDR image data with a frame frequency of 50 Hz, is supplied to an LDR photoelectric conversion unit 102. In this LDR photoelectric conversion unit 102, photoelectric conversion characteristics for LDR images (LDR OETF curve) are applied to the basic format image data Vb to obtain basic format image data Vb' for transmission. This basic format image data Vb' is supplied to encoding units 106-0, 106-1, and 106-2 of a video encoder 106.

[0087] Furthermore, the high-quality format image data Vh1, which is LDR image data with a frame frequency of 100 Hz, is supplied to the LDR photoelectric conversion unit 103. In this LDR photoelectric conversion unit 103, photoelectric conversion characteristics for LDR images (LDR OETF curve) are applied to the high-quality format image data Vh1 to obtain high-quality format image data Vh1' for transmission. This high-quality format image data Vh1' is supplied to the encoding unit 106-1 of the video encoder 106.

[0088] Moreover, high-quality format image data Vh2, which is HDR image data with a frame frequency of 50 Hz, is supplied to the HDR photoelectric conversion unit 104. In this HDR photoelectric conversion unit 104, photoelectric conversion characteristics for HDR images (HDR OETF curve) are applied to the high-quality format image data Vh2 to obtain high-quality format image data Vh2' for transmission. This high-quality format image data Vh2' is supplied to encoding units 106-2 and 106-3 of the video encoder 106.

[0089] Moreover, high-quality format image data Vh3, which is HDR image data with a frame frequency of 100 Hz, is supplied to the HDR photoelectric conversion unit 105. In this HDR photoelectric conversion unit 105, photoelectric conversion characteristics for HDR images (HDR OETF curve) are applied to the high-quality format image data Vh3 to obtain high-quality format image data Vh3' for transmission. This high-quality format image data Vh3' is supplied to the encoding unit 106-3 of the video encoder 106.

[0090] In the video encoder 106, the basic format image data Vb' and the high quality format image data Vh1', Vh2', and Vh3' are each subjected to encoding processing to generate a video stream. That is, in the encoding unit 106-0, predictive encoding processing such as H.264 / AVC and H.265 / HEVC is performed on the basic format image data Vb' for transmission, and a basic video stream STb including encoded image data of each picture is generated. In this case, the encoding unit 106-0 performs prediction within the image data Vb'.

[0091] Furthermore, the encoding unit 106-1 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh1' for transmission, and generates an extended video stream STe1 including the encoded image data of each picture. In this case, the encoding unit 106-1 selectively performs prediction within the image data Vh1' or prediction between the image data Vb' for each encoding block in order to reduce the prediction residual.

[0092] Moreover, the encoding unit 106-2 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh2' for transmission, and generates an extended video stream STe2 including encoded image data of each picture. In this case, the encoding unit 106-2 selectively performs prediction within the image data Vh2' or prediction between the image data Vb' for each encoding block in order to reduce the prediction residual.

[0093] Moreover, the encoding unit 106-3 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh3' for transmission, and generates an extended video stream STe3 including encoded image data of each picture. In this case, the encoding unit 106-3 selectively performs prediction within the image data Vh3' or prediction between the image data Vh2' for each encoding block in order to reduce a prediction residual.

[0094] Furthermore, the video encoder 106 inserts identification information of the high-quality formats corresponding to the layers of the extended video streams STe1, STe2, and STe3. That is, the video encoder 106 sets identification information of the high-quality formats corresponding to each extended video stream (extension category information of the stream) in the "nuh_layer_id" field of the header of the NAL unit (see FIGS. 5 and 9).

[0095] The basic video stream STb and the extended video streams STe1, STe2, and STe generated by the video encoder 106 are supplied to a system encoder 107. In this system encoder 107, a transport stream TS including each video stream is generated.

[0096] In this system encoder 107, identification information of the high-quality formats corresponding to each of the extended video streams STe1, STe2, and STe3 is inserted into the layer of the container (transport stream). That is, in the system encoder 107, a scalable extension descriptor including identification information (extension category information of the stream) is inserted into a video elementary stream loop corresponding to each extended video stream existing under the PMT (see FIGS. 7 and 9).

[0097] The transport stream TS generated by the system encoder 107 is sent to the transmitting unit 108. In the transmitting unit 108, this transport stream TS is transmitted to the receiving device 200 via broadcast waves or net packets.

[0098] "Configuration of receiving device" Fig. 11 shows a configuration example of a receiving device 200. This receiving device 200 corresponds to the configuration example of the transmitting device 100 of Fig. 2. This receiving device 200 has a control unit 201, a receiving unit 202, a system decoder 203, a video decoder 204, LDR electric-optical conversion units 205 and 206L, HDR electric-optical conversion units 207 and 208, and a display unit (display device) 209. The control unit 201 is configured with a CPU (Central Processing Unit) and controls the operation of each unit of the receiving device 200 based on a control program stored in a storage device (not shown).

[0099] The receiving unit 202 receives the transport stream TS transmitted by broadcast waves or network packets from the transmitting device 100. The system decoder 203 extracts the basic video stream STb and the extended video streams STe1, STe2, and STe3 from this transport stream TS.

[0100] The system decoder 203 also extracts various information inserted into the container (transport stream) layer and sends it to the control unit 201. This information includes the above-mentioned scalable extension descriptor. The control unit 201 can grasp the identification information of the high-quality format (stream extension category information) corresponding to each of the extended video streams STe1, STe2, and STe3 from the "type of enhancement" field of this descriptor.

[0101] Also, the control unit 201 can grasp, from the "scalable_priority" field of this descriptor, the priority order of each of the enhanced video streams STe1, STe2, and STe3 within the same enhancement category, i.e., whether it is a first priority stream that references a basic stream or a second priority stream that references the first priority stream. Furthermore, the control unit 201 can grasp, from the "enhancement reference PID" field of this descriptor, the PID value of the video stream referenced by each of the enhanced video streams STe1, STe2, and STe3.

[0102] The video decoder 204 has four decoding units 204-0, 204-1, 204-2, and 204-3. The decoding unit 204-0 performs a decoding process on the basic video stream STb to generate basic format image data Vb'. In this case, the decoding unit 204-0 performs prediction compensation within the image data Vb'.

[0103] The decoding unit 204-1 performs a decoding process on the extended video stream STe1 to generate high-quality format image data Vh1'. In this case, the decoding unit 204-1 performs prediction compensation within the image data Vh1' or prediction compensation between the image data Vb' for each coding block in correspondence with the prediction at the time of coding.

[0104] The decoding unit 204-2 performs a decoding process on the extended video stream STe2 to generate high-quality format image data Vh2'. In this case, the decoding unit 204-2 performs prediction compensation within the image data Vh2' or prediction compensation between the image data Vb' for each coding block in correspondence with the prediction at the time of coding.

[0105] The decoding unit 204-3 performs a decoding process on the extended video stream STe3 to generate high-quality format image data Vh3'. In this case, the decoding unit 204-3 performs prediction compensation within the image data Vh3' or prediction compensation between the image data Vh2' for each coding block in correspondence with the prediction at the time of coding.

[0106] 12 shows a configuration example of the main part of the decoding unit 240. This decoding unit 240 can be applied to the decoding units 204-1, 204-2, and 204-3. This decoding unit 240 performs processing that is the opposite of the processing of the encoding unit 165 in FIG. 4. This decoding unit 240 has a decoding function unit 241, an intra-layer prediction compensation unit 242, an inter-layer prediction compensation unit 243, a prediction adjustment unit 244, and a selection unit 245.

[0107] The decode function unit 241 performs a decoding process other than prediction compensation on the video stream ST to obtain prediction residual data. The intra-layer prediction compensation unit 242 performs prediction compensation (intra-layer prediction compensation) on the prediction residual data within the image data V1 to obtain image data V1. The inter-layer prediction compensation unit 243 performs prediction compensation (inter-layer prediction compensation) on the prediction residual data with the reference image data V2 to obtain image data V1.

[0108] Although detailed description is omitted, the prediction adjustment unit 244 performs processing according to the type of scalable extension of the image data V1 to the image data V2, similar to the prediction adjustment unit 163 of the encoding unit 160 in Fig. 4. The selection unit 245 selectively extracts the image data V1 obtained by the intra-layer prediction compensation unit 242 or the image data V1 obtained by the inter-layer prediction compensation unit 243 for each coding block in correspondence with the prediction at the time of coding, and outputs it.

[0109] 11, the video decoder 204 sends header information of the NAL units of each video stream to the control unit 201. The control unit 201 can grasp the identification information of the high quality format (extended category information of the stream) corresponding to each of the extended video streams STe1, STe2, and STe3 from the "nuh_layer_id" field of this header information.

[0110] The LDR electric-to-optical converter 205 performs electric-to-optical conversion with the reverse characteristics to those of the LDR electric-to-electrical converter 102 in the above-described transmitting device 100 on the basic format image data Vb' obtained by the decoding unit 204-0, and obtains basic format image data Vb. This basic format image data is LDR image data with a frame frequency of 50 Hz.

[0111] Moreover, the LDR electric-to-optical converter 206 performs electric-to-optical conversion with the reverse characteristics to those of the LDR electric-to-electrical converter 103 in the above-mentioned transmitting device 100 on the high-quality format image data Vh1' obtained by the decoding unit 204-1, and obtains high-quality format image data Vh1. This high-quality format image data Vh1 is LDR image data with a frame frequency of 100 Hz.

[0112] Moreover, the HDR electric-to-optical converter 207 performs electric-to-optical conversion with the reverse characteristics to those of the HDR optical-to-electrical converter 104 in the above-described transmission device 100 on the high-quality format image data Vh2′ obtained by the decoder 204-2, and obtains high-quality format image data Vh2. This high-quality format image data Vh2 is HDR image data with a frame frequency of 50 Hz.

[0113] Moreover, the HDR electric-to-optical converter 208 performs electric-to-optical conversion with the reverse characteristics to those of the HDR optical-to-electrical converter 105 in the above-described transmission device 100 on the high-quality format image data Vh3′ obtained by the decoder 204-3, and obtains high-quality format image data Vh3. This high-quality format image data Vh3 is HDR image data with a frame frequency of 100 Hz.

[0114] The display unit 209 is configured with, for example, an LCD (Liquid Crystal Display), an organic EL (Organic Electro-Luminescence) panel, etc. The display unit 209 displays an image based on either the basic format image data Vb or the high quality format image data Vh1, Vh2, or Vh3 according to its display capability.

[0115] In this case, the control unit 201 controls the image data to be supplied to the display unit 209. This control is performed based on the identification information (extension category information of the stream) of the high-quality format corresponding to each of the extended video streams STe1, STe2, and STe3 grasped by the control unit 201 as described above, and the display capability information of the display unit 209.

[0116] That is, when the display unit 209 is not capable of displaying a high frame frequency or a high dynamic range, the control unit 201 controls the display unit 209 to be supplied with basic format image data Vb related to the decoding of the basic video stream STb. In this case, the control unit 201 controls the decoding unit 204-0 to decode the basic video stream STb, and the LDR electric-optical converter 205 to output the basic format image data Vb.

[0117] Furthermore, when the display unit 209 is capable of displaying a high frame frequency but is not capable of displaying a high dynamic range, the control unit 201 controls the display unit 209 to be supplied with high-quality format image data Vh1 related to the decoding of the extended video stream STe1. In this case, the control unit 201 controls the decoding unit 204-0 to decode the basic video stream STb, the decoding unit 204-1 to decode the extended video stream STe1, and the LDR electric-optical converter 206 to output the high-quality format image data Vh1.

[0118] Furthermore, when the display unit 209 is not capable of displaying at a high frame frequency but is capable of displaying at a high dynamic range, the control unit 201 controls so that high-quality format image data Vh2 related to the decoding of the extended video stream STe2 is supplied to the display unit 209. In this case, the control unit 201 controls so that the decoding unit 204-0 decodes the basic video stream STb, the decoding unit 204-2 decodes the extended video stream STe2, and the HDR electric-optical conversion unit 207 outputs the high-quality format image data Vh2.

[0119] Furthermore, when the display unit 209 is capable of both high frame frequency display and high dynamic range display, the control unit 201 controls so that high-quality format image data Vh3 related to the decoding of the extended video stream STe3 is supplied to the display unit 209. In this case, the control unit 201 controls so that the decoding unit 204-0 decodes the basic video stream STb, the decoding unit 204-2 decodes the extended video stream STe2, the decoding unit 204-3 decodes the extended video stream STe3, and the HDR electric-optical conversion unit 208 outputs the high-quality format image data Vh3.

[0120] The operation of the receiving device 200 shown in Fig. 11 will be briefly described below. The receiving unit 202 receives the transport stream TS transmitted from the transmitting device 100 via broadcast waves or network packets. This transport stream TS is supplied to a system decoder 203. The system decoder 203 extracts the basic video stream STb and the extended video streams STe1, STe2, and STe3 from this transport stream TS.

[0121] Furthermore, the system decoder 203 extracts various pieces of information inserted into the layer of the container (transport stream) and sends them to the control unit 201. This information also includes a scalable extension descriptor. The control unit 201 grasps the identification information of the high-quality format (stream extension category information) corresponding to each of the extended video streams STe1, STe2, and STe3 from the "type of enhancement" field of this descriptor.

[0122] When the display unit 209 is not capable of displaying a high frame frequency or a high dynamic range, the basic format image data Vb is supplied from the LDR electro-optical converter 205 to the display unit 209. The display unit 209 displays an image based on this basic format image data Vb, that is, LDR image data with a frame frequency of 50 Hz.

[0123] In this case, the basic video stream STb extracted by the system decoder 203 is supplied to the decoding unit 204-0. The decoding unit 204-0 performs a decoding process on the basic video stream STb, and generates basic format image data Vb'. Here, the decoding unit 204-0 can confirm that the supplied video stream is the basic video stream STb from the "nuh_layer_id" field in the header of the NAL unit.

[0124] The basic format image data Vb' generated by the decode unit 204-0 is supplied to the LDR electric-optical converter 205. In the LDR electric-optical converter 205, the basic format image data Vb' is subjected to electric-optical conversion to obtain the basic format image data Vb. The basic format image data Vb is supplied to the display unit 209.

[0125] On the other hand, when the display unit 209 is capable of displaying at a high frame frequency but is not capable of displaying at a high dynamic range, high-quality format image data Vh1 is supplied from the LDR electro-optical conversion unit 206 to the display unit 209. The display unit 209 displays an image based on this high-quality format image data Vh1, that is, LDR image data with a frame frequency of 100 Hz.

[0126] In this case, the basic video stream STb extracted by the system decoder 203 is supplied to the decoding unit 204-0. In the decoding unit 204-0, a decoding process is performed on the basic video stream STb, and basic format image data Vb' is generated. Also, the extended video stream STe1 extracted by the system decoder 203 is supplied to the decoding unit 204-1. In the decoding unit 204-1, a decoding process is performed on the extended video stream STe1 with reference to the basic format image data Vb', and high quality format image data Vh1' is generated.

[0127] Here, the decoding unit 204-0 can confirm that the supplied video stream is the basic video stream STb from the "nuh_layer_id" field in the header of the NAL unit. Also, the decoding unit 204-1 can confirm that the supplied video stream is the extended video stream STe1 from the "nuh_layer_id" field in the header of the NAL unit.

[0128] The high-quality format image data Vh1′ generated by the decode unit 204-1 is supplied to the LDR electric-optical converter 206. In the LDR electric-optical converter 206, the high-quality format image data Vh1′ is subjected to electric-optical conversion to obtain high-quality format image data Vh1, which is supplied to the display unit 209.

[0129] Furthermore, when the display unit 209 is not capable of displaying at a high frame frequency but is capable of displaying at a high dynamic range, high-quality format image data Vh2 is supplied from the HDR electro-optical conversion unit 207 to the display unit 209. The display unit 209 displays an image based on this high-quality format image data Vh2, that is, HDR image data with a frame frequency of 50 Hz.

[0130] In this case, the basic video stream STb extracted by the system decoder 203 is supplied to a decoding unit 204-0. In the decoding unit 204-0, a decoding process is performed on the basic video stream STb, and basic format image data Vb' is generated. Also, the extended video stream STe2 extracted by the system decoder 203 is supplied to a decoding unit 204-2. In the decoding unit 204-2, a decoding process is performed on the extended video stream STe2 with reference to the basic format image data Vb', and high quality format image data Vh2' is generated.

[0131] Here, the decoding unit 204-0 can confirm that the supplied video stream is the basic video stream STb from the "nuh_layer_id" field in the header of the NAL unit. Also, the decoding unit 204-2 can confirm that the supplied video stream is the extended video stream STe2 from the "nuh_layer_id" field in the header of the NAL unit.

[0132] The high-quality format image data Vh2′ generated by the decoding unit 204-2 is supplied to the HDR electric-to-optical converter 207. In the HDR electric-to-optical converter 207, the high-quality format image data Vh2′ is subjected to electric-to-optical conversion to obtain high-quality format image data Vh2, which is supplied to the display unit 209.

[0133] Furthermore, when the display unit 209 is capable of both high frame frequency display and high dynamic range display, high quality format image data Vh3 is supplied from the HDR electro-optical conversion unit 208 to the display unit 209. The display unit 209 displays this high quality format image data Vh3, that is, an image based on HDR image data with a frame frequency of 100 Hz.

[0134] In this case, the basic video stream STb extracted by the system decoder 203 is supplied to a decoding unit 204-0. In the decoding unit 204-0, a decoding process is performed on the basic video stream STb, and basic format image data Vb' is generated. Also, the extended video stream STe2 extracted by the system decoder 203 is supplied to a decoding unit 204-2. In the decoding unit 204-2, a decoding process is performed on the extended video stream STe2 with reference to the basic format image data Vb', and high quality format image data Vh2' is generated.

[0135] Furthermore, the extended video stream STe3 extracted by the system decoder 203 is supplied to a decoding unit 204-3. In the decoding unit 204-3, the extended video stream STe3 is decoded with reference to the high-quality format image data Vh2', and high-quality format image data Vh3' is generated.

[0136] Here, the decoding unit 204-0 can confirm that the supplied video stream is the basic video stream STb from the "nuh_layer_id" field in the NAL unit header. Also, the decoding unit 204-2 can confirm that the supplied video stream is the extended video stream STe2 from the "nuh_layer_id" field in the NAL unit header. Also, the decoding unit 204-3 can confirm that the supplied video stream is the extended video stream STe3 from the "nuh_layer_id" field in the NAL unit header.

[0137] The high-quality format image data Vh3′ generated by the decoding unit 204-3 is supplied to the HDR electric-optical conversion unit 208. In the HDR electric-optical conversion unit 208, the high-quality format image data Vh3′ is subjected to electric-optical conversion to obtain high-quality format image data Vh3, which is supplied to the display unit 209.

[0138] As described above, in the transmission / reception system 10 shown in Fig. 1, in the transmission device 100, identification information (extended category information of the stream) of the high-quality format corresponding to each of a predetermined number of extended video streams included in the transport stream TS is inserted into a container or a layer of the video stream and transmitted. Therefore, on the receiving side, based on this identification information, it becomes easy to selectively perform a decoding process on a predetermined video stream to obtain image data according to the display capability.

[0139] <2. Modifications> In the above embodiment, an example has been shown in which identification information (extended category information of streams) of high-quality formats corresponding to a predetermined number of extended video streams included in a transport stream TS is inserted into both the container and video stream layers and transmitted. However, it is also possible to insert this identification information only into the container layer or only into the video stream layer.

[0140] Also, instead of transmitting an ID indicating the layer extension type of the stream, the extension category of the stream, and information indicating the priority within the extension category, it is possible to indicate a combination of these with the value of "stream_type." For example, as shown in Fig. 10, the basic stream can be "Stream_type = ST0", the first stream of the frame rate scalable extension can be "Stream_type = ST1", the first stream of the dynamic range scalable extension can be "Stream_type = ST2", and the stream of the frame rate / dynamic range scalable extension (second extension stream) can be "Stream_type = ST3".

[0141] In the above embodiment, the transmission / reception system 10 including the transmission device 100 and the reception device 200 is shown, but the configuration of the transmission / reception system to which the present technology can be applied is not limited to this. For example, the reception device 200 may be configured as a set-top box and a monitor connected by a digital interface such as HDMI (High-Definition Multimedia Interface). In this case, the set-top box can obtain display capability information by obtaining EDID (Extended display identification data) from the monitor. Note that "HDMI" is a registered trademark.

[0142] In the above embodiment, an example was given in which the container was a transport stream (MPEG-2 TS). However, the present technology can be similarly applied to a system configured to distribute to a receiving terminal using a network such as the Internet. In Internet distribution, distribution is often performed in a container of MP4 or other format. In other words, containers of various formats such as the transport stream (MPEG-2 TS) adopted in the digital broadcasting standard and MP4 used in Internet distribution are applicable.

[0143] The present technology can also be configured as follows. (1) an image encoding unit for generating a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, respectively; a transmitting unit that transmits a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated by the image encoding unit; an identification information inserting unit that inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the container; Transmitting device. (2) The image encoding unit is With respect to the basic format image data, a predictive coding process is performed within the basic format image data to generate the basic video stream; With respect to the high quality format image data, a predictive coding process is selectively performed within the high quality format image data or between the basic format image data and other high quality format image data to generate the extended video stream. The transmitting device according to (1) above. (3) The identification information to be inserted into the layer of the container includes: Information indicating whether each of the predetermined number of extended video streams is generated by performing a predictive coding process with the basic format image data or a predictive coding process with the high quality format image data is added. The transmitting device according to (2) above. (4) The identification information to be inserted into the layer of the container includes: Information indicating a video stream corresponding to image data referenced in a predictive coding process between the basic format image data or other high quality format image data performed when generating each of the predetermined number of extended video streams is added. A transmitting device as described in (2) or (3). (5) The container is an MPEG2-TS. The identification information insertion unit The identification information is inserted into each video elementary stream loop corresponding to the predetermined number of extended video streams existing under a program map table. A transmitting device according to any one of (1) to (4). (6) The identification information insertion unit Further inserting identification information of high quality formats corresponding to the predetermined number of extended video streams into a layer of the video streams. A transmitting device according to any one of (1) to (5). (7) The video stream has a NAL unit structure, The identification information insertion unit The identification information is inserted into the header of the NAL unit. The transmitting device according to (6) above. (8) an image encoding step of generating a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, respectively; a transmitting step of transmitting, by a transmitting unit, a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated in the image encoding step; an identification information inserting step of inserting identification information of high quality formats corresponding to the predetermined number of extended video streams into a layer of the container; Transmission method. (9) an image encoding unit for generating a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, respectively; a transmitting unit that transmits a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated by the image encoding unit; an identification information inserting unit that inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the video stream; Transmitting device. (10) The image encoding unit includes: With respect to the basic format image data, a predictive coding process is performed within the basic format image data to generate the basic video stream; With respect to the high quality format image data, a predictive coding process is selectively performed within the high quality format image data or between the basic format image data and other high quality format image data to generate the extended video stream. The transmitting device according to (9) above. (11) The video stream has a NAL unit structure, The identification information insertion unit The identification information is inserted into the header of the NAL unit. The transmitting device according to (9) or (10). (12) an image encoding step of generating a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, respectively; a transmitting step of transmitting, by a transmitting unit, a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated in the image encoding step; an identification information inserting step of inserting identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the video stream; Transmission method. (13) A receiving unit for receiving a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the container, The video stream receiving device further includes a processor configured to process each of the video streams included in the received container based on the identification information. Receiving device. (14) The processing unit is Decoding the basic video stream and a predetermined extended video stream based on the identification information and the display capability information to obtain image data corresponding to the display capability. The receiving device according to (13) above. (15) The basic video stream is generated by performing a predictive coding process on the basic format image data, The extended video stream is generated by selectively performing a predictive coding process within the high-quality format image data or a predictive coding process between the high-quality format image data and the basic format image data or another high-quality format image data. A receiving device according to (13) or (14) above. (16) A receiving step of receiving, by a receiving unit, a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the container, The method further comprises the step of processing each of the video streams contained in the received container based on the identification information. Receiving method. (17) A receiving unit for receiving a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the video stream; The video stream receiving device further includes a processor configured to process each of the video streams included in the received container based on the identification information. Receiving device. (18) The processing unit is Decoding the basic video stream and a predetermined extended video stream based on the identification information and the display capability information to obtain image data corresponding to the display capability. The receiving device according to (17) above. (19) The basic video stream is generated by performing a predictive coding process on the basic format image data, The extended video stream is generated by selectively performing a predictive coding process within the high-quality format image data or a predictive coding process between the high-quality format image data and the basic format image data or another high-quality format image data. A receiving device according to (17) or (18). (20) A receiving step of receiving, by a receiving unit, a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the video stream; The method further comprises the step of processing each of the video streams contained in the received container based on the identification information. Receiving method.

[0144] The main feature of this technology is that it makes it easy for the receiving side to obtain image data according to its display capabilities by inserting identification information (extended category information of the stream) of the high-quality format corresponding to each of a predetermined number of extended video streams included in the transport stream TS into the container or video stream layer and transmitting it (see Figure 10).

[0145] 10. Transmitting and receiving system 100 Transmitting device 101...Control section 102, 103: LDR photoelectric conversion section 104, 105...HDR photoelectric conversion section 106 Video Encoder 106-0, 106-1, 106-1, 106-1... Encoding section 107 System Encoder 108 Transmitter 150 Image data generating unit 151···HDR Camera 152,154...Frame rate conversion section 153 Dynamic range conversion section 160 Encoding section 161... Intra-layer prediction unit 162... Inter-layer prediction unit 163 Forecast Adjustment Section 164...Selection section 165 Encoding function section 200 Receiving device 201...Control section 202 Receiving unit 203 System Decoder 204: Video decoder 204-0, 204-1, 204-1, 204-1... Decoder 205, 206...LDR electro-optical conversion unit 207, 208... HDR electro-optical converter 209...Display section 240 Decoder 241 Decoding function unit 242... Intra-layer prediction compensation unit 243... Inter-layer prediction compensation unit 244... Forecast Adjustment Section 245...Selection section

Claims

1. an image encoding unit for generating a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data; a transmitting unit that transmits a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated by the image encoding unit; an identification information inserting unit that inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the container; Transmitting device.

2. The image encoding unit is With respect to the basic format image data, a predictive coding process is performed within the basic format image data to generate the basic video stream; With respect to the high quality format image data, a predictive coding process is selectively performed within the high quality format image data or between the basic format image data and other high quality format image data to generate the extended video stream. The transmitting device according to claim 1 .

3. The identification information inserted into the layer of the container includes: Information indicating whether each of the predetermined number of extended video streams is generated by performing a predictive coding process with the basic format image data or a predictive coding process with the high quality format image data is added. The transmitting device according to claim 2.

4. The identification information inserted into the layer of the container includes: Information indicating a video stream corresponding to image data referenced in a predictive coding process between the basic format image data or other high quality format image data performed when generating each of the predetermined number of extended video streams is added. The transmitting device according to claim 2.

5. The container is MPEG2-TS, The identification information insertion unit The identification information is inserted into each video elementary stream loop corresponding to the predetermined number of extended video streams existing under a program map table. The transmitting device according to claim 1 .

6. The identification information insertion unit Further inserting identification information of high quality formats corresponding to the predetermined number of extended video streams into a layer of the video streams. The transmitting device according to claim 1 .

7. The video stream has a NAL unit structure, The identification information insertion unit The identification information is inserted into the header of the NAL unit. The transmitting device according to claim 6.

8. an image encoding step of generating a basic video stream obtained by encoding basic format image data and a predetermined number of enhanced video streams obtained by encoding a predetermined number of high quality format image data, respectively; a transmitting step of transmitting, by a transmitting unit, a container in a predetermined format including the basic video stream generated in the image encoding step and the predetermined number of extended video streams; an identification information inserting step of inserting identification information of high quality formats corresponding to the predetermined number of extended video streams into a layer of the container; Transmission method.

9. an image encoding unit for generating a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data; a transmitting unit that transmits a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated by the image encoding unit; an identification information inserting unit that inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the video stream; Transmitting device.

10. The image encoding unit is With respect to the basic format image data, a predictive coding process is performed within the basic format image data to generate the basic video stream; With respect to the high quality format image data, a predictive coding process is selectively performed within the high quality format image data or between the basic format image data and other high quality format image data to generate the extended video stream. The transmitting device according to claim 9.

11. The video stream has a NAL unit structure, The identification information insertion unit The identification information is inserted into the header of the NAL unit. The transmitting device according to claim 9.

12. an image encoding step of generating a basic video stream obtained by encoding basic format image data and a predetermined number of enhanced video streams obtained by encoding a predetermined number of high quality format image data, respectively; a transmitting step of transmitting, by a transmitting unit, a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated in the image encoding step; an identification information inserting step of inserting identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the video stream; Transmission method.

13. a receiving unit for receiving a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the container, The video stream receiving device further includes a processor configured to process each of the video streams included in the received container based on the identification information. Receiving device.

14. The processing unit includes: Decoding the basic video stream and a predetermined extended video stream based on the identification information and the display capability information to obtain image data corresponding to the display capability.

14. A receiving device according to claim 13.

15. the basic video stream is generated by performing a predictive coding process on the basic format image data, The extended video stream is generated by selectively performing a predictive coding process within the high-quality format image data or a predictive coding process between the high-quality format image data and the basic format image data or another high-quality format image data. A receiving device according to claim 13.

16. The method includes a receiving step of receiving, by a receiving unit, a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the container, The method further comprises the step of processing each of the video streams contained in the received container based on the identification information. Receiving method.

17. a receiving unit for receiving a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the video stream; The video stream receiving device further includes a processor configured to process each of the video streams included in the received container based on the identification information. Receiving device.

18. The processing unit includes: Decoding the basic video stream and a predetermined extended video stream based on the identification information and the display capability information to obtain image data corresponding to the display capability.

20. The receiving device according to claim 17.

19. the basic video stream is generated by performing a predictive coding process on the basic format image data, The extended video stream is generated by selectively performing a predictive coding process within the high-quality format image data or a predictive coding process between the high-quality format image data and the basic format image data or another high-quality format image data. A receiving device according to claim 17.

20. The method includes a receiving step of receiving, by a receiving unit, a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the video stream; The method further comprises the step of processing each of the video streams contained in the received container based on the identification information. Receiving method.

Citation Information

Patent Citations

  • Multiplexing device for hierarchized elementary stream, demultiplexing device, multiplexing method, and program

    JP2009267537A

  • Level signaling for layered video coding

    WO2013151814A1

  • Image data transmission device, image data transmission method, image data reception device, and image data reception method

    WO2013161442A1

  • Transmission device, transmission method, reception device, and reception method

    WO2014034463A1

  • Transmitting device, transmitting method, receiving device and receiving method

    WO2014038522A1