Receiving method and receiving device

Abstract

To allow a predetermined number of pieces of high-quality format image data to be transmitted favorably together with basic format image data.SOLUTION: A basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of pieces of high-quality format image data are generated. A container of a predetermined format containing each video stream is transmitted. Identification information of the high-quality format to which each of the predetermined number of extended video streams corresponds is inserted into the container and / or a layer of the video stream.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present technology relates to a transmitting device, a transmitting method, a receiving device, and a receiving method, and relates to a transmitting device or the like that transmits a predetermined number of high-quality format image data together with basic format image data. [Background technology]

[0002] It has been known that high-quality format image data is transmitted together with basic format image data, and the receiving side selectively uses the basic format image data or the high-quality format image data. For example, Patent Document 1 describes scalable media coding to generate a base layer stream for a low-resolution video service and an enhancement layer stream for a high-resolution video service, and transmitting a broadcast signal including these. Note that high-quality formats include high resolution, high frame frequency, high dynamic range, wide color gamut, high bit length, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2008-543142 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present technology is to successfully transmit a predetermined number of high-quality format image data together with basic format image data. [Means for solving the problem]

[0005] The concept of this technology is: an image encoding unit that generates a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data; a transmitter that transmits a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated by the image encoding unit; an identification information inserting unit that inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into the container and / or a layer of the video stream; Located in the transmitting device.

[0006] In the present technology, an image encoding unit generates a basic video stream and a predetermined number of extended video streams. Here, the basic video stream is obtained by encoding basic format image data, and each of the predetermined number of extended video streams is obtained by encoding a predetermined number of high-quality format image data.

[0007] For example, the image encoding unit may be configured to, with respect to basic format image data, perform predictive encoding processing within this basic format image data to generate a basic video stream, and with respect to high-quality format image data, selectively perform predictive encoding processing within this high-quality format image data or predictive encoding processing between basic format image data or other high-quality format image data to generate an extended video stream.

[0008] The transmitting unit transmits a container in a predetermined format that includes the basic video stream generated by the image encoding unit and a predetermined number of extended video streams. For example, the container may be a transport stream (MPEG-2 TS) adopted in the digital broadcasting standard. Furthermore, for example, the container may be an MP4 container used for Internet distribution, or a container in another format.

[0009] The identification information inserting unit inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into a container or a layer of the video stream. For example, the container may be MPEG2-TS, and when inserting the identification information into the layer of the container, the identification information inserting unit may insert the identification information into each video elementary stream loop (video ES loop) corresponding to the predetermined number of extended video streams existing under a program map table (PMT). Also, for example, the video stream may have a NAL (Network Abstraction Layer) unit structure, and the identification information inserting unit may insert the identification information into a header of the NAL unit.

[0010] In this way, in this technology, identification information of high-quality formats corresponding to a predetermined number of extended video streams is inserted into a container or a video stream layer and transmitted, which makes it easy for the receiving side to selectively decode predetermined video streams based on the identification information to obtain image data according to the display capabilities.

[0011] In the present technology, for example, the identification information inserted into the layer of the container may be added with information indicating whether each of the predetermined number of extended video streams has been generated by performing predictive coding processing with basic format image data or high-quality format image data. In this case, the receiving side can easily recognize whether basic format image data or other high-quality format image data has been referenced in the predictive coding processing when generating each of the predetermined number of extended video streams.

[0012] Furthermore, in the present technology, for example, the identification information inserted into the layer of the container may be added with information indicating a video stream corresponding to image data referenced in a predictive coding process between basic format image data or other high-quality format image data performed when generating each of the predetermined number of extended video streams. In this case, the receiving side can easily recognize which video stream corresponds to the image data referenced in the predictive coding process when generating each of the predetermined number of extended video streams.

[0013] Another concept of the present technology is a receiving unit for receiving a container in a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into the container layer and / or the video stream layer; The video stream processing device further includes a processing unit configured to process each of the video streams included in the received container based on the identification information. It is in the receiving device.

[0014] In this technology, a receiving unit receives a container including a basic video stream and a predetermined number of extended video streams. Here, the basic video stream is obtained by encoding basic format image data. Also, the predetermined number of extended video streams are each obtained by encoding a predetermined number of high-quality format image data. Identification information of the high-quality formats corresponding to the predetermined number of extended video streams is inserted into layers of the container and the video stream.

[0015] For example, the basic video stream may be generated by performing predictive coding processing within basic format image data on this basic format image data, and the extended video stream may be generated by selectively performing predictive coding processing within this high-quality format image data or predictive coding processing between basic format image data or other high-quality format image data on high-quality format image data.

[0016] The processing unit processes each video stream included in the received container based on the identification information. For example, the processing unit may perform a decoding process on the basic video stream and a predetermined extended video stream based on the identification information and the display capability information to obtain image data corresponding to the display capability.

[0017] In this way, with this technology, a predetermined number of extended video streams inserted into a container or a video stream layer are processed based on the identification information of the corresponding high-quality format, making it easy to selectively decode predetermined video streams to obtain image data according to the receiving capability. [Effects of the Invention]

[0018] According to the present technology, it is possible to transmit a predetermined number of high-quality format image data together with basic format image data. Note that the effects described herein are not necessarily limited to those described herein, and may be any of the effects described in the present disclosure. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a block diagram showing an example of the configuration of a transmission / reception system according to an embodiment; [Figure 2] FIG. 2 is a block diagram illustrating an example of the configuration of a transmitting device. [Figure 3]FIG. 10 is a block diagram showing an example of the configuration of an image data generating unit that generates basic format image data Vb and three high-quality format image data Vh1, Vh2, and Vh3. [Figure 4] FIG. 2 is a block diagram showing an example of the configuration of the main part of an encoding unit. [Figure 5] 1 is a diagram showing an example structure of a NAL unit header and the contents of main parameters in the example structure. [Figure 6] FIG. 10 is a diagram showing an example of the configuration of a basic video stream STb and extended video streams STe1, STe2, and STe3. [Figure 7] FIG. 10 is a diagram illustrating an example of the structure of a scalable extension descriptor. [Figure 8] FIG. 10 is a diagram showing the contents of main information in an example structure of a scalable extension descriptor. [Figure 9] 10 is a diagram showing the correspondence between the value of the "type of enhancement" field of the scalable extension descriptor and the value of the "nuh_layer_id" field of the NAL unit header. [Figure 10] FIG. 1 is a diagram illustrating an example of the configuration of a transport stream TS. [Figure 11] FIG. 2 is a block diagram illustrating an example of the configuration of a receiving device. [Figure 12] FIG. 2 is a block diagram showing an example of the configuration of the main part of a decoding unit. DETAILED DESCRIPTION OF THE INVENTION

[0020] The following describes modes for carrying out the invention (hereinafter referred to as "embodiments") in the following order: 1. Embodiment 2. Variations

[0021] <1. Embodiment> [Transmitting and receiving system] 1 shows an example of the configuration of a transmission / reception system 10 according to an embodiment. The transmission / reception system 10 includes a transmission device 100 and a reception device 200.

[0022] The transmitting device 100 transmits a transport stream TS as a container via broadcast waves or network packets. This transport stream TS includes a basic video stream and a predetermined number of extended video streams.

[0023] The basic video stream is generated by encoding basic format image data using, for example, H.264 / AVC, H.265 / HEVC, etc. Here, predictive encoding processing is performed within the basic format image data to generate the basic video stream.

[0024] The predetermined number of extended video streams are generated by encoding a predetermined number of high-quality image data using, for example, H.264 / AVC, H.265 / HEVC, etc. Here, with respect to the high-quality format image data, a predictive encoding process is selectively performed within this high-quality format image data, or a predictive encoding process between basic format image data or other high-quality format image data, to generate the extended video stream.

[0025] Identification information of the high-quality formats corresponding to each of the predetermined number of extended video streams is inserted into a layer of the container. This identification information allows the receiving side to easily determine the high-quality formats corresponding to each of the predetermined number of extended video streams in the layer of the container. In this embodiment, the identification information is inserted into each video elementary stream loop corresponding to the predetermined number of extended video streams that exists under the program map table.

[0026] This identification information includes information indicating whether each of the predetermined number of extended video streams has been generated by performing predictive coding with basic format image data or high-quality format image data. This information allows the receiving side to easily recognize, at the container layer, whether basic format image data or other high-quality format image data was referenced in the predictive coding process when generating each of the predetermined number of extended video streams.

[0027] Furthermore, information indicating the video stream corresponding to the image data referenced in the predictive coding process between the basic format image data or other high-quality format image data performed when generating each of the predetermined number of extended video streams is added to this identification information. This information allows the receiving side to easily recognize, in the container layer, which video stream corresponds to the image data referenced in the predictive coding process when generating each of the predetermined number of extended video streams.

[0028] Identification information of the high-quality formats to which the predetermined number of extended video streams respectively correspond is inserted into the layer of the video stream. This identification information allows the receiving side to easily grasp the high-quality formats to which the predetermined number of extended video streams respectively correspond. In this embodiment, the identification information is inserted into the header of the NAL unit.

[0029] The receiving device 200 receives the above-mentioned transport stream TS transmitted by broadcast waves or network packets from the transmitting device 100. As described above, identification information of the high-quality formats corresponding to a predetermined number of extended video streams included in the transport stream TS is inserted into the container or video stream layer. Based on this identification information, the receiving device 200 processes each video stream included in the transport stream TS and obtains image data according to its display capabilities.

[0030] "Configuration of transmitting device" FIG. 2 shows an example of the configuration of a transmitting device 100. This transmitting device 100 handles basic format image data Vb and three high-quality format image data Vh1, Vh2, and Vh3 as transmission image data. Here, the basic format image data Vb is LDR (Low Dynamic Range) image data with a frame frequency of 50 Hz. The high-quality format image data Vh1 is LDR image data with a frame frequency of 100 Hz. The LDR image data has a brightness range of 0% to 100% relative to the brightness of the white peak of a conventional LDR image.

[0031] The high-quality format image data Vh2 is HDR (High Dynamic Range) image data with a frame frequency of 50 Hz. The high-quality format image data Vh3 is HDR image data with a frame frequency of 100 Hz. If the brightness of the white peak of a conventional LDR image is 100%, this HDR image data has a brightness range of 0 to 100%*N, for example, 0 to 400% or 0 to 800%.

[0032] 3 shows an example of the configuration of an image data generation unit 150 that generates basic format image data Vb and three high-quality format image data Vh1, Vh2, and Vh3. The image data generation unit 150 includes an HDR camera 151, a frame rate conversion unit 152, a dynamic range conversion unit 153, and a frame rate conversion unit 154.

[0033] The HDR camera 151 captures an image of a subject and outputs HDR image data with a frame frequency of 100 Hz, i.e., high-quality format image data Vh3. The frame rate conversion unit 152 performs processing to convert the frame frequency of the high-quality format image data Vh3 output from the HDR camera 151 from 100 Hz to 50 Hz, and outputs HDR image data with a frame frequency of 50 Hz, i.e., high-quality format image data Vh2.

[0034] The dynamic range conversion unit 153 performs processing to convert the high-quality format image data Vh3 output from the HDR camera 151 from HDR to LDR, and outputs LDR image data with a frame frequency of 100 Hz, i.e., high-quality format image data Vh1. The frame rate conversion unit 154 performs processing to convert the frame frequency of the high-quality format image data Vh1 output from the dynamic range conversion unit 153 from 100 Hz to 50 Hz, and outputs LDR image data with a frame frequency of 50 Hz, i.e., basic format image data Vb.

[0035] 2, the transmitting device 100 includes a control unit 101, LDR photoelectric conversion units 102 and 103, HDR photoelectric conversion units 104 and 105, a video encoder 106, a system encoder 107, and a transmitting unit 108. The control unit 101 is configured to include a CPU (Central Processing Unit), and controls the operation of each unit of the transmitting device 100 based on a control program.

[0036] The LDR photoelectric conversion unit 102 applies photoelectric conversion characteristics for LDR images (LDR OETF curve) to the basic format image data Vb to obtain basic format image data Vb' for transmission. The LDR photoelectric conversion unit 103 applies photoelectric conversion characteristics for LDR images to the high-quality format image data Vh1 to obtain high-quality format image data Vh1' for transmission.

[0037] The HDR photoelectric conversion unit 104 applies photoelectric conversion characteristics for HDR images (HDR OETF curve) to the high-quality format image data Vh2 to obtain high-quality format image data Vh2' for transmission. The HDR photoelectric conversion unit 105 applies photoelectric conversion characteristics for HDR images to the high-quality format image data Vh3 to obtain high-quality format image data Vh3' for transmission.

[0038] The video encoder 106 has four encoding units 106-0, 106-1, 106-2, and 106-3. The encoding unit 106-0 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on basic format image data Vb' for transmission to generate a basic video stream STb. In this case, the encoding unit 106-0 performs prediction within the image data Vb'.

[0039] The encoding unit 106-1 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh1' for transmission to generate an extended video stream STe1. In this case, the encoding unit 106-1 selectively performs prediction within the image data Vh1' or prediction between the image data Vb' for each coding block in order to reduce prediction residuals.

[0040] The encoding unit 106-2 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh2' for transmission to generate an extended video stream STe2. In this case, the encoding unit 106-2 selectively performs prediction within the image data Vh2' or prediction between the image data Vb' for each coding block in order to reduce prediction residuals.

[0041] The encoding unit 106-3 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh3' for transmission to generate an extended video stream STe3. In this case, the encoding unit 106-3 selectively performs prediction within the image data Vh3' or prediction between the image data Vh2' for each coding block in order to reduce prediction residuals.

[0042] 4 shows an example configuration of the main parts of the encoding unit 160. This encoding unit 160 can be applied to the encoding units 106-1, 106-2, and 106-3. This encoding unit 160 has an intra-layer prediction unit 161, an inter-layer prediction unit 162, a prediction adjustment unit 163, a selection unit 164, and an encoding function unit 165.

[0043] The intra-layer prediction unit 161 performs prediction (intra-layer prediction) on image data V1 to be coded within the image data V1 to obtain prediction residual data. The inter-layer prediction unit 162 performs prediction (inter-layer prediction) on image data V1 to be coded between the image data V1 to be coded and reference image data V2 to obtain prediction residual data.

[0044] In order to efficiently perform inter-layer prediction in the inter-layer prediction unit 162, the prediction adjustment unit 163 performs the following processing depending on the type of scalable extension of the image data V1 to the image data V2. In the case of dynamic range extension, level adjustment is performed to convert from LDR to HDR. In the case of spatial scalable extension, blocks are enlarged to a predetermined size. In the case of frame rate extension, a bypass is performed. In the case of color gamut extension, mapping is performed for both luminance and chrominance. In the case of bit length extension, conversion is performed to align the MSBs of pixels.

[0045] For example, in the case of the encoding unit 106-1, the image data V1 is high-quality format image data Vh1' (100 Hz, LDR), the image data V2 is basic format image data Vb' (50 Hz, LDR), and the scalable extension type corresponds to frame rate extension. Therefore, the prediction adjustment unit 163 bypasses the image data Vb' as is.

[0046] For example, in the case of the encoding unit 106-2, the image data V1 is high-quality format image data Vh2' (50 Hz, HDR), the image data V2 is basic format image data Vb' (50 Hz, LDR), and the scalable extension type corresponds to dynamic range extension. Therefore, the prediction adjustment unit 163 performs level adjustment on the image data Vb' to convert it from LDR to HDR.

[0047] For example, in the case of the encoding unit 106-3, the image data V1 is high-quality format image data Vh3' (100 Hz, HDR), the image data V2 is high-quality format image data Vh2' (50 Hz, HDR), and the scalable extension type corresponds to frame rate extension. Therefore, the prediction adjustment unit 163 bypasses the image data Vb' as is.

[0048] The selection unit 164 selectively extracts the prediction residual data obtained by the intra-layer prediction unit 161 or the prediction residual data obtained by the inter-layer prediction unit 162 for each coding block, and sends this to the encoding function unit 165. In this case, the selection unit 164 extracts, for example, the prediction residual with the smaller value. The encoding function unit 165 performs encoding processes such as transform coding, quantization, and entropy coding on the prediction residual data extracted by the selection unit 164, to obtain a video stream ST.

[0049] 2, the video encoder 106 inserts identification information of the corresponding high-quality formats into the layers of the extended video streams STe1, STe2, and STe3. The video encoder 106 inserts this identification information into, for example, the header of the NAL unit.

[0050] Figure 5(a) shows an example of the structure (Syntax) of a NAL unit header, and Figure 5(b) shows the contents (Semantics) of the main parameters in that example structure. The 1-bit field "Forbidden_zero_bit" must be 0. The 6-bit field "nal_unit_type" indicates the NAL unit type. The 6-bit field "Nuh_layer_id" is an ID indicating the layer extension type of the stream. The 3-bit field "nuh_temporal_id_plus1" indicates temporal_id (0 to 6), and takes on a value (1 to 7) with 1 added.

[0051] In this embodiment, the 6-bit field of "nuh_layer_id" indicates the identification information of the high-quality format that each extended video stream corresponds to (extension category information of the stream). For example, "0" indicates the base stream; "1 to 4" indicate a spatial extension stream; "5 to 8" indicate a frame-rate extension stream; "9 to 12" indicate a dynamic range extension stream; "13 to 16" indicate a color gamut extension stream; "17 to 20" indicate a bit-length extension stream; "21 to 24" indicate spatial extension and frame-rate extension; and "25 to 28" indicate frame-rate extension and dynamic range extension.

[0052] For example, the basic video stream STb corresponds to a base stream, and therefore the "nuh_layer_id" in the header of the NAL unit that constitutes this basic video stream STb is set to "0." Also, for example, the extended video stream STe1 corresponds to a frame rate extended stream, and therefore the "nuh_layer_id" in the header of the NAL unit that constitutes this extended video stream STe1 is set to any value in the range of "5 to 8."

[0053] For example, the extended video stream STe2 corresponds to an extended dynamic range stream, and therefore the "nuh_layer_id" in the header of the NAL unit that constitutes this extended video stream STe2 is set to any value in the range of "9 to 12." For example, the extended video stream STe3 corresponds to an extended frame rate and extended dynamic range stream, and therefore the "nuh_layer_id" in the header of the NAL unit that constitutes this extended video stream STe3 is set to any value in the range of "25 to 28."

[0054] 6 shows an example of the configuration of a basic video stream STb and extended video streams STe1, STe2, and STe3. The horizontal axis indicates the picture order of composition (POC), with the left side representing earlier display times and the right side representing later display times. Each rectangular box represents a picture, and the solid arrows indicate the reference relationship between pictures in predictive coding.

[0055] The basic video stream STb is composed of coded image data for pictures "00", "01", etc. The extended video stream STe1 is composed of coded image data for pictures "10", "11", etc. that are located between each picture in the basic video stream STb. The extended video stream STe2 is composed of coded image data for pictures "20", "21", etc. that are located in the same positions as each picture in the basic video stream STb. The extended video stream STe3 is composed of coded image data for pictures "30", "31", etc. that are located between each picture in the extended video stream STe2.

[0056] 2, the system encoder 107 generates a transport stream TS including the basic video stream STb and the extended video streams STe1, STe2, and STe3 generated by the video encoder 106. Then, the transmitting unit 108 transmits this transport stream TS to the receiving device 200 via broadcast waves or network packets.

[0057] At this time, the system encoder 107 inserts identification information of the high-quality formats corresponding to each of the extended video streams STe1, STe2, and STe3 into the layer of the container (transport stream). In this embodiment, for example, a scalable extension descriptor including the identification information is inserted into a video elementary stream loop corresponding to each extended video stream existing under a PMT (Program Map Table).

[0058] Figure 7 shows an example of the structure (Syntax) of this scalable extension descriptor. Figure 8 shows the main information content (Semantics) of the structure example shown in Figure 7. The 8-bit field "descriptor_tag" indicates the descriptor type, and in this case, indicates that it is a scalable extension descriptor. The 8-bit field "descriptor_length" indicates the length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor.

[0059] The 4-bit field of "type of enhancement" indicates the identification information of the high-quality format that each enhanced video stream corresponds to (extension category information of the stream). For example, "1" indicates spatial scalable enhancement, "2" indicates frame rate scalable enhancement, "3" indicates dynamic range scalable enhancement, "4" indicates color gamut scalable enhancement, "5" indicates bit depth scalable enhancement, "6" indicates spatial / frame rate scalable enhancement, and "7" indicates frame rate / dynamic range scalable enhancement.

[0060] For example, the extended video stream STe1 corresponds to the frame rate scalable extension, and therefore the "type of enhancement" of the scalable extension descriptor corresponding to this extended video stream STe1 is set to "2".

[0061] For example, the extended video stream STe2 corresponds to the dynamic range scalable extension, and therefore the "type of enhancement" of the scalable extension descriptor corresponding to this extended video stream STe2 is set to "3."

[0062] For example, the extended video stream STe3 corresponds to the frame rate / dynamic range scalable extension, and therefore the "type of enhancement" of the scalable extension descriptor corresponding to this extended video stream STe3 is set to "7".

[0063] Figure 9 shows the correspondence between the value of this "type of enhancement" field and the value of the "nuh_layer_id" field in the NAL unit header. As shown above, it is possible to determine the identification information of the high-quality format corresponding to each enhanced video stream (extension category information of the stream) in the same way from either field.

[0064] 7, the 4-bit field of "scalable_priority" indicates the priority of each extended video stream within the same extended category. That is, this field indicates whether each extended video stream is generated by performing predictive coding processing with basic format image data or high-quality format image data.

[0065] For example, "0" indicates that it is a first priority stream that references the basic stream, i.e., that it has been generated by performing predictive coding processing with basic format image data. Also, for example, "1" indicates that it is a second priority stream that references the first priority stream, i.e., that it has been generated by performing predictive coding processing with high quality format image data.

[0066] For example, the extended video stream STe1 is related to the encoding of high-quality format image data Vh1' and is generated by performing predictive encoding processing with basic format image data Vb'. Therefore, the "scalable_priority" of the scalable extension descriptor corresponding to this extended video stream STe1 is set to "0."

[0067] For example, the extended video stream STe2 relates to the encoding of high-quality format image data Vh2' and is generated by performing predictive encoding processing with basic format image data Vb'. Therefore, the "scalable_priority" of the scalable extension descriptor corresponding to this extended video stream STe2 is set to "0."

[0068] For example, the extended video stream STe3 relates to the encoding of high-quality format image data Vh3' and is generated by performing predictive encoding processing with high-quality format image data Vh2'. Therefore, the "scalable_priority" of the scalable extension descriptor corresponding to this extended video stream STe3 is set to "1."

[0069] The 32-bit field of "enhancement reference PID" indicates the PID value of the reference stream. That is, this field indicates the PID value of the video stream corresponding to the image data referenced in the predictive coding process between the basic format image data or other high-quality format image data performed when generating each enhanced video stream.

[0070] For example, the extended video stream STe1 is related to the encoding of high-quality format image data Vh1' and is generated by performing predictive encoding processing with basic format image data Vb'. Therefore, the "enhancement reference PID" of the scalable extension descriptor corresponding to this extended video stream STe1 indicates the PID value of the basic video stream STb.

[0071] For example, the extended video stream STe2 is related to the encoding of high-quality format image data Vh2' and is generated by performing predictive encoding processing with basic format image data Vb'. Therefore, the "enhancement reference PID" of the scalable extension descriptor corresponding to this extended video stream STe2 indicates the PID value of the basic video stream STb.

[0072] For example, the extended video stream STe3 relates to the encoding of high-quality format image data Vh3' and is generated by performing predictive encoding processing with high-quality format image data Vh2'. Therefore, the "enhancement reference PID" of the scalable extension descriptor corresponding to this extended video stream STe3 indicates the PID value of the extended video stream STe2.

[0073] [Transport Stream TS Structure] 10 shows an example of the structure of a transport stream TS. This transport stream TS includes four video streams: a basic video stream STb and extended video streams STe1, STe2, and STe3. In this example, each video stream has a PES packet called "video PES."

[0074] The packet identifier (PID) of the basic video stream STb is, for example, PID1. The coded image data of each picture in this video stream includes NAL units such as AUD, VPS, SPS, PPS, PSEI, SLICE, SSEI, and EOS. The "nuh_layer_id" in the header of these NAL units is set to "0," indicating that they are basic video streams (see FIG. 9).

[0075] The packet identifier (PID) of the extended video stream STe1 is, for example, PID2. The coded image data of each picture in this video stream includes NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS. The "nuh_layer_id" in the header of these NAL units is, for example, "5," indicating that this is a frame rate extended stream (see FIG. 9).

[0076] The packet identifier (PID) of the extended video stream STe2 is, for example, PID3. The coded image data of each picture in this video stream includes NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS. The "nuh_layer_id" in the headers of these NAL units is, for example, "9," indicating that this is an extended dynamic range stream (see FIG. 9).

[0077] Furthermore, the packet identifier (PID) of the extended video stream STe3 is, for example, PID4. The coded image data of each picture in this video stream includes NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS. The "nuh_layer_id" in the headers of these NAL units is, for example, "25," indicating that this is a frame rate extended and dynamic range extended stream (see FIG. 9).

[0078] The transport stream TS also includes a Program Map Table (PMT) as Program Specific Information (PSI), which is information that indicates to which program each elementary stream included in the transport stream belongs.

[0079] The PMT contains a program loop that describes information related to the entire program. The PMT also contains elementary stream loops that contain information related to each elementary stream. In this configuration example, there are four video elementary stream loops (video ES loops) corresponding to four video streams: basic video stream STb and extended video streams STe1, STe2, and STe3. The video elementary stream loop corresponding to basic video stream STb contains information such as the stream type (ST0) and packet identifier (PID1).

[0080] In addition, the video elementary stream loop corresponding to the extended video stream STe1 contains information such as the stream type (ST1) and packet identifier (PID2), as well as descriptors describing information related to this extended video stream STe1. One of these descriptors is the scalable extension descriptor mentioned above.

[0081] The "type of enhancement" in this descriptor is set to "2", indicating that it is a frame rate enhancement stream (frame rate scalable enhancement) (see FIG. 9). Also, the "scalable_priority" in this descriptor is set to "0", indicating that it is a first priority stream that references the basic stream. Also, the "enhancement reference PID" in this descriptor is set to "PID1", indicating that it references the basic video stream STb.

[0082] In addition, the video elementary stream loop corresponding to the extended video stream STe2 contains information such as the stream type (ST2) and packet identifier (PID3), as well as descriptors describing information related to this extended video stream STe2. The scalable extension descriptor mentioned above is inserted as one of these descriptors.

[0083] The "type of enhancement" in this descriptor is set to "3", indicating that it is a dynamic range enhancement stream (dynamic range scalable enhancement) (see FIG. 9). Also, the "scalable_priority" in this descriptor is set to "0", indicating that it is a first-priority stream that references the basic stream. Also, the "enhancement reference PID" in this descriptor is set to "PID1", indicating that it references the basic video stream STb.

[0084] In addition, the video elementary stream loop corresponding to the extended video stream STe3 contains information such as the stream type (ST3) and packet identifier (PID4), as well as descriptors describing information related to this extended video stream STe3. The scalable extension descriptor mentioned above is inserted as one of these descriptors.

[0085] The "type of enhancement" in this descriptor is set to "7", indicating that it is a frame rate and dynamic range extended stream (frame rate / dynamic range scalable extension) (see Figure 9). Also, the "scalable_priority" in this descriptor is set to "1", indicating that it is a second priority stream that references the first priority stream. Also, the "enhancement reference PID" in this descriptor is set to "PID3", indicating that it references the enhanced video stream STe2.

[0086] The operation of the transmitting device 100 shown in Fig. 2 will be briefly described. Basic format image data Vb, which is LDR image data with a frame frequency of 50 Hz, is supplied to the LDR photoelectric conversion unit 102. In this LDR photoelectric conversion unit 102, photoelectric conversion characteristics for LDR images (LDR OETF curve) are applied to the basic format image data Vb, thereby obtaining basic format image data Vb' for transmission. This basic format image data Vb' is supplied to encoding units 106-0, 106-1, and 106-2 of the video encoder 106.

[0087] Furthermore, the high-quality format image data Vh1, which is LDR image data with a frame frequency of 100 Hz, is supplied to the LDR photoelectric conversion unit 103. In this LDR photoelectric conversion unit 103, photoelectric conversion characteristics for LDR images (LDR OETF curve) are applied to the high-quality format image data Vh1, thereby obtaining high-quality format image data Vh1' for transmission. This high-quality format image data Vh1' is supplied to the encoding unit 106-1 of the video encoder 106.

[0088] Furthermore, high-quality format image data Vh2, which is HDR image data with a frame frequency of 50 Hz, is supplied to the HDR photoelectric conversion unit 104. In this HDR photoelectric conversion unit 104, photoelectric conversion characteristics for HDR images (HDR OETF curve) are applied to the high-quality format image data Vh2, and high-quality format image data Vh2' for transmission is obtained. This high-quality format image data Vh2' is supplied to encoding units 106-2 and 106-3 of the video encoder 106.

[0089] Furthermore, high-quality format image data Vh3, which is HDR image data with a frame frequency of 100 Hz, is supplied to the HDR photoelectric conversion unit 105. In this HDR photoelectric conversion unit 105, photoelectric conversion characteristics for HDR images (HDR OETF curve) are applied to the high-quality format image data Vh3, and high-quality format image data Vh3' for transmission is obtained. This high-quality format image data Vh3' is supplied to the encoding unit 106-3 of the video encoder 106.

[0090] The video encoder 106 performs encoding processing on each of the basic format image data Vb' and the high-quality format image data Vh1', Vh2', and Vh3' to generate a video stream. That is, the encoding unit 106-0 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the basic format image data Vb' for transmission, and generates a basic video stream STb including encoded image data of each picture. In this case, the encoding unit 106-0 performs prediction within the image data Vb'.

[0091] Furthermore, the encoding unit 106-1 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh1' for transmission, and generates an extended video stream STe1 including encoded image data of each picture. In this case, the encoding unit 106-1 selectively performs prediction within the image data Vh1' or prediction with respect to the image data Vb' for each coding block in order to reduce prediction residuals.

[0092] Furthermore, the encoding unit 106-2 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh2' for transmission, and generates an extended video stream STe2 including encoded image data of each picture. In this case, the encoding unit 106-2 selectively performs prediction within the image data Vh2' or prediction with respect to the image data Vb' for each coding block in order to reduce prediction residuals.

[0093] Furthermore, the encoding unit 106-3 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh3' for transmission, and generates an extended video stream STe3 including encoded image data of each picture. In this case, the encoding unit 106-3 selectively performs prediction within the image data Vh3' or prediction between the image data Vh2' for each coding block in order to reduce prediction residuals.

[0094] Furthermore, the video encoder 106 inserts identification information of the corresponding high-quality formats into the layers of the extended video streams STe1, STe2, and STe3. That is, the video encoder 106 sets identification information of the corresponding high-quality format for each extended video stream (extension category information of the stream) in the "nuh_layer_id" field of the header of the NAL unit (see FIGS. 5 and 9).

[0095] The basic video stream STb and extended video streams STe1, STe2, and STe generated by the video encoder 106 are supplied to a system encoder 107. The system encoder 107 generates a transport stream TS including each video stream.

[0096] The system encoder 107 inserts identification information of the high-quality formats corresponding to each of the extended video streams STe1, STe2, and STe3 into the container (transport stream) layer. That is, the system encoder 107 inserts a scalable extension descriptor including identification information (extension category information of the stream) into the video elementary stream loop corresponding to each extended video stream existing under the PMT (see FIGS. 7 and 9).

[0097] The transport stream TS generated by the system encoder 107 is sent to the transmitting unit 108. In the transmitting unit 108, this transport stream TS is transmitted to the receiving device 200 via broadcast waves or network packets.

[0098] "Configuration of receiving device" Fig. 11 shows an example configuration of a receiving device 200. This receiving device 200 corresponds to the example configuration of the transmitting device 100 in Fig. 2. This receiving device 200 has a control unit 201, a receiving unit 202, a system decoder 203, a video decoder 204, LDR electric-to-optical conversion units 205 and 206L, HDR electric-to-optical conversion units 207 and 208, and a display unit (display device) 209. The control unit 201 is configured to include a CPU (Central Processing Unit), and controls the operation of each unit of the receiving device 200 based on a control program stored in storage (not shown).

[0099] The receiving unit 202 receives the transport stream TS transmitted by broadcast waves or network packets from the transmitting device 100. The system decoder 203 extracts the basic video stream STb and the extended video streams STe1, STe2, and STe3 from this transport stream TS.

[0100] The system decoder 203 also extracts various pieces of information inserted into the container (transport stream) layer and sends them to the control unit 201. This information includes the scalable extension descriptor described above. The control unit 201 can determine the high-quality format identification information (stream extension category information) corresponding to each of the extended video streams STe1, STe2, and STe3 from the "type of enhancement" field of this descriptor.

[0101] The control unit 201 can also determine, from the "scalable_priority" field of this descriptor, the priority of each of the enhanced video streams STe1, STe2, and STe3 within the same enhancement category, i.e., whether it is a first-priority stream that references the basic stream or a second-priority stream that references the first-priority stream. Furthermore, the control unit 201 can determine, from the "enhancement reference PID" field of this descriptor, the PID value of the video stream referenced by each of the enhanced video streams STe1, STe2, and STe3.

[0102] The video decoder 204 has four decoding units 204-0, 204-1, 204-2, and 204-3. The decoding unit 204-0 performs a decoding process on the basic video stream STb to generate basic format image data Vb'. In this case, the decoding unit 204-0 performs prediction compensation within the image data Vb'.

[0103] The decoding unit 204-1 performs a decoding process on the extended video stream STe1 to generate high-quality format image data Vh1'. In this case, the decoding unit 204-1 performs prediction compensation within the image data Vh1' or prediction compensation between the image data Vb' for each coding block, corresponding to the prediction during encoding.

[0104] The decoding unit 204-2 performs a decoding process on the extended video stream STe2 to generate high-quality format image data Vh2'. In this case, the decoding unit 204-2 performs prediction compensation within the image data Vh2' or prediction compensation between the image data Vb' for each coding block, corresponding to the prediction during encoding.

[0105] The decoding unit 204-3 performs a decoding process on the extended video stream STe3 to generate high-quality format image data Vh3'. In this case, the decoding unit 204-3 performs prediction compensation within the image data Vh3' or prediction compensation between the image data Vh2' and the image data Vh3' for each coding block, corresponding to the prediction during encoding.

[0106] 12 shows an example configuration of the main parts of the decoding unit 240. This decoding unit 240 can be applied to the decoding units 204-1, 204-2, and 204-3. This decoding unit 240 performs processing that is the reverse of the processing of the encoding unit 165 in FIG. 4. This decoding unit 240 has a decoding function unit 241, an intra-layer prediction compensation unit 242, an inter-layer prediction compensation unit 243, a prediction adjustment unit 244, and a selection unit 245.

[0107] The decoding function unit 241 performs decoding processing other than prediction compensation on the video stream ST to obtain prediction residual data. The intra-layer prediction compensation unit 242 performs prediction compensation within the image data V1 (intra-layer prediction compensation) on the prediction residual data to obtain image data V1. The inter-layer prediction compensation unit 243 performs prediction compensation between the prediction residual data and reference image data V2 (inter-layer prediction compensation) to obtain image data V1.

[0108] Although detailed description will be omitted, the prediction adjustment unit 244 performs processing according to the type of scalable extension of image data V1 to image data V2, similar to the prediction adjustment unit 163 of the encoding unit 160 in Fig. 4. The selection unit 245 selectively extracts, for each coding block, the image data V1 obtained by the intra-layer prediction compensation unit 242 or the image data V1 obtained by the inter-layer prediction compensation unit 243, corresponding to the prediction during encoding, and outputs it.

[0109] 11, the video decoder 204 sends header information of the NAL units of each video stream to the control unit 201. The control unit 201 can determine the identification information of the high-quality format (extension category information of the stream) corresponding to each of the extended video streams STe1, STe2, and STe3 from the "nuh_layer_id" field of this header information.

[0110] The LDR electric-to-optical converter 205 performs electric-to-optical conversion with the opposite characteristics to those of the LDR electric-to-electrical converter 102 in the transmitting device 100 on the basic format image data Vb' obtained by the decoder 204-0, thereby obtaining basic format image data Vb. This basic format image data is LDR image data with a frame frequency of 50 Hz.

[0111] Furthermore, the LDR electric-to-optical converter 206 performs electric-to-optical conversion with the opposite characteristics to those of the LDR electric-to-electrical converter 103 in the transmitting device 100 described above on the high-quality format image data Vh1' obtained by the decoding unit 204-1, thereby obtaining high-quality format image data Vh1. This high-quality format image data Vh1 is LDR image data with a frame frequency of 100 Hz.

[0112] Furthermore, the HDR electric-to-optical converter 207 performs electric-to-optical conversion with characteristics opposite to those of the HDR electric-to-electrical converter 104 in the transmission device 100 described above on the high-quality format image data Vh2' obtained by the decoding unit 204-2, thereby obtaining high-quality format image data Vh2. This high-quality format image data Vh2 is HDR image data with a frame frequency of 50 Hz.

[0113] Furthermore, the HDR electric-to-optical converter 208 performs electric-to-optical conversion with characteristics opposite to those of the HDR electric-to-electrical converter 105 in the transmission device 100 described above on the high-quality format image data Vh3' obtained by the decoding unit 204-3, thereby obtaining high-quality format image data Vh3. This high-quality format image data Vh3 is HDR image data with a frame frequency of 100 Hz.

[0114] The display unit 209 is configured with, for example, an LCD (Liquid Crystal Display), an organic EL (Organic Electro-Luminescence) panel, etc. The display unit 209 displays an image based on either basic format image data Vb or high-quality format image data Vh1, Vh2, or Vh3, depending on its display capabilities.

[0115] In this case, the control unit 201 controls the image data to be supplied to the display unit 209. This control is performed based on the identification information of the high-quality formats (extension category information of the streams) corresponding to each of the extended video streams STe1, STe2, and STe3 grasped by the control unit 201 as described above, and the display capability information of the display unit 209.

[0116] That is, if the display unit 209 is not capable of displaying a high frame frequency or a high dynamic range, the control unit 201 controls the display unit 209 to supply basic format image data Vb related to the decoding of the basic video stream STb to the display unit 209. In this case, the control unit 201 controls the decoding unit 204-0 to decode the basic video stream STb and the LDR electric-to-optical converter 205 to output the basic format image data Vb.

[0117] On the other hand, if the display unit 209 is capable of displaying at a high frame frequency but is not capable of displaying at a high dynamic range, the control unit 201 controls the display unit 209 to supply high-quality format image data Vh1 related to the decoding of the extended video stream STe1 to the display unit 209. In this case, the control unit 201 controls the decoding unit 204-0 to decode the basic video stream STb, the decoding unit 204-1 to decode the extended video stream STe1, and the LDR electric-to-optical converter 206 to output the high-quality format image data Vh1.

[0118] Furthermore, if the display unit 209 is not capable of displaying at a high frame frequency but is capable of displaying at a high dynamic range, the control unit 201 controls the display unit 209 so that high-quality format image data Vh2 related to the decoding of the extended video stream STe2 is supplied to the display unit 209. In this case, the control unit 201 controls the decoding unit 204-0 to decode the basic video stream STb, the decoding unit 204-2 to decode the extended video stream STe2, and the HDR electric-to-optical conversion unit 207 to output the high-quality format image data Vh2.

[0119] Furthermore, if the display unit 209 is capable of both high frame frequency display and high dynamic range display, the control unit 201 controls the display unit 209 so that high-quality format image data Vh3 related to the decoding of the extended video stream STe3 is supplied to the display unit 209. In this case, the control unit 201 controls the decoding unit 204-0 to decode the basic video stream STb, the decoding unit 204-2 to decode the extended video stream STe2, the decoding unit 204-3 to decode the extended video stream STe3, and the HDR electric-to-optical conversion unit 208 to output the high-quality format image data Vh3.

[0120] The operation of the receiving device 200 shown in Fig. 11 will now be briefly described. The receiving unit 202 receives the transport stream TS transmitted by broadcast waves or network packets from the transmitting device 100. This transport stream TS is supplied to the system decoder 203. The system decoder 203 extracts the basic video stream STb and the extended video streams STe1, STe2, and STe3 from this transport stream TS.

[0121] The system decoder 203 also extracts various pieces of information inserted into the container (transport stream) layer and sends them to the control unit 201. This information includes a scalable extension descriptor. The control unit 201 determines the high-quality format identification information (stream extension category information) corresponding to each of the extended video streams STe1, STe2, and STe3 from the "type of enhancement" field of this descriptor.

[0122] If the display unit 209 is not capable of displaying at a high frame frequency or a high dynamic range, the basic format image data Vb is supplied from the LDR electro-optical converter 205 to the display unit 209. The display unit 209 displays an image based on this basic format image data Vb, i.e., LDR image data with a frame frequency of 50 Hz.

[0123] In this case, the basic video stream STb extracted by the system decoder 203 is supplied to the decoding unit 204-0. The decoding unit 204-0 performs a decoding process on the basic video stream STb, and basic format image data Vb' is generated. Here, the decoding unit 204-0 can confirm that the supplied video stream is the basic video stream STb from the "nuh_layer_id" field in the header of the NAL unit.

[0124] The basic format image data Vb' generated by the decoding unit 204-0 is supplied to the LDR electric-to-optical converter 205. The LDR electric-to-optical converter 205 performs electric-to-optical conversion on the basic format image data Vb' to obtain basic format image data Vb, which is supplied to the display unit 209.

[0125] On the other hand, if the display unit 209 is capable of displaying at a high frame frequency but is not capable of displaying at a high dynamic range, the high-quality format image data Vh1 is supplied from the LDR electro-optical converter 206 to the display unit 209. The display unit 209 displays an image based on this high-quality format image data Vh1, that is, LDR image data with a frame frequency of 100 Hz.

[0126] In this case, the basic video stream STb extracted by the system decoder 203 is supplied to a decoding unit 204-0. The decoding unit 204-0 performs a decoding process on the basic video stream STb, generating basic format image data Vb'. The extended video stream STe1 extracted by the system decoder 203 is supplied to a decoding unit 204-1. The decoding unit 204-1 performs a decoding process on the extended video stream STe1 with reference to the basic format image data Vb', generating high-quality format image data Vh1'.

[0127] Here, the decoding unit 204-0 can confirm that the supplied video stream is the basic video stream STb from the "nuh_layer_id" field in the header of the NAL unit, while the decoding unit 204-1 can confirm that the supplied video stream is the extended video stream STe1 from the "nuh_layer_id" field in the header of the NAL unit.

[0128] The high-quality format image data Vh1' generated by the decoding unit 204-1 is supplied to the LDR electric-to-optical converter 206. The LDR electric-to-optical converter 206 performs electric-to-optical conversion on the high-quality format image data Vh1' to obtain high-quality format image data Vh1, which is supplied to the display unit 209.

[0129] Furthermore, if the display unit 209 is not capable of displaying at a high frame frequency but is capable of displaying at a high dynamic range, the high-quality format image data Vh2 is supplied from the HDR electro-optical conversion unit 207 to the display unit 209. The display unit 209 displays an image based on this high-quality format image data Vh2, that is, HDR image data with a frame frequency of 50 Hz.

[0130] In this case, the basic video stream STb extracted by the system decoder 203 is supplied to a decoding unit 204-0. The decoding unit 204-0 performs a decoding process on the basic video stream STb, generating basic format image data Vb'. The extended video stream STe2 extracted by the system decoder 203 is supplied to a decoding unit 204-2. The decoding unit 204-2 performs a decoding process on the extended video stream STe2 with reference to the basic format image data Vb', generating high-quality format image data Vh2'.

[0131] Here, the decoding unit 204-0 can confirm that the supplied video stream is the basic video stream STb from the "nuh_layer_id" field in the header of the NAL unit, while the decoding unit 204-2 can confirm that the supplied video stream is the extended video stream STe2 from the "nuh_layer_id" field in the header of the NAL unit.

[0132] The high-quality format image data Vh2′ generated by the decoding unit 204-2 is supplied to the HDR electric-to-optical converter 207. The HDR electric-to-optical converter 207 performs electric-to-optical conversion on the high-quality format image data Vh2′ to obtain high-quality format image data Vh2, which is supplied to the display unit 209.

[0133] Furthermore, if the display unit 209 is capable of both high frame frequency display and high dynamic range display, high-quality format image data Vh3 is supplied from the HDR electro-optical conversion unit 208 to the display unit 209. The display unit 209 displays this high-quality format image data Vh3, that is, an image based on HDR image data with a frame frequency of 100 Hz.

[0134] In this case, the basic video stream STb extracted by the system decoder 203 is supplied to a decoding unit 204-0. The decoding unit 204-0 performs a decoding process on the basic video stream STb, generating basic format image data Vb'. The extended video stream STe2 extracted by the system decoder 203 is supplied to a decoding unit 204-2. The decoding unit 204-2 performs a decoding process on the extended video stream STe2 with reference to the basic format image data Vb', generating high-quality format image data Vh2'.

[0135] Furthermore, the extended video stream STe3 extracted by the system decoder 203 is supplied to a decoding unit 204-3. The decoding unit 204-3 performs a decoding process on the extended video stream STe3 with reference to the high-quality format image data Vh2', and generates high-quality format image data Vh3'.

[0136] Here, the decoding unit 204-0 can confirm that the supplied video stream is the basic video stream STb from the "nuh_layer_id" field in the NAL unit header. The decoding unit 204-2 can confirm that the supplied video stream is the extended video stream STe2 from the "nuh_layer_id" field in the NAL unit header. The decoding unit 204-3 can confirm that the supplied video stream is the extended video stream STe3 from the "nuh_layer_id" field in the NAL unit header.

[0137] The high-quality format image data Vh3' generated by the decoding unit 204-3 is supplied to the HDR electric-to-optical converter 208. The HDR electric-to-optical converter 208 performs electric-to-optical conversion on the high-quality format image data Vh3' to obtain high-quality format image data Vh3, which is supplied to the display unit 209.

[0138] As described above, in the transmission / reception system 10 shown in Fig. 1, the transmission device 100 transmits identification information (extension category information of the stream) of the high-quality format corresponding to each of a predetermined number of extended video streams included in the transport stream TS, inserted into the container or video stream layer. Therefore, on the receiving side, it becomes easy to selectively perform decoding processing on predetermined video streams based on this identification information to obtain image data according to the display capability.

[0139] <2. Modifications> In the above embodiment, an example has been shown in which identification information (extended category information of the stream) of the high-quality format corresponding to each of a predetermined number of extended video streams included in the transport stream TS is inserted into both the container layer and the video stream layer and transmitted. However, it is also possible to insert this identification information only into the container layer or only into the video stream layer.

[0140] Also, instead of transmitting an ID indicating the layer extension type of the stream, the extension category of the stream, and information indicating the priority within the extension category, it is possible to indicate a combination of these with the value of "stream_type." For example, as shown in Figure 10, the basic stream can be "Stream_type = ST0," the first stream of frame rate scalable extension can be "Stream_type = ST1," the first stream of dynamic range scalable extension can be "Stream_type = ST2," and the stream of frame rate / dynamic range scalable extension (second extension stream) can be "Stream_type = ST3."

[0141] Furthermore, in the above-described embodiment, the transmission / reception system 10 including the transmission device 100 and the reception device 200 has been described, but the configuration of the transmission / reception system to which the present technology can be applied is not limited to this. For example, the reception device 200 may be configured as a set-top box and a monitor connected via a digital interface such as HDMI (High-Definition Multimedia Interface). In this case, the set-top box can obtain display capability information by, for example, obtaining EDID (Extended display identification data) from the monitor. Note that "HDMI" is a registered trademark.

[0142] In addition, in the above-described embodiment, an example has been shown in which the container is a transport stream (MPEG-2 TS). However, the present technology can be similarly applied to a system configured to distribute to a receiving terminal using a network such as the Internet. In Internet distribution, distribution is often performed using containers in MP4 or other formats. In other words, containers in various formats such as the transport stream (MPEG-2 TS) adopted in the digital broadcasting standard and MP4 used in Internet distribution are applicable.

[0143] The present technology can also be configured as follows. (1) an image encoding unit that generates a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data; a transmitter that transmits a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated by the image encoding unit; an identification information inserting unit that inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the container; Transmitting device. (2) The image encoding unit With respect to the basic format image data, a predictive coding process is performed within the basic format image data to generate the basic video stream; With respect to the high-quality format image data, a predictive coding process is selectively performed within the high-quality format image data or between the high-quality format image data and the basic format image data or other high-quality format image data to generate the extended video stream. The transmitting device according to (1) above. (3) The identification information inserted into the layer of the container includes: Information indicating whether each of the predetermined number of extended video streams is generated by performing predictive coding processing with the basic format image data or with the high quality format image data is added. The transmitting device according to (2) above. (4) The identification information inserted into the layer of the container includes: Information indicating a video stream corresponding to image data referenced in a predictive coding process between the basic format image data or other high-quality format image data performed when generating each of the predetermined number of extended video streams is added. The transmitting device according to (2) or (3). (5) The container is MPEG2-TS, The identification information insertion unit The identification information is inserted into each video elementary stream loop corresponding to the predetermined number of extended video streams existing under the program map table. The transmitting device according to any one of (1) to (4). (6) The identification information insertion unit and further inserting identification information of the high-quality formats corresponding to the predetermined number of extended video streams into a layer of the video streams. The transmitting device according to any one of (1) to (5). (7) The video stream has a NAL unit structure, The identification information insertion unit The identification information is inserted into the header of the NAL unit. The transmitting device according to (6) above. (8) an image encoding step of generating a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, respectively; a transmitting step of transmitting, by a transmitting unit, a container in a predetermined format including the basic video stream generated in the image encoding step and the predetermined number of extended video streams; an identification information inserting step of inserting identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the container; Sending method. (9) an image encoding unit that generates a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data; a transmitter that transmits a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated by the image encoding unit; an identification information inserting unit that inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into layers of the video streams; Transmitting device. (10) The image encoding unit With respect to the basic format image data, a predictive coding process is performed within the basic format image data to generate the basic video stream; With respect to the high-quality format image data, a predictive coding process is selectively performed within the high-quality format image data or between the high-quality format image data and the basic format image data or other high-quality format image data to generate the extended video stream. The transmitting device according to (9) above. (11) The video stream has a NAL unit structure, The identification information insertion unit The identification information is inserted into the header of the NAL unit. The transmitting device according to (9) or (10) above. (12) an image encoding step of generating a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, respectively; a transmitting step of transmitting, by a transmitting unit, a container in a predetermined format including the basic video stream generated in the image encoding step and the predetermined number of extended video streams; an identification information inserting step of inserting identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the video stream; Sending method. (13) A receiving unit is provided for receiving a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high-quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into the layer of the container, The video stream processing device further includes a processing unit configured to process each of the video streams included in the received container based on the identification information. Receiving device. (14) The processing unit Decoding the basic video stream and a predetermined extended video stream based on the identification information and display capability information to obtain image data corresponding to the display capability. The receiving device according to (13) above. (15) The basic video stream is generated by performing predictive coding processing on the basic format image data, The extended video stream is generated by selectively performing predictive coding processing within the high-quality format image data or predictive coding processing between the high-quality format image data and the basic format image data or other high-quality format image data on the high-quality format image data. A receiving device according to (13) or (14). (16) A receiving step of receiving, by a receiving unit, a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high-quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into the layer of the container, The method further comprises the step of processing each of the video streams contained in the received container based on the identification information. Receiving method. (17) A receiving unit is provided for receiving a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the video stream; The video stream processing device further includes a processing unit configured to process each of the video streams included in the received container based on the identification information. Receiving device. (18) The processing unit Decoding the basic video stream and a predetermined extended video stream based on the identification information and display capability information to obtain image data corresponding to the display capability. The receiving device according to (17) above. (19) The basic video stream is generated by performing predictive coding processing on the basic format image data, The extended video stream is generated by selectively performing predictive coding processing within the high-quality format image data or predictive coding processing between the high-quality format image data and the basic format image data or other high-quality format image data on the high-quality format image data. A receiving device according to (17) or (18). (20) A receiving step of receiving, by a receiving unit, a container of a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high-quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the video stream; The method further comprises the step of processing each of the video streams contained in the received container based on the identification information. Receiving method.

[0144] The main feature of this technology is that it makes it easy for the receiving side to obtain image data according to the display capabilities by inserting identification information (extended category information of the stream) of the high-quality format corresponding to each of the predetermined number of extended video streams included in the transport stream TS into the container or video stream layer and transmitting it (see Figure 10).

[0145] 10. Transmitting and receiving system 100 Transmitting device 101 Control unit 102, 103: LDR photoelectric conversion unit 104,105 HDR photoelectric conversion unit 106...Video Encoder 106-0, 106-1, 106-1, 106-1... Encoding section 107 System Encoder 108 Transmitter 150 Image data generation unit 151···HDR Camera 152,154...Frame rate conversion section 153 Dynamic range conversion section 160 Encoding section 161...Intra-layer prediction unit 162...Inter-layer prediction unit 163···Forecast Adjustment Section 164...Selection section 165 Encoding function unit 200 Receiving device 201 Control unit 202 Receiving unit 203 System Decoder 204...Video decoder 204-0, 204-1, 204-1, 204-1... Decoder 205, 206···LDR electro-optical conversion unit 207, 208···HDR electro-optical converter 209...Display section 240 Decoder 241 Decoding function unit 242 Intra-layer prediction compensation unit 243 Inter-layer prediction compensation unit 244···Prediction Adjustment Section 245···Selection section

Claims

1. an image encoding unit that generates a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data; a transmitter that transmits a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated by the image encoding unit; an identification information inserting unit that inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the container; Transmitting device.

2. The image encoding unit With respect to the basic format image data, a predictive coding process is performed within the basic format image data to generate the basic video stream; With respect to the high-quality format image data, a predictive coding process is selectively performed within the high-quality format image data or between the high-quality format image data and the basic format image data or other high-quality format image data to generate the extended video stream. The transmitting device according to claim 1 .

3. The identification information inserted into the layer of the container includes: Information indicating whether each of the predetermined number of extended video streams is generated by performing predictive coding processing with the basic format image data or with the high quality format image data is added. The transmitting device according to claim 2 .

4. The identification information inserted into the layer of the container includes: Information indicating a video stream corresponding to image data referenced in a predictive coding process between the basic format image data or other high-quality format image data performed when generating each of the predetermined number of extended video streams is added. The transmitting device according to claim 2 .

5. The container is MPEG2-TS, The identification information insertion unit The identification information is inserted into each video elementary stream loop corresponding to the predetermined number of extended video streams existing under the program map table. The transmitting device according to claim 1 .

6. The identification information insertion unit and further inserting, into a layer of the video stream, identification information of the high-quality formats to which the predetermined number of extended video streams respectively correspond. The transmitting device according to claim 1 .

7. The video stream has a NAL unit structure, The identification information insertion unit Inserting the identification information into the header of the NAL unit. The transmitting device according to claim 6.

8. an image encoding step of generating a basic video stream obtained by encoding basic format image data and a predetermined number of enhanced video streams obtained by encoding a predetermined number of high quality format image data; a transmitting step of transmitting, by a transmitting unit, a container in a predetermined format including the basic video stream generated in the image encoding step and the predetermined number of extended video streams; an identification information inserting step of inserting identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the container; Sending method.

9. an image encoding unit that generates a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data; a transmitter that transmits a container in a predetermined format including the basic video stream and the predetermined number of extended video streams generated by the image encoding unit; an identification information inserting unit that inserts identification information of high-quality formats corresponding to the predetermined number of extended video streams into layers of the video streams; Transmitting device.

10. The image encoding unit With respect to the basic format image data, a predictive coding process is performed within the basic format image data to generate the basic video stream; With respect to the high-quality format image data, a predictive coding process is selectively performed within the high-quality format image data or between the high-quality format image data and the basic format image data or other high-quality format image data to generate the extended video stream. The transmitting device according to claim 9.

11. The video stream has a NAL unit structure, The identification information insertion unit Inserting the identification information into the header of the NAL unit. The transmitting device according to claim 9.

12. an image encoding step of generating a basic video stream obtained by encoding basic format image data and a predetermined number of enhanced video streams obtained by encoding a predetermined number of high quality format image data; a transmitting step of transmitting, by a transmitting unit, a container in a predetermined format including the basic video stream generated in the image encoding step and the predetermined number of extended video streams; an identification information inserting step of inserting identification information of high-quality formats corresponding to the predetermined number of extended video streams into a layer of the video stream; Sending method.

13. a receiving unit for receiving a container in a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into the layer of the container, The video stream processing device further includes a processing unit configured to process each of the video streams included in the received container based on the identification information. Receiving device.

14. The processing unit Decoding the basic video stream and a predetermined extended video stream based on the identification information and display capability information to obtain image data corresponding to the display capability.

14. The receiving device according to claim 13.

15. the basic video stream is generated by performing predictive coding processing within the basic format image data on the basic format image data, The extended video stream is generated by selectively performing predictive coding processing within the high-quality format image data or predictive coding processing between the high-quality format image data and the basic format image data or other high-quality format image data on the high-quality format image data. A receiving device according to claim 13.

16. a receiving step of receiving, by a receiving unit, a container in a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into the layer of the container, The method further comprises the step of processing each of the video streams contained in the received container based on the identification information. Receiving method.

17. a receiving unit for receiving a container in a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the video stream; The video stream processing device further includes a processing unit configured to process each of the video streams included in the received container based on the identification information. Receiving device.

18. The processing unit Decoding the basic video stream and a predetermined extended video stream based on the identification information and display capability information to obtain image data corresponding to the display capability.

18. The receiving device according to claim 17.

19. the basic video stream is generated by performing predictive coding processing within the basic format image data on the basic format image data, The extended video stream is generated by selectively performing predictive coding processing within the high-quality format image data or predictive coding processing between the high-quality format image data and the basic format image data or other high-quality format image data on the high-quality format image data. A receiving device according to claim 17.

20. a receiving step of receiving, by a receiving unit, a container in a predetermined format including a basic video stream obtained by encoding basic format image data and a predetermined number of extended video streams obtained by encoding a predetermined number of high quality format image data, Identification information of high-quality formats corresponding to the predetermined number of extended video streams is inserted into a layer of the video stream; The method further comprises the step of processing each of the video streams contained in the received container based on the identification information. Receiving method.

Images