Transmission method, transmission device, receiving device, and receiving method

Abstract

To successfully transmit a predetermined number of pieces of high-quality format image data together with basic format image data.SOLUTION: A basic video stream including encoded image data of basic format image data and an extended video stream including encoded image data of each of a predetermined number of pieces of high-quality format image data are generated, or a video stream including encoded image data of basic format image data and encoded image data of each of a predetermined number of pieces of high-quality format image data are generated. A container of a predetermined format including the generated video stream is transmitted. Identification information identifying a corresponding format is inserted into the encoded image data of the basic format image data and each of a predetermined number of pieces of high-quality format image data.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] This technology transmits method ,send Device, Receiving device and how to receive it Regarding. [Background technology]

[0002] Conventionally, it is known that high-quality format image data is transmitted together with basic format image data, and the receiving side selectively uses the basic format image data or the high-quality format image data. For example, Patent Document 1 describes that media coding is performed in a scalable manner to generate a base layer stream for a low-resolution video service and an enhancement layer stream for a high-resolution video service, and a broadcast signal including these is transmitted. In addition to high resolution, the high-quality format also includes high frame frequency, high dynamic range, wide color gamut, and high bit length. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2008-543142 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present technology is to successfully transmit a predetermined number of high-quality format image data together with basic format image data. [Means for solving the problem]

[0005] The concept of this technology is as follows: an image encoding unit for generating a basic video stream including encoded image data of basic format image data and an extended video stream including encoded image data of each of a predetermined number of high quality format image data; a transmission unit that transmits a container in a predetermined format including the basic video stream and the extended video stream generated by the image encoding unit, The image encoding unit inserts identification information for identifying a corresponding format into each of the encoded image data of the basic format image data and the predetermined number of high quality format image data. Located in the transmitting device.

[0006] In the present technology, an image encoding unit generates a basic video stream including encoded image data of basic format image data and an extended video stream including encoded image data of each of a predetermined number of high quality format image data. For example, predictive encoding processing is performed within the basic format image data for the basic format image data. Also, predictive encoding processing is performed within the high quality format image data or predictive encoding processing between the high quality format image data and the basic format image data or other high quality format image data for the high quality format image data.

[0007] The transmitting unit transmits a container in a predetermined format including the basic video stream and the extended video stream generated by the image encoding unit. For example, the container may be a transport stream (MPEG-2 TS) adopted in the digital broadcasting standard. Also, for example, the container may be MP4 used for Internet distribution or a container in another format.

[0008] In the image encoding unit, identification information for identifying the corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of high quality format image data. For example, the encoded image data may have a NAL unit structure, and the image encoding unit may insert the identification information into the header of the NAL unit. In this case, for example, the image encoding unit may insert the identification information using the "nuh_layer_id" field of the header of the NAL unit. Also, in this case, for example, the image encoding unit may insert the identification information using the "nuh_layer_id" and "nuh_temporal_id_plus1" fields of the header of the NAL unit.

[0009] In this way, in the present technology, identification information for identifying the corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of high-quality format image data, so that the receiving side can easily obtain image data according to the display capability by selectively performing a decoding process on the predetermined encoded image data based on the identification information.

[0010] In addition, in the present technology, for example, the container layer may further include an information inserting unit that inserts information defining what format of encoded image data the identification information inserted into the encoded image data indicates. In this case, for example, the container may be MPEG2-TS, and the information inserting unit may insert information into a video elementary stream loop corresponding to an extended video stream present under the program map table. In this case, the receiving side can grasp in advance, in the container layer, what format of encoded image data the identification information inserted into the encoded image data indicates.

[0011] Another concept of the present technology is a receiving unit for receiving a container of a predetermined format having a basic video stream including encoded image data of basic format image data and an enhanced video stream including encoded image data of each of a predetermined number of high quality format image data; Identification information for identifying a corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of the high quality format image data, The video stream processing device further includes a processing unit for processing each video stream contained in the received container based on the identification information and the display capability information. in the receiving device.

[0012] In the present technology, a receiving unit receives a container of a predetermined format having a basic video stream and an extended video stream, where the basic video stream includes encoded image data of the basic format image data, and the extended video stream includes encoded image data of each of a predetermined number of high quality format image data.

[0013] Identification information for identifying the corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of high quality format image data. For example, the encoded image data may have a NAL unit structure, and the identification information may be inserted into a header of the NAL unit. Each video stream included in the received container is processed by the processing unit based on the identification information and the display capability information.

[0014] In this way, in this technology, the extended video stream contained in the received container is processed based on the identification information for identifying the corresponding format, which is inserted in each of the encoded image data of the basic format image data and the predetermined number of high-quality format image data, and the display capability information. Therefore, it is possible to easily obtain image data according to the display capability by selectively performing a decoding process on the predetermined encoded image data.

[0015] In addition, in the present technology, for example, information defining what format of encoded image data the identification information inserted into the encoded image data indicates is inserted into the container layer, and the processing unit may be configured to determine what format of encoded image data the identification information inserted into the encoded image data indicates based on the information inserted into the container layer.

[0016] Another concept of the present technology is an image encoding unit that generates a video stream including encoded image data of basic format image data and encoded image data of each of a predetermined number of high quality format image data; a transmission unit that transmits a container in a predetermined format including the video stream generated by the image encoding unit, The image encoding unit inserts identification information for identifying a corresponding format into the encoded image data of the basic format image data and into each of the encoded image data of the predetermined number of high quality format image data. Located in the transmitting device.

[0017] In the present technology, an image encoding unit generates a video stream including encoded image data of each of basic format image data and a predetermined number of high quality format image data. For example, predictive encoding processing is performed within the basic format image data for the basic format image data. Also, predictive encoding processing is performed within the high quality format image data or predictive encoding processing between the high quality format image data and the basic format image data or other high quality format image data for the high quality format image data.

[0018] The transmitting unit transmits a container in a predetermined format including the video stream generated by the image encoding unit. For example, the container may be a transport stream (MPEG-2 TS) adopted in the digital broadcasting standard. Also, for example, the container may be MP4 used for Internet distribution or a container in another format.

[0019] In the image encoding unit, identification information for identifying the corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of high quality format image data. For example, the encoded image data may have a NAL unit structure, and the image encoding unit may insert the identification information into the header of the NAL unit. In this case, for example, the image encoding unit may insert the identification information using the "nuh_layer_id" field of the header of the NAL unit. Also, in this case, for example, the image encoding unit may insert the identification information using the "nuh_layer_id" and "nuh_temporal_id_plus1" fields of the header of the NAL unit.

[0020] In this way, in the present technology, identification information for identifying the corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of high-quality format image data, so that the receiving side can easily obtain image data according to the display capability by selectively performing a decoding process on the predetermined encoded image data based on the identification information.

[0021] In addition, in the present technology, for example, the container layer may further include an information inserting unit that inserts information defining what format of encoded image data the identification information inserted into the encoded image data indicates. In this case, for example, the container may be MPEG2-TS, and the information inserting unit may insert information into a video elementary stream loop corresponding to a video stream present under a program map table. In this case, the receiving side can grasp in advance, in the container layer, what format of encoded image data the identification information inserted into the encoded image data indicates.

[0022] Another concept of the present technology is a receiving unit for receiving a container of a predetermined format having a video stream including encoded image data of basic format image data and encoded image data of each of a predetermined number of high quality format image data; Identification information for identifying a corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of the high quality format image data, The video stream stored in the received container is processed based on the identification information and the display capability information. in the receiving device.

[0023] In this technique, a container of a predetermined format having a video stream including encoded image data of basic format image data and encoded image data of each of a predetermined number of high quality format image data is received by a receiving unit.

[0024] Identification information for identifying the corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of high quality format image data. For example, the encoded image data may have a NAL unit structure, and the identification information may be inserted into a header of the NAL unit. The processing unit processes the video stream contained in the received container based on the identification information and the display capability information.

[0025] In this way, in this technology, the video stream contained in the received container is processed based on the identification information for identifying the corresponding format and the display capability information, which are inserted in each of the encoded image data of the basic format image data and the predetermined number of high-quality format image data. Therefore, it is possible to easily obtain image data according to the display capability by selectively performing a decoding process on the predetermined encoded image data.

[0026] In addition, in the present technology, for example, information defining what format of encoded image data the identification information inserted into the encoded image data indicates is inserted into the container layer, and the processing unit may be configured to determine what format of encoded image data the identification information inserted into the encoded image data indicates based on the information inserted into the container layer. Effect of the Invention

[0027] According to the present technology, a predetermined number of high-quality format image data can be successfully transmitted together with basic format image data. Note that the effects described herein are not necessarily limited to those described herein, and may be any of the effects described in the present disclosure. [Brief description of the drawings]

[0028] [Figure 1] 1 is a block diagram showing an example of the configuration of a transmission / reception system according to an embodiment; [Diagram 2] FIG. 2 is a block diagram showing a configuration example of a transmission device. [Diagram 3] 11 is a block diagram showing an example of the configuration of an image data generating unit that generates basic format image data Vb and three high quality format image data Vh1, Vh2, and Vh3. FIG. [Figure 4] 2 is a block diagram showing an example of the configuration of a main part of an encoding unit. FIG. [Diagram 5] A diagram showing an example of the structure of a NAL unit header and the contents of main parameters in the example structure. [Figure 6] FIG. 2 is a diagram showing an example of the configuration of encoded image data Cb, Ch1, Ch2, and Ch3. [Figure 7] FIG. 13 is a diagram showing an example of the structure of a scalable extension descriptor. [Figure 8] FIG. 13 is a diagram showing the contents of main information in an example of the structure of a scalable extension descriptor. [Figure 9] 13 is a diagram showing the correspondence between the value of "nuh_layer_id" in the NAL unit header and the description in the scalable extension descriptor. [Figure 10] FIG. 2 is a diagram showing an example of the configuration of a transport stream TS (in the case of two streams). [Figure 11] FIG. 2 is a diagram showing an example of the configuration of a transport stream TS (in the case of one stream). [Figure 12] FIG. 2 is a block diagram showing a configuration example of a receiving device. [Figure 13] 4 is a block diagram showing an example of the configuration of a main part of a decoding unit; [Figure 14] FIG. 13 is a diagram showing the output of the compressed data buffer (cpb) in the case of a two-stream configuration, and the allocation of encoded image data Cb, Ch1, Ch2, and Ch3 to the corresponding decoding units according to "nuh_layer_id." [Figure 15] FIG. 13 is a diagram showing the output of the compressed data buffer (cpb) in the case of a one-stream configuration, and the allocation of encoded image data Cb, Ch1, Ch2, and Ch3 to the corresponding decoding units according to "nuh_layer_id." [Figure 16] 13 is a flowchart showing an example of a process for determining a decoding range from display capability information (display performance information). [Figure 17] FIG. 2 is a diagram showing an example of the configuration of encoded image data Cb, Ch1, Ch2, and Ch3. [Figure 18] 13 is a diagram showing the correspondence between the values ​​of "nuh_layer_id" and "nuh_temporal_id_plus1" in the NAL unit header and the descriptions in the scalable extension descriptor. [Figure 19] 13 is a flowchart showing another example of the process of determining the decoding range from the display capability information (display performance information). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Hereinafter, modes for carrying out the invention (hereinafter, referred to as "embodiments") will be described in the following order. 1. Embodiment 2. Variations

[0030] <1. Preferred embodiment> [Transmitting and receiving system] 1 shows an example of the configuration of a transmission / reception system 10 according to an embodiment. This transmission / reception system 10 includes a transmission device 100 and a reception device 200. A transport stream TS as a container is transmitted from the transmission device 100 to the reception device 200 on broadcast waves or network packets. In this embodiment, there are two cases: (1) a case where the transport stream TS has a two-stream configuration having two video streams, a basic video stream and an extended video stream, and (2) a case where the transport stream TS has a one-stream configuration having one video stream.

[0031] "For 2-stream configuration" The transmitting device 100 transmits a transport stream TS as a container on a broadcast wave or a network packet. This transport stream TS includes two video streams, a basic video stream and an extended video stream. The basic video stream includes encoded image data of basic format image data. The basic video stream is generated by, for example, applying a predictive encoding process such as H.264 / AVC or H.265 / HEVC to basic format image data.

[0032] The extended video stream includes encoded image data of each of a predetermined number of high-quality format image data, and is generated by performing predictive encoding processing, such as H.264 / AVC or H.265 / HEVC, on each of the predetermined number of high-quality image data.

[0033] Identification information for identifying the corresponding format is inserted into each of the coded image data of the basic format image data and the predetermined number of high quality format image data. On the receiving side, based on this identification information, it is possible to easily perform a decoding process on the predetermined coded image data to obtain image data according to the display capability. In this embodiment, the identification information is inserted into the header of the NAL unit.

[0034] Information is inserted into the container layer to define what format of encoded image data the identification information to be inserted into the encoded image data indicates. This information enables the receiving side to previously know, at the container layer, what format of encoded image data the identification information to be inserted into the encoded image data indicates. In this embodiment, the information is inserted into each video elementary stream loop corresponding to the extended video stream present under the program map table.

[0035] The receiving device 200 receives the transport stream TS transmitted by broadcast waves or network packets from the transmitting device 100. As described above, this transport stream TS has a basic video stream including coded image data of basic format image data, and an extended video stream including coded image data of each of a predetermined number of high quality format image data.

[0036] As described above, identification information for identifying the corresponding format is inserted into each of the coded image data of the predetermined number of high-quality format image data. The receiving device 200 processes each video stream included in the transport stream TS based on the identification information and the display capability information, and obtains image data according to the display capability.

[0037] "For one-stream configuration" The transmitting device 100 transmits a transport stream TS as a container on a broadcast wave or a packet of a network. This transport stream TS includes one video stream. This video stream includes encoded image data of basic format image data and encoded image data of a predetermined number of high-quality format image data. This video stream is generated by performing predictive encoding processing such as H.264 / AVC and H.265 / HEVC on the basic format image data and the predetermined number of high-quality format image data, respectively.

[0038] Identification information for identifying the basic format or the corresponding high-quality format is inserted into each of the coded image data of the basic format image data and the predetermined number of high-quality format image data. On the receiving side, based on this identification information, it is possible to selectively perform a decoding process on the predetermined coded image data to easily obtain image data according to the display capability. In this embodiment, the identification information is inserted into the header of the NAL unit.

[0039] Information is inserted into the container layer that defines the format of the coded image data indicated by the identification information to be inserted into the coded image data. This information enables the receiving side to know in advance, at the container layer, the format of the coded image data indicated by the identification information to be inserted into the coded image data. In this embodiment, the information is inserted into each video elementary stream loop corresponding to the video stream present under the program map table.

[0040] The receiving device 200 receives the transport stream TS transmitted by broadcast waves or network packets from the transmitting device 100. As described above, this transport stream TS has a video stream including the coded image data of basic format image data and each of the coded image data of a predetermined number of high quality format image data.

[0041] As described above, identification information for identifying the corresponding format is inserted into each of the coded image data of the basic format image data and the predetermined number of high-quality format image data. Based on this identification information and the display capability information, the receiving device 200 processes the video stream included in the transport stream TS and obtains image data according to the display capability.

[0042] "Transmitter Configuration" 2 shows an example of the configuration of the transmitting device 100. This transmitting device 100 handles basic format image data Vb and three high-quality format image data Vh1, Vh2, and Vh3 as transmission image data. Here, the basic format image data Vb is LDR (Low Dynamic Range) image data with a frame frequency of 50 Hz. The high-quality format image data Vh1 is LDR image data with a frame frequency of 100 Hz. The LDR image data has a luminance range of 0% to 100% with respect to the brightness of the white peak of a conventional LDR image.

[0043] The high-quality format image data Vh2 is HDR (High Dynamic Range) image data with a frame frequency of 50 Hz. The high-quality format image data Vh3 is HDR image data with a frame frequency of 100 Hz. If the brightness of the white peak of a conventional LDR image is 100%, this HDR image data has a brightness range of 0 to 100%*N, for example, 0 to 1000% or more.

[0044] 3 shows an example of the configuration of an image data generation unit 150 that generates basic format image data Vb and three high-quality format image data Vh1, Vh2, and Vh3. The image data generation unit 150 includes an HDR camera 151, a frame rate conversion unit 152, a dynamic range conversion unit 153, and a frame rate conversion unit 154.

[0045] The HDR camera 151 captures an image of a subject and outputs HDR image data having a frame frequency of 100 Hz, i.e., high-quality format image data Vh3. The frame rate conversion unit 152 converts the frame frequency of the high-quality format image data Vh3 output from the HDR camera 151 from 100 Hz to 50 Hz, and outputs HDR image data having a frame frequency of 50 Hz, i.e., high-quality format image data Vh2.

[0046] The dynamic range conversion unit 153 performs HDR to LDR conversion processing on the high-quality format image data Vh3 output from the HDR camera 151, and outputs LDR image data with a frame frequency of 100 Hz, i.e., high-quality format image data Vh1. The frame rate conversion unit 154 performs a process of converting the frame frequency of the high-quality format image data Vh1 output from the dynamic range conversion unit 153 from 100 Hz to 50 Hz, and outputs LDR image data with a frame frequency of 50 Hz, i.e., basic format image data Vb.

[0047] Returning to Fig. 2, the transmitting device 100 has a control unit 101, LDR photoelectric conversion units 102 and 103, HDR photoelectric conversion units 104 and 105, a video encoder 106, a system encoder 107, and a transmitting unit 108. The control unit 101 is configured with a CPU (Central Processing Unit), and controls the operation of each unit of the transmitting device 100 based on a control program.

[0048] The LDR photoelectric conversion unit 102 applies photoelectric conversion characteristics for LDR images (LDR OETF curve) to the basic format image data Vb to obtain basic format image data Vb' for transmission. The LDR photoelectric conversion unit 103 applies photoelectric conversion characteristics for LDR images to the high quality format image data Vh1 to obtain high quality format image data Vh1' for transmission.

[0049] The HDR photoelectric conversion unit 104 applies photoelectric conversion characteristics for HDR images (HDR OETF curve) to the high-quality format image data Vh2 to obtain high-quality format image data Vh2' for transmission. The HDR photoelectric conversion unit 105 applies photoelectric conversion characteristics for HDR images to the high-quality format image data Vh3 to obtain high-quality format image data Vh3' for transmission.

[0050] The video encoder 106 has four encoding units 106-0, 106-1, 106-2, and 106-3. The encoding unit 106-0 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on basic format image data Vb' for transmission to obtain encoded image data Cb. In this case, the encoding unit 106-0 performs prediction within the image data Vb'.

[0051] The encoding unit 106-1 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on high-quality format image data Vh1' for transmission to obtain encoded image data Ch1. In this case, the encoding unit 106-1 selectively performs prediction within the image data Vh1' or prediction between the image data Vb' for each encoding block in order to reduce a prediction residual.

[0052] The encoding unit 106-2 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh2' for transmission to obtain encoded image data Ch2. In this case, the encoding unit 106-2 selectively performs prediction within the image data Vh2' or prediction between the image data Vb' for each encoding block in order to reduce a prediction residual.

[0053] The encoding unit 106-3 performs predictive encoding processing such as H.264 / AVC or H.265 / HEVC on the high-quality format image data Vh3' for transmission to obtain encoded image data Ch3. In this case, the encoding unit 106-3 selectively performs prediction within the image data Vh3' or prediction between the image data Vh2' for each encoding block in order to reduce a prediction residual.

[0054] 4 shows a configuration example of the main part of the encoding unit 160. This encoding unit 160 can be applied to the encoding units 106-1, 106-2, and 106-3. This encoding unit 160 has an intra-layer prediction unit 161, an inter-layer prediction unit 162, a prediction adjustment unit 163, a selection unit 164, and an encoding function unit 165.

[0055] The intra-layer prediction unit 161 performs prediction (intra-layer prediction) on the image data V1 to be coded to obtain prediction residual data. The inter-layer prediction unit 162 performs prediction (inter-layer prediction) on the image data V1 to be coded with reference image data V2 to obtain prediction residual data.

[0056] In order to efficiently perform inter-layer prediction in the inter-layer prediction unit 162, the prediction adjustment unit 163 performs the following processing according to the type of scalable extension of the image data V1 to the image data V2. In the case of dynamic range extension, level adjustment is performed to convert from LDR to HDR. In the case of spatial scalable extension, blocks of other layers that have been scaled to a predetermined size are targeted. In the case of frame rate extension, bypass is performed. In the case of color gamut extension, mapping is performed for both luminance and chrominance. In the case of bit length extension, conversion is performed to align the MSB of pixels.

[0057] For example, in the case of the encoding unit 106-1, the image data V1 is high-quality format image data Vh1' (100 Hz, LDR), the image data V2 is basic format image data Vb' (50 Hz, LDR), and the type of scalable extension corresponds to frame rate extension. Therefore, in the prediction adjustment unit 163, the image data Vb' is bypassed as it is.

[0058] Also, for example, in the case of the encoding unit 106-2, the image data V1 is high-quality format image data Vh2' (50 Hz, HDR), the image data V2 is basic format image data Vb' (50 Hz, LDR), and the type of scalable extension corresponds to dynamic range extension. Therefore, the prediction adjustment unit 163 performs level adjustment on the image data Vb' to convert from LDR to HDR. Note that the level adjustment may be performed based on information supplied from the dynamic range conversion unit 153.

[0059] Also, for example, in the case of the encoding unit 106-3, the image data V1 is high-quality format image data Vh3' (100 Hz, HDR), the image data V2 is high-quality format image data Vh2' (50 Hz, HDR), and the type of scalable extension corresponds to frame rate extension. Therefore, in the prediction adjustment unit 163, the image data Vb' is bypassed as it is.

[0060] The selection unit 164 selectively extracts the prediction residual data obtained by the intra-layer prediction unit 161 or the prediction residual data obtained by the inter-layer prediction unit 162 for each coding block, and sends it to the encoding function unit 165. In this case, the selection unit 164 extracts, for example, the prediction residual having a smaller value. The encoding function unit 165 performs encoding processes such as transform coding, quantization, and entropy coding on the prediction residual data extracted from the selection unit 164 to obtain encoded image data CV.

[0061] 2, the video encoder 106 inserts identification information for identifying the corresponding format into each of the encoded image data Cb, Ch1, Ch2, and Ch3. The video encoder 106 inserts this identification information into, for example, the header of the NAL unit.

[0062] Figure 5(a) shows an example structure (Syntax) of a NAL unit header, and Figure 5(b) shows the contents (Semantics) of the main parameters in that example structure. The 1-bit field "forbidden_zero_bit" must be 0. The 6-bit field "nal_unit_type" indicates the NAL unit type. The 6-bit field "nuh_layer_id" is an ID indicating the layer extension type of the stream. The 3-bit field "nuh_temporal_id_plus1" indicates temporal_id (0-6) and takes on a value of 1 plus 1 (1-7).

[0063] In this embodiment, the 6-bit field of "nuh_layer_id" indicates identification information for identifying the format to which the NAL unit (encoded image data) corresponds. For example, "0" indicates basic. "1" indicates spatial extension. "2" indicates frame rate extension. "3" indicates bit length extension. "4" indicates dynamic range extension. "5" indicates wide color gamut extension. "6" indicates frame rate extension and dynamic range extension. "7" indicates spatial extension and frame rate extension.

[0064] The coded image data Cb corresponds to the basic format image data Vb, and the "nuh_layer_id" of this coded image data Cb is set to "0". The coded image data Ch1 corresponds to the frame rate extension format image data Vh1, and the "nuh_layer_id" of this coded image data Ch1 is set to "2". The coded image data Ch2 corresponds to the dynamic range extension format image data Vh2, and the "nuh_layer_id" of this coded image data Ch2 is set to "4". The coded image data Ch3 corresponds to the frame rate extension and dynamic range extension format image data Vh3, and the "nuh_layer_id" of this coded image data Ch3 is set to "6".

[0065] Fig. 6 shows an example of the composition of encoded image data Cb, Ch1, Ch2, and Ch3. The horizontal axis shows the picture order of composition (POC), with the left side showing earlier display times and the right side showing later display times. Each rectangular frame shows a picture, and the arrows show an example of a picture reference relationship in predictive coding processing. For both inter-layer and intra-layer prediction, the target picture changes for each block, and the prediction direction and the number of references are not limited to the example shown in the figure.

[0066] The coded image data Cb is composed of coded image data of pictures "00", "01", .... The coded image data Ch1 is composed of coded image data of pictures "10", "11", ..., which are located between each picture in the coded image data Cb. The coded image data Ch2 is composed of coded image data of pictures "20", "21", ..., which are located in the same positions as each picture in the coded image data Cb. And the coded image data Ch3 is composed of coded image data of pictures "30", "31", ..., which are located between each picture in the coded image data Ch2.

[0067] 2, the system encoder 107 generates a video stream using the encoded image data Cb, Ch1, Ch2, and Ch3 generated by the video encoder 106, and performs PES packetization and TS packetization to generate a transport stream TS. Then, the transmitter 108 transmits this transport stream TS to the receiving device 200 via broadcast waves or net packets.

[0068] In the case of a two-stream configuration, the system encoder 107 generates a basic video stream including the coded image data Cb and an extended video stream including the coded image data Ch1, Ch2, and Ch3. That is, in this case, the transport stream TS has two video streams, a basic video stream including the coded image data Cb and an extended video stream including the coded image data Ch1, Ch2, and Ch3.

[0069] In addition, in the case of one stream configuration, the system encoder 107 generates a video stream including the coded image data Cb, Ch1, Ch2, and Ch3. That is, in this case, the transport stream TS has one video stream including the coded image data Cb, Ch1, Ch2, and Ch3.

[0070] The system encoder 107 inserts, into the layer of the container (transport stream), information that defines what format the identification information inserted into the encoded image data indicates.

[0071] In this embodiment, in the case of a two-stream configuration, a scalable extension descriptor is inserted into a video elementary stream loop corresponding to an extended video stream (including encoded data Ch1, Ch2, and Ch3) existing under a PMT (Program Map Table). In this case, the identification information inserted into the encoded image data Ch1, Ch2, and Ch3 defines the format of the encoded image data.

[0072] In this embodiment, in the case of a single stream configuration, a scalable extension descriptor is inserted into a video elementary stream loop corresponding to a video stream (including coded image data Cb, Ch1, Ch2, Ch3) existing under a PMT (Program Map Table). In this case, the identification information inserted into the coded image data Cb, Ch1, Ch2, Ch3 defines the format of the coded image data.

[0073] Figure 7 shows an example of the structure (Syntax) of this scalable extension descriptor. Figure 8 shows the main information contents (Semantics) in the example structure shown in Figure 7. The 8-bit field "descriptor_tag" indicates the descriptor type, which indicates that it is a scalable extension descriptor in this case. The 8-bit field "descriptor_length" indicates the length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor.

[0074] The "Extended_spatial_resolution_flag" flag indicates whether a spatial resolution extension component is included. "1" indicates that a spatial resolution extension component is included, and "0" indicates that a spatial resolution extension component is not included. The "Extended_frame_rate_flag" flag indicates whether a frame rate extension component is included. "1" indicates that a frame rate extension component is included, and "0" indicates that a frame rate extension component is not included.

[0075] The "Extended_bit_depth_flag" flag indicates whether a bit depth extension component is included. "1" indicates that a bit depth extension component is included, and "0" indicates that a bit depth extension component is not included. The "Extended_dynamic_range_flag" flag indicates whether a dynamic range extension component is included. "1" indicates that a dynamic range extension component is included, and "0" indicates that a dynamic range extension component is not included. The "Extended_color_gamut_flag" flag indicates whether a color gamut extension component is included. "1" indicates that a color gamut extension component is included, and "0" indicates that a color gamut extension component is not included.

[0076] The 8-bit field "number_of_layerIDs" indicates the number of layers included in the stream. There are as many 6-bit "layerID" fields as there are layers. This "layerID" field indicates the layer ID (Layer_id).

[0077] Here, in the case of a two-stream configuration, when the extended video stream includes encoded image data Ch1, Ch2, and Ch3, "Extended_spatial_resolution_flag", "Extended_bit_depth_flag", and "Extended_color_gamut_flag" are set to "0", and "Extended_frame_rate_flag" and "Extended_dynamic_range_flag" are set to "1". In addition, "number_of_layerIDs" is set to "3", and "layerID" is set to "2", "4", and "6", respectively.

[0078] With these settings, "layerID"="2" indicates frame rate extension, and therefore "nuh_layer_id"="2" in the NAL unit header is defined as indicating frame rate extension coded image data. Also, "layerID"="4" indicates dynamic range extension, and therefore "nuh_layer_id"="4" in the NAL unit header is defined as indicating dynamic range extension coded image data. Furthermore, "layerID"="6" indicates frame rate extension and dynamic range extension, and therefore "nuh_layer_id"="6" in the NAL unit header is defined as indicating frame rate extension and dynamic range extension coded image data.

[0079] In addition, in the case of a one-stream configuration, when the video stream includes encoded data Cb, Ch1, Ch2, and Ch3, "Extended_spatial_resolution_flag", "Extended_bit_depth_flag", and "Extended_color_gamut_flag" are set to "0", and "Extended_frame_rate_flag" and "Extended_dynamic_range_flag" are set to "1". Furthermore, "number_of_layerIDs" is set to "4", and "layerID" is set to "0", "2", "4", and "6", respectively.

[0080] With these settings, "layerID"="0" indicates basic, and therefore "nuh_layer_id"="0" in the header of the NAL unit is defined to indicate coded image data in the basic format. Furthermore, "layerID"="2" indicates frame rate extension, and therefore "nuh_layer_id"="2" in the header of the NAL unit is defined to indicate coded image data in the frame rate extension. Furthermore, "layerID"="4" indicates dynamic range extension, and therefore "nuh_layer_id"="4" in the header of the NAL unit is defined to indicate coded image data in the dynamic range extension. Furthermore, "layerID"="6" indicates frame rate extension and dynamic range extension, and therefore "nuh_layer_id"="6" in the header of the NAL unit is defined to indicate coded image data in the frame rate extension and dynamic range extension.

[0081] Figure 9 shows the correspondence between the value of "nuh_layer_id" in the NAL unit header and the description in the scalable extension descriptor. That is, if a stream contains coded image data (basic component) in the basic format with "nuh_layer_id" = "0", "0" is set as "layerID".

[0082] Furthermore, if the stream contains spatially extended coded image data (spatial extension component) with "nuh_layer_id"="1", "Extended_spatial_resolution_flag" is set to "1", and "layerID" is set to "1". If the stream contains frame rate extended coded image data (frame rate extension component) with "nuh_layer_id"="2", "Extended_frame_rate_flag" is set to "1", and "layerID" is set to "2". If the stream contains frame rate extended coded image data (bit length extension component) with "nuh_layer_id"="3", "Extended_bit_depth_flag" is set to "1", and "layerID" is set to "3".

[0083] Furthermore, if the stream contains coded image data with extended dynamic range (dynamic range extended component) with "nuh_layer_id"="4", "Extended_dynamic_range_flag" is set to "1", and "layerID" is set to "4". Furthermore, if the stream contains coded image data with extended color gamut (color gamut extended component) with "nuh_layer_id"="5", "Extended_color_gamut_flag" is set to "1", and "layerID" is set to "5".

[0084] Furthermore, if the stream contains frame rate extended and dynamic range extended encoded image data (frame rate extended component and dynamic range extended component) with "nuh_layer_id"="6", "Extended_frame_rate_flag" and "Extended_dynamic_range_flag" are set to "1", and "layerID" is set to "6". Furthermore, if the stream contains spatial extended and frame rate extended encoded image data (spatial extended component and frame rate extended component) with "nuh_layer_id"="7", "Extended_spatial_resolution_flag" and "Extended_frame_rate_flag" are set to "1", and "layerID" is set to "7".

[0085] [Transport Stream TS Structure] 10 shows an example of the configuration of a transport stream TS in the case of a two-stream configuration. This transport stream TS includes two video streams: a basic video stream STb and an extended video stream STe. In this configuration example, a PES packet "video PES" exists for each video stream.

[0086] The packet identifier (PID) of the basic video stream STb is, for example, PID1. This basic video stream STb includes coded image data of each picture in the basic format. The coded image data of each picture in the basic format includes NAL units such as AUD, VPS, SPS, PPS, PSEI, SLICE, SSEI, and EOS. The "nuh_layer_id" in the headers of these NAL units is, for example, "0", indicating that the coded image data is related to the basic format.

[0087] The packet identifier (PID) of the extended video stream STe is, for example, PID2. The extended video stream STe includes coded image data of each picture in three high-quality formats: frame rate extension, dynamic range extension, and frame rate extension and dynamic range extension. The coded image data of each picture in the high-quality format includes NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS.

[0088] Note that the SPS in the coded image data of each picture in the basic format and the SPS in the coded image data of each picture in the high-quality format have the same value of "nal_unit_type", but differ in whether or not an extension is included. In other words, the SPS in the coded image data of each picture in the high-quality format includes an SPS extension. Note that it is also possible for the SPS in the basic format and the SPS in the high-quality format to have different "nal_unit_type" values.

[0089] "nuh_layer_id" in the header of the NAL unit constituting the coded image data of each picture with frame rate extension is set to "2", indicating that the coded image data relates to frame rate extension. Also, "nuh_layer_id" in the header of the NAL unit constituting the coded image data of each picture with dynamic range extension is set to "4", indicating that the coded image data relates to dynamic range extension. Also, "nuh_layer_id" in the header of the NAL unit constituting the coded image data of each picture with frame rate extension and dynamic range extension is set to "6", indicating that the coded image data relates to frame rate extension and dynamic range extension.

[0090] The transport stream TS also includes a program map table (PMT) as program specific information (PSI). This PSI is information that indicates which program each elementary stream included in the transport stream belongs to.

[0091] The PMT contains a program loop that describes information related to the entire program. The PMT also contains elementary stream loops that contain information related to each elementary stream. In this configuration example, there are two video elementary stream loops (video ES loops) corresponding to two video streams, the basic video stream STb and the extended video stream STe. The video elementary stream loop corresponding to the basic video stream STb contains information such as the stream type (ST0) and packet identifier (PID1).

[0092] In addition, in the video elementary stream loop corresponding to the extended video stream STe, information such as a stream type (ST1) and a packet identifier (PID2) is placed, and a descriptor describing information related to the extended video stream STe is also placed. As one of these descriptors, the above-mentioned scalable extension descriptor (see FIG. 7) is inserted.

[0093] In this descriptor, "Extended_frame_rate_flag" and "Extended_dynamic_range_flag" are set to "1", "number_of_layerIDs" is set to "3", and "layerID" is set to "2", "4", and "6", respectively. As a result, "nuh_layer_id"="2" in the NAL unit header is defined as indicating coded image data with frame rate extension. Also, "nuh_layer_id"="4" in the NAL unit header is defined as indicating coded image data with dynamic range extension. Furthermore, "nuh_layer_id"="6" in the NAL unit header is defined as indicating coded image data with frame rate extension and dynamic range extension.

[0094] 11 shows an example of the configuration of a transport stream TS in the case of a single stream configuration. This transport stream TS includes one video stream ST. In this example, a PES packet "video PES" for this video stream ST exists.

[0095] The packet identifier (PID) of this video stream ST is, for example, PID 1. This video stream ST includes coded image data of each picture in a basic format, as well as coded image data of each picture in three high-quality formats: frame rate extension, dynamic range extension, and frame rate and dynamic range extension.

[0096] In the coded image data of each picture in the basic format, there are NAL units such as AUD, VPS, SPS, PPS, PSEI, SLICE, SSEI, EOS, etc. "nuh_layer_id" in the header of these NAL units is, for example, "0", which indicates that the coded image data is related to the basic format.

[0097] Also, the coded image data of each picture in the high-quality format includes NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS. Note that the SPS in the coded image data of each picture in the basic format and the SPS in the coded image data of each picture in the high-quality format have the same value of "nal_unit_type", but differ in whether or not an extension is included. In other words, the SPS in the coded image data of each picture in the high-quality format includes an SPS extension.

[0098] "nuh_layer_id" in the header of the NAL unit constituting the coded image data of each picture with frame rate extension is set to "2", indicating that the coded image data relates to frame rate extension. Also, "nuh_layer_id" in the header of the NAL unit constituting the coded image data of each picture with dynamic range extension is set to "4", indicating that the coded image data relates to dynamic range extension. Also, "nuh_layer_id" in the header of the NAL unit constituting the coded image data of each picture with frame rate extension and dynamic range extension is set to "6", indicating that the coded image data relates to frame rate extension and dynamic range extension.

[0099] The transport stream TS also includes a program map table (PMT) as program specific information (PSI). This PSI is information that indicates which program each elementary stream included in the transport stream belongs to.

[0100] The PMT contains a program loop that describes information related to the entire program. The PMT also contains elementary stream loops that contain information related to each elementary stream. In this configuration example, there is one video elementary stream loop (video ES loop) corresponding to one video stream ST.

[0101] In this video elementary stream loop, information such as the stream type (ST0) and packet identifier (PID1) is placed, and also a descriptor describing information related to this video stream ST is placed. As one of these descriptors, the above-mentioned scalable extension descriptor (see Figure 7) is inserted.

[0102] In this descriptor, "Extended_frame_rate_flag" and "Extended_dynamic_range_flag" are set to "1", "number_of_layerIDs" is set to "4", and "layerID" is set to "0", "2", "4", and "6", respectively. As a result, "nuh_layer_id"="0" in the header of the NAL unit is defined to indicate coded image data in the basic format. "nuh_layer_id"="2" in the header of the NAL unit is defined to indicate coded image data with extended frame rate. "nuh_layer_id"="4" in the header of the NAL unit is defined to indicate coded image data with extended dynamic range. "nuh_layer_id"="6" in the header of the NAL unit is defined to indicate coded image data with extended frame rate and extended dynamic range.

[0103] The operation of the transmitting device 100 shown in Fig. 2 will be briefly described. Basic format image data Vb, which is LDR image data with a frame frequency of 50 Hz, is supplied to an LDR photoelectric conversion unit 102. In this LDR photoelectric conversion unit 102, photoelectric conversion characteristics for LDR images (LDR OETF curve) are applied to the basic format image data Vb to obtain basic format image data Vb' for transmission. This basic format image data Vb' is supplied to encoding units 106-0, 106-1, and 106-2 of a video encoder 106.

[0104] Furthermore, the high-quality format image data Vh1, which is LDR image data with a frame frequency of 100 Hz, is supplied to the LDR photoelectric conversion unit 103. In this LDR photoelectric conversion unit 103, photoelectric conversion characteristics for LDR images (LDR OETF curve) are applied to the high-quality format image data Vh1 to obtain high-quality format image data Vh1' for transmission. This high-quality format image data Vh1' is supplied to the encoding unit 106-1 of the video encoder 106.

[0105] Moreover, high-quality format image data Vh2, which is HDR image data with a frame frequency of 50 Hz, is supplied to the HDR photoelectric conversion unit 104. In this HDR photoelectric conversion unit 104, photoelectric conversion characteristics for HDR images (HDR OETF curve) are applied to the high-quality format image data Vh2 to obtain high-quality format image data Vh2' for transmission. This high-quality format image data Vh2' is supplied to encoding units 106-2 and 106-3 of the video encoder 106.

[0106] Moreover, high-quality format image data Vh3, which is HDR image data with a frame frequency of 100 Hz, is supplied to the HDR photoelectric conversion unit 105. In this HDR photoelectric conversion unit 105, photoelectric conversion characteristics for HDR images (HDR OETF curve) are applied to the high-quality format image data Vh3 to obtain high-quality format image data Vh3' for transmission. This high-quality format image data Vh3' is supplied to the encoding unit 106-3 of the video encoder 106.

[0107] In the video encoder 106, the basic format image data Vb' and the high quality format image data Vh1', Vh2', and Vh3' are each subjected to an encoding process to generate encoded image data Cb, Ch1, Ch2, and Ch3. That is, in the encoding unit 106-0, a predictive encoding process such as H.264 / AVC or H.265 / HEVC is performed on the basic format image data Vb' for transmission to obtain encoded image data Cb.

[0108] Moreover, in the encoding unit 106-1, predictive encoding processing such as H.264 / AVC, H.265 / HEVC, etc. is performed on the high-quality format image data Vh1' for transmission to obtain encoded image data Ch1. Moreover, in the encoding unit 106-2, predictive encoding processing such as H.264 / AVC, H.265 / HEVC, etc. is performed on the high-quality format image data Vh2' for transmission to obtain encoded image data Ch2. Furthermore, in the encoding unit 106-3, predictive encoding processing such as H.264 / AVC, H.265 / HEVC, etc. is performed on the high-quality format image data Vh3' for transmission to obtain encoded image data Ch3.

[0109] In the video encoder 106, identification information for identifying the corresponding format is inserted into each of the encoded image data Cb, Ch1, Ch2, and Ch3. That is, in the video encoder 106, identification information for identifying the format corresponding to the NAL unit (encoded image data) is inserted into the “nuh_layer_id” field of the header of the NAL unit.

[0110] The encoded image data Cb, Ch1, Ch2, and Ch3 obtained by the video encoder 106 are supplied to a system encoder 107. In the system encoder 107, a video stream is generated using the encoded image data Cb, Ch1, Ch2, and Ch3, and PES packetization and TS packetization are performed to generate a transport stream TS.

[0111] In the case of a two-stream configuration, two video streams are generated: a basic video stream including the coded image data Cb, and an extended video stream including the coded image data Ch1, Ch2, and Ch3. In the case of a one-stream configuration, one video stream including the coded image data Cb, Ch1, Ch2, and Ch3 is generated.

[0112] In the system encoder 107, information is inserted into the layer of the container (transport stream) that defines what format the identification information inserted into the encoded image data indicates.

[0113] In the case of a two-stream configuration, a scalable extension descriptor is inserted into the video elementary stream loop corresponding to the extended video stream (including coded image data Ch1, Ch2, Ch3) under the PMT. In the case of a one-stream configuration, a scalable extension descriptor is inserted into the video elementary stream loop corresponding to the video stream (including coded image data Cb, Ch1, Ch2, Ch3) under the PMT.

[0114] The transport stream TS generated by the system encoder 107 is sent to the transmitting unit 108. In the transmitting unit 108, this transport stream TS is transmitted to the receiving device 200 via broadcast waves or net packets.

[0115] "Configuration of receiving device" Fig. 12 shows a configuration example of a receiving device 200. This receiving device 200 corresponds to the configuration example of the transmitting device 100 of Fig. 2. This receiving device 200 has a control unit 201, a receiving unit 202, a system decoder 203, a compressed data buffer (cpb) 204, a video decoder 205, LDR electric-optical conversion units 206 and 207, HDR electric-optical conversion units 208 and 209, and a display unit (display device) 210.

[0116] The control unit 201 includes a CPU (Central Processing Unit) and controls the operation of each unit of the receiving device 200 based on a control program. The receiving unit 202 receives a transport stream TS transmitted by broadcast waves or network packets from the transmitting device 100. The system decoder 203 extracts a video stream from this transport stream TS.

[0117] In the case of a two-stream configuration (see FIG. 10), two video streams are extracted: a basic video stream including coded image data Cb of basic format image data, and an extended video stream including coded image data Ch1, Ch2, and Ch3 of high-quality format image data. In the case of a one-stream configuration (see FIG. 11), one video stream is extracted including coded image data Cb of basic format image data and coded image data Ch1, Ch2, and Ch3 of high-quality format image data.

[0118] Furthermore, the system decoder 203 extracts various pieces of information inserted in the layer of the container (transport stream) and sends them to the control unit 201. This information includes the above-mentioned scalable extension descriptor. Based on this descriptor, the control unit 201 can determine in advance, at the layer of the container, what format the encoded image data is indicated by the identification information inserted in the encoded image data (in this embodiment, "nuh_layer_id" in the header of the NAL unit).

[0119] The compressed data buffer 204 temporarily stores the video stream extracted by the system decoder 203. The video decoder 205 has four decoding units 205-0, 205-1, 205-2, and 205-3. The decoding unit 205-0 performs a decoding process on the basic format coded image data (basic components) Cb selectively read from the compressed data buffer 204, and generates basic format image data Vb'. In this case, the decoding unit 205-0 performs prediction compensation within the image data Vb'.

[0120] The decoding unit 205-1 generates high-quality format image data Vh1' by performing a decoding process on the frame-rate extended encoded image data (frame-rate extended component) Ch1 selectively read out from the compressed data buffer 204. In this case, the decoding unit 205-1 performs prediction compensation within the image data Vh1' or prediction compensation between the image data Vb' for each coding block in correspondence with the prediction at the time of encoding.

[0121] The decoding unit 205-2 generates high-quality format image data Vh2' by performing a decoding process on the dynamic range extended encoded image data (dynamic range extended component) Ch2 selectively read out from the compressed data buffer 204. In this case, the decoding unit 205-2 performs prediction compensation within the image data Vh2' or prediction compensation between the image data Vb' for each coding block in correspondence with the prediction at the time of encoding.

[0122] The decoding unit 205-3 generates high-quality format image data Vh3' by performing a decoding process on the frame rate extended and dynamic range extended encoded image data (frame rate extended component and dynamic range extended component) Ch3 selectively read out from the compressed data buffer 204. In this case, the decoding unit 205-3 performs prediction compensation within the image data Vh3' or prediction compensation between the image data Vh2' for each coding block in correspondence with the prediction at the time of encoding.

[0123] 13 shows a configuration example of the main part of the decoding unit 250. This decoding unit 250 can be applied to the decoding units 205-1, 205-2, and 205-3. This decoding unit 250 performs processing that is the opposite of the processing of the encoding unit 165 in FIG. 4. This decoding unit 250 has a decoding function unit 251, an intra-layer prediction compensation unit 252, an inter-layer prediction compensation unit 253, a prediction adjustment unit 254, and a selection unit 255.

[0124] The decode function unit 251 performs a decoding process other than prediction compensation on the encoded image data CV to obtain prediction residual data. The intra-layer prediction compensation unit 252 performs prediction compensation (intra-layer prediction compensation) on the prediction residual data within the image data V1 to obtain image data V1. The inter-layer prediction compensation unit 253 performs prediction compensation (inter-layer prediction compensation) on the prediction residual data between the image data V2 to be referenced and obtains image data V1.

[0125] Although detailed description is omitted, the prediction adjustment unit 254 performs processing according to the type of scalable extension of the image data V1 to the image data V2, similar to the prediction adjustment unit 163 of the encoding unit 160 in Fig. 4. The selection unit 255 selectively extracts the image data V1 obtained by the intra-layer prediction compensation unit 252 or the image data V1 obtained by the inter-layer prediction compensation unit 253 for each coding block in correspondence with the prediction at the time of coding, and outputs it.

[0126] 12, the LDR electric-to-optical converter 206 performs electric-to-optical conversion with the reverse characteristics to those of the LDR electric-to-electrical converter 102 in the above-described transmitting device 100 on the basic format image data Vb' obtained by the decoding unit 205-0, and obtains basic format image data Vb. This basic format image data is LDR image data with a frame frequency of 50 Hz.

[0127] Moreover, the LDR electric-to-optical converter 207 performs electric-to-optical conversion with the reverse characteristics to those of the LDR optical-to-electrical converter 103 in the above-described transmitting device 100 on the high-quality format image data Vh1' obtained by the decoding unit 205-1, and obtains high-quality format image data Vh1. This high-quality format image data Vh1 is LDR image data with a frame frequency of 100 Hz.

[0128] Moreover, the HDR electric-to-optical converter 208 performs electric-to-optical conversion with the reverse characteristics to those of the HDR optical-to-electrical converter 104 in the above-described transmission device 100 on the high-quality format image data Vh2′ obtained by the decoder 205-2, and obtains high-quality format image data Vh2. This high-quality format image data Vh2 is HDR image data with a frame frequency of 50 Hz.

[0129] Moreover, the HDR electric-to-optical converter 209 performs electric-to-optical conversion with the reverse characteristics to those of the HDR optical-to-electrical converter 105 in the transmission device 100 described above on the high-quality format image data Vh3′ obtained by the decoder 205-3, to obtain high-quality format image data Vh3. This high-quality format image data Vh3 is HDR image data with a frame frequency of 100 Hz.

[0130] The display unit 210 is configured with, for example, an LCD (Liquid Crystal Display), an organic EL (Organic Electro-Luminescence) panel, etc. The display unit 210 displays an image based on either the basic format image data Vb or the high-quality format image data Vh1, Vh2, or Vh3 according to its display capability.

[0131] In this case, the control unit 201 controls the image data to be supplied to the display unit 210. This control is performed based on identification information of the basic format and high quality format inserted in each encoded image data, and on display capability information of the display unit 209.

[0132] That is, when the display unit 210 is not capable of displaying a high frame frequency or a high dynamic range, the control unit 201 controls the display unit 210 to be supplied with basic format image data Vb related to the decoding of basic format coded image data (basic components) Cb. In this case, the control unit 201 selectively extracts basic format coded image data Cb from the compressed data buffer 204 and sends it to the decoding unit 205-0. Then, the control unit 201 controls the decoding unit 205-0 to decode the coded image data Cb and the LDR electric-optical conversion unit 206 to output the basic format image data Vb.

[0133] In addition, when the display unit 210 is capable of displaying a high frame frequency but is not capable of displaying a high dynamic range, the display unit 210 is controlled so that high-quality format image data Vh1 related to the decoding of the frame rate extended encoded image data (frame rate extended component) Ch1 is supplied to the display unit 210.

[0134] In this case, the control unit 201 selectively retrieves the basic format coded image data Cb from the compressed data buffer 204 and sends it to the decoding unit 205-0, and also selectively retrieves the frame rate extended coded image data Ch1 from the compressed data buffer 204 and sends it to the decoding unit 205-1. Then, the control unit 201 controls the decoding unit 205-0 to decode the coded image data Cb, the decoding unit 205-1 to decode the coded image data Ch1, and the LDR electric-optical conversion unit 207 to output the high quality format image data Vh1.

[0135] In addition, when the display unit 210 is not capable of displaying a high frame frequency but is capable of displaying a high dynamic range, the display unit 210 is controlled so that high-quality format image data Vh2 related to the decoding of the dynamic range extension encoded image data (dynamic range extension component) Ch2 is supplied to the display unit 210.

[0136] In this case, the control unit 201 selectively retrieves the basic format coded image data Cb from the compressed data buffer 204 and sends it to the decoding unit 205-0, and also selectively retrieves the dynamic range extended coded image data Ch2 from the compressed data buffer 204 and sends it to the decoding unit 205-2. Then, the control unit 201 controls the decoding unit 205-0 to decode the coded image data Cb, the decoding unit 205-2 to decode the coded image data Ch2, and the LDR electric-optical conversion unit 208 to output the high quality format image data Vh2.

[0137] In addition, when the display unit 210 is capable of displaying both high frame frequencies and high dynamic ranges, the display unit 210 is controlled so that high-quality format image data Vh3 related to the decoding of frame rate extended and dynamic range extended encoded image data (frame rate extended components and dynamic range extended components) Ch3 is supplied to the display unit 210.

[0138] In this case, the control unit 201 selectively extracts basic format coded image data Cb from the compressed data buffer 204 and sends it to the decoding unit 205-0, selectively extracts dynamic range extended coded image data Ch2 from the compressed data buffer 204 and sends it to the decoding unit 205-1, and further selectively extracts frame rate extended and dynamic range extended coded image data Ch3 from the compressed data buffer 204 and sends it to the decoding unit 205-3.

[0139] Then, the control unit 201 controls the decoding unit 205-0 to decode the encoded image data Cb, the decoding unit 205-2 to decode the encoded image data Ch2, the decoding unit 205-3 to decode the encoded image data Ch3, and the HDR electrical-to-optical conversion unit 209 to output high-quality format image data Vh3.

[0140] FIG. 14 shows an outline of the output of the compressed data buffer (cpb) 204 in the case of a two-stream configuration, and the allocation of the coded image data Cb, Ch1, Ch2, and Ch3 to the corresponding decoding units according to "nuh_layer_id."

[0141] In the case of a two-stream configuration, as shown in FIG. 14(a), the coded image data Cb in the basic format contained in the basic video stream (PID1) and the coded image data of each picture Ch1, Ch2, and Ch3 in the high-quality format contained in the extended video stream (PID2) are sequentially read out from the compressed data buffer (cpb) 204.

[0142] Here, "00", "01", ... indicate the coded image data of each picture constituting the coded image data Cb, and "nuh_layer_id" in the header of the NAL unit is set to "0." Since this coded image data Cb was included in the basic video stream, the control unit 201 understands that "nuh_layer_id"="0" indicates coded image data in the basic format.

[0143] Furthermore, "10", "11", ... indicate the coded image data of each picture constituting the coded image data Ch1, and "nuh_layer_id" in the header of the NAL unit is set to "2." The control unit 201 understands from the definition of the scalable extension descriptor that "nuh_layer_id" = "2" indicates coded image data of frame rate extension.

[0144] Furthermore, "20", "21", ... indicate the coded image data of each picture constituting the coded image data Ch2, and "nuh_layer_id" in the header of the NAL unit is set to "4." The control unit 201 understands that "nuh_layer_id"="4" indicates coded image data with dynamic range extension from the definition in the scalable extension descriptor.

[0145] Furthermore, "30", "31", ... indicate the coded image data of each picture constituting the coded image data Ch3, and "nuh_layer_id" in the header of the NAL unit is set to "6." From the definition in the scalable extension descriptor, the control unit 201 understands that "nuh_layer_id"="6" indicates coded image data with frame rate extension and dynamic range extension.

[0146] The coded image data of each picture read from the compressed data buffer 204 is sent to the corresponding decoding unit based on "nuh_layer_id" as shown in FIG. 14(b). In this case, the coded image data of the layer not related to the decoding is read and discarded. The illustrated example shows a case where all is decoded.

[0147] FIG. 15 shows an outline of the output of the compressed data buffer (cpb) 204 in the case of a one-stream configuration, and the allocation of the encoded image data Cb, Ch1, Ch2, and Ch3 to the corresponding decoding units according to "nuh_layer_id."

[0148] In the case of a single stream configuration, as shown in Figure 15(a), the coded image data Cb in basic format and the coded image data of each picture Ch1, Ch2, and Ch3 in high quality format contained in one video stream (PID1) are sequentially read out from the compressed data buffer (cpb) 204.

[0149] Here, "00", "01", ... indicate the coded image data of each picture constituting the coded image data Cb, and "nuh_layer_id" in the header of the NAL unit is set to "0." The control unit 201 understands from the definition of the scalable extension descriptor that "nuh_layer_id" = "0" indicates coded image data in the basic format.

[0150] Furthermore, "10", "11", ... indicate the coded image data of each picture constituting the coded image data Ch1, and "nuh_layer_id" in the header of the NAL unit is set to "2." The control unit 201 understands from the definition of the scalable extension descriptor that "nuh_layer_id" = "2" indicates coded image data of frame rate extension.

[0151] Furthermore, "20", "21", ... indicate the coded image data of each picture constituting the coded image data Ch2, and "nuh_layer_id" in the header of the NAL unit is set to "4." The control unit 201 understands that "nuh_layer_id"="4" indicates coded image data with dynamic range extension from the definition in the scalable extension descriptor.

[0152] Furthermore, "30", "31", ... indicate the coded image data of each picture constituting the coded image data Ch3, and "nuh_layer_id" in the header of the NAL unit is set to "6." From the definition in the scalable extension descriptor, the control unit 201 understands that "nuh_layer_id"="6" indicates coded image data with frame rate extension and dynamic range extension.

[0153] The coded image data of each picture read from the compressed data buffer 204 is sent to the corresponding decoding unit based on "nuh_layer_id" as shown in FIG. 15(b). In this case, the coded image data of the layer not related to the decoding is read and discarded. The illustrated example shows a case where all is decoded.

[0154] 16 shows an example of a process in which the control unit 201 determines the decoding range from the display capability information (display performance information). The control unit 201 starts the process in step ST1.

[0155] Next, in step ST2, the control unit 201 refers to the scalable extension descriptor to grasp the "nuh_layer_id" of each format. In this embodiment, it is grasped that "nuh_layer_id"="0" in the basic format, "nuh_layer_id"="2" in the frame rate extension, "nuh_layer_id"="4" in the dynamic range rate extension, and "nuh_layer_id"="6" in the frame rate extension and dynamic range rate extension.

[0156] Next, in step ST3, the control unit 201 determines whether or not 100pHDR display, that is, HDR display with a frame frequency of 100Hz, is possible. If possible, in step ST4, the control unit 201 sets the coded image data in which "nuh_layer_id" is "0", "4", or "6", that is, the coded image data Cb, Ch2, and Ch3, as the decoding range, and then ends the process in step ST11.

[0157] If it is not possible in step ST3, the control unit 201 determines in step ST5 whether or not it is possible to display 50pHDR, that is, whether or not it is possible to display HDR with a frame frequency of 50 Hz. If it is possible, the control unit 201 determines in step ST6 that the coded image data in which "nuh_layer_id" is "0" or "4", that is, the coded image data Cb, Ch2, is the decoding range, and then ends the process in step ST11.

[0158] If it is not possible in step ST5, the control unit 201 determines in step ST7 whether or not it is possible to display 100pLDR, that is, whether or not it is possible to display LDR with a frame frequency of 100 Hz. If it is possible, the control unit 201 determines in step ST8 that the coded image data in which "nuh_layer_id" is "0" or "2", that is, the coded image data Cb, Ch1, is the decoding range, and then ends the process in step ST11.

[0159] If it is not possible in step ST7, the control unit 201 determines in step ST9 whether or not it is possible to display 50pLDR, that is, whether or not it is possible to display LDR with a frame frequency of 50 Hz. If it is possible, the control unit 201 determines in step ST10 that the coded image data in which "nuh_layer_id" is "0", that is, the coded image data Cb, is the decoding range, and then ends the process in step ST11. Note that if it is not possible in step ST9, the control unit 201 ends the process in step ST11.

[0160] The operation of the receiving device 200 shown in Fig. 12 will be briefly described. The receiving unit 202 receives the transport stream TS transmitted from the transmitting device 100 via broadcast waves or network packets. This transport stream TS is supplied to a system decoder 203. The system decoder 203 extracts a video stream from this transport stream TS. This video stream is temporarily stored in a compressed data buffer 204.

[0161] In the case of a two-stream configuration (see FIG. 10), two video streams are extracted: a basic video stream including coded image data Cb of basic format image data, and an extended video stream including coded image data Ch1, Ch2, and Ch3 of high-quality format image data. In the case of a one-stream configuration (see FIG. 11), one video stream is extracted including coded image data Cb of basic format image data and coded image data Ch1, Ch2, and Ch3 of high-quality format image data.

[0162] Furthermore, the system decoder 203 extracts various pieces of information inserted into layers of the container (transport stream) and sends them to the control unit 201. This information includes a scalable extension descriptor. Based on this descriptor, the control unit 201 determines what format the coded image data is indicated by the identification information inserted into the coded image data (in this embodiment, "nuh_layer_id" in the header of the NAL unit).

[0163] When the display unit 210 is not capable of displaying a high frame frequency or a high dynamic range, the basic format image data Vb is supplied from the LDR electro-optical converter 206 to the display unit 210. The display unit 210 displays this basic format image data Vb, that is, a 50pLDR image based on LDR image data with a frame frequency of 50 Hz.

[0164] In this case, basic format coded image data Cb in which "nuh_layer_id" in the header of the NAL unit is "0" is selectively extracted from the compressed data buffer 204 and supplied to the decoding unit 205-0. The decoding unit 205-0 performs a decoding process on the coded image data Cb to generate basic format image data Vb'. This basic format image data Vb' is supplied to the LDR electric-optical converter 206. The LDR electric-optical converter 206 performs electric-optical conversion on the basic format image data Vb' to obtain basic format image data Vb, which is supplied to the display unit 210.

[0165] In addition, when the display unit 210 is capable of displaying at a high frame frequency but is not capable of displaying at a high dynamic range, high-quality format image data Vh1 is supplied from the LDR electro-optical conversion unit 207 to the display unit 210. The display unit 210 displays an image based on this high-quality format image data Vh1, that is, LDR image data with a frame frequency of 100 Hz.

[0166] In this case, basic format coded image data Cb in which "nuh_layer_id" in the header of the NAL unit is "0" is selectively extracted from the compressed data buffer 204 and supplied to the decoding unit 205-0. The decoding unit 205-0 performs a decoding process on the coded image data Cb to generate basic format image data Vb'.

[0167] Also, frame rate extended coded image data Ch1, in which the "nuh_layer_id" in the header of the NAL unit is "2", is selectively extracted from the compressed data buffer 204 and supplied to the decoding unit 205-1. In the decoding unit 205-1, the coded image data Ch1 is decoded with reference to the basic format image data Vb', and high quality format image data Vh1' is generated.

[0168] The high-quality format image data Vh1′ generated by the decoding unit 205-1 is supplied to the LDR electric-optical conversion unit 207. In the LDR electric-optical conversion unit 207, the high-quality format image data Vh1′ is subjected to electric-optical conversion to obtain high-quality format image data Vh1, which is supplied to the display unit 210.

[0169] Furthermore, when the display unit 210 is not capable of displaying at a high frame frequency but is capable of displaying at a high dynamic range, high-quality format image data Vh2 is supplied from the HDR electro-optical conversion unit 208 to the display unit 210. The display unit 210 displays an image based on this high-quality format image data Vh2, that is, HDR image data with a frame frequency of 50 Hz.

[0170] In this case, basic format coded image data Cb in which "nuh_layer_id" in the header of the NAL unit is "0" is selectively extracted from the compressed data buffer 204 and supplied to the decoding unit 205-0. The decoding unit 205-0 performs a decoding process on the coded image data Cb to generate basic format image data Vb'.

[0171] Also, dynamic range extended coded image data Ch2, in which the "nuh_layer_id" in the header of the NAL unit is "4", is selectively extracted from the compressed data buffer 204 and supplied to the decoding unit 205-2. In the decoding unit 205-2, the coded image data Ch2 is decoded with reference to the basic format image data Vb', and high quality format image data Vh2' is generated.

[0172] The high-quality format image data Vh2′ generated by the decoding unit 205-2 is supplied to the HDR electric-to-optical converter 208. In the HDR electric-to-optical converter 208, the high-quality format image data Vh2′ is subjected to electric-to-optical conversion to obtain high-quality format image data Vh2, which is supplied to the display unit 210.

[0173] Furthermore, when the display unit 210 is capable of both high frame frequency display and high dynamic range display, high quality format image data Vh3 is supplied from the HDR electro-optical conversion unit 209 to the display unit 210. The display unit 210 displays an image based on this high quality format image data Vh3, that is, HDR image data with a frame frequency of 100 Hz.

[0174] In this case, basic format coded image data Cb in which "nuh_layer_id" in the header of the NAL unit is "0" is selectively extracted from the compressed data buffer 204 and supplied to the decoding unit 205-0. The decoding unit 205-0 performs a decoding process on the coded image data Cb to generate basic format image data Vb'.

[0175] Also, dynamic range extended coded image data Ch2, in which the "nuh_layer_id" in the header of the NAL unit is "4", is selectively extracted from the compressed data buffer 204 and supplied to the decoding unit 205-2. In the decoding unit 205-2, the coded image data Ch2 is decoded with reference to the basic format image data Vb', and high quality format image data Vh2' is generated.

[0176] Also, the frame rate extended and dynamic range extended encoded image data Ch3, whose NAL unit header "nuh_layer_id" is "6", is selectively extracted from the compressed data buffer 204 and supplied to the decoding unit 205-3. In the decoding unit 205-3, the encoded image data Ch2 is decoded with reference to the high quality format image data Vh2', and high quality format image data Vh3' is generated.

[0177] The high-quality format image data Vh3′ generated by the decoding unit 205-3 is supplied to the HDR electric-to-optical converter 209. In the HDR electric-to-optical converter 209, the high-quality format image data Vh3′ is subjected to electric-to-optical conversion to obtain high-quality format image data Vh3, which is supplied to the display unit 210.

[0178] As described above, in the transmitting / receiving system 10 shown in Fig. 1, the transmitting device 100 inserts identification information for identifying the corresponding format into each of the coded image data of the basic format image data and the predetermined number of high quality format image data. Therefore, the receiving side can easily obtain image data according to the display capability by selectively performing a decoding process on the predetermined coded image data based on the identification information.

[0179] 1, information defining what format of encoded image data is indicated by the identification information to be inserted into the encoded image data is inserted into the container layer in the transmitting device 100. Therefore, the receiving side can know in advance from the container layer what format of encoded image data is indicated by the identification information to be inserted into the encoded image data.

[0180] <2. Modifications> In the above embodiment, an example is shown in which the “nuh_layer_id” field in the header of the NAL unit is used to insert identification information into encoded image data, but it is also possible to use two fields, “nuh_layer_id” and “nuh_temporal_id_plus1”.

[0181] For example, "nuh_layer_id" and "nuh_temporal_id_plus1" of the coded image data Cb, Ch1, Ch2, and Ch3 are set as shown in Fig. 17. That is, for the coded image data Cb of the basic format, "nuh_layer_id" is set to "0", and "nuh_temporal_id_plus1" is set to "1 to 6". Also, for the coded image data Ch1 of the frame rate extension, "nuh_layer_id" is set to "0", and "nuh_temporal_id_plus1" is set to "7".

[0182] For the dynamic range extended coded image data Ch2, "nuh_layer_id" is set to "4", and "nuh_temporal_id_plus1" is set to "1 to 6". For the frame rate extended and dynamic range extended coded image data Ch3, "nuh_layer_id" is set to "4", and "nuh_temporal_id_plus1" is set to "7".

[0183] In this case, the scalable extension descriptor (see FIG. 7) is set as follows: In other words, in the case of a two-stream configuration, when the extended video stream includes encoded data Ch2 and Ch3, "Extended_spatial_resolution_flag", "Extended_bit_depth_flag", and "Extended_color_gamut_flag" are set to "0", and "Extended_frame_rate_flag" and "Extended_dynamic_range_flag" are set to "1". In addition, "number_of_layer IDs" is set to "3", and "layerID" is set to "4", "4", in that order.

[0184] With these settings, "nuh_layer_id"="0" and "nuh_temporal_id_plus1"="7" are defined to indicate coded image data with frame rate extension. Also, "nuh_layer_id"="4" and "nuh_temporal_id_plus1"="1-6" are defined to indicate coded image data with dynamic range extension. Furthermore, "nuh_layer_id"="4" and "nuh_temporal_id_plus1"="7" are defined to indicate coded image data with frame rate extension and dynamic range extension.

[0185] Furthermore, in the case of a one-stream configuration, when the extended video stream includes encoded data Cb, Ch1, Ch2, and Ch3, "Extended_spatial_resolution_flag", "Extended_bit_depth_flag", and "Extended_color_gamut_flag" are set to "0", and "Extended_frame_rate_flag" and "Extended_dynamic_range_flag" are set to "1". Furthermore, "number_of_layer IDs" is set to "4", and "layerID" is set to "0", "0", "4", and "4", in that order.

[0186] With these settings, "nuh_layer_id"="0" and "nuh_temporal_id_plus1"="1 to 6" are defined to indicate coded image data in the basic format. Furthermore, "nuh_layer_id"="0" and "nuh_temporal_id_plus1"="7" are defined to indicate coded image data with frame rate extension. Furthermore, "nuh_layer_id"="4" and "nuh_temporal_id_plus1"="1 to 6" are defined to indicate coded image data with dynamic range extension. Furthermore, "nuh_layer_id"="4" and "nuh_temporal_id_plus1"="7" are defined to indicate coded image data with frame rate extension and dynamic range extension.

[0187] FIG. 18 shows the correspondence between the values ​​of "nuh_layer_id" and "nuh_temporal_id_plus1" in the NAL unit header and the descriptions in the scalable extension descriptor. That is, if a stream contains coded image data (basic component) in a basic format with "nuh_layer_id"="0" and "nuh_temporal_id_plus1"="1-6", "layerID" is set to "0". Also, if a stream contains coded image data (frame rate extension component) in a frame rate extension with "nuh_layer_id"="0" and "nuh_temporal_id_plus1"="7", "Extended_spatial_resolution_flag" is set to "1" and "layerID" is set to "0".

[0188] Furthermore, if the stream contains coded image data (frame rate extension component) with dynamic range extension where "nuh_layer_id"="4" and "nuh_temporal_id_plus1"="1 to 6", "Extended_dynamic_range_flag" is set to "1", and "layerID" is set to "4". Furthermore, if the stream contains coded image data (frame rate extension component and dynamic range extension component) with frame rate extension and dynamic range extension where "nuh_layer_id"="4" and "nuh_temporal_id_plus1"="7", "Extended_frame_rate_flag" and "Extended_dynamic_range_flag" are set to "1", and "layerID" is set to "4".

[0189] The flowchart in Figure 19 shows an example of a process in which the control unit 201 determines the decoding range from display capability information (display performance information) when two fields, "nuh_layer_id" and "nuh_temporal_id_plus1", in the header of the NAL unit are used to insert identification information into the encoded image data as described above.

[0190] In step ST21, the control unit 201 starts the process. Next, in step ST22, the control unit 201 refers to the scalable extension descriptor and grasps "nuh_layer_id" and "nuh_temporal_id_plus1" of each format.

[0191] Here, it is understood that in the basic format, “nuh_layer_id” = “0”, “nuh_temporal_id_plus1” = “1 to 6”, in the frame rate extension, “nuh_layer_id” = “0”, “nuh_temporal_id_plus1” = “7”, in the dynamic range extension, “nuh_layer_id” = “4”, “nuh_temporal_id_plus1” = “1 to 6”, and in the frame rate extension and dynamic range extension, “nuh_layer_id” = “4”, “nuh_temporal_id_plus1” = “7”.

[0192] Next, in step ST23, the control unit 201 determines whether or not 100pHDR display, that is, HDR display with a frame frequency of 100Hz, is possible. If possible, in step ST24, the control unit 201 sets the coded image data Cb with "nuh_layer_id"="0" and "nuh_temporal_id_plus1"="1 to 6" and the coded image data Ch2 and Ch3 with "nuh_layer_id"="4" and "nuh_temporal_id_plus1"="1 to 7" as the decoding range, and then ends the process in step ST31.

[0193] If it is not possible in step ST23, the control unit 201 determines in step ST25 whether or not it is possible to display 50pHDR, that is, whether or not it is possible to display HDR with a frame frequency of 50 Hz. If it is possible, the control unit 201 determines in step ST26 that the coded image data Cb where "nuh_layer_id"="0" and "nuh_temporal_id_plus1"="1 to 6" and the coded image data Ch2 where "nuh_layer_id"="4" and "nuh_temporal_id_plus1"="1 to 6" are the decoding range, and then ends the process in step ST31.

[0194] If it is not possible in step ST25, the control unit 201 determines in step ST27 whether or not it is possible to display 100pLDR, that is, whether or not it is possible to display LDR with a frame frequency of 100 Hz. If it is possible, the control unit 201 determines in step ST28 that the coded image data Cb, Ch1 where "nuh_layer_id"="0" and "nuh_temporal_id_plus1"="1 to 7" are the decoding range, and then ends the process in step ST31.

[0195] If it is not possible in step ST27, the control unit 201 determines in step ST29 whether or not it is possible to display 50pLDR, that is, whether or not it is possible to display LDR with a frame frequency of 50 Hz. If it is possible, the control unit 201 determines in step ST30 that the coded image data Cb where "nuh_layer_id"="0" and "nuh_temporal_id_plus1"="1 to 6" is the decoding range, and then ends the process in step ST31. Note that if it is not possible in step ST29, the control unit 201 ends the process in step ST31.

[0196] In the above embodiment, the transmission / reception system 10 including the transmission device 100 and the reception device 200 is shown, but the configuration of the transmission / reception system to which the present technology can be applied is not limited to this. For example, the reception device 200 may be configured as a set-top box and a monitor connected by a digital interface such as HDMI (High-Definition Multimedia Interface). In this case, the set-top box can obtain display capability information by obtaining EDID (Extended display identification data) from the monitor. Note that "HDMI" is a registered trademark.

[0197] In the above embodiment, an example was given in which the container was a transport stream (MPEG-2 TS). However, the present technology can be similarly applied to a system configured to distribute to a receiving terminal using a network such as the Internet. In Internet distribution, distribution is often performed in a container of MP4 or other format. In other words, containers of various formats such as the transport stream (MPEG-2 TS) adopted in the digital broadcasting standard and MP4 used in Internet distribution are applicable.

[0198] The present technology can also be configured as follows. (1) an image encoding unit that generates a basic video stream including encoded image data of basic format image data and an extended video stream including encoded image data of each of a predetermined number of high quality format image data; a transmission unit that transmits a container in a predetermined format including the basic video stream and the extended video stream generated by the image encoding unit, The image encoding unit inserts identification information for identifying a corresponding format into each of the encoded image data of the basic format image data and the predetermined number of high quality format image data. Transmitting device. (2) The encoded image data has a NAL unit structure, The image encoding unit inserts the identification information into a header of the NAL unit. The transmitting device according to (1) above. (3) The image encoding unit inserts the identification information into the NAL unit header using the “nuh_layer_id” field. The transmitting device according to (2) above. (4) The image encoding unit inserts the identification information into the “nuh_layer_id” and “nuh_temporal_id_plus1” fields of the header of the NAL unit. The transmitting device according to (2) above. (5) The method further includes an information inserting unit that inserts, into a layer of the container, information that defines a format of the encoded image data that the identification information to be inserted into the encoded image data indicates. A transmitting device according to any one of (1) to (4). (6) The container is an MPEG2-TS. The information insertion unit is The information is inserted into a video elementary stream loop corresponding to the extended video stream existing under a program map table. The transmitting device according to (5) above. (7) an image encoding step of generating a basic video stream including encoded image data of the basic format image data and an enhanced video stream including encoded image data of each of a predetermined number of high quality format image data; a transmitting step of transmitting a container in a predetermined format including the basic video stream and the extended video stream generated in the image encoding step by a transmitting unit; In the image encoding step, identification information for identifying a corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of high quality format image data. Transmission method. (8) A receiving unit for receiving a container of a predetermined format having a basic video stream including encoded image data of basic format image data and an extended video stream including encoded image data of each of a predetermined number of high quality format image data, Identification information for identifying a corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of the high quality format image data, The video stream processing device further includes a processing unit for processing each video stream contained in the received container based on the identification information and the display capability information. Receiving device. (9) The encoded image data has a NAL unit structure, The identification information is inserted in the header of the NAL unit. The receiving device according to (8) above. (10) information defining what format of encoded image data the identification information inserted in the encoded image data indicates is inserted in a layer of the container; The processing unit determines, based on information inserted in the layer of the container, what format of the encoded image data the identification information inserted in the encoded image data indicates. A receiving device according to (8) or (9). (11) an image encoding unit that generates a video stream including encoded image data of basic format image data and encoded image data of each of a predetermined number of high quality format image data; a transmission unit that transmits a container in a predetermined format including the video stream generated by the image encoding unit, The image encoding unit inserts identification information for identifying a corresponding format into each of the encoded image data of the basic format image data and the predetermined number of high quality format image data. Transmitting device. (12) The encoded image data has a NAL unit structure, The image encoding unit inserts the identification information into a header of the NAL unit. The transmitting device according to (11) above. (13) The image encoding unit inserts the identification information using a “nuh_layer_id” field in a header of the NAL unit. The transmitting device according to (12) above. (14) The image encoding unit inserts the identification information using the “nuh_layer_id” and “nuh_temporal_id_plus1” fields of the header of the NAL unit. The transmitting device according to (12) above. (15) The method further includes an information inserting unit that inserts, into a layer of the container, information that defines a format of the encoded image data that the identification information to be inserted into the encoded image data indicates. A transmitting device according to any one of (11) to (14). (16) The container is an MPEG2-TS. The information insertion unit is The information is inserted into a video elementary stream loop corresponding to the video stream existing under a program map table. The transmitting device according to (15) above. (17) an image encoding step of generating a video stream including encoded image data of the basic format image data and encoded image data of each of the predetermined number of high quality format image data; a transmitting step of transmitting a container in a predetermined format including the video stream generated in the image encoding step by a transmitting unit; In the image encoding step, identification information for identifying a corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of high quality format image data. Transmission method. (18) A receiving unit for receiving a container of a predetermined format having a video stream including encoded image data of basic format image data and encoded image data of each of a predetermined number of high quality format image data, Identification information for identifying a corresponding format is inserted into each of the encoded image data of the basic format image data and the predetermined number of the high quality format image data, The video stream stored in the received container is processed based on the identification information and the display capability information. Receiving device. (19) The encoded image data has a NAL unit structure, The identification information is inserted in the header of the NAL unit. The receiving device according to (18) above. (20) Information is inserted into a layer of the container to define a format of the encoded image data indicated by the identification information inserted into the encoded image data; The processing unit determines, based on information inserted in the layer of the container, what format of the encoded image data the identification information inserted in the encoded image data indicates. The receiving device according to (18) or (19).

[0199] The main feature of this technology is that by inserting identification information for identifying the corresponding format into each of the encoded image data of basic format image data and a predetermined number of high-quality format image data and transmitting the encoded image data, it is made easy to selectively perform decoding processing on the predetermined encoded image data on the receiving side to obtain image data according to the display capabilities (see Figs. 10 and 11).

[0200] 10. Transmitting and receiving system 100 Transmitting device 101...Control section 102, 103: LDR photoelectric conversion section 104, 105...HDR photoelectric conversion section 106 Video Encoder 106-0, 106-1, 106-2, 106-3...Encoding section 107 System Encoder 108 Transmitter 150 Image data generating unit 151···HDR Camera 152,154...Frame rate conversion section 153 Dynamic range conversion section 160 Encoding section 161... Intra-layer prediction unit 162... Inter-layer prediction unit 163 Forecast Adjustment Section 164...Selection section 165 Encoding function section 200 Receiving device 201... Control unit 202 Receiving unit 203 System Decoder 204 Compressed data buffer 205...Video decoder 205-0, 205-1, 205-2, 205-3... Decoder 206, 207···LDR electro-optical conversion unit 208, 209...HDR electro-optical converter 210...Display section 250 Decoder 251 Decode function unit 252 Intra-layer prediction compensation unit 253 Inter-layer prediction compensation unit 254... Forecast Adjustment Section 255...Selection section

Claims

1. generating a base video stream including encoded image data of the base format image data and an enhanced video stream including encoded image data of a predetermined number of high quality format image data corresponding to a predetermined number of types of scalable enhancements; transmitting said base video stream and said enhanced video stream; the extended video stream includes encoded image data of high quality format image data corresponding to at least a frame rate extension; The coded image data included in the basic video stream and the extended video stream includes an NAL unit into which a value indicating a Temporal ID corresponding to high-quality format image data corresponding to the frame rate extension or a value indicating a Temporal ID not corresponding to high-quality format image data corresponding to the frame rate extension is inserted. Transmission method.

2. an image encoding unit for generating a basic video stream including encoded image data of basic format image data and an extended video stream including encoded image data of a predetermined number of high quality format image data corresponding to a predetermined number of types of scalable extensions; a transmitter for transmitting the basic video stream and the extended video stream, the extended video stream includes encoded image data of high quality format image data corresponding to at least a frame rate extension; The coded image data included in the basic video stream and the extended video stream includes an NAL unit into which a value indicating a Temporal ID corresponding to high-quality format image data corresponding to the frame rate extension or a value indicating a Temporal ID not corresponding to high-quality format image data corresponding to the frame rate extension is inserted. Transmitting device.

3. a receiving unit for receiving a basic video stream including encoded image data of basic format image data and an enhanced video stream including encoded image data of a predetermined number of high quality format image data corresponding to a predetermined number of types of scalable extensions; the extended video stream includes encoded image data of high quality format image data corresponding to at least a frame rate extension; The coded image data included in the basic video stream and the extended video stream includes an NAL unit into which a value indicating a Temporal ID corresponding to high quality format image data corresponding to the frame rate extension or a value indicating a Temporal ID not corresponding to high quality format image data corresponding to the frame rate extension is inserted, The video stream receiving device further includes a processor configured to process each of the received video streams based on a value indicating the Temporal ID. Receiving device.

4. The processing unit processes each of the received video streams based on the value indicating the Temporal ID and the display capability information.

4. The receiving device according to claim 3.

5. receiving a base video stream including encoded image data of a base format image data and an enhanced video stream including encoded image data of a predetermined number of high quality format image data corresponding to a predetermined number of types of scalable enhancements; the extended video stream includes encoded image data of high quality format image data corresponding to at least a frame rate extension; The coded image data included in the basic video stream and the extended video stream includes an NAL unit into which a value indicating a Temporal ID corresponding to high quality format image data corresponding to the frame rate extension or a value indicating a Temporal ID not corresponding to high quality format image data corresponding to the frame rate extension is inserted, The method further comprises the step of processing each of the received video streams based on a value indicative of the Temporal ID. Receiving method.

Images