Reproduction device and reproduction method

Abstract

The technology of the present invention relates to a reproduction device and a reproduction method. The reproduction device includes: a decoding unit that decodes encoded video data or encoded audio data; a zoom area selection unit that selects one or more zoom area information from multiple zoom area information that specify an area to be zoomed; and a data processing unit that performs a cropping process on the video data obtained by decoding or performs an audio conversion process on the audio data obtained by decoding based on the selected zoom area information, wherein the zoom area information includes zoom area auxiliary information, wherein the zoom area auxiliary information includes information about a reproduction device.

Description

[0001] This application is a divisional application of the application with national application number 201580053817.8, international application date September 28, 2015, national entry date April 1, 2017, and invention name “Encoding device and method, reproduction device and method, and program”. Technical Field

[0002] The present technology relates to an encoding device, an encoding method, a reproducing device, a reproducing method, and a program, and more specifically, to an encoding device, an encoding method, a reproducing device, a reproducing method, and a program that enable each reproducing device to reproduce appropriate content in a simplified manner. Background Art

[0003] In recent years, high-resolution video content called 4K or 8K has become known. Such 4K or 8K video content is often generated in consideration of a wide viewing angle, that is, reproduction on a large screen.

[0004] In addition, since 4K or 8K video content has a high resolution, the resolution is sufficient even when a portion of the screen of such video content is cropped, and therefore, such video content can be cropped and reproduced (for example, see non-patent document 1).

[0005] Reference List

[0006] Non-patent literature

[0007] Non-patent document 1: FDR-AX100, [Online], [Retrieved on September 24, 2014], Internet<URL:http: / / www.sony.net / Products / di / en-us / products / j4it / index.html> Summary of the invention

[0008] Problems to be solved by the present invention

[0009] Meanwhile, video reproduction devices are diversified, and reproduction with various screen sizes from large screens to smartphones (multi-function mobile phones) is considered. However, in the current situation, the same content is reproduced in enlargement or reduction to match each screen size.

[0010] Meanwhile, the above-mentioned 4K or 8K video content is often generated in consideration of reproduction on a large screen. Therefore, it is not appropriate to reproduce such video content using a reproduction device with a relatively small screen, such as a tablet personal computer (PC) or a smartphone.

[0011] Therefore, for example, for reproduction devices having screen sizes and the like different from each other, in order to provide content suitable for each screen size, screen shape, and the like, it is necessary to separately prepare content suitable for each screen size, screen shape, and the like.

[0012] The present technology takes these circumstances into consideration and enables each reproducing device to reproduce appropriate content in a simplified manner.

[0013] Solution to the problem

[0014] A reproduction device according to a first aspect of the present technology includes: a decoding unit that decodes encoded video data or encoded audio data; a zoom area selection unit that selects one or more zoom area information from multiple zoom area information that specify an area to be zoomed; and a data processing unit that performs a cropping process on the video data obtained by decoding or performs an audio conversion process on the audio data obtained by decoding based on the selected zoom area information.

[0015] Among the plurality of pieces of zoom area information, zoom area information specifying an area for each type of reproduction target device may be included.

[0016] Among the plurality of pieces of zoom area information, zoom area information specifying an area for each rotation direction of the reproduction target device may be included.

[0017] Among the plurality of pieces of zoom area information, zoom area information specifying an area for each specific video object may be included.

[0018] The zoom area selection unit may be caused to select the zoom area information according to an operation input by a user.

[0019] The zoom area selection unit may be caused to select the zoom area information based on information related to the reproduction device.

[0020] The zoom area selection unit may be caused to select the zoom area information by using at least any one of information indicating the type of the reproduction device and information indicating the rotation direction of the reproduction device as the information related to the reproduction device.

[0021] According to the first aspect of the present technology, the reproduction method or program includes the following steps: decoding encoded video data or encoded audio data; selecting one or more zoom area information from multiple zoom area information specifying the area to be zoomed; and based on the selected zoom area information, performing cropping processing on the video data obtained by decoding or performing audio conversion processing on the audio data obtained by decoding.

[0022] According to a first aspect of the present technology, encoded video data or encoded audio data is decoded; one or more pieces of zoom area information are selected from multiple pieces of zoom area information specifying an area to be zoomed; and based on the selected zoom area information, a cropping process is performed on the video data obtained by decoding or an audio conversion process is performed on the audio data obtained by decoding.

[0023] According to the second aspect of the present technology, the encoding device includes: an encoding unit that encodes video data or encodes audio data; and a multiplexer that generates a bit stream by multiplexing the encoded video data or the encoded audio data with multiple scaling area information specifying the area to be scaled.

[0024] According to the second aspect of the present technology, the encoding method or program includes the following steps: encoding video data or encoding audio data; and generating a bit stream by multiplexing the encoded video data or the encoded audio data with multiple scaling area information specifying the area to be scaled.

[0025] According to a second aspect of the present technology, video data is encoded or audio data is encoded; and a bit stream is generated by multiplexing the encoded video data or the encoded audio data with a plurality of pieces of scaling area information specifying an area to be scaled.

[0026] Effects of the Invention

[0027] According to the first aspect and the second aspect of the present technology, each reproducing device can reproduce appropriate content in a simplified manner.

[0028] Note that the effects of the present technology are not limited to the effects described here, but may be any effects described in the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a diagram showing an example of the configuration of an encoding device.

[0030] Figure 2 is a diagram showing the configuration of encoded content data.

[0031] Figure 3 is a diagram showing zoom area information.

[0032] Figure 4 is a diagram showing the syntax of the zoom area information presence flag.

[0033] Figure 5 is a diagram showing the syntax of zoom area information.

[0034] Figure 6is a diagram showing the syntax of zoom area information.

[0035] Figure 7 is a diagram showing the syntax of zoom area information.

[0036] Figure 8 is a diagram showing the syntax of zoom area information.

[0037] Fig. 9 is a diagram showing the syntax of zoom area information.

[0038] Fig.10 is a diagram showing the syntax of zoom area information.

[0039] Fig.11 is a diagram showing zoom area information.

[0040] Fig.12 is a diagram showing zoom area information.

[0041] Fig.13 is a diagram showing the syntax of zoom area information.

[0042] Fig.14 This is a diagram showing the syntax of the zoom area information existence flag, etc.

[0043] Fig.15 is a diagram showing the syntax of zoom area information.

[0044] Fig.16 It is a diagram showing the syntax of zoom area auxiliary information, etc.

[0045] Fig.17 is a diagram showing scaling specifications.

[0046] Fig.18 is a diagram showing an example of reproduced content.

[0047] Fig.19 is a flowchart showing the encoding process.

[0048] Fig. 20 is a diagram showing an example of the configuration of a reproduction device.

[0049] Fig.21 is a flowchart showing the reproduction process.

[0050] Fig. 22 is a diagram showing an example of the configuration of a reproduction device.

[0051] Fig.23 is a flowchart showing the reproduction process.

[0052] Fig.24 is a diagram showing an example of the configuration of a reproduction device.

[0053] Fig.25 is a flowchart showing the reproduction process.

[0054] Fig.26 is a diagram showing an example of the configuration of a reproduction device.

[0055] Fig. 27 is a flowchart showing the reproduction process.

[0056] Fig.28 is a diagram showing an example of the configuration of a computer. DETAILED DESCRIPTION

[0057] Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

[0058] <First Embodiment>

[0059] <Example of Configuration of Encoding Device>

[0060] The present technology enables reproduction devices such as TV receivers and smart phones with display screen sizes different from each other to reproduce appropriate content such as content suitable for such reproduction devices in a simplified manner. The content described here may be, for example, content formed by video and audio or content formed by either video and audio. Hereinafter, the description will be continued using an example of a case where the content is formed by a video and audio accompanying the video.

[0061] Figure 1 is a diagram showing an example of the configuration of an encoding device according to the present technology.

[0062] This encoding device 11 encodes content generated by a content generator, and outputs a bit stream (code string) in which encoded data obtained as a result is stored.

[0063] The encoding device 11 includes: a video data encoding unit 21; an audio data encoding unit 22; a metadata encoding unit 23; a multiplexer 24; and an output unit 25.

[0064] In this example, video data of video and audio data of audio constituting content are supplied to a video data encoding unit 21 and an audio data encoding unit 22 , respectively, and metadata of the content is supplied to a metadata encoding unit 23 .

[0065] The video data encoding unit 21 encodes the video data of the supplied content and supplies the encoded video data obtained as a result thereof to the multiplexer 24. The audio data encoding unit 22 encodes the audio data of the supplied content and supplies the encoded audio data obtained as a result thereof to the multiplexer 24.

[0066] The metadata encoding unit 23 encodes metadata of the supplied content, and supplies the encoded metadata obtained as a result thereof to the multiplexer 24 .

[0067] The multiplexer 24 generates a bit stream by multiplexing the encoded video data supplied from the video data encoding unit 21, the encoded audio data supplied from the audio data encoding unit 22, and the encoded metadata supplied from the metadata encoding unit 23, and supplies the generated bit stream to the output unit 25. The output unit 25 outputs the bit stream supplied from the multiplexer 24 to a reproducing device or the like.

[0068] Note that, hereinafter, the bit stream output from the output unit 25 will also be referred to as encoded content data.

[0069] <Encoded Content Data>

[0070] Content encoded by the encoding device 11 is generated in consideration of cropping and reproduction as needed. In other words, the content producer generates content in consideration of directly reproducing the content or cropping and reproducing a part of the entire area of ​​the video constituting the content.

[0071] For example, the content producer selects a partial area to be cropped and reproduced, that is, an area to be scaled and reproduced by cropping, as a scaled area from the entire area of ​​a video (image) constituting the content.

[0072] Note that, for example, the zoom area used to achieve the purpose of the viewing angle suitable for the reproduction device under consideration can be freely determined by the content producer. In addition, the zoom area can be determined based on the zoom purpose, such as zooming in and tracking a specific object such as a singer or performer in the video of the content.

[0073] In this way, when a plurality of zoom areas are designated for content by the producer side, zoom area information designating the zoom area is stored as metadata in the bit stream output from the encoding device 11, that is, in the encoded content data. At this time, when it is desired to designate a zoom area for each predetermined time unit, the zoom area information can be stored in the encoded content data for each of the above-mentioned time units.

[0074] More specifically, for example, Figure 2 As shown, in the case where the content is stored in the bitstream for each frame, the scaling area information may be stored in the bitstream for each frame.

[0075] exist Figure 2In the example shown, a header sector HD in which header information and the like are stored is arranged at the beginning of a bit stream, ie, encoded content data, and a data sector DA in which encoded video data and encoded audio data are stored is arranged after the header sector HD.

[0076] In the header segment HD, a video information header segment PHD is set in which header information related to the video constituting the content is stored, an audio information header segment AHD is stored in which header information related to the audio constituting the content, and a meta information header segment MHD is stored in which header information related to the metadata of the content is stored.

[0077] In addition, in the meta information header segment MHD, a zoom area information header segment ZHD is set in which information related to the zoom area information is stored. For example, in the zoom area information header segment ZHD, a zoom area information existence flag indicating whether the zoom area information is stored in the data segment DA is stored.

[0078] In addition, in the data segment DA, a data segment in which data of the encoded content is stored for each frame of the content is set. In this example, a data segment DAF-1 in which data of the first frame is stored is set at the beginning of the data segment DA, and a data segment DAF-2 in which data of the second frame of the content is stored is set after the data segment DAF-1. In addition, here, the data segments of the third frame and subsequent frames are not shown in the drawings. Hereinafter, in the case where the data segment DAF-1 or the data segment DAF-2 of each frame does not need to be particularly distinguished from each other, each of the data segment DAF-1 and the data segment DAF-2 will be simply referred to as the data segment DAF.

[0079] In the data sector DAF-1 of the first frame, a video information data sector PD-1 in which encoded video data is stored, an audio information data sector AD-1 in which encoded audio data is stored, and a meta information data sector MD-1 in which encoded metadata is stored are set.

[0080] For example, the meta information data segment MD-1 includes position information of a video object and a sound source object included in the first frame of the content. In addition, a zoom area information data segment ZD-1 in which the encoded zoom area information in the encoded metadata is stored is set in the meta information data segment MD-1. The position information of the video object and the sound source object, the zoom area information, etc. are set as metadata of the content.

[0081] Likewise, similar to the data segment DAF-1, in the data segment DAF-2, a video information data segment PD-2 storing the encoded video data, an audio information data segment AD-2 storing the encoded audio data, and a meta information data segment MD-2 storing the encoded metadata are set. In addition, in the meta information data segment MD-2, a zoom area information data segment ZD-2 storing the encoded zoom area information is set.

[0082] Furthermore, hereinafter, in the case where the video information data segment PD-1 and the video information data segment PD-2 do not need to be particularly distinguished from each other, each of the video information data segment PD-1 and the video information data segment PD-2 will also be referred to simply as the video information data segment PD, and in the case where the audio information data segment AD-1 and the audio information data segment AD-2 do not need to be particularly distinguished from each other, each of the audio information data segment AD-1 and the audio information data segment AD-2 will also be referred to simply as the audio information data segment AD. In addition, in the case where the meta information data segment MD-1 and the meta information data segment MD-2 do not need to be particularly distinguished from each other, each of the meta information data segment MD-1 and the meta information data segment MD-2 will also be referred to simply as the meta information data segment MD, and in the case where the zoom area information data segment ZD-1 and the zoom area information data segment ZD-2 do not need to be particularly distinguished from each other, each of the zoom area information data segment ZD-1 and the zoom area information data segment ZD-2 will also be referred to simply as the zoom area information data segment ZD.

[0083] In addition, Figure 2 In the case of, in each data segment DAF, an example of setting the video information data segment PD, the audio information data segment AD, and the meta information data segment MD is described. However, the meta information data segment MD may be set in each or one of the video information data segment PD and the audio information data segment AD. In this case, the zoom area information is stored in the zoom area information data segment ZD of the meta information data segment MD set in the video information data segment PD or the audio information data segment AD.

[0084] Similarly, although an example is described in which the video information header segment PHD, the audio information header segment AHD, and the meta information header segment MHD are set in the header segment HD, the meta information header segment MHD may be set in both or either of the video information header segment PHD and the audio information header segment AHD.

[0085] In addition, in the case where the zoom area information in each frame of the content is the same, the zoom area information may be configured to be stored in the header segment HD. In this case, it is not necessary to set the zoom area information data segment ZD in each data segment DAF.

[0086] <Specific example 1 of zoom area information>

[0087] Subsequently, a more specific example of the zoom area information will be described.

[0088] The above-mentioned zoom area information is information that specifies the zoom area of ​​the area to be zoomed. More specifically, the zoom area information is information that indicates the position of the zoom area. Figure 3 The zoom area shown can be specified using the coordinates of the center position of the zoom area, the coordinates of the start point, the coordinates of the end point, the vertical width, the horizontal width, etc.

[0089] exist Figure 3 In the case shown, the area of ​​the entire video (image) of the content is the original area OR, and a rectangular zoom area ZE is specified within the original area OR. In this example, the width of the zoom area ZE in the transverse direction (horizontal direction) of the figure is the horizontal width XW, and the width of the zoom area ZE in the longitudinal direction (vertical direction) of the figure is the vertical width YW.

[0090] Here, in the figure, a point having the lateral direction (horizontal direction) as the X direction and the longitudinal direction (vertical direction) as the Y direction in the XY coordinate system will be represented as coordinates (X, Y).

[0091] Now, when the coordinates of the point P11 of the center position (center position) of the zoom area ZE are (XC, YC), the zoom area ZE can be specified using the center coordinates (XC, YC) and the horizontal width XW and vertical width YW of the zoom area ZE. Therefore, the center coordinates (XC, YC), the horizontal width XW, and the vertical width YW can be set as the zoom area information.

[0092] In addition, for example, in the case where the zoom area ZE is a rectangular area, the upper left vertex P12 of the zoom area ZE in the figure is set as the starting point, and the lower right vertex P13 of the zoom area ZE in the figure is set as the end point, and the zoom area ZE can also be specified using the coordinates (X0, Y0) of the starting point (vertex P12) and the coordinates (X1, Y1) of the end point (vertex P13). Therefore, the coordinates (X0, Y0) of the starting point and the coordinates (X1, Y1) of the end point can be set as the zoom area information.

[0093] More specifically, the coordinates (X0, Y0) of the start point and the coordinates (X1, Y1) of the end point are set as the zoom area information. In this case, for example, it can be configured so that according to the value of the zoom area information existence flag, Figure 4 The zoom area information shown is stored in the above zoom area information header section ZHD, and Figure 5 The zoom area information shown is stored in each zoom area information data section ZD.

[0094] Figure 4 : is a diagram showing the syntax of the zoom area information existence flag. In this example, "hasZoomAreaInfo" represents the zoom area information existence flag, and the value of the zoom area information existence flag hasZoomAreaInfo is one of "0" and "1".

[0095] Here, when the value of the zoom area information existence flag hasZoomAreaInfo is "0", it means that the encoded content data does not include zoom area information. On the contrary, when the value of the zoom area information existence flag hasZoomAreaInfo is "1", it means that the encoded content data includes zoom area information.

[0096] In addition, when the value of the zoom area information existence flag hasZoomAreaInfo is "1", the zoom area information is stored in the zoom area information data segment ZD of each frame. Figure 5 The syntax shown stores the zoom region information in the zoom region information data segment ZD.

[0097] exist Figure 5 In the figure, "ZoomAreaX0" and "ZoomAreaY0" respectively represent the X coordinate X0 and Y coordinate Y0 of the start point of the zoom area ZE. In addition, "ZoomAreaX1" and "ZoomAreaY1" respectively represent the X coordinate X1 and Y coordinate Y1 of the end point of the zoom area ZE.

[0098] For example, in the case where the video of the content to be encoded is an 8K video, each of the values ​​of “ZoomAreaX0” and “ZoomAreaX1” is set to one of values ​​0 to 7679, and each of the values ​​of “ZoomAreaY0” and “ZoomAreaY1” is set to one of values ​​0 to 4319.

[0099] <Specific example 2 of zoom area information>

[0100] In addition, for example, also in the case where the center coordinates (XC, YC), the horizontal width XW, and the vertical width YW are set as the zoom area information, Figure 4The zoom area information existence flag hasZoomAreaInfo shown is stored in the zoom area information header segment ZHD. When the value of the zoom area information existence flag hasZoomAreaInfo is "1", the zoom area information is stored in the zoom area information data segment ZD of each frame. In this case, for example, Figure 6 The syntax shown stores the zoom region information in the zoom region information data segment ZD.

[0101] exist Figure 6 In the case of , “ZoomAreaXC” and “ZoomAreaYC” respectively represent the X coordinate XC and the Y coordinate YC of the center coordinates (XC, YC) of the zoom area ZE.

[0102] In addition, “ZoomAreaXW” and “ZoomAreaYW” represent the horizontal width XW and the vertical width YW of the zoom area ZE, respectively.

[0103] Also in this example, for example, in the case where the video of the content to be encoded is an 8K video, each of the values ​​of “ZoomAreaXC” and “ZoomAreaXW” is set to one of values ​​0 to 7679, and each of the values ​​of “ZoomAreaYC” and “ZoomAreaYW” is set to one of values ​​0 to 4319.

[0104] <Specific example 3 of zoom area information>

[0105] In addition, for example, in the case where the zoom area is specified using the center coordinates (XC, YC), the horizontal width XW, and the vertical width YW, and the horizontal width XW and the vertical width YW are set to fixed values, only the difference in the center coordinates (XC, YC) may be stored as zoom area information in the zoom area information data segment ZD.

[0106] In this case, for example, in the zoom area information data segment ZD-1 set in the data segment DAF-1 of the first frame, it is stored Figure 6 In addition, in the zoom area information data segment ZD set in the data segment DAF of each frame in the second frame and subsequent frames, Figure 7 The syntax shown is used to store the zoom region information.

[0107] exist Figure 7In the case of "nbits", "ZoomAreaXCshift" and "ZoomAreaYCshift" are stored as zoom area information. "nbits" is the number of bits of information indicating each of "ZoomAreaXCshift" and "ZoomAreaYCshift".

[0108] In addition, "ZoomAreaXCshift" indicates the difference between XC, which is the X coordinate of the center coordinate (XC, YC), and a predetermined reference value. For example, the reference value of the coordinate XC may be the X coordinate of the center coordinate (XC, YC) in the first frame or the X coordinate of the center coordinate (XC, YC) in the previous frame of the current frame.

[0109] "ZoomAreaYCshift" indicates the difference between the YC as the Y coordinate of the center coordinate (XC, YC) and a predetermined reference value. For example, similar to the reference value of the coordinate XC, the reference value of the coordinate YC may be the Y coordinate of the center coordinate (XC, YC) in the first frame or the Y coordinate of the center coordinate (XC, YC) in the previous frame of the current frame.

[0110] Such “ZoomAreaXCshift” and “ZoomAreaYCshift” indicate the shift amount from the reference value of the center coordinates (XC, YC).

[0111] Note that, for example, in the case where the reference value of the center coordinates (XC, YC) is known on the reproduction side of the content, in the case where the reference value of the center coordinates (XC, YC) is stored in the zoom area information header section ZHD, etc., Figure 7 The zoom area information shown may be stored in the zoom area information data section ZD of each frame.

[0112] <Specific example 4 of zoom area information>

[0113] In addition, for example, when the zoom area is specified using the center coordinates (XC, YC), the horizontal width XW and the vertical width YW and the center coordinates (XC, YC) are set to a fixed value, only the difference, that is, the change in the horizontal width XW and the vertical width YW, can be stored in the zoom area information data segment ZD as the zoom area information.

[0114] In this case, for example, in the zoom area information data segment ZD-1 set in the data segment DAF-1 of the first frame, it is stored Figure 6 In addition, in the zoom area information data segment ZD set in the data segment DAF set in each of the second frame and the subsequent frames, Figure 8 The syntax shown is used to store the zoom region information.

[0115] exist Figure 8 In the zoom area information, "nbits", "ZoomAreaXWshift", and "ZoomAreaYWshift" are stored. "nbits" is the number of bits of information indicating each of "ZoomAreaXWshift" and "ZoomAreaYWshift".

[0116] In addition, "ZoomAreaXWshift" indicates a change amount relative to a predetermined reference value of the horizontal width XW. For example, the reference value of the horizontal width XW may be the horizontal width XW in the first frame or the horizontal width XW of the previous frame of the current frame.

[0117] "ZoomAreaYWshift" indicates a change amount relative to a reference value of the vertical width YW. For example, similar to the reference value of the horizontal width XW, the reference value of the vertical width YW may be the vertical width YW in the first frame or the vertical width YW of the previous frame of the current frame.

[0118] Note that, for example, in the case where the reference values ​​of the horizontal width XW and the vertical width YW are known on the reproduction side of the content, in the case where the reference values ​​of the horizontal width XW and the vertical width YW are stored in the zoom area information header section ZHD or the like, Figure 8 The zoom area information shown may be stored in the zoom area information data section ZD of each frame.

[0119] <Specific example 5 of zoom area information>

[0120] In addition, for example, in the case where the zoom area is specified using the center coordinates (XC, YC), the horizontal width XW, and the vertical width YW, as shown in FIG. Figure 7 and Figure 8 As in the case of FIG. 4 , the difference between the center coordinates (XC, YC), the horizontal width XW, and the vertical width YW may be stored as zoom area information in the zoom area information data segment ZD.

[0121] In this case, for example, in the zoom area information data segment ZD-1 set in the data segment DAF-1 set in the first frame, the Figure 6 In addition, in the zoom area information data segment ZD set in the data segment DAF in each of the second frame and the subsequent frames, Fig. 9 The syntax shown is used to store the zoom region information.

[0122] exist Fig. 9In the case of , “nbits”, “ZoomAreaXCshift”, “ZoomAreaYCshift”, “ZoomAreaXWshift”, and “ZoomAreaYWshift” are stored as the zoom area information.

[0123] “nbits” is bit number information indicating the number of bits of information of each of “ZoomAreaXCshift”, “ZoomAreaYCshift”, “ZoomAreaXWshift”, and “ZoomAreaYWshift”.

[0124] like Figure 7 In the case of , "ZoomAreaXCshift" and "ZoomAreaYCshift" respectively represent the differences from the reference values ​​of the X coordinate and the Y coordinate of the center coordinate (XC, YC).

[0125] In addition, if Figure 8 In the case of FIG. 5 , “ZoomAreaXWshift” and “ZoomAreaYWshift” represent the amount of change relative to the reference values ​​of the horizontal width XW and the vertical width YW, respectively.

[0126] Here, the reference values ​​of the center coordinates (XC, YC), the horizontal width XW, and the vertical width YW may be set to the center coordinates (XC, YC), the horizontal width XW, and the vertical width YW in the first frame or the previous frame of the current frame. In addition, in the case where the reference values ​​of the center coordinates (XC, YC), the horizontal width XW, and the vertical width YW are known on the reproduction side of the content, or in the case where the reference values ​​are stored in the zoom area information header segment ZHD, Fig. 9 The zoom area information shown may be stored in the zoom area information data section ZD of each frame.

[0127] <Specific Example 6 of Zoom Area Information>

[0128] In addition, by Figures 6 to 9 The examples shown can be combined, for example, Fig.10 The syntax shown stores the zoom area information in each zoom area information data segment ZD.

[0129] in this case, Figure 4 The zoom area information existence flag hasZoomAreaInfo shown is stored in the zoom area information header segment ZHD. In addition, when the value of the zoom area information existence flag hasZoomAreaInfo is "1", the zoom area information is stored in the zoom area information data segment ZD of each frame. Fig.10The syntax shown stores the zoom region information in the zoom region information data segment ZD.

[0130] exist Fig.10 In the case shown, the coding mode information is arranged at the beginning of the zoom area information, and the coding mode information indicates Figures 6 to 9 The format of describing the zoom area information (more specifically, the information specifying the position of the zoom area) in the format shown in FIG. Fig.10 In the encoding mode, "mode" indicates the encoding mode information.

[0131] Here, the value of the encoding mode information mode is set to one of values ​​0 to 3.

[0132] For example, when the value of the coding mode information mode is "0", as shown in "case 0" and below, Figure 6 Similar to the illustrated example, “ZoomAreaXC” indicating the coordinates XC, “ZoomAreaYC” indicating the coordinates YC, “ZoomAreaXW” indicating the horizontal width XW, and “ZoomAreaYW” indicating the vertical width YW are stored as the zoom area information.

[0133] On the other hand, when the value of the coding mode information mode is "1", as shown in "case 1" and below in the figure, similar to Figure 7 In the illustrated example, “nbits” indicating bit number information, “ZoomAreaXCshift” indicating a difference in coordinates XC, and “ZoomAreaYCshift” indicating a difference in coordinates YC are stored as zoom area information.

[0134] When the value of the coding mode information mode is "2", as shown in "case 2" and below, Figure 8 Similarly to the example shown, “nbits” indicating bit number information, “ZoomAreaXWshift” indicating the amount of change in the horizontal width XW, and “ZoomAreaYWshift” indicating the amount of change in the vertical width YW are stored as zoom area information.

[0135] In addition, when the value of the coding mode information mode is "3", as shown in "case 3" and below, Fig. 9Similar to the example shown, “nbits” indicating the number of bits information, “ZoomAreaXCshift” indicating the difference in coordinates XC, “ZoomAreaYCshift” indicating the difference in coordinates YC, “ZoomAreaXWshift” indicating the amount of change in horizontal width XW, and “ZoomAreaYWshift” indicating the amount of change in vertical width YW are stored as zoom area information.

[0136] <Specific example 7 of zoom area information>

[0137] In addition, although an example in which coordinate information is stored as zoom area information is described above, angle information specifying a zoom area may be stored as zoom area information in each zoom area information data section ZD.

[0138] For example, Fig.11 As shown, the original region OR is located at a position having the same height as the center position CP and facing the center position CP. Fig.11 A point separated by a predetermined distance from the front side of the image is set as a reference viewing point WP when viewing the content. In addition, it is assumed that the positional relationship between the center position CP and the viewing point WP is always the same regardless of the frame of the content. Fig.11 In the Figure 3 The parts corresponding to the situation shown, and their description will not be presented appropriately.

[0139] exist Fig.11 , the straight line connecting the center position CP and the viewing point WP is set as the straight line L11. In addition, the midpoint of the left side of the zoom area ZE in the figure is set as point P21, and the straight line connecting point P21 and the viewing point WP is set as the straight line L12. In addition, the angle formed by the straight lines L11 and L12 is set as the horizontal angle φ 左 .

[0140] Similarly, the midpoint of the right side of the zoom area ZE in the figure is set as point P22, and the straight line connecting point P22 and the viewing point WP is set as straight line L13. In addition, the angle formed by the straight line L11 and the straight line L13 is set as the horizontal angle φ 右 .

[0141] In addition, a point having the same Y coordinate as the center position CP of the right side of the zoom area ZE in the figure is set as point P23, and a straight line connecting point P23 and the viewing point WP is set as straight line L14. In addition, the upper right vertex of the zoom area ZE in the figure is set as point P24, a straight line connecting point P24 and the viewing point WP is set as straight line L15, and an angle formed by straight line L14 and straight line L15 is set as pitch angle θ 顶.

[0142] Similarly, the lower right vertex of the zoom area ZE in the figure is set as point P25, the straight line connecting point P25 and the viewing point WP is set as straight line L16, and the angle formed by straight line L14 and straight line L16 is set as pitch angle θ 底 .

[0143] At this time, the horizontal angle φ can be used 左 , horizontal angle φ 右 , pitch angle θ 顶 and the pitch angle θ 底 To specify the zoom area ZE. Accordingly, the horizontal angle φ can be 左 , horizontal angle φ 右 , pitch angle θ 顶 and the pitch angle θ 底 The zoom area information is stored in Figure 2 In each zoom area information data segment ZD shown. In addition, the horizontal angle φ 左 , horizontal angle φ 右 , pitch angle θ 顶 and the pitch angle θ 底 Some or all of the changes in are set as zoom area information.

[0144] <Specific example 8 of zoom area information>

[0145] In addition, for example, Fig.12 As shown, the angle information determined based on the positional relationship between the center position CP, the point P11 located at the center position of the zoom area ZE, and the viewing point WP can be set as the zoom area information. Fig.12 In the Figure 3 or Fig.11 The situation shown corresponds to the part, and its description will not be presented appropriately.

[0146] exist Fig.12 , a straight line connecting the point P11 located at the center position of the zoom area ZE and the viewing point WP is set as a straight line L21. In addition, a point having the same X coordinate as the point P11 located at the center position of the zoom area ZE and the same Y coordinate as the center position CP of the original area OR is set as a point P31, and a straight line connecting the point P31 and the viewing point WP is set as a straight line L22.

[0147] In addition, the midpoint of the upper side of the zoom area ZE in the figure is set to point P32, the straight line connecting point P32 and the viewing point WP is set to straight line L23, the midpoint of the lower side of the zoom area ZE in the figure is set to point P33, and the straight line connecting point P33 and the viewing point WP is set to straight line L24.

[0148] In addition, the angle formed by the straight line L12 and the straight line L13 is set as the horizontal viewing angle φ W , and the angle formed by the straight line L11 and the straight line L22 is set as the horizontal angle φ C In addition, the angle formed by the straight line L23 and the straight line L24 is set as the vertical viewing angle θ W , and the angle formed by the straight line L21 and the straight line L22 is set as the pitch angle θ C .

[0149] Here, the horizontal angle φ C and the pitch angle θ C denote the horizontal angle and the elevation angle from the viewing point WP relative to the point P11 located at the center of the zoom area ZE, respectively.

[0150] At this time, the horizontal viewing angle φ can be used W , horizontal angle φ C , vertical viewing angle θ W and the pitch angle θ C To specify the zoom area ZE. Therefore, the horizontal viewing angle φ W , horizontal angle φ C , vertical viewing angle θ W and the pitch angle θ C Or the changes in these angles are stored as zoom area information Figure 2 In each zoom area information data segment ZD shown.

[0151] In this case, for example, Figure 4 The zoom area information existence flag hasZoomAreaInfo shown is stored in the zoom area information header segment ZHD. In addition, when the value of the zoom area information existence flag hasZoomAreaInfo is "1", the zoom area information is stored in the zoom area information data segment ZD of each frame. Fig.13 The syntax shown stores the zoom region information in the zoom region information data segment ZD.

[0152] exist Fig.13 In the illustrated case, encoding mode information is arranged at the beginning of the zoom area information, the encoding mode information indicating one format among a plurality of formats in which the zoom area information (more specifically, information on the position of the zoom area) is described.

[0153] exist Fig.13 In the encoding mode information, “mode” represents encoding mode information, and the value of the encoding mode information mode is set to one of values ​​0 to 3.

[0154] For example, when the value of the coding mode information mode is "0", as shown in "case 0" and below in the figure, it means that the horizontal angle φ C "ZoomAreaAZC" indicates the pitch angle θ C "ZoomAreaELC" indicates the horizontal viewing angle φ W "ZoomAreaAZW" and represents the vertical viewing angle θ W The "ZoomAreaELW" is stored as the zoom area information.

[0155] When the value of the coding mode information is "1", as shown in "case 1" and below in the figure, "nbits" indicating the number of bits, "nbits" indicating the horizontal angle φ C The shift angle of "ZoomAreaAZCshift" and the pitch angle θ C The offset angle of "ZoomAreaELCshift" is stored as the zoom area information.

[0156] Here, the bit number information nbits is information indicating the number of bits of each of “ZoomAreaAZCshift” and “ZoomAreaELCshift”.

[0157] In addition, "ZoomAreaAZCshift" is set to the horizontal angle φ of the previous frame of the current frame C Or the horizontal angle φ as a predetermined reference C The horizontal angle φ with the current frame C The difference between the two, "ZoomAreaELCshift" is set to the pitch angle θ of the frame before the current frame. C Or the pitch angle θ as a predetermined reference C The pitch angle θ with the current frame C The difference between, etc.

[0158] When the value of the coding mode information mode is "2", as shown in "case 2" and below in the figure, "nbits" indicating the bit information, "nbits" indicating the horizontal viewing angle φ W The amount of change "ZoomAreaAZWshift" and the vertical viewing angle θ W The amount of change "ZoomAreaELWshift" is stored as the zoom area information.

[0159] Here, the bit number information nbits is information indicating the number of bits of information of each of “ZoomAreaAZWshift” and “ZoomAreaELWshift”.

[0160] In addition, "ZoomAreaAZWshift" is set to the horizontal viewing angle φ of the previous frame of the current frame W Or the horizontal angle of view φ as a predetermined reference W The horizontal viewing angle φ of the current frame W The difference between the vertical angles of the previous frame and the vertical angle of the previous frame is set to θ. W Or the vertical viewing angle θ as a predetermined reference W The vertical viewing angle θ of the current frame W The difference between, etc.

[0161] In addition, when the value of the coding mode information mode is "3", as shown in "case 3" and below in the figure, "nbits" indicating the number of bits, "nbits" indicating the horizontal angle φ C The "ZoomAreaAZCshift" of the offset angle, indicating the pitch angle θ C "ZoomAreaELCshift" of the offset angle, indicating the horizontal viewing angle φ W The amount of change "ZoomAreaAZWshift" and the vertical viewing angle θ W The amount of change "ZoomAreaELWshift" is stored as the zoom area information.

[0162] In this case, the bit number information nbits is information of the number of bits of information representing each of “ZoomAreaAZCshift”, “ZoomAreaELCshift”, “ZoomAreaAZWshift”, and “ZoomAreaELWshift”.

[0163] Note that the configuration of the zoom area information is not limited to Fig.13 In the example shown, only "ZoomAreaAZC", "ZoomAreaELC", "ZoomAreaAZW", and "ZoomAreaELW" can be set as the zoom area information. In addition, both sides of "ZoomAreaAZCshift", "ZoomAreaELCshift", "ZoomAreaAZWshift", and "ZoomAreaELWshift" or only one side can be set as the zoom area information.

[0164] <Specific Example 9 of Zoom Area Information>

[0165] In addition, although the above describes the case where there is only one piece of zoom area information, multiple pieces of zoom area information may be stored in the zoom area information data segment ZD. In other words, by specifying multiple zoom areas for one content, zoom area information may be stored in the zoom area information data segment ZD for each zoom area.

[0166] In this case, for example, Fig.14 The syntax shown stores each information in the zoom area information header section ZHD, and further in Fig.15 The syntax shown stores the zoom region information in the zoom region information data segment ZD of each frame.

[0167] exist Fig.14 In the example shown, "hasZoomAreaInfo" indicates a zoom area information existence flag. When the value of the zoom area information existence flag is "1", "numZoomAreas" is stored after the zoom area information existence flag hasZoomAreaInfo.

[0168] Here, "numZoomAreas" indicates zoom area number information, which indicates the number of zoom area information described in the zoom area information data segment ZD, that is, the number of zoom areas set for the content. In this example, the value of the zoom area number information numZoomAreas is one of values ​​0 to 15.

[0169] In the encoded content data, zoom area information, more specifically, information specifying the position of each zoom area corresponding to a value obtained by adding 1 to the value of the zoom area number information numZoomAreas is stored in the zoom area information data segment ZD.

[0170] Accordingly, for example, in the case where the value of the zoom area number information numZoomAreas is "0", in the zoom area information data segment ZD, for one zoom area, information specifying the position of the zoom area is stored.

[0171] In addition, when the value of the zoom area information existence flag hasZoomAreaInfo is "1", the zoom area information is stored in the zoom area information data segment ZD. Fig.15 The syntax shown describes the zoom area information in the zoom area information data section ZD.

[0172] exist Fig.15 In the example shown, zoom area information corresponding to the number indicated by the zoom area number information numZoomAreas is stored.

[0173] exist Fig.15 In , "mode[idx]" represents encoding mode information of the zoom area specified by the index idx, and the value of the encoding mode information mode[idx] is set to one of values ​​0 to 3. Note that the index idx is each value from 0 to numZoomAreas.

[0174] For example, when the value of the encoding mode information mode[idx] is "0", as shown in "case0" and below in the figure, "ZoomAreaXC[idx]" representing the coordinate XC, "ZoomAreaYC[idx]" representing the coordinate YC, "ZoomAreaXW[idx]" representing the horizontal width XW, and "ZoomAreaYW[idx]" representing the vertical width YW are stored as zoom area information of the zoom area specified by the index idx.

[0175] In addition, when the value of the encoding mode information mode[idx] is "1", as shown in "case1" and below in the figure, "nbits" as the number of bits information, "ZoomAreaXCshift[idx]" indicating the difference in coordinates XC, and "ZoomAreaYCshift[idx]" indicating the difference in coordinates YC are stored as the zoom area information of the zoom area specified by the index idx. Here, the number of bits information nbits indicates the number of bits of information of each of "ZoomAreaXCshift[idx]" and "ZoomAreaYCshift[idx]".

[0176] In the case where the value of the encoding mode information mode[idx] is "2", as shown in "case 2" and below in the figure, "nbits" indicating the number of bits information, "ZoomAreaXWshift[idx]" indicating the amount of change in the horizontal width XW, and "ZoomAreaYWshift[idx]" indicating the amount of change in the vertical width YW are stored as the zoom area information of the zoom area specified by the index idx. Here, the number of bits information nbits indicates the number of bits of information of each of "ZoomAreaXWshift[idx]" and "ZoomAreaYWshift[idx]".

[0177] In addition, in the case where the value of the encoding mode information mode[idx] is "3", as shown in "case3" and below in the figure, "nbits" as the bit information, "ZoomAreaXCshift[idx]" indicating the difference in coordinates XC, "ZoomAreaYCshift[idx]" indicating the difference in coordinates YC, "ZoomAreaXWshift[idx]" indicating the amount of change in the horizontal width XW, and "ZoomAreaYWshift[idx]" indicating the amount of change in the vertical width YW are stored as the zoom area information of the zoom area specified by the index idx. Here, the bit information nbits indicates the number of bits of information of each of "ZoomAreaXCshift[idx]", "ZoomAreaYCshift[idx]", "ZoomAreaXWshift[idx]", and "ZoomAreaYWshift[idx]".

[0178] exist Fig.15 In the example shown, encoding mode information mode[idx] and zoom region information corresponding to the number of zoom regions are stored in the zoom region information data segment ZD.

[0179] Note that, alternatively, the zoom area information may consist of only the coordinates XC and YC, the horizontal angle φ C and the pitch angle θ C , the difference between the coordinates XC and YC or the horizontal angle φ C The difference and pitch angle θ C The difference constitutes.

[0180] In this case, the horizontal width XW and the vertical width YW and the horizontal viewing angle φ W and vertical viewing angle θ W It can be set on the reproduction side. At this time, the horizontal width XW and the vertical width YW and the horizontal viewing angle φ W and vertical viewing angle θ W It can be automatically set in the reproduction side device or can be specified by the user.

[0181] In such an example, for example, where the content is video and audio of ball sports, coordinates XC and YC indicating the position of a ball are set as zoom area information, and a fixed or user-specified horizontal width XW and vertical width YW are used on the reproduction side device.

[0182] <Zoom area auxiliary information>

[0183] In addition, in the zoom area information header section ZHD, as zoom area auxiliary information, supplementary information such as an ID indicating a reproduction target device or a zoom purpose and other text information may be included.

[0184] In this case, in the zoom area information header segment ZHD, for example, Fig.16 The syntax shown is to store the zoom area information when the flag hasZoomAreaInfo and the zoom area auxiliary information exist.

[0185] exist Fig.16 In the illustrated example, the zoom area information existence flag hasZoomAreaInfo is arranged at the beginning, and in the case where the value of the zoom area information existence flag hasZoomAreaInfo is "1", each information such as the zoom area auxiliary information is stored thereafter.

[0186] In other words, in this example, after the zoom area information existence flag hasZoomAreaInfo, zoom area number information "numZoomAreas" indicating the number of zoom area information described in the zoom area information data segment ZD is stored. Here, the value of the zoom area number information numZoomAreas is set to one of values ​​0 to 15.

[0187] In addition, after the zoom area number information numZoomAreas, information of each zoom area specified by the index idx corresponding to the number indicated by the zoom area number information numZoomAreas is arranged. Here, the index idx is set to each value from 0 to numZoomAreas.

[0188] In other words, "hasExtZoomAreaInfo[idx]" after the zoom area number information numZoomAreas indicates an auxiliary information flag indicating whether the zoom area auxiliary information of the zoom area specified by the index idx is stored. Here, the value of the auxiliary information flag hasExtZoomAreaInfo[idx] is set to one of "0" and "1".

[0189] In the case where the value of the auxiliary information flag hasExtZoomAreaInfo[idx] is "0", the zoom area auxiliary information indicating the zoom area specified by the index idx is not stored in the zoom area information header segment ZHD. On the contrary, in the case where the value of the auxiliary information flag hasExtZoomAreaInfo[idx] is "1", the zoom area auxiliary information indicating the zoom area specified by the index idx is stored in the zoom area information header segment ZHD.

[0190] In a case where the value of the auxiliary information flag hasExtZoomAreaInfo[idx] is "1", a specification ID indicating the specification of the zoom area specified by the index idx is arranged after the auxiliary information flag hasExtZoomAreaInfo[idx].

[0191] Furthermore, “hasZoomAreaCommentary” indicates a supplementary information flag indicating whether new supplementary information other than the specification ID exists for the zoom area specified by the index idx, for example, text information including a description of the zoom area.

[0192] For example, when the value of the supplementary information flag hasZoomAreaCommentary is "0", it indicates that there is no supplementary information. On the contrary, when the value of the supplementary information flag hasZoomAreaCommentary is "1", it indicates that there is supplementary information, and after the supplementary information flag hasZoomAreaCommentary, "nbytes" as byte number information and "ZoomAreaCommentary[idx]" as supplementary information are arranged.

[0193] Here, the byte number information nbytes indicates the number of bytes of the supplementary information ZoomAreaCommentary[idx]. In addition, the supplementary information ZoomAreaCommentary[idx] is set to text information describing the zoom area specified by the index idx.

[0194] More specifically, for example, assuming that the content consists of a live video and its audio, and the zoom area specified by the index idx is a zoom area for continuously zooming in on a singer as a video object. In this case, for example, text information such as "singer zoom" is set as supplementary information ZoomAreaCommentary[idx].

[0195] In the zoom area information header segment ZHD, as necessary, the settings of the following items corresponding to the number indicated by the zoom area number information numZoomAreas are stored: the auxiliary information flag hasExtZoomAreaInfo[idx], ZoomAreaSpecifiedID[idx] as the specification ID, the supplementary information flag hasZoomAreaCommentary, the number of bytes information nbytes, and the supplementary information ZoomAreaCommentary[idx]. However, for a zoom area whose auxiliary information flag hasExtZoomAreaInfo[idx] has a value of "0", ZoomAreaSpecifiedID[idx], the supplementary information flag hasZoomAreaCommentary, the number of bytes information nbytes, and the supplementary information ZoomAreaCommentary[idx] are not stored. Similarly, for a zoom area whose supplementary information flag hasZoomAreaCommentary has a value of "0", the number of bytes information nbytes and the supplementary information ZoomAreaCommentary[idx] are not stored.

[0196] In addition, ZoomAreaSpecifiedID[idx] as a specification ID is information indicating zoom specifications such as a reproduction target device and a zoom purpose for a zoom area, and, for example, Fig.17 As shown, the zoom specification is set for each value of ZoomAreaSpecifiedID[idx].

[0197] In this example, for example, in the case where the value of ZoomAreaSpecifiedID[idx] is "1", the zoom area indicating the zoom specification indicated by the specification ID is a zoom area assuming that the reproduction target device is a projector.

[0198] In addition, when the value of ZoomAreaSpecifiedID[idx] is 2 to 4, these values ​​respectively indicate that the zoom area of ​​the zoom specification represented by the specification ID is the zoom area assuming that the reproduction target device is a television receiver with a screen of more than 50, 30 to 50, and less than 30.

[0199] In this way, in Fig.17 In the illustrated example, the zoom area information whose value of ZoomAreaSpecifiedID[idx] is one of “1” to “4” is information indicating a zoom area set for each type of reproduction target device.

[0200] Also, for example, in the case where the value of ZoomAreaSpecifiedID[idx] is "7", the zoom area indicating the zoom specification indicated by the specification ID is a zoom area assuming that the reproduction target device is a smartphone and the rotation direction of the smartphone is the vertical direction.

[0201] Here, the rotation direction of the smartphone being the vertical direction means that the direction of the smartphone is the vertical direction when the user uses the smartphone to view content, that is, from the user's perspective, the longitudinal direction of the display screen of the smartphone is the vertical direction (upward / downward direction). Therefore, for example, in the case where the value of ZoomAreaSpecifiedID[idx] is "7", the zoom area is considered to be an area that is long in the vertical direction.

[0202] In addition, for example, when the value of ZoomAreaSpecifiedID[idx] is "8", the zoom area indicating the zoom specification indicated by the specification ID is a zoom area assuming that the reproduction target device is a smartphone and the rotation direction of the smartphone is horizontal. In this case, for example, the zoom area is considered to be a region that is long in the horizontal direction.

[0203] In this way, in Fig.17 In the example shown, the zoom area information whose value of ZoomAreaSpecifiedID[idx] is one of "5" to "8" is information indicating a zoom area set for the type of reproduction target device and the rotation direction of the reproduction target device.

[0204] In addition, for example, when the value of ZoomAreaSpecifiedID[idx] is "9", it indicates that the zoom area of ​​the zoom specification indicated by the specification ID is a zoom area with a predetermined zoom purpose set by the content producer. Here, the predetermined zoom purpose is, for example, displaying a specific zoom view, such as displaying a zoom of a predetermined video object.

[0205] Therefore, for example, in the case where the value "9" of ZoomAreaSpecifiedID[idx] indicates a zoom specification for continuously zooming in on a singer, the supplementary information ZoomAreaCommentary[idx] of the index idx is set to text information such as "singer zoom". The user can obtain the content of the zoom specification represented by each specification ID based on the specification ID or information associated with the specification ID, the supplementary information of the specification ID, etc.

[0206] In this way, in Fig.17In the example shown, each zoom area information whose value of ZoomAreaSpecifiedID[idx] is one of "9" to "15" is information indicating an arbitrary zoom area freely set by the content generator side, for example, a zoom area set for each specific video object.

[0207] As described above, by setting one or more zoom areas for a content, for example Fig.18 As shown, content matching the user's preference or content suitable for each reproduction device can be provided in a simplified manner.

[0208] exist Fig.18 In the figure, image Q11 shows a video (image) of a predetermined content. The content is a live video content, and image Q11 is a wide-angle image in which the live performers, namely singer M11, guitarist M12, and bassist M13, are projected, and the entire state, audience, etc. are projected.

[0209] For the image Q11 constituting such content, the content producer sets one or more zoom areas according to the zoom specification or zoom purpose of the reproduction target device.

[0210] For example, in order to display a zoomed view of the singer M11 as a video object, the image Q12 may be displayed on the reproduction side while setting the area centered on the singer M11 on the image Q11 as the zoom area.

[0211] Similarly, for example, in order to display a zoomed view enlarging the guitarist M12 as a video object, a reproduced image Q13 can be reproduced as content on the reproduction side with the area centered on the guitarist M12 on the image Q11 set as a zoom area.

[0212] In addition, for example, by selecting a plurality of zoom areas on the reproduction side and by aligning these zoom areas to constitute one screen, the reproduction image Q14 can be reproduced as content on the reproduction side.

[0213] In this example, the image Q14 is composed of an image Q21 having a zoom area slightly smaller than the viewing angle of the image Q11, an image Q22 having a zoom area enlarging the singer M11, an image Q23 having a zoom area enlarging the guitarist M12, and an image Q24 having a zoom area enlarging the bassist M13. That is, the image Q14 has a multi-screen configuration. In the case where a plurality of zoom areas are set in advance on the content provider side, on the content reproduction side, by selecting a plurality of zoom areas, the content can be reproduced by adopting a multi-screen configuration such as the image Q14.

[0214] In addition, for example, when the viewing angle of half the viewing angle of Q11 is set in consideration of a reproduction device such as a tablet PC having a not so large display screen, that is, when an area having about half the area of ​​the entire image Q11 including the center of the image Q11 is set as the zoom area, the image Q15 can be reproduced as content on the reproduction side. In this example, each performer can also be displayed in a sufficiently large size in a reproduction device having a not so large display screen.

[0215] In addition, for example, in the case where a relatively narrow horizontally long area including the center of image Q11 within image Q11 is set as a zoom area in consideration of a smart phone whose rotation direction is horizontal, that is, the display screen is in a horizontally long state, image Q16 can be reproduced as content on the reproduction side.

[0216] For example, considering that the rotation direction of a smart phone is vertical, i.e., its display screen is in a vertically long state, and a vertically long area near the center of image Q11 is set as a zoom area, image Q17 can be reproduced as content on the reproduction side.

[0217] In the image Q17, singer M11, one of the performers, is displayed in a zoomed-in manner. In this example, since it is appropriate to display one performer in a zoomed-in manner for the reproduction target device, rather than displaying all the performers arranged in the horizontal direction, considering the small vertically long display screen, such a zoom area is set.

[0218] In addition, for example, considering that the reproduction device has a relatively large display screen such as a large-size television receiver, image Q18 can be reproduced as content on the reproduction side when the viewing angle is set to be slightly smaller than the viewing angle of image Q11, that is, when a relatively large area within image Q11 including the center of image Q11 is set as the zoom area.

[0219] As described above, by setting a zoom area on the content provider side and generating encoded content data including zoom area information indicating the zoom area on the reproduction side, a user who is viewing the content can choose to directly reproduce the content or perform zoom reproduction, i.e., cropped reproduction, based on the zoom area information.

[0220] Specifically, in the case where there are a plurality of pieces of zoom area information, the user may select zoom reproduction according to specific zoom area information among the plurality of pieces of zoom area information.

[0221] In addition, in the case where zoom area auxiliary information is stored in the encoded content data, on the reproduction side, by referring to the reproduction target device, the zoom purpose, the zoom specification such as the zoom content and the auxiliary information, a zoom area suitable for the reproduction device or the user's preference can be selected. The selection of the zoom area can be specified by the user or can be automatically performed by the reproduction device.

[0222] <Description of Encoding Process>

[0223] Next, the specific operation of the encoding device 11 will be described.

[0224] When video data and audio data constituting content and metadata of the content are provided from the outside, the encoding device 11 performs encoding processing and outputs the encoded content data. Fig.19 The encoding process performed by the encoding device 11 is described with reference to the flowchart shown.

[0225] In step S11 , the video data encoding unit 21 encodes the video data of the supplied content, and supplies the encoded video data obtained as a result thereof to the multiplexer 24 .

[0226] In step S12 , the audio data encoding unit 22 encodes the audio data of the supplied content, and supplies the encoded audio data obtained as a result thereof to the multiplexer 24 .

[0227] In step S13 , the metadata encoding unit 23 encodes metadata of the supplied content, and supplies the encoded metadata obtained as a result thereof to the multiplexer 24 .

[0228] In this article, for example, the above-mentioned zoom area information is included in the metadata to be encoded. The zoom area information can be, for example, a reference Figures 5 to 10 , Fig.13 and Fig.15 Any information other than that described in the description.

[0229] In addition, the metadata encoding unit 23 also encodes header information of the zoom area information such as the zoom area information existence flag hasZoomAreaInfo, the zoom area number information numZoomAreas, and the zoom area auxiliary information as necessary, and supplies the encoded header information to the multiplexer 24 .

[0230] In step S14, the multiplexer 24 generates a bit stream by multiplexing the encoded video data supplied from the video data encoding unit 21, the encoded audio data supplied from the audio data encoding unit 22, and the encoded metadata supplied from the metadata encoding unit 23, and supplies the generated bit stream to the output unit 25. At this time, the multiplexer 24 also stores the encoded header information of the scaling area information supplied from the metadata encoding unit 23 in the bit stream.

[0231] Thus, for example, one can obtain Figure 2 The coded content data shown is as a bit stream. Note that the configuration of the zoom area information header section ZHD of the coded content data may be any configuration, such as Figure 4 , Fig.14 or Fig.16 Configuration shown.

[0232] In step S15 , the output unit 25 outputs the bit stream supplied from the multiplexer 24 , and the encoding process ends.

[0233] As described above, the encoding device 11 encodes metadata including the zoom region information together with the content, thereby generating a bit stream.

[0234] In this way, by generating a bit stream including zoom area information for specifying a zoom area without preparing content for each reproducing device, content matching the user's preference or content suitable for each reproducing device can be provided in a simplified manner.

[0235] In other words, content producers can provide content considered optimal for user preferences, the screen size of the reproduction device, the rotation direction of the reproduction device, etc. in a simplified manner simply by specifying a zoom area without preparing content for each preference or each reproduction device.

[0236] In addition, on the reproduction side, by selecting a zoom area and cropping the content as needed, it is possible to view content that is optimal for the user's preferences, the screen size of the reproduction device, the rotation direction of the reproduction device, and the like.

[0237] <Example of Configuration of Reproduction Apparatus>

[0238] Next, a reproducing device that receives a bit stream output from the encoding device 11, that is, encoded content data and reproduces the content will be described.

[0239] Fig. 20 It is a diagram showing an example of the configuration of a reproduction device according to an embodiment of the present technology.

[0240] In this example, a display device 52 that displays information when a zoom area is selected, a video output device 53 that outputs video of content, and an audio output device 54 that outputs audio of content are connected to the reproduction device 51 as necessary.

[0241] Note that the display device 52, the video output device 53, and the audio output device 54 may be provided in the reproduction device 51. In addition, the display device 52 and the video output device 53 may be the same device.

[0242] The reproduction device 51 includes: a content data decoding unit 61; a zoom area selection unit 62; a video data decoding unit 63; a video dividing unit 64; an audio data decoding unit 65; and an audio conversion unit 66.

[0243] The content data decoding unit 61 receives the bit stream, that is, the encoded content data transmitted from the encoding device 11 , and separates the encoded video data, the encoded audio data, and the encoded metadata from the encoded content data.

[0244] The content data decoding unit 61 supplies the encoded video data to the video data decoding unit 63 , and supplies the encoded audio data to the audio data decoding unit 65 .

[0245] The content data decoding unit 61 obtains metadata by decoding the encoded metadata, and supplies the obtained metadata as needed to each unit of the reproducing device 51. In addition, in the case where the zoom area information is included in the metadata, the content data decoding unit 61 supplies the zoom area information to the zoom area selection unit 62. Furthermore, in the case where the zoom area auxiliary information is stored in the bit stream, the content data decoding unit 61 reads the zoom area auxiliary information, decodes the zoom area auxiliary information as needed, and supplies the obtained zoom area auxiliary information to the zoom area selection unit 62.

[0246] The zoom area selection unit 62 selects one piece of zoom area information from the one or more pieces of zoom area information supplied from the content data decoding unit 61, and supplies the selected zoom area information as selected zoom area information to the video splitting unit 64 and the audio converting unit 66. In other words, in the zoom area selection unit 62, a zoom area is selected based on the zoom area information supplied from the content data decoding unit 61.

[0247] For example, in the case where the zoom area auxiliary information is supplied from the content data decoding unit 61, the zoom area selecting unit 62 supplies the zoom area auxiliary information to the display device 52 to be displayed on the display device 52. In this way, for example, the following supplementary information is displayed on the display device 52 as the zoom area auxiliary information, such as the purpose and content of the zoom area, a specification ID indicating the zoom specification such as a reproduction target device, etc., information based on the specification ID, and text information.

[0248] Then, the user checks the zoom area auxiliary information displayed on the display device 52, and selects a desired zoom area by operating an input unit not shown in the figure. The zoom area selection unit 62 selects a zoom area based on a signal according to the user's operation provided from the input unit, and outputs selected zoom area information indicating the selected zoom area. In other words, the zoom area information of the zoom area specified by the user is selected, and the selected zoom area information is output as the selected zoom area information.

[0249] Note that the selection of the zoom area may be performed using any method, such as the zoom area selection unit 62 generating information indicating the position and size of each zoom area based on the zoom area information and displaying the information on the display device 52, and the user selecting a zoom area based on the display.

[0250] Note that in a case where selection of a zoom area is not performed, that is, in a case where reproduction of the original content is selected, the selected zoom area information is set to information indicating that cropping or the like is not performed.

[0251] Furthermore, for example, in the case where the reproducing apparatus 51 has previously recorded reproducing device information indicating the type of the own device such as a smartphone or a television receiver, the zoom area information (zoom area) can be selected by using the reproducing device information.

[0252] In this case, for example, the zoom area selection unit 62 obtains the reproduction device information and selects the zoom area information by using the obtained reproduction device information and the zoom area auxiliary information.

[0253] More specifically, the zoom area selection unit 62 selects a specification ID indicating that the reproduction target device is a device of the type indicated by the reproduction device information from the specification ID as the zoom area auxiliary information. Then, the zoom area selection unit 62 sets the zoom area information corresponding to the selected specification ID, that is, the zoom area information whose index idx is the same as the selected specification ID, as the selected zoom area information.

[0254] In addition, for example, in the case where the reproduction device 51 is a mobile device such as a smart phone or a tablet PC, the zoom area selection unit 62 can obtain direction information indicating the rotation direction of the reproduction device 51 from a gyro sensor not shown in the figure, and select the zoom area information by using the direction information.

[0255] In this case, for example, the zoom area selection unit 62 selects a specification ID indicating that the reproduction target device is a device of the type indicated by the reproduction device information, and the assumed rotation direction is the direction indicated by the direction information obtained in the specification ID as the zoom area auxiliary information. Then, the zoom area selection unit 62 sets the zoom area information corresponding to the selected specification ID as the selected zoom area information. In this way, in both the state in which the user uses the reproduction device 51 in the vertical direction (the screen is long in the vertical direction) and the state in which the user uses the reproduction device 51 in the horizontal direction (the screen is long in the horizontal direction), the zoom area information of the zoom area that is optimal for the current state is selected.

[0256] Note that, in addition to this, the zoom area information may be selected using only one of the reproduction device information and the direction information, or the zoom area information may be selected using any other information related to the reproduction apparatus 51 .

[0257] The video data decoding unit 63 decodes the encoded video data supplied from the content data decoding unit 61 , and supplies video data obtained as a result thereof to the video dividing unit 64 .

[0258] The video dividing unit 64 crops (divides) the zoom area indicated by the selected zoom area information supplied from the zoom area selecting unit 62 from the video (image) based on the video data supplied from the video data decoding unit 63 , and outputs the video data obtained as a result thereof to the video output device 53 .

[0259] Note that in the case where the selection zoom region information is information indicating that cropping is not to be performed, the video segmentation unit 64 does not perform cropping processing on the video data, and outputs the video data directly to the video output device 53 as zoomed video data.

[0260] The audio data decoding unit 65 decodes the encoded audio data supplied from the content data decoding unit 61 , and supplies audio data obtained as a result thereof to the audio conversion unit 66 .

[0261] The audio conversion unit 66 performs audio conversion processing on the audio data supplied from the audio data decoding unit 65 based on the selected zoom area information supplied from the zoom area selection unit 62 , and supplies zoomed audio data obtained as a result to the audio output device 54 .

[0262] Here, the audio conversion process is conversion for audio reproduction suitable for scaling the video of the content.

[0263] For example, according to the cropping process of the zoom area, that is, the division process of the zoom area, the distance from the object inside the video to the viewing point that becomes the reference changes. Therefore, for example, in the case where the audio data is object-based audio, the audio conversion unit 66 converts the position information of the object provided from the content data decoding unit 61 through the audio data decoding unit 65 into metadata based on the selected zoom area information. In other words, the audio conversion unit 66 moves the position of the object as the sound source based on the selected zoom area information, that is, changes the distance from the object.

[0264] Then, the audio conversion unit 66 performs rendering processing based on the audio data in which the position of the object has been moved, and supplies scaled audio data obtained as a result thereof to the audio output device 54 , thereby reproducing audio.

[0265] Note that such audio conversion processing is described in detail in, for example, PCT / JP2014 / 067508 or the like.

[0266] In addition, in the case where the selection zoom region information is information indicating that cropping is not to be performed, the audio conversion unit 66 does not perform audio conversion processing on the audio data, and directly outputs the audio data as zoomed audio data to the audio output device 54 .

[0267] <Description of Reproduction Processing>

[0268] Subsequently, the operation of the reproduction device 51 will be described.

[0269] When receiving the encoded content data output from the encoding device 11, the reproducing device 51 performs a reproducing process in which the received encoded content data is decoded, and reproduces the content. Fig.21 The flowchart shown is used to describe the reproduction processing performed by the reproduction device 51.

[0270] In step S41 , the content data decoding unit 61 separates the encoded video data, the encoded audio data, and the encoded metadata from the received encoded content data, and decodes the encoded metadata.

[0271] Then, the content data decoding unit 61 supplies the encoded video data to the video data decoding unit 63, and supplies the encoded audio data to the audio data decoding unit 65. In addition, the content data decoding unit 61 supplies metadata obtained by decoding to each unit of the reproduction device 51 as necessary.

[0272] At this time, the content data decoding unit 61 supplies the zoom area information obtained as metadata to the zoom area selection unit 62. In addition, in the case where zoom area auxiliary information as header information of metadata is stored in the encoded content data, the content data decoding unit 61 reads the zoom area auxiliary information and supplies the read zoom area auxiliary information to the zoom area selection unit 62. For example, as the zoom area auxiliary information, the above-mentioned supplementary information ZoomAreaCommentary[idx], ZoomAreaSpecifiedID[idx] as the specification ID, and the like are read.

[0273] In step S42, the zoom area selection unit 62 selects one piece of zoom area information from the zoom area information supplied from the content data decoding unit 61, and supplies the selected zoom area information to the video splitting unit 64 and the audio converting unit 66 according to the selection result.

[0274] For example, when zoom area information is selected, the zoom area selection unit 62 provides zoom area auxiliary information to the display device 52 for display on the display device 52 , and selects the zoom area information based on a signal provided by an operation input of a user who sees the display.

[0275] In addition, as described above, by using not only the zoom area auxiliary information and the operation input from the user but also the reproduction device information or the direction information, the zoom area information can be selected.

[0276] In step S43 , the video data decoding unit 63 decodes the encoded video data supplied from the content data decoding unit 61 , and supplies video data obtained as a result thereof to the video dividing unit 64 .

[0277] In step S44, the video dividing unit 64 divides (crops) the zoom area indicated by the selected zoom area information supplied from the zoom area selecting unit 62 for the video based on the video data supplied from the video data decoding unit 63. In this way, zoom video data for reproducing the video of the zoom area indicated by the selected zoom area information is obtained.

[0278] The video dividing unit 64 supplies the scaled video data obtained by the division to the video output device 53, thereby reproducing the video of the cropped content. The video output device 53 reproduces (displays) the video based on the scaled video data supplied from the video dividing unit 64.

[0279] In step S45 , the audio data decoding unit 65 decodes the encoded audio data supplied from the content data decoding unit 61 , and supplies the audio data obtained as a result thereof to the audio conversion unit 66 .

[0280] In step S46, the audio conversion unit 66 performs an audio conversion process on the audio data supplied from the audio data decoding unit 65 based on the selected zoom area information supplied from the zoom area selection unit 62. In addition, the audio conversion unit 66 supplies the scaled audio data obtained by the audio conversion process to the audio output device 54, thereby outputting the audio. The audio output device 54 reproduces the audio of the content on which the audio conversion process has been performed based on the scaled audio data supplied from the audio conversion unit 66, and the reproduction process ends.

[0281] Note that, more specifically, the processing of steps S43 and S44 and the processing of steps S45 and S46 are executed in parallel with each other.

[0282] As described above, the reproducing device 51 selects appropriate zoom area information, performs cropping of video data and audio conversion processing on audio data based on the selected zoom area information according to the result of the selection, and reproduces the content.

[0283] In this way, by selecting the zoom area information, content that is appropriately cropped and has converted audio, such as content that matches the user's preferences or content that is suitable for the size of the display screen of the reproduction device 51, the rotation direction of the reproduction device 51, etc., can be reproduced in a simplified manner. In addition, in the case where the user selects a zoom area based on the zoom area auxiliary information presented by the display device 52, the user can select a desired zoom area in a simplified manner.

[0284] Note that in reference Fig.21 In the described reproduction processing, although the case is described in which both the cropping of the video constituting the content and the audio conversion processing of the audio constituting the content are performed based on the selection zoom area information, only one of them may be performed.

[0285] In addition, also in the case where the content consists of only video or audio, cropping or audio conversion processing is performed on such video or audio, and the video or audio can be reproduced.

[0286] For example, also in the case where the content consists only of audio, by selecting zoom area information representing the area to be zoomed and changing the distance from the sound source object through audio conversion processing according to the selected zoom area information, it is possible to reproduce the content suitable for the user's preferences, reproduction equipment, etc.

[0287] <Second Embodiment>

[0288] <Example of Configuration of Reproduction Apparatus>

[0289] Note that while the above describes an example in which a zoom area is cut out from a video of content according to one piece of selected zoom area information by the video splitting unit 64, it may be configured such that a plurality of zoom areas are selected and such a plurality of zoom areas are output in a multi-screen arrangement.

[0290] In this case, for example, the reproduction device 51 is configured as follows Fig. 22 Note that Fig. 22 In the Fig. 20 The situation shown corresponds to the part, and its description will not be presented appropriately.

[0291] Fig. 22 The illustrated reproducing apparatus 51 includes: a content data decoding unit 61; a zoom region selecting unit 62; a video data decoding unit 63; a video dividing unit 64; a video arranging unit 91; an audio data decoding unit 65; and an audio converting unit 66.

[0292] Fig. 22 The configuration of the reproduction device 51 shown is similar to Fig. 20 The reproduction device 51 shown is different in that a video arrangement unit 91 is newly provided at the stage after the video division unit 64, and is the same as the reproduction device 51 in other respects. Fig. 20 The configuration of the reproduction device 51 shown is the same.

[0293] In this example, the zoom area selection unit 62 selects one or more pieces of zoom area information and supplies such zoom area information as selected zoom area information to the video segmentation unit 64. In addition, the zoom area selection unit 62 selects one piece of zoom area information and supplies the zoom area information to the audio conversion unit 66 as selected zoom area information.

[0294] Note that with Fig. 20 As in the case of the reproduction apparatus 51 shown, the selection of zoom area information performed by the zoom area selection unit 62 may be performed according to a user's input operation, or may be performed based on zoom area auxiliary information, reproduction device information, direction information, or the like.

[0295] In addition, the zoom area information as the selected zoom area information provided to the audio conversion unit 66 may be selected according to the input operation of the user, or may be zoom area information arranged at a predetermined position, such as the start position of the encoded content data. In addition to this, the zoom area information may be zoom area information having a representative zoom area, such as a zoom area having a maximum size.

[0296] The video dividing unit 64 crops the zoom area represented by each of the one or more pieces of selected zoom area information supplied from the zoom area selecting unit 62 from the video (image) based on the video data supplied from the video data decoding unit 63, thereby generating zoomed video data of each zoom area. In addition, the video dividing unit 64 supplies the zoomed video data of each zoom area obtained by the cropping to the video arranging unit 91.

[0297] Note that the video dividing unit 64 may directly supply the non-cropped video data to the video arranging unit 91 as a piece of scaled video data.

[0298] The video arrangement unit 91 generates multi-screen video data to be reproduced based on a video of such multi-screen video data arranged on a plurality of screens based on one or more pieces of scaled video data supplied from the video division unit 64, and supplies the generated multi-screen video data to the video output device 53. Here, the video reproduced based on the multi-screen video data is similar to, for example, Fig.18 The shown image Q14 is a video in which the videos (images) of the selected zoom areas are arranged to be aligned.

[0299] In addition, the audio conversion unit 66 performs an audio conversion process on the audio data supplied from the audio data decoding unit 65 based on the selected zoom area information supplied from the zoom area selection unit 62, and supplies the scaled audio data obtained as a result thereof as the audio data of the representative audio of the multi-screen arrangement to the audio output device 54. In addition, the audio conversion unit 66 may directly supply the audio data supplied from the audio data decoding unit 65 as the audio data of the representative audio (scaled audio data) to the audio output device 54.

[0300] <Description of Reproduction Processing>

[0301] Next, refer to Fig.23 The flowchart shown is used to describe the Fig. 22 The reproduction process is performed by the reproduction device 51 shown in FIG. Fig.21 The processing of step S41 shown is similar, so its description is omitted.

[0302] In step S72, the zoom area selection unit 62 selects one or more pieces of zoom area information from the zoom area information supplied from the content data decoding unit 61, and supplies the selected zoom area information to the video splitting unit 64 according to the selection result.

[0303] Note that the process of selecting zoom area information described here is basically similar to that described in the previous section except that the number of zoom area information selected is different. Fig.21 The process of step S42 is shown.

[0304] In addition, the zoom area selection unit 62 selects zoom area information of one representative zoom area from the zoom area information supplied from the content data decoding unit 61, and supplies the selected zoom area information according to the selection result to the audio conversion unit 66. Here, the selected zoom area information supplied to the audio conversion unit 66 is the same as one of the one or more pieces of selected zoom area information supplied to the video splitting unit 64.

[0305] When the zoom area information is selected, thereafter, the processing of steps S73 and S74 is performed, and decoding of the encoded video data and cropping of the zoom area from the video are performed. However, such processing is similar to Fig.21 However, in step S74, for each of the one or more selected zoom area information, cropping (splitting) of the zoom area represented by the selected zoom area information from the video based on the video data is performed, and the zoomed video data of each zoom area is provided to the video arrangement unit 91.

[0306] In step S75, the video arrangement unit 91 performs video arrangement processing based on one or more pieces of zoomed video data supplied from the video segmentation unit 64. In other words, the video arrangement unit 91 generates multi-screen video data based on one or more pieces of zoomed video data, and supplies the generated multi-screen video data to the video output device 53, thereby reproducing the video of each zoomed area of ​​the content. The video output device 53 reproduces (displays) the video arranged in a plurality of screens based on the multi-screen video data supplied from the video arrangement unit 91. For example, in the case where a plurality of zoomed areas are selected, the video is displayed in a manner similar to Fig.18 The image Q14 shows a multi-screen configuration for reproducing content.

[0307] When the video arrangement processing is performed, thereafter, the processing of steps S76 and S77 is performed, and the reproduction processing ends. However, this processing is similar to Fig.21 The processing of steps S45 and S46 is shown, and therefore the description thereof is omitted.

[0308] As described above, the reproducing device 51 selects one or more pieces of zoom area information, performs cropping of video data and audio conversion processing of audio data based on the selected zoom area information according to the result of the selection, and reproduces the content.

[0309] In this way, by selecting one or more pieces of zoom area information, appropriate content, such as content matching the user's preference or content suitable for the size of the display screen of the reproduction device 51, etc. can be reproduced in a simplified manner. In particular, in the case where multiple pieces of zoom area information are selected, content video can be reproduced in a multi-screen display matching the user's preference, etc.

[0310] In addition, in the case where the user selects a zoom area based on the zoom area auxiliary information presented by the display device 52, the user can select a desired zoom area in a simplified manner.

[0311] <Third Embodiment>

[0312] <Example of Configuration of Reproduction Apparatus>

[0313] In addition, in the case where the above content is transmitted via a network, the reproduction side device may be configured to effectively receive data necessary only for the reproduction of the selected zoom area. In this case, for example, the reproduction device is configured as follows Fig.24 Note that Fig.24 In the Fig. 20 The situation shown corresponds to the part, and its description will not be presented appropriately.

[0314] exist Fig.24 In the illustrated case, the reproduction device 121 that reproduces the content receives a supply of desired encoded video data and encoded audio data from the content data distribution server 122 in which the content and metadata are recorded. In other words, the content data distribution server 122 records the content and the metadata of the content in an encoded state or an unencoded state, and distributes the content in response to a request from the reproduction device 121.

[0315] In this example, the reproduction device 121 includes: a communication unit 131; a metadata decoding unit 132; a video / audio data decoding unit 133; a zoom area selection unit 62; a video data decoding unit 63; a video splitting unit 64; an audio data decoding unit 65; and an audio conversion unit 66.

[0316] The communication unit 131 transmits and receives various types of data to and from the content data distribution server 122 through the network.

[0317] For example, the communication unit 131 receives encoded metadata from the content data distribution server 122 and supplies the received encoded metadata to the metadata decoding unit 132, or receives encoded video data and encoded audio data from the content data distribution server 122 and supplies the received data to the video / audio data decoding unit 133. Furthermore, the communication unit 131 transmits the selected zoom area information supplied from the zoom area selection unit 62 to the content data distribution server 122.

[0318] The metadata decoding unit 132 obtains metadata by decoding the encoded metadata supplied from the communication unit 131 , and supplies the obtained metadata to each unit of the reproduction device 121 as necessary.

[0319] In addition, when the metadata includes zoom area information, the metadata decoding unit 132 supplies the zoom area information to the zoom area selection unit 62. In addition, when zoom area auxiliary information is received from the content data distribution server 122, the metadata decoding unit 132 supplies the zoom area auxiliary information to the zoom area selection unit 62.

[0320] When the encoded video data and the encoded audio data are supplied from the communication unit 131, the video / audio data decoding unit 133 supplies the encoded video data to the video data decoding unit 63, and supplies the encoded audio data to the audio data decoding unit 65,

[0321] <Description of Reproduction Processing>

[0322] Subsequently, the operation of the reproduction device 121 will be described.

[0323] The reproduction device 121 requests the content data distribution server 122 to transmit the encoded metadata. Then, when the encoded metadata is transmitted from the content data distribution server 122, the reproduction device 121 reproduces the content by performing the reproduction process. Fig.25 The flowchart shown is used to describe the reproduction processing performed by the reproduction device 121.

[0324] In step S101, the communication unit 131 receives the encoded metadata transmitted from the content data distribution server 122, and supplies the received metadata to the metadata decoding unit 132. Note that, more specifically, the communication unit 131 also receives header information of the metadata such as the zoom area number information and the zoom area auxiliary information from the content data distribution server 122 as needed, and supplies the received header information to the metadata decoding unit 132.

[0325] In step S102, the metadata decoding unit 132 decodes the encoded metadata supplied from the communication unit 131, and supplies the metadata obtained by the decoding to each unit of the reproduction device 121 as necessary. In addition, the metadata decoding unit 132 supplies the zoom area information obtained as the metadata to the zoom area selection unit 62, and in the case where the zoom area auxiliary information as the header information of the metadata exists, also supplies the zoom area auxiliary information to the zoom area selection unit 62.

[0326] In this way, in the case where the metadata is obtained, the zoom area information is subsequently selected by performing the process of step S103. However, the process of step S103 is similar to Fig.21 However, in step S103, the selected zoom area information obtained by selecting the zoom area information is supplied to the video dividing unit 64, the audio converting unit 66, and the communication unit 131.

[0327] In step S104 , the communication unit 131 transmits the selected zoom area information supplied from the zoom area selection unit 62 to the content data distribution server 122 through the network.

[0328] The content data distribution server 122 that has received the selected zoom area information performs cropping (division) of the zoom area indicated by the selected zoom area information on the video data of the recorded content, thereby generating zoomed video data. The zoomed video data obtained in this way is video data that reproduces only the zoom area indicated by the selected zoom area information in the entire video of the original content.

[0329] The content data distribution server 122 transmits the encoded video data obtained by encoding the scaled video data and the encoded audio data obtained by encoding the audio data, which constitute the content, to the reproduction device 121 .

[0330] Note that in the content data distribution server 122, zoom video data for each zoom area may be prepared in advance. In addition, in the content data distribution server 122, regarding audio data constituting the content, although all audio data is generally encoded and the encoded audio data is output regardless of the selected zoom area, it may be configured to output only the encoded audio data of a part of the audio data. For example, in the case where the audio data constituting the content is audio data of each object, only the audio data of the object in the zoom area indicated by the selected zoom area information may be encoded and sent to the reproduction device 121.

[0331] In step S105, the communication unit 131 receives the encoded video data and the encoded audio data transmitted from the content data distribution server 122, and supplies the encoded video data and the encoded audio data to the video / audio data decoding unit 133. In addition, the video / audio data decoding unit 133 supplies the encoded video data supplied from the communication unit 131 to the video data decoding unit 63, and supplies the encoded audio data supplied from the communication unit 131 to the audio data decoding unit 65.

[0332] When the encoded video data and the encoded audio data are obtained, thereafter, the processing of steps S106 to S109 is performed, and the reproduction processing ends. However, such processing is similar to Fig.21 The processing of steps S43 to S46 is shown, and therefore the description thereof is omitted.

[0333] However, since the signal obtained by decoding the encoded video data by the video data decoding unit 63 is scaled video data that has been cropped, basically no cropping process is performed by the video segmentation unit 64. Only when additional cropping is required, the video segmentation unit 64 crops the scaled video data supplied from the video data decoding unit 63 based on the selected zoom area information supplied from the zoom area selection unit 62.

[0334] In this way, when the content is reproduced based on the scaled video data and the scaled audio data by the video output device 53 and the audio output device 54, the content according to the selected scaled area is reproduced, for example, Fig.18 The content shown.

[0335] As described above, the reproducing device 121 selects appropriate zoom area information, transmits the selected zoom area information to the content data distribution server 122 according to the selection result, and receives the encoded video data and the encoded audio data.

[0336] In this way, by receiving the encoded video data and the encoded audio data according to the selected zoom area information, appropriate content can be reproduced in a simplified manner, such as content matching the user's preference or content suitable for the size of the display screen of the reproduction device 121, the rotation direction of the reproduction device 121, etc. In addition, only data that needs to be reproduced in the content can be obtained efficiently.

[0337] <Fourth Embodiment>

[0338] <Example of Configuration of Reproduction Apparatus>

[0339] In addition, the above describes an example in which zoom area information is included in the encoded content data. However, for example, the content may be cropped and reproduced based on zoom area information disclosed on a network such as the Internet or zoom area information recorded on a predetermined recording medium, that is, in a manner in which the zoom area information is separated from the content. In this case, for example, cropped reproduction may be performed by obtaining zoom area information generated not only by the content producer but also by a third party other than the content producer, that is, other users.

[0340] In this way, in the case where the content and the metadata including the zoom area information are separately obtained, for example, the reproduction apparatus is configured as follows. Fig.26 Note that Fig.26 In the Fig. 20 The situation shown corresponds to the part, and its description will not be presented appropriately.

[0341] Fig.26 The illustrated reproducing apparatus 161 includes: a metadata decoding unit 171; a content data decoding unit 172; a zoom region selecting unit 62; a video data decoding unit 63; a video dividing unit 64; an audio data decoding unit 65; and an audio converting unit 66.

[0342] The metadata decoding unit 171 obtains encoded metadata including metadata of the zoom area information from, for example, a device on a network, a recording medium connected to the reproduction device 161 , or the like, and decodes the obtained encoded metadata.

[0343] In addition, the metadata decoding unit 171 supplies metadata obtained by decoding the encoded metadata to each unit of the reproducing device 161 as needed, and supplies the zoom area information included in the metadata to the zoom area selection unit 62. Furthermore, the unit 171 obtains header information of the metadata such as zoom area auxiliary information together with the encoded metadata as needed, and supplies the obtained header information to the zoom area selection unit 62.

[0344] The content data decoding unit 172 obtains the encoded video data and the encoded audio data of the content, for example, from a device on the network, a recording medium connected to the reproduction device 161, etc. In addition, the content data decoding unit 172 supplies the obtained encoded video data to the video data decoding unit 63, and supplies the obtained encoded audio data to the audio data decoding unit 65. Note that in this example, the encoded video data and the encoded audio data, and the encoded metadata are obtained from mutually different devices, recording media, etc.

[0345] <Description of Reproduction Processing>

[0346] Subsequently, the operation of the reproduction device 161 will be described.

[0347] When the reproduction of the content is instructed, the reproduction device 161 performs a reproduction process of obtaining the encoded metadata and the encoded content, and reproduces the content. Fig. 27 The flowchart shown describes the reproduction processing performed by the reproduction device 161.

[0348] In step S131, the metadata decoding unit 171 obtains encoded metadata including zoom area information from, for example, a device on a network, a recording medium connected to the reproducing device 161, etc. Note that the encoded metadata may be obtained in advance before the reproducing process starts.

[0349] In step S132, the metadata decoding unit 171 decodes the obtained encoded metadata, and supplies the metadata obtained as a result thereof as necessary to each unit of the reproducing device 161. In addition, the metadata decoding unit 171 supplies the zoom area information included in the metadata to the zoom area selection unit 62, and also supplies the header information of the metadata such as zoom area auxiliary information to the zoom area selection unit 62 as necessary.

[0350] When metadata is obtained by decoding, the process of step S133 is performed, and the zoom area information is selected. However, the process of step S133 is similar to Fig.21 The processing of step S42 is shown, and therefore its description is omitted.

[0351] In step S134, the content data decoding unit 172 obtains the encoded video data and the encoded audio data of the content from, for example, a device on the network, a recording medium connected to the reproduction device 161, etc. In addition, the content data decoding unit 172 supplies the obtained encoded video data to the video data decoding unit 63, and supplies the obtained encoded audio data to the audio data decoding unit 65.

[0352] In this way, when the encoded video data and the encoded audio data of the content are obtained, thereafter, the processing of steps S135 to S138 is performed, and the reproduction processing ends. However, such processing is similar to Fig.21 The processing of steps S43 to S46 is shown, and therefore the description thereof is omitted.

[0353] As above, the reproduction device 161 separately obtains the encoded video data and the encoded audio data of the content and the encoded metadata including the zoom area information. Then, the reproduction device 161 selects appropriate zoom area information, and performs cropping of the video data and audio conversion processing of the audio data based on the selected zoom area information according to the result of the selection, and reproduces the content.

[0354] In this way, by separately obtaining the encoded metadata including the zoom area information from the encoded video data and the encoded audio data, it is possible to crop and reproduce the zoom area set not only by the content producer but also by other users and the like.

[0355] Meanwhile, the above series of processing can be performed by hardware or software. In the case of performing a series of processing by software, a program configuring the software is installed to a computer. Here, the computer includes a computer built in dedicated hardware, such as a general-purpose personal computer capable of performing various functions by installing various programs thereto, etc.

[0356] Fig.28 : is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by using a program.

[0357] In the computer, a central processing unit (CPU) 501 , a read only memory (ROM) 502 , and a random access memory (RAM) 503 are interconnected by a bus 504 .

[0358] In addition, an input / output interface 505 is connected to the bus 504 . An input unit 506 , an output unit 507 , a recording unit 508 , a communication unit 509 , and a drive 510 are connected to the input / output interface 505 .

[0359] The input unit 506 is configured by a keyboard, a mouse, a microphone, an imaging device, etc. The output unit 507 is configured by a display, a speaker, etc. The recording unit 508 is configured by a hard disk, a nonvolatile memory, etc. The communication unit 509 is configured by a network interface, etc. The drive 510 drives a removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0360] In the computer configured as above, the CPU 501 loads the program recorded in the recording unit 508 into the RAM 503 through the input / output interface 505 and the bus 504, for example, and executes the loaded program, thereby performing a series of processes.

[0361] For example, the program executed by the computer (CPU 501) can be provided as a package medium or the like in the form of being recorded on the removable medium 511. In addition, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0362] In the computer, by loading the removable medium 511 into the drive 510, the program can be installed to the recording unit 508 through the input / output interface 505. In addition, the program can be received by the communication unit 509 through a wired or wireless transmission medium and installed to the recording unit 508. In addition, the program can also be pre-installed to the ROM 502 or the recording unit 508.

[0363] Note that the program executed by the computer may be a program that executes processing in time series according to the order described in this specification, or a program that executes processing in a parallel manner or at necessary timing such as when called.

[0364] In addition, the embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made within the scope not departing from the concept of the present technology.

[0365] For example, the present technology may adopt a configuration of cloud computing in which one function is shared by a plurality of devices through a network and is processed together by all the devices.

[0366] In addition, each step described in the above flowchart can be performed not only by one device but also by a plurality of devices in a shared manner.

[0367] Furthermore, in the case where a plurality of processes are included in one step, the plurality of processes included in one step may be executed not only by one device but also by a plurality of devices in a shared manner.

[0368] Additionally, the present technology can adopt the following configurations. [1]

[0370] A reproduction device, comprising:

[0371] a decoding unit, the decoding unit decoding the encoded video data or the encoded audio data;

[0372] a zoom area selection unit, the zoom area selection unit selecting one or more zoom area information from a plurality of zoom area information specifying an area to be zoomed; and

[0373] A data processing unit performs a cropping process on the video data obtained by decoding or an audio conversion process on the audio data obtained by decoding based on the selected scaling region information. [2]

[0375] The reproducing apparatus according to [1], wherein the plurality of pieces of zoom area information include zoom area information that specifies the area for each type of the reproduction target device. [3]

[0377] The reproducing apparatus according to [1] or [2], wherein the plurality of pieces of zoom area information include zoom area information that specifies the area for each rotation direction of the reproduction target device. [4]

[0379] The reproduction device according to any one of [1] to [3], wherein the plurality of pieces of zoom area information include zoom area information that specifies an area for each specific video object. [5]

[0381] The reproduction device according to any one of [1] to [4], wherein the zoom area selection unit selects the zoom area information according to an operation input by a user. [6]

[0383] The reproduction device according to any one of [1] to [4], wherein the zoom area selection unit selects the zoom area information based on information related to the reproduction device. [7]

[0385] The reproduction device according to [6], wherein the zoom area selection unit selects the zoom area information by using at least any one of information indicating a type of the reproduction device and information indicating a rotation direction of the reproduction device as the information related to the reproduction device. [8]

[0387] A reproduction method comprises the following steps:

[0388] Decoding the encoded video data or the encoded audio data;

[0389] Selecting one or more pieces of zoom area information from a plurality of pieces of zoom area information that specify an area to be zoomed; and

[0390] Based on the selected zoom region information, a cropping process is performed on the video data obtained by decoding or an audio conversion process is performed on the audio data obtained by decoding. [9]

[0392] A program causing a computer to execute a process comprising the following steps:

[0393] Decoding the encoded video data or the encoded audio data;

[0394] Selecting one or more pieces of zoom area information from a plurality of pieces of zoom area information that specify an area to be zoomed; and

[0395] Based on the selected zoom region information, a cropping process is performed on the video data obtained by decoding or an audio conversion process is performed on the audio data obtained by decoding.

[10]

[0397] A coding device, comprising:

[0398] an encoding unit that encodes video data or encodes audio data; and

[0399] A multiplexer generates a bit stream by multiplexing the encoded video data or the encoded audio data with a plurality of pieces of scaling region information specifying a region to be scaled.

[11]

[0401] A coding method comprises the following steps:

[0402] encoding video data or encoding audio data; and

[0403] A bit stream is generated by multiplexing the encoded video data or the encoded audio data with a plurality of pieces of scaling region information specifying a region to be scaled.

[12]

[0405] A program causing a computer to execute a process comprising the following steps:

[0406] encoding video data or encoding audio data; and

[0407] A bit stream is generated by multiplexing the encoded video data or the encoded audio data with a plurality of pieces of scaling region information specifying a region to be scaled.

[0408] Reference numerals list

[0409] 11 Encoding device

[0410] 21 Video data encoding unit

[0411] 22 Audio data encoding unit

[0412] 23 Metadata encoding unit

[0413] 24 Multiplexer

[0414] 25 Output Units

[0415] 51 Reproduction Device

[0416] 61 Content data decoding unit

[0417] 62 Zoom area selection unit

[0418] 63 Video data decoding unit

[0419] 64 Video Segmentation Unit

[0420] 65 Audio data decoding unit

[0421] 66 Audio conversion unit

Claims

1. A reproduction device, comprising: a decoding unit, the decoding unit decoding the encoded content data to obtain: metadata; and video data or audio data; a zoom area selection unit, the zoom area selection unit selecting one or more zoom area information from a plurality of zoom area information pieces specifying an area to be zoomed included in the metadata; as well as a data processing unit that performs a cropping process on the video data obtained by decoding, or performs an audio conversion process on the audio data obtained by decoding, based on the selected scaling area information, wherein the plurality of pieces of zoom area information include zoom area information that specifies an area for each type of reproduction target device; and The zoom area selection unit selects the zoom area information based on information related to the type of the reproduction device.

2. A reproduction method, comprising: decoding the encoded content data to obtain: metadata; and video data or audio data; selecting one or more pieces of zoom area information from a plurality of pieces of zoom area information specifying an area to be zoomed included in the metadata; as well as performing cropping processing on the video data obtained by decoding, or performing audio conversion processing on the audio data obtained by decoding based on the selected scaling area information, wherein the plurality of pieces of zoom area information include zoom area information that specifies an area for each type of reproduction target device; and The zoom area information is selected based on information related to the type of the reproduction device.

Images