File generation device, file generation method, playback device, and playback processing method

Abstract

Provided are an information processing device, an information processing method, a reproduction processing device, and a reproduction processing method, which provide a user with a high-quality audiovisual experience. A metadata generation unit generates transition identification information for identifying a second viewpoint which is the viewpoint position in a second image which can be transitioned to from a first image, the viewpoint position of the first　image being a first viewpoint. A file generation unit generates a file that includes data of the first image and the transition identification information.

Description

[Technical field]

[0001] The present invention relates to File Generation Device, File Generation The present invention relates to a method, a reproduction processing device, and a reproduction processing method. [Background technology]

[0002] MPEG-DASH (Moving Picture Experts Group - Dynamic Adaptive Streaming over HTTP) exists as a standard for adaptive content delivery technology using HTTP (Hypertext Transfer Protocol). In addition, ISOBMFF (International Organization for Standardization Base Media File Format), which is a file container specification for the international standard video compression technology "MPEG-4", is provided as the file format for MPEG-DASH.

[0003] Incidentally, there is a type of video that reproduces a spherical image by mapping a three-dimensional image, which is an image projected onto a three-dimensional structure with a horizontal circumference of 360 degrees and a vertical period of 180 degrees, as a flat image, such as a so-called spherical image. A spherical image is also called a 3DoF (Degrees of Freedom) image, and a spherical image is also called a projected flat image or a 3DoF image. In MPEG-I OMAF (Omnidirectional Media Format), the use of MPEG-DASH for the distribution of three-dimensional images that form a spherical image is being considered.

[0004] Furthermore, the MPEG-I Phase 1b requirement proposes a technology called Multiple Viewpoint for delivering spherical images from multiple viewpoints. By using this Multiple Viewpoint, it becomes possible to provide content to users by switching from a spherical image from one viewpoint to a spherical image from a different viewpoint. A viewpoint is the viewpoint position in a spherical image. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] “ISO / IEC” 14496-12:2015 Information technology. Coding of audio-visual objects. Part 12:ISO base media file format, 2015-12 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the client device does not have information indicating the intention of the producer, such as which other celestial sphere video a client device can transition to from the celestial sphere video currently being viewed. In a distribution system in which the client device does not have information indicating celestial sphere videos to which transitions can be made, it is difficult to provide a viewing experience including transitions to celestial sphere videos to which transitions can be made, and there is a risk of impairing the quality of the viewing experience of the viewer.

[0007] In addition, since the functionality for achieving video transitions depends on the type of functionality implemented in the client device, the creator's intent may not be properly reflected when the content is played back, which may result in a loss of quality in the user's viewing experience.

[0008] Therefore, the present disclosure provides a high-quality viewing experience to users. File Generation Device, File GenerationA method, a recycling device and a recycling method are provided. [Means for solving the problem]

[0009] According to the present disclosure, a metadata generation unit generates transition identification information for identifying a second viewpoint that is a viewpoint position in a second video to which a transition can be made from a first video having a first viewpoint as a viewpoint position, and transition execution region information indicating a transition execution region on the first video that accepts an operation for transitioning from the first viewpoint to the second viewpoint. A file generation unit generates a file including data of the first video, the transition identification information, and the transition execution region information. The file generation unit stores the transition execution region information in a ViewpointTrackGroupBox in a TrackBox of an ISOBMFF file. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a system configuration diagram of an example of a distribution system. [Diagram 2] FIG. 2 is a block diagram of a file generating device. [Diagram 3] FIG. 13 is a diagram for explaining storage of transition identification information. [Figure 4] FIG. 2 is a block diagram of a client device. [Diagram 5] 13 is a flowchart of a file generation process performed by the file generation device. [Figure 6] 13 is a flowchart of a metadata creation process. [Figure 7] 11 is a flowchart of a reproduction process executed by a client device. [Figure 8] 13 is a flowchart of a viewpoint transition process. [Figure 9] FIG. 13 is a diagram showing an example of an extended VWPT descriptor. [Figure 10] 13 is a diagram showing an example of an MPD file in which ViewpintInfo.SwitchableContent is included in the VWPT descriptor. [Figure 11] 13 is a diagram showing an example of storage of transition identification information in which a transition destination is a viewpoint group; FIG. [Figure 12] FIG. 13 is a diagram showing an example of SphereRegionStruct() and RegionForViewpoint(). [Figure 13] FIG. 13 is a diagram showing the control structure of an overlay. [Figure 14] FIG. 13 is a diagram showing an example of SphereRelativeOmniOverlay() and AssociatiedSphereRegion(). [Figure 15] FIG. 13 is a diagram illustrating an example of overlay_control_struct() to which a new control structure for storing transition execution region information is added. [Figure 16] FIG. 13 is a diagram illustrating an example of Associatedviewpointregion(). [Figure 17] FIG. 13 is a diagram illustrating an example of an overlay_control_struct() to which a new control structure for storing transition identification information is added. [Figure 18] FIG. 13 is a diagram showing an example of Associatedviewpointregion() and AssociatedViewpoint(). [Figure 19] 13 is a diagram showing an example of AssociatedSphereRegion() when the video display area is used as the transition execution area. FIG. [Figure 20] 13 is a diagram illustrating an example of RegionForViewpoint() when transition execution region information is displayed after a certain time period. FIG. [Figure 21] FIG. 13 is a diagram illustrating an example of RegionForViewpoint() when switching user interactions. [Figure 22] This is a diagram showing the format of Matroska Media Container. [Figure 23] FIG. 2 is a diagram illustrating the hardware configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings. In each of the following embodiments, the same parts are denoted by the same reference numerals to avoid repetitive description. The scope of the disclosure of the present technology is not limited to the contents of the embodiments, but also includes the contents described in the following non-patent documents that were publicly known at the time of filing the application.

[0012] Non-patent document 1: (mentioned above) Non-patent document 2: ISO / IEC 23090-2:2019 Information technology. Coded representation of immersive media. Part 2: Omnidirectional media format, 2019-01 Non-patent document 3: N18393, WD5 of ISO / IEC 23090-2 OMAF 2nd edition, 2019-05-16 Non-Patent Document 4: m47385, [OMAF] Multiviewpoint switching transitions, 2019-03 Non-Patent Document 5: "Matoska Media Container", [Retrieved March 12, 2020], Internet<URL : https: / / www.matroscka.org / >

[0013] The contents described in the above non-patent documents are also incorporated by reference into the present embodiment. In other words, the contents described in the above non-patent documents are also the basis for determining the support requirements. For example, even if the file structures described in non-patent documents 1 to 5 are not directly defined in the detailed description of the invention, they are within the scope of this disclosure and meet the support requirements of the claims. Similarly, for example, technical terms such as parsing, syntax, and semantics are also within the scope of this disclosure and meet the support requirements of the claims, even if they are not directly defined in the detailed description of the invention.

[0014] The present disclosure will be described in the following order. 1. First embodiment 1.1 Modification of the first embodiment (1) 1.2 Modification of the first embodiment (2) 2. Second embodiment 2.1 Modification of the second embodiment (1) 2.2 Modification of the second embodiment (2) 2.3 Modification of the second embodiment (3) 2.3 Modification of the second embodiment (4) 3. Third embodiment 4. Fourth embodiment

[0015] [1. First embodiment] (Configuration of distribution system according to the first embodiment) Fig. 1 is a system configuration diagram of an example of a distribution system. The distribution system 100 includes a file generation device 1 which is an information processing device, a client device 2 which is a playback processing device, and a Web server 3. The file generation device 1, the client device 2, and the Web server 3 are connected to a network 4. The file generation device 1, the client device 2, and the Web server 3 can communicate with each other via the network 4. Here, while Fig. 1 shows one of each device, the distribution system 100 may include multiple file generation devices 1 and multiple client devices 2.

[0016] The file generation device 1 generates 3DoF content, which is data for providing 3DoF video. The file generation device 1 uploads the generated 3DoF content to the Web server 3. Here, in this embodiment, a configuration in which the Web server 3 provides the 3DoF content to the client device 2 will be described, but the distribution system 100 can also adopt other configurations. For example, the file generation device 1 may be configured to include the function of the Web server 3, store the generated 3DoF content in its own device, and provide it to the client device 2.

[0017] The Web server 3 holds the 3DoF content uploaded from the file generation device 1. Then, the Web server 3 provides the specified 3DoF content in response to a request from the client device 2.

[0018] The client device 2 transmits a transmission request for 3DoF content to the Web server 3. Then, the client device 2 acquires the 3DoF content specified in the transmission request from the Web server 3. Then, the client device 2 decodes the 3DoF content to generate an image, and displays the image on a display device such as a monitor.

[0019] (Configuration of the file generation device according to the first embodiment) Next, details of the file generation device 1 will be described. Fig. 2 is a block diagram of the file generation device. As shown in Fig. 2, the file generation device 1, which is an information processing device, has a file generation processing unit 10, a control unit 11, and a communication unit 12. The control unit 11 executes processes related to the control of the file generation processing unit 10. For example, the control unit 11 performs overall control of the operation timing of each unit of the file generation processing unit 10. The file generation processing unit 10 has a data acquisition unit 101, an encoding unit 102, a metadata generation unit 103, and a file generation unit 104.

[0020] The data acquisition unit 101 accepts input of original data of 3DoF content that displays 3DoF video. The original data of the 3DoF content includes image data for generating a 3DoF image and control information for generating metadata. The control information includes, for example, time information of each image data, viewpoint position information of each image data, and information of image data to which each image data can be transitioned. The control information also includes information indicating a viewpoint corresponding to each image data, position information of each viewpoint, and information indicating a correspondence with a viewpoint to which a transition can be made. The data acquisition unit 101 outputs image data included in the original data of the acquired 3DoF content to the encoding unit 102. The data acquisition unit 101 also outputs control information included in the original data of the acquired 3DoF content to the metadata generation unit 103.

[0021] The encoding unit 102 receives input of image data from the data acquisition unit 101. Then, the encoding unit 102 performs encoding processing on the image data to generate 3DoF image data. Then, the encoding unit 102 outputs the generated 3DoF image data to the file generation unit 104.

[0022] In the following, the viewpoint position may be called a viewpoint. Furthermore, when emphasizing that the 3DoF image data is an image seen from various viewpoint positions, it may be called "viewpoint image data", and the omnidirectional image displayed when the viewpoint image data is played back may be called a "viewpoint image".

[0023] The metadata generating unit 103 receives control information as an input from the data acquiring unit 101. Next, the metadata generating unit 103 generates metadata based on the control information. The metadata includes control information related to image generation and playback, such as the codec to be used for compression. Furthermore, the metadata generating unit 103 generates the following information as metadata:

[0024] The metadata generating unit 103 generates transition identification information for identifying a viewpoint to which a transition can be made from each viewpoint, from the acquired control information. Specifically, the metadata generating unit 103 assigns identification information, viewpoint_id, to each viewpoint. Furthermore, the metadata generating unit 103 associates identification information of a transitionable viewpoint with identification information of a transition source viewpoint. By using this transition identification information, the client device 2 can identify a transitionable viewpoint from a specific viewpoint, and can realize a transition to a transitionable viewpoint. Then, the metadata generating unit 103 associates transition identification information, which designates the viewpoint as a transition source, with viewpoint image data at each viewpoint.

[0025] Furthermore, the metadata generating unit 103 calculates the display position of the transition execution area corresponding to the transition destination viewpoint in the transition source viewpoint image from the viewpoint position information included in the control information. Next, the metadata generating unit 103 generates transition execution area information representing the transition execution area on each viewpoint image using the calculated display position of the transition execution area.

[0026] Next, the metadata generating unit 103 generates transition trigger information indicating an operation that triggers execution of a transition specified by the control information. After that, the metadata generating unit 103 associates transition execution region information and transition trigger information corresponding to each viewpoint image data. In this way, the metadata generating unit 103 generates 3DoF image data indicating a transitionable viewpoint by associating the transition identification information and the transition execution region information with the corresponding viewpoint image data.

[0027] Thereafter, the metadata generating unit 103 outputs the metadata including the transition identification information, the transition execution area information, and the transition trigger information to the file generating unit 104.

[0028] The file generating unit 104 receives an input of 3DoF image data from the encoding unit 102. The file generating unit 104 also receives an input of metadata including transition identification information, transition execution area information, and transition trigger information from the metadata generating unit 103. The file generating unit 104 then stores the acquired 3DoF image data in an ISOBMFF file for each segment, thereby creating a file and generating a segment file of the 3DoF image data. The file generating unit 104 also stores various information included in the metadata, such as transition identification information, transition execution area information, and transition trigger information, in each ISOBMFF file. Each TrackBox in the IOSBMFF file includes information about one viewpoint image. In the following, each TrackBox in the IOSBMFF file may simply be referred to as a track. For example, each track in the IOSBMFF file includes a link to a specific viewpoint image data.

[0029] Here, a description will be given of the storage of transition identification information in an ISOBMFF file by the file generation unit 104. In order to store the transition identification information, the file generation unit 104 generates a ViewpointTrackGroupBox shown in syntax 111 in Fig. 3. Fig. 3 is a diagram for explaining the storage of transition identification information.

[0030] A ViewpointTrackGroupBox is a box used when using OMAF's Multiple Viewpoint. A ViewpointTrackGroupBox is a box included in the TrackBox of an ISOBMFF file, and stores information about the viewpoint of the image included in the TrackBox, position information, group information, coordinate rotation information, etc. In other words, a ViewpointTrackGroupBox corresponds to one viewpoint image data.

[0031] The signed viewpoint_lavel in the syntax 112 represents information about the viewpoint of the video included in the track. The viewpoint information is, for example, viewpoint_id, which is identification information of the viewpoint, and coincides with track_group_id. Also, ViewPosStruct() represents position information of the viewpoint.

[0032] ViewpointGroupStruct() is information for grouping viewpoints, and has VWPT_group_id and VWPT_group_description. A viewpoint group is, for example, a group of viewpoints in a building. The same coordinates are used within a group.

[0033] ViewpointGlobalCoordinateSysRotationStruct() has rotation information in the global coordinate system of the coordinate system referenced by the viewpoint. ViewpointGlobalCoordinateSysRotationStruct() depends on the settings at the time of shooting. ViewpointGlobalCoordinateSysRotationStruct() can be used to express the relationship between two viewpoints.

[0034] Furthermore, the file generation unit 104 according to this embodiment extends ViewpointTrackGroupBox as shown in syntax 111, and stores SwitchableContentStruct() representing transition identification information in ViewpointTrackGroupBox.

[0035] The file generation unit 104 newly defines SwitchableContentStruct() indicated by syntax 112. The file generation unit 104 expresses transition identification information for identifying a viewpoint to which a transition can be made from a viewpoint image that is a omnidirectional video image at a certain viewpoint, using SwitchableContentStruct(). In this manner, by storing SwitchableContentStruct() that is the transition identification information in a ViewpointTrackGroupBox, transition period information corresponding to specific viewpoint image data is stored in an ISOBMFF file that stores the viewpoint image data.

[0036] In this embodiment, the file generator 104 stores the SwitchableContentStruct(), which is the transition identification information, in the ViewpointTrackGroupBox, but this is not limiting, and it is also possible to store the SwitchableContentStruct() in the TrackGroupTypeBox(). Furthermore, the file generator 104 stores the transition execution region information and transition trigger information corresponding to a particular viewpoint image data in an ISOBMFF file that stores the viewpoint image data. In this way, the file generator 104 completes the segment file of the 3DoF content.

[0037] Furthermore, the file generating unit 104 generates an MPD (Media Presentation Description) file based on the metadata acquired from the metadata generating unit 103. The MPD file stores meta information of the 3DoF content, such as the media type and information on video and audio segment files. The file generating unit 104 then outputs the segment files and MPD file of the generated 3DoF content to the communication unit 12.

[0038] The communication unit 12 acquires the segment files and MPD files of the 3DoF content in which the 3DoF image data, metadata, transition identification information, transition execution area information, and transition trigger information are stored, from the file generation unit 104. Then, the communication unit 12 transmits the acquired segment files and MPD files of the 3DoF content to the Web server 3 and uploads them.

[0039] (Configuration of a client device according to the first embodiment) Fig. 4 is a block diagram of a client device. As shown in Fig. 4, the client device 2 has a playback processing unit 20, a control unit 21, and a communication unit 22. The control unit 21 controls the operation of each unit of the playback processing unit 20. For example, the control unit 21 comprehensively controls the timing of the operation of each unit of the playback processing unit 20. The playback processing unit 20 has a file acquisition unit 201, a measurement unit 202, a file processing unit 203, a decoding processing unit 204, a display information generation unit 205, and a display unit 206.

[0040] The communication unit 22 acquires the segment files and MPD file of the 3DoF content from the Web server 3. Then, the communication unit 22 outputs the acquired segment files and MPD file of the 3DoF content to the file acquisition unit 201.

[0041] The file acquisition unit 201 acquires an MPD file corresponding to the 3DoF content to be played from the Web server 3 via the communication unit 22. The MPD file includes information on the storage destination of the 3DoF content data and information on video playback such as the encoding speed and image size. The file acquisition unit 201 then outputs the acquired MPD file to the file processing unit 203. After that, the file acquisition unit 201 receives an input of information on the storage destination of the 3DoF content data to be played from the file processing unit 203.

[0042] The file acquisition unit 201 uses the acquired information on the storage destination of the 3DoF content data to make an acquisition request for the 3DoF content data to the Web server 3. Then, the file acquisition unit 201 acquires the segment files of the 3DoF content. After that, the file acquisition unit 201 outputs the segment files of the 3DoF content to the file processing unit 203.

[0043] The measurement unit 202 measures the transmission bandwidth of the transmission path between the client device 2 and the WEB server. Then, the measurement unit 202 outputs the measurement result of the transmission bandwidth to the file processing unit 203. The measurement unit 202 also receives input of an operation by an operator, such as moving the viewpoint. For example, when a transition execution area is displayed in a 3DoF image displayed on a monitor, the operator performs an operation such as clicking on the transition execution area. Then, the measurement unit 202 outputs the user's operation to the file processing unit 203.

[0044] The file processing unit 203 receives an input of an MPD file corresponding to the 3DoF content to be played from the file acquisition unit 201. Then, the file processing unit 203 parses the acquired MPD file to acquire information indicating the data of the 3DoF content to be played. The file processing unit 203 also recognizes a plurality of data used for adaptive distribution. For example, in the case of adaptive distribution in which the bit rate is switched, the file processing unit 203 acquires information of 3DoF image data corresponding to each bit rate. After that, the file processing unit 203 selects the 3DoF content to be played and outputs information of the selected 3DoF content to the file acquisition unit 201.

[0045] The file processing unit 203 receives an input of a segment file in which data of the 3DoF content to be played is stored from the file acquisition unit 201. The file processing unit 203 parses the acquired segment file. Then, the file processing unit 203 acquires 3DoF image data, metadata, transition identification information, transition execution area information, and transition trigger information.

[0046] The file processing unit 203 also receives an input of the measurement result of the transmission bandwidth from the measurement unit 202. Then, the file processing unit 203 selects 3DoF image data to be played back based on the parsing result, information indicating the transmission bandwidth acquired from the measurement unit 202, and the like. Then, the file processing unit 203 outputs the selected 3DoF image data to the decoding processing unit 204. Furthermore, the file processing unit 203 outputs the acquired metadata and transition execution region information to the display information generation unit 205.

[0047] The file processing unit 203 also receives input of an operation by the operator from the measurement unit 202. Then, the file processing unit 203 uses transition execution area information to determine whether the input operation is an operation for a transition execution area that accepts operations to transition the viewpoint. Furthermore, the file processing unit 203 determines from the transition trigger information whether the operation by the operator is an operation that becomes a transition trigger that causes a transition of the viewpoint.

[0048] When the operation by the operator is an operation that triggers a transition to the transition execution region, the file processing unit 203 acquires a viewpoint of a transition destination when a transition corresponding to the input operation is performed from the acquired transition identification information. Then, the file processing unit 203 specifies 3DoF content according to the acquired viewpoint.

[0049] If the identified 3DoF content has not been acquired, the file processing unit 203 outputs information on the identified 3DoF content to the file acquisition unit 201 and requests acquisition of the identified 3DoF content. After that, when the file processing unit 203 acquires a segment file of the identified 3DoF content, it executes parsing to acquire 3DoF image data, metadata, and transition identification information of the 3DoF content. Then, the file processing unit 203 outputs the acquired 3DoF image data to the decoding processing unit 204. In addition, the file processing unit 203 outputs metadata to the display information generation unit 205 to instruct switching of the 3DoF content.

[0050] The decoding processing unit 204 receives the input of the 3DoF image data from the file processing unit 203. Then, the decoding processing unit 204 performs a decoding process on the acquired 3DoF image data. After that, the decoding processing unit 204 outputs the decoded 3DoF image data to the display information generation unit 205.

[0051] The display information generating unit 205 receives an input of the decoded 3DoF image data from the decoding processing unit 204. The display information generating unit 205 also receives an input of the metadata and transition execution area information from the file processing unit 203. Then, the display information generating unit 205 generates a display image from the 3DoF image data using information on the viewpoint position, viewpoint direction, and time specified in the metadata. Furthermore, the display information generating unit 205 sets the area specified by the transition execution area information of the display image as the transition execution area. Thereafter, the display information generating unit 205 provides the generated display image to the display unit 206 to display it.

[0052] Further, the display information generating unit 205 may receive an instruction to switch 3DoF content from the file processing unit 203 while displaying an image for display generated from 3DoF image data on the display unit 206. In this case, the display information generating unit 205 acquires 3DoF image data from the 3DoF content specified by the file processing unit 203. Then, the display information generating unit 205 causes the display unit 206 to display the image for display generated from the acquired 3DoF image data, thereby switching the viewpoint. In this case as well, the display information generating unit 205 provides a transition execution area in the image for display after the transition of the viewpoint.

[0053] The display unit 206 has a display device such as a monitor. The display unit 206 receives an input of a display image generated by the display information generating unit 205. Then, the display unit 206 displays the acquired display image on the display device.

[0054] (File Generation Procedure According to the First Embodiment) Next, the flow of the file generation process by the file generation device 1 will be described in detail with reference to Fig. 5. Fig. 5 is a flowchart of the file generation process by the file generation device.

[0055] The data acquisition unit 101 acquires original data of the 3DoF content. This original data includes image data of images seen from a plurality of viewpoints and control information. Then, the data acquisition unit 101 outputs the control information included in the acquired original data to the metadata generation unit 103. The metadata generation unit 103 uses the control information to generate metadata including transition identification information, transition execution region information, and transition trigger information (step S101).

[0056] Furthermore, the data acquisition unit 101 outputs image data included in the acquired original data to the encoding unit 102. The encoding unit 102 uses the image data to generate a plurality of viewpoint image data, which are data of images seen from a plurality of viewpoints (step S102).

[0057] Next, the metadata generation unit 103 associates transition identification information, transition execution area information, and transition trigger information with each viewpoint image data generated by the encoding unit 102, and generates viewpoint image data indicating viewpoints to which transitions can be made (step S103).

[0058] The file generation unit 104 acquires the encoded viewpoint image data from the encoding unit 102. Furthermore, the file generation unit 104 acquires metadata including information associating transition identification information, transition execution area information, and transition trigger information with each viewpoint image data from the metadata generation unit 103. Then, the file generation unit 104 stores the 3DoF image data including multiple viewpoint image data in an ISOBMFF file for each segment. Furthermore, the file generation unit 104 stores the transition identification information, transition execution area information, and transition trigger information corresponding to each viewpoint image data in the ISOBMFF file to generate a segment file of the 3DoF content (step S104).

[0059] Next, the file generation unit 104 outputs the generated segment file of the 3DoF content to the communication unit 12. The communication unit 12 outputs the segment file of the 3DoF content generated by the file generation unit 104 to the Web server 3 (step S105).

[0060] Next, the metadata creation process will be described in detail with reference to Fig. 6. Fig. 6 is a flowchart of the metadata creation process. The process shown in the flowchart of Fig. 6 corresponds to an example of the process executed in step S101 in Fig. 5.

[0061] The metadata generating unit 103 assigns identification information to each of the viewpoints notified in the control information acquired from the data acquiring unit 101. Next, the metadata generating unit 103 generates transition identification information indicating a transitionable viewpoint from the viewpoint serving as the transition source, for each viewpoint, using the identification information, based on the correspondence relationship of the transitionable viewpoints included in the control information (step S111).

[0062] Next, the metadata generating unit 103 calculates the display position of the transition execution area corresponding to the transition destination viewpoint in the transition source viewpoint image, from the viewpoint position information included in the control information (step S112).

[0063] Next, the metadata generating unit 103 generates transition execution area information indicating the transition execution area on each viewpoint image, using the calculated display position of the transition execution area (step S113).

[0064] Next, the metadata generating unit 103 generates transition trigger information indicating an operation that triggers execution of the transition specified by the control information (step S114). After that, the metadata generating unit 103 generates control information related to image generation and playback, such as the codec to be used, and finishes creating metadata including the transition identification information, transition execution area information, and transition trigger information.

[0065] (Reproduction Processing Procedure According to the First Embodiment) Next, the flow of the reproduction process executed by the client device 2 will be described with reference to Fig. 7. Fig. 7 is a flowchart of the reproduction process executed by the client device.

[0066] The file acquisition unit 201 acquires an MPD file corresponding to the 3DoF content to be played from the Web server 3 via the communication unit 22. Next, the file processing unit 203 parses the MPD file and executes an analysis process. Then, the file acquisition unit 201 acquires a segment file of the 3DoF content to be played, which is specified by the file processing unit 203 from the analysis result, from the Web server 3 (step S201).

[0067] The file processing unit 203 parses the segment file of the 3DoF content acquired by the file acquisition unit 201. Next, the file processing unit 203 executes an analysis process of the metadata acquired by the parsing (step S202).

[0068] Then, when file processing unit 203 receives input of the operator's action from measurement unit 202, it also takes that information into consideration and determines the content configuration to be played back from the results of the metadata analysis process (step S203).

[0069] The file processing unit 203 acquires segment files of the 3DoF content according to the determined content configuration (step S204).

[0070] The file processing unit 203 acquires 3DoF image data from the segment file of the acquired 3DoF content and outputs it to the decoding processing unit 204. The decoding processing unit 204 performs a decoding process on the 3DoF image data. After that, the decoding processing unit 204 outputs the decoded 3DoF image data to the display information generation unit 205. The display information generation unit 205 executes a viewing process to generate a display image including a transition execution area using information included in the metadata and display the image on the display unit 206 (step S205).

[0071] Next, the viewpoint transition process will be described in detail with reference to Fig. 8. Fig. 8 is a flowchart of the viewpoint transition process.

[0072] The file processing unit 203 parses the segment file of the 3DoF content to acquire metadata, and acquires transition identification information, transition execution area information, and transition trigger information from the metadata (step S211).

[0073] Next, the file processing unit 203 obtains the 3DoF image data to be displayed (step S212).

[0074] Next, the file processing unit 203 outputs the acquired 3DoF image data to the decoding processing unit 204. In addition, the file processing unit 203 outputs the acquired metadata to the display information generation unit 205. The decoding processing unit 204 decodes the 3DoF image data, and outputs the decoded 3DoF image data to the display information generation unit 205. The display information generation unit 205 generates a display image from the 3DoF image data and metadata, and displays it on the display unit 206 (step S213).

[0075] Thereafter, file processing unit 203 receives a notification of the operation by the operator from measurement unit 202 (step S214).

[0076] Next, the file processing unit 203 confirms that the input operation is an operation that will be a transition trigger for the transition execution area. Then, the file processing unit 203 identifies a viewpoint of the transition destination according to the operation. Next, the file processing unit 203 obtains a segment file including viewpoint image data of the identified viewpoint of the transition destination (step S215).

[0077] Next, the file processing unit 203 acquires the viewpoint image data of the transition destination, causes the decoding processing unit 204 to decode it, and transmits it to the display information generation unit 205 to instruct the content switching. The display information generation unit 205 receives the content switching instruction together with the viewpoint image data of the transition destination from the file processing unit 203. Thereafter, the display information generation unit 205 generates a display image from the acquired viewpoint image data, and displays it on the display unit 206, thereby switching the viewpoint of the display image to the viewpoint of the transition destination (step S216).

[0078] As described above, the file generation device according to the present embodiment generates information representing a transitionable viewpoint for each 3DoF image viewpoint, associates it with the viewpoint, and stores it in the 3DoF content. This allows the client device to identify a transitionable viewpoint from the viewpoint of the 3DoF image being displayed and provide it to the operator. In addition, when the operator instructs switching by specifying a transition destination viewpoint from among the transitionable viewpoints provided, the client device can switch from the 3DoF image seen from the current viewport to the 3DoF image seen from the specified viewpoint. That is, in the distribution system according to the present embodiment, the operator can recognize transitionable viewpoints and can transition to a desired image by specifying a transition destination viewpoint from among the transitionable viewpoints. This allows the user to be provided with a viewing experience such as a transition to an image to which the producer wants to transition, and can provide the user with a high-quality viewing experience.

[0079] [1.1 Modification of the first embodiment (1)] Next, a modified example (1) of the first embodiment will be described. In this modified example, transition identification information is stored in a VWPT descriptor defined in MPEG-DASH.

[0080] The metadata generating unit 103 according to this modification extends the VWPT descriptor in the MPD file as shown in Fig. 9 in order to store transition identification information. Fig. 9 is a diagram showing an example of an extended VWPT descriptor. The metadata generating unit 103 newly defines ViewPointInfo.SwitchableContent and ViewPointInfo.SwitchableContent@content in the two lines from the bottom in Fig. 9 in the VWPT descriptor. ViewPointInfo.SwitchableContent@content indicates identification information of the viewpoint.

[0081] The file generation unit 104 generates an MPD file of 3DoF content represented by the syntax shown in Fig. 10, and stores ViewpintInfo.SwitchableContent in the VWPT descriptor. Fig. 10 is a diagram showing an example of an MPD file in which ViewpintInfo.SwitchableContent is included in the VWPT descriptor. In the MPD file in Fig. 10, AdaptationoSet is provided for each viewpoint, and the value of SwitchableContent content therein indicates the ID of a viewpoint to which the viewpoint can be transitioned.

[0082] As described above, the transition identification information can also be stored in the VWPT descriptor. In this way, even if stored in the VWPT descriptor, it is possible to provide the client device with identification information of the viewport that can be transitioned, and it is possible to provide the user with a high-quality viewing experience.

[0083] [1.2 Modification of the first embodiment (2)] Next, a modified example (2) of the first embodiment will be described. In this modified example, information on a viewpoint group including a plurality of viewpoints to which a transition can be made is used as transition identification information.

[0084] The metadata generating unit 103 according to this modification adds identification information to each viewpoint. Furthermore, the metadata generating unit 103 generates a viewpoint group that groups a plurality of viewpoints and adds identification information to the viewpoint group. For example, the metadata generating unit 103 groups the viewpoint identification information to generate a viewpoint group. Then, for each viewpoint, the metadata generating unit 103 selects a viewpoint group that groups viewpoints to which a transition can be made from the viewpoint. Then, the metadata generating unit 103 generates transition identification information for identifying a viewpoint group to which a transition can be made from viewpoint image data of each viewpoint, using the viewpoint and viewpoint group identification information. Thereafter, the metadata generating unit 103 outputs the transition identification information to the file generating unit 104.

[0085] The file generation unit 104 receives an input of transition identification information from the metadata generation unit 103. Then, the file generation unit 104 newly defines SwitchableContentStruct() representing the transition identification information indicated by syntax 131 in FIG. 11. In syntax 131, VQPT_group_id corresponds to the identification information of the viewpoint group. FIG. 11 is a diagram showing an example of storage of transition identification information with a viewpoint group as the transition destination. Furthermore, the metadata generation unit 103 stores SwitchableContentStruct() in the ViewpointTrackGroup of the ISOBMFF file. As a result, the file generation unit 104 associates the viewpoint image data stored in the ISOBMFF file with the transition identification information, making it possible to identify a viewpoint group to which a transition can be made from the viewpoint image data.

[0086] In the above, a viewpoint group is set as a transition destination, but information of another video can also be set as a transition destination. For example, the metadata generating unit 103 can use access information to the other video specified by a URL, such as a URL (Uniform Resource Locator) representing an MPD or a URL representing a location on the Web, as information representing the other video at the transition destination. In this case, the metadata generating unit 103 generates transition identification information with the other video as the transition destination, and outputs the information to the file generating unit 104.

[0087] The file generation unit 104 receives an input of transition identification information with another video as the transition destination from the metadata generation unit 103. Then, the file generation unit 104 newly defines SwitchableContentStruct() shown in syntax 132 in Fig. 11 and stores the transition identification information. In syntax 32, MPD_URL corresponds to the identification information of the other video as the transition destination.

[0088] As described above, in the file generation device according to this modification, a transition destination other than a viewpoint is specified, making it possible for the client device to identify the transition destination. In this way, by specifying a transition destination other than a viewpoint, it is possible to provide the user with a transition destination from an image represented by specific viewpoint image data, and to provide the user with a high-quality viewing experience.

[0089] [2. Second embodiment] In this embodiment, the generation of a transition execution area indicating a transition execution area that is an area that accepts operations used to transition a video will be described in detail. The file generation device 1 according to this embodiment provides the transition execution area as an interactive area for transitioning a viewpoint. The file generation device 1 according to this example is also represented by the block diagram of FIG. 2. In the following description, the description of the processing of each unit similar to that of the first embodiment may be omitted.

[0090] The metadata generating unit 103 calculates a display position of a transition execution area corresponding to a transition destination viewpoint in a viewpoint image of a transition source from the position information and the direction information of the viewpoint included in the control information. For example, the metadata generating unit 103 obtains a size of the transition execution area and a display position in a celestial sphere image which is a viewport image, and sets the size as transition execution area information. In the present embodiment, the metadata generating unit 103 sets an interactive rectangular area on the celestial sphere image as the transition execution area. When the transition execution area is rectangular, the size of the transition execution area is expressed by length and width. For example, the metadata generating unit 103 determines the transition execution area as an area displaying information of a viewpoint which can be transitioned in the viewpoint direction on the celestial sphere image, and generates transition execution area information. Furthermore, the metadata generating unit 103 generates information that associates the transition execution area with identification information of the viewpoint corresponding to the transition execution area.

[0091] Then, the metadata generation unit 103 outputs the transition execution area information and information associating the transition execution area with the viewpoint corresponding to the transition execution area to the file generation unit 104. At this time, the metadata generation unit 103 also outputs the transition identification information to the file generation unit 104.

[0092] The file generating unit 104 receives an input of transition identification information, transition execution area information, and information associating the transition execution area with a viewpoint corresponding to the transition execution area from the metadata generating unit 103. Next, the file generating unit 104 generates SphereRegionStruct() indicating the transition execution area which is a rectangular area. Then, the file generating unit 104 newly defines RegionForViewpoint() storing SphereRegionStruct() and transition identification information shown in FIG. 12. FIG. 12 is a diagram showing an example of SphereRegionStruct() and RegionForViewpoint(). SphereRegionStruct() in the syntax 211 corresponds to transition execution area information. In addition, in the syntax 211, transition identification information indicating a viewpoint to which a transition can be made is stored by viewpoint_id. That is, both transition identification information and transition execution area information are stored in RegionForViewpoint().

[0093] Furthermore, the file generation unit 104 extends ViewpointTrackGroup as shown in syntax 212, and stores the newly defined RegionForViewpoint() in ViewpointTrackGroup. That is, the file generation unit 104 stores the transition execution region information in ViewpointTrackGroupBox in the TrackBox of the ISOBMFF file. The client device 2 can provide information about the transition destination video as interactive visual information to a user watching the 3DoF video, using the transition execution region information stored in ViewpointTrackGroupBox.

[0094] As described above, the file device according to the present embodiment arranges an interactive area for transitioning a viewpoint as a transition execution area on a 3DoF image, i.e., a spherical video image. The client device arranges and displays transition execution area information on the spherical image, thereby providing the user with an interactive transition execution area and improving the quality of the user's viewing experience.

[0095] [2.1 Modification of the second embodiment (1)] The file generation device 1 according to this modification uses the OMAF technology to provide the transition execution area to the client device 2. Here, the OMAF overlay will be described.

[0096] Existing OMAF technology defines overlay as a technology for overlaying other images on a spherical image. Images overlaid by overlay can be either videos or still images. ISOBMFF defines OverlayStruct() as a storage destination for information about the images to be overlaid. OverlayStruct() is stored in VisualDample() in the ISOBMFF file if it is a video, and in ItemPropertyContainerBox() in the ISOBMFF file if it is a still image.

[0097] FIG. 13 is a diagram showing the control structure of an overlay. In OverlayStruct(), Overlay_control_struct[i] is specified, which contains the control structure shown in Table 221 in FIG. 13. In Overlay_control_struct[i], property information is provided that indicates various overlay structures depending on the value of i. Among these properties, SphereRelativeOmniOverlay() when i=1 is a property that indicates the overlay area on the spherical image. Also, AssociatedSphereRegion() when i=10 is a property that indicates the area associated with the overlay, and clicking this area makes it possible to switch between displaying and hiding the associated overlay.

[0098] The file generation device 1 according to this modified example uses the above-described OMAF overlay technology to provide a transition execution area to the client device 2. The process of each unit in this modified example will be described below.

[0099] The metadata generating unit 103 calculates a transition execution area. Then, the metadata generating unit 103 generates transition execution information so as to correspond to the overlay of OMAF. After that, the metadata generating unit 103 outputs the transition execution information to the file generating unit 104.

[0100] The file generation unit 104 receives an input of transition execution region information from the metadata generation unit 103. Then, the file generation unit 104 displays, by overlay, a video display region on the spherical image displaying the transition execution region, using SphereRelativeOmniOverlay() represented by syntax 222 in Fig. 14. Fig. 14 is a diagram illustrating an example of SphereRelativeOmniOverlay() and AssociatiedSpereRegion().

[0101] SphereRelativeOmniOverlay() indicates the image display area of ​​the overlay on the spherical image. proj_picture_width and proj_picture_hight in syntax 222 indicate the image to be pasted in the image display area of ​​the overlay. proj_reg_width, roj_reg_hight, proj_reg_top, and proj_reg_reft in syntax 222 indicate the image display area of ​​the overlay to which the image is pasted.

[0102] Furthermore, the file generator 104 according to this embodiment uses AssociatedSphereRegion() in the case of Overlay_control_struct

[10] in the control structure of the overlay to store the transition execution region. AssociatedSphereRegion() indicates the region associated with the overlay. The region indicated by AssociatedSphereRegion() is the region where the display and non-display of the associated overlay can be switched when an operation such as clicking is performed.

[0103] In this case, the file generation unit 104 generates AssociatedSpereRegion() represented by syntax 223 in Fig. 14. Here, in order to associate the transition execution region with a transitionable viewpoint, the file generation unit 104 extends AssociatedSpereRegion() as shown in syntax 223 and stores viewpoint_id, which is identification information of the associated viewpoint.

[0104] As described above, in this modification, the transition execution area is displayed on the spherical video by using the OMAF overlay technology. This allows the user to check the transition execution area displayed on the spherical video and easily move the viewpoint.

[0105] [2.2 Modification of the second embodiment (2)] The file generating device 1 according to this modification adds a new definition to the overlay structure and stores the transition execution region information using the structure. The method of storing the transition execution region information according to this modification will be described in detail below.

[0106] Instead of using AssociatedSphereRegion(), the file generation unit 104 prescribes a new control structure for storing transition execution region information in overlay_control_struct() representing the control structure of the overlay, as shown in Table 231 in Fig. 15. Fig. 15 is a diagram showing an example of overlay_control_struct() with a new control structure added for storing transition execution region information. In this modification, the file generation unit 104 adds Associatedviewpointregion() at i=12 as a new control structure to overlay_control_struct().

[0107] Then, the file generating unit 104 generates an Associatedviewpointregion() represented by syntax 232 as shown in Fig. 16. Fig. 16 is a diagram showing an example of the Associatedviewpointregion(). In this case, the file generating unit 104 also stores information linking the transition execution region to the viewpoint in the Associatedviewpointregion().

[0108] As described above, in this modified example, a new control structure is defined in the overlay to display the transition execution area on the spherical video. Even with this method, the user can check the transition execution area displayed on the spherical video, and can easily perform viewpoint transition.

[0109] [2.3 Modification of the second embodiment (3)] In the above-mentioned modified example (1) of the second embodiment, AssociatedSphereRedion is extended in order to link the transition execution area and the transitionable viewpoint. In contrast, the file generating device 1 according to this modified example adds a new definition to the overlay structure and links the transition execution area and the transitionable viewpoint using the structure. The details of the method of storing the transition execution area in this modified example will be described below.

[0110] The file generating unit 104 according to this embodiment uses AssociatedSphereRegion() in the case of Overlay_control_struct

[10] in the control structure of the overlay to store transition execution region information. In addition, the file generating unit 104 prescribes a new control structure for storing transition identification information in overlay_control_struct() as shown in Table 241 of FIG. 17. FIG. 17 is a diagram showing an example of overlay_control_struct() with a new control structure added to store transition identification information. In this modification, the file generating unit 104 adds Associatedviewpoint() at i=12 as a new control structure to overlay_control_struct().

[0111] Fig. 18 is a diagram showing an example of Associatedviewpointregion() and AssociatedViewpoint(). For example, the file generation unit 104 generates Associatedviewpointregion() represented by syntax 242 shown in Fig. 18. The file generation unit 104 also generates Associatedviewpoint() represented by syntax 243. Then, the file generation unit 104 links the transition execution region with the transitionable viewpoint by linking AssociatedSphereRegion() with Associatedviewpoint().

[0112] As described above, in this modified example, a new control structure is defined in the overlay to store information on the viewpoints to which transitions can be made, and the information is associated with the transition execution area. Even with this method, the user can check the transition execution area displayed on the spherical video and can easily move the viewpoint.

[0113] [2.4 Modification of the second embodiment (4)] In the above-mentioned modified examples (1) to (3) of the second embodiment, the video display area and the transition execution area are specified separately, but if these areas are in the same position and the same size, the same area is defined in two places, which is inefficient. Therefore, the file generating device 1 according to this modified example reuses the area by using a flag indicating that the transition execution area is the same as the video display area. The method of storing the transition execution area in this modified example will be described in detail below.

[0114] The metadata generating unit 103 determines the transition execution area so that the video display area and the transition execution area in the OMAF overlay are the same. Then, the metadata generating unit 103 generates a flag indicating that the video display area and the transition execution area match. Then, the metadata generating unit 103 outputs the transition execution area information including the flag indicating that the video display area and the transition execution area match to the file generating unit 104.

[0115] The file generating unit 104 receives an input of transition execution area information including a flag indicating that the video display area and the transition execution area match from the metadata generating unit 103. Then, the file generating unit 104 specifies the video display area in ShareRalativeOverlay(). Next, the file generating unit 104 generates AssociatedSphereRegion() that stores the transition execution area information. Furthermore, as shown in syntax 251 of FIG. 19, the file generating unit 104 extends AssociatedSphereRegion() to set a flag indicating that the transition execution area is the same as the video display area. FIG. 19 is a diagram showing an example of AssociatedSphereRegion() when the video display area is diverted to the transition execution area. The file generating unit 104 divertes the information of the video display area to the transition execution area without storing the information of the transition execution area by indicating that the transition execution area matches the video display area using the flag set in AssociatedSphereRegion().

[0116] Here, in this modified example, the case where AssociatedSphereRegion() is used has been described, but the file generating unit 104 can also use AssociatedViewpointRegion() to indicate the transition execution region. In this case, too, the file generating unit 104 can reuse the information of the video display region for the transition execution region by setting a flag in AssociatedViewpointRegion() indicating that the transition execution region is the same as the video display region.

[0117] As described above, in this modification, the information of the image display region is used as the transition execution region by using a new flag provided in AssociatedSphereRegion(), which makes it possible to reduce data.

[0118] [Modification (4) of the second embodiment] In the second embodiment described above, the transition execution area information is always displayed on the spherical image, which is a 3DoF image. Therefore, unnecessary information is presented to a user who is not interested in the transition of the viewpoint, and the quality of the user's viewing experience may be degraded. Therefore, the file generation device 1 according to this modification displays the transition execution area information on the spherical image when the user views the same spherical image including the transition execution area for a certain period of time. The following describes details of a method for storing the transition execution area information in this modification.

[0119] The metadata generating unit 103 calculates transition execution area information. Furthermore, the metadata generating unit 103 generates time information for controlling the display of the transition execution area on the spherical video. For example, the metadata generating unit 103 generates time information representing the time until the transition execution area information associated with the displayed spherical video is displayed. Then, the metadata generating unit 103 outputs the time information to the file generating unit 104 together with the transition execution area information.

[0120] The file generation unit 104 receives the transition execution region information and time information from the metadata generation unit 103. Then, the file generation unit 104 newly defines a SphereRegionStruct() and a RegionForViewpoint() that stores the transition identification information. Furthermore, the file generation unit 104 stores the time information in the RegionForViewpoint().

[0121] For example, the file generation unit 104 stores the time until transition execution region information associated with the displayed spherical image is displayed in RegionForViewpoint() as shown in syntax 261 in Fig. 20. Fig. 20 is a diagram illustrating an example of RegionForViewpoint() in the case where the transition execution region information is displayed after a certain time. The time_to_ovelay in the syntax 261 is the time until the transition execution region information is displayed.

[0122] The file processing unit 203 of the client device 2 acquires the time until the transition execution region information included in RegionForViewpoint() is displayed. Then, when a specified time has elapsed while the 3DoF image is being displayed after the file processing unit 203 has caused the display information generating unit 205 to display the 3DoF image, the file processing unit 203 instructs the display information generating unit 205 to display the execution region. As a result, the transition execution region is displayed on the client device 2 a certain time after the 3DoF image is displayed, making it possible to transition the viewpoint.

[0123] As described above, in this modification, the transition execution area information on the spherical video is not displayed at first, but is displayed after a certain time has passed. This allows the transition execution area information to be displayed when the user is interested in the transition execution area and is likely to transition the viewpoint. This makes it possible to reduce the display of unnecessary information and improve the quality of the user's viewing experience.

[0124] [3. Third embodiment] In this embodiment, the generation of transition trigger information, which is information on an operation when transitioning a viewpoint, will be described in detail. A file generation device 1 according to this embodiment provides information for switching an operation (user interaction) on an area when transitioning a viewpoint to a client device 2. The file generation device 1 according to this embodiment is also represented by the block diagram of FIG. 2. In the following description, the description of the processing of each part similar to that of the first embodiment may be omitted.

[0125] The metadata generating unit 103 generates transition trigger information indicating a user interaction according to information on the user interaction specified by the operator. When a plurality of user interactions are specified, the metadata generating unit 103 generates transition trigger information indicating that switching between each of the user interactions is possible. For example, when click and zoom are specified as the user interaction, the metadata generating unit 103 generates transition trigger information including information indicating that click and zoom are switchable. In addition, when zoom is used as the user interaction, for example, the metadata generating unit 103 may set a zoom threshold at which a transition occurs and add it to the transition trigger information. Thereafter, the metadata generating unit 103 outputs the generated transition trigger information to the file generating unit 104.

[0126] The file generation unit 104 receives an input of transition trigger information from the metadata generation unit 103. Next, the file generation unit 104 newly defines a SphereRegionStruct() and a RegionForViewpoint() that stores transition identification information. Furthermore, the file generation unit 104 stores information for switching user interaction for a region in RegionForViewpoint(), as shown in syntax 311 in Fig. 21. Fig. 21 is a diagram showing an example of RegionForViewpoint() when switching user interaction.

[0127] The action_control_flag in the syntax 311 is a flag for switching user interaction with respect to the transition execution area. For example, the file generating unit 104 defines the action_control_flag as follows. When the value of the action_control_flag is 0, clicking on the transition execution area switches to a celestial sphere video seen from the viewpoint specified by the viewpoint_id. When the value of the action_control_flag is 1, zooming in on the transition execution area switches to a celestial sphere video seen from the viewpoint specified by the viewpoint_id.

[0128] Furthermore, when the user interaction is zooming, the file generation unit 104 sets the zoom threshold at which a transition occurs as shown in syntax 311. Zoom_threshold in syntax 311 is a threshold for executing a transition when the user interaction that triggers a transition is zooming. For example, the file generation unit 104 defines Zoom_threshold as follows. When Zoom_threshold>1 is defined, a transition is executed when the zoom rate of zooming in to a transition possible area is greater than 1. When Zoom_threshold<1 is defined, a transition is executed when the zoom rate of zooming out to a transition possible area is greater than 1.

[0129] The setting of the information for switching the user interaction and the setting of the threshold value for causing the transition can also be applied to AssociatedSphereRegion() in the modified example (2) of the second embodiment. In this embodiment, the case where clicking and zooming are used as the user interaction has been described, but similar settings can also be made for other operations.

[0130] [4. Fourth embodiment] In the above embodiments and their modified examples, the transition identification information, transition execution area information, and transition trigger information are stored in ISOBMFF. However, it is possible to provide transition identification information, transition execution area information, and transition trigger information even when transmitting using the Matroska Media Container (http: / / www.matroska.org / ) shown in Fig. 22. Fig. 22 is a diagram showing the format of the Matroska Media Container. In this case, the file generation unit 104 stores the transition identification information, transition execution area information, and transition trigger information in an element newly defined in the Track Entry element.

[0131] [Hardware configuration] Fig. 23 is a hardware configuration diagram of a computer. The file generation device 1 and the client device 2 can be realized by a computer 90 shown in Fig. 23. In the computer 90, a processor 91, a memory 92, a network interface 93, a non-volatile storage 94, an input / output interface 95, and a display interface 86 are connected to each other via a bus.

[0132] To the input / output interface 95, external devices such as an input device, an output device, a storage device, and a drive are connected. The input device is, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output device is, for example, a speaker, an output terminal, etc. The storage device is, for example, a hard disk, a RAM (Random Access Memory) disk, etc. The drive drives removable media such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory. In addition, to the display interface 96, a display 98 which is a display device is connected.

[0133] The network interface 93 is connected to an external network. The file generation device 1 and the client device 2 are connected to each other via the network interface 93. In addition, the file generation device 1 and the client device 2 are connected to the Web server 3 via the network interface 93. The non-volatile storage 94 is a built-in auxiliary storage device such as a hard disk or an SSD (Solid State Drive).

[0134] In the computer 90 configured as above, the processor 91 loads, for example, a program stored in the non-volatile storage 94 into the memory 92 via the bus and executes the program, thereby performing the above-mentioned series of processes. The memory 92 also stores data and the like necessary for the processor 91 to execute various processes as appropriate.

[0135] The program executed by the processor 91 can be applied by recording it on a removable medium such as a package medium. In this case, the program can be installed in the non-volatile storage 94 via the input / output interface 95 by mounting the removable medium on a drive that is an external device 97.

[0136] The program may also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program may be received by the network interface 93 and installed in the non-volatile storage 94.

[0137] Alternatively, this program may be pre-installed in the non-volatile storage 94 .

[0138] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present disclosure. In addition, components of different embodiments and modifications may be appropriately combined.

[0139] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0140] The present technology can also be configured as follows. (1) a metadata generation unit that generates transition identification information for identifying a second viewpoint that is a viewpoint position in a second video to which a transition can be made from a first video having a first viewpoint as a viewpoint position; a file generating unit that generates a file including the first image data and the transition identification information; An information processing device comprising: (2) The information processing device according to claim 1, wherein the metadata generation unit generates the transition identification information including identification information for identifying the second viewpoint. (3) The information processing device according to any one of claims 1 to 2, wherein the metadata generation unit generates the transition identification information including identification information of a viewpoint group including a plurality of the second viewpoints. (4) The information processing device according to any one of appendices (1) to (3), wherein the metadata generating unit generates the transition identification information including access information to another video specified by a uniform resource locator (URL). (5) The information processing device according to any one of (1) to (4), wherein the file generation unit stores the identification information in a ViewpointTrackGroupBox in a TrackBox of an ISOBMFF file. (6) The information processing device according to any one of appendices (1) to (4), wherein the file generation unit stores the identification information in a VWPT descriptor of a Media Presentation Description (MPD) file. (7) the metadata generation unit generates transition execution area information indicating a transition execution area on the first image that accepts an operation for transitioning from the first viewpoint to the second viewpoint; The file generation unit causes the file to include the transition execution region information. An information processing device according to claim 1. (8) The information processing device according to claim 7, wherein the metadata generation unit generates the transition execution area information including a size of the transition execution area and a display position in the first video. (9) The information processing device according to any one of claims 7 to 8, wherein the file generation unit stores the transition execution area information in a ViewpointTrackGroupBox in a TrackBox of an ISOBMFF file. (9) The information processing device according to claim 7 or 8, wherein the metadata generation unit indicates the transition execution area information by an overlay in OMAF. (11) The information processing device according to claim 10, wherein the metadata generation unit generates the transition execution area information including a flag indicating that the transition execution area is the same as the video display area in the overlay. (12) The information processing device according to any one of appendices (7) to (11), wherein the metadata generation unit includes time information for controlling display of the transition execution region on the first video. (13) The metadata generation unit generates transition trigger information indicating an operation that causes a transition from the first viewpoint to the second viewpoint; The file generation unit causes the file to include the transition execution region information. An information processing device according to claim 1. (14) The information processing device according to claim 13, wherein the metadata generation unit generates the transition trigger information indicating that the transition is to be caused by a click. (15) The information processing device according to any one of claims 13 to 14, wherein the metadata generation unit generates the transition trigger information indicating that the transition is to occur by zooming. (16) The information processing device according to claim 15, wherein the metadata generation unit generates the transition trigger information including the zoom threshold at which the transition occurs. (17) generating transition identification information for identifying a second viewpoint, which is a viewpoint position in a second image to which a transition can be made from the first image having the first viewpoint as a viewpoint position; A file including the first image data and the transition identification information is generated. An information processing method for causing a computer to execute processing. (18) a file acquisition unit that acquires a file including transition identification information for identifying a second viewpoint that is a viewpoint position in a second image to which a transition can be made from a first image having the first viewpoint as a viewpoint position; a file processing unit that acquires the transition identification information from the file acquired by the file acquisition unit, and identifies the second viewpoint to which a transition can be made from the first viewpoint based on the acquired transition identification information; a playback unit that plays back the first video and outputs information about the second viewpoint specified by the file processing unit; A regeneration processing device comprising: (19) acquiring a file including transition identification information for identifying a second viewpoint, which is a viewpoint position in a second image to which a transition can be made from a first image having the first viewpoint as a viewpoint position; acquiring the transition identification information from the acquired file, and identifying the second viewpoint to which a transition can be made from the first viewpoint based on the acquired transition identification information; The first video is played back, and information on the identified second viewpoint is output. A reproduction processing method for causing a computer to execute processing. [Explanation of symbols]

[0141] 1 File Generator 2. Client Device 3. Web Server 4. Network 10 File generation processing section 11 Control section 12 Communications Department 20 Playback Processing Section 21 Control section 22 Communications Department 100 Distribution System 101 Data Acquisition Section 102 Encoding section 103 Metadata Generation Unit 104 File Generation Unit 201 File Acquisition Department 202 Measurement Division 203 File Processing Unit 204 Decryption processing unit 205 Display information generation section 206 Display section

Claims

1. a metadata generation unit that generates transition identification information for identifying a second viewpoint that is a viewpoint position in a second image to which a transition can be made from a first image having a first viewpoint as a viewpoint position, and transition execution area information that indicates a transition execution area on the first image that accepts an operation for transitioning from the first viewpoint to the second viewpoint; a file generating unit that generates a file including the data of the first video, the transition identification information, and the transition execution region information; Equipped with The file generation unit stores the transition execution area information in a ViewpointTrackGroupBox in a TrackBox of an ISOBMFF file. File generator.

2. A metadata generation unit that generates transition identification information for identifying a second viewpoint which is a viewpoint position in a second image to which a transition can be made from a first image having a first viewpoint as a viewpoint position, and transition execution area information indicating a transition execution area on the first image which accepts an operation for transitioning from the first viewpoint to the second viewpoint; a file generating unit that generates a file including the data of the first video, the transition identification information, and the transition execution region information; Equipped with The metadata generation unit indicates the transition execution region information by an overlay in OMAF. File generator.

3. The file generating device according to claim 1 , wherein the metadata generating section generates the transition identification information including identification information for identifying the second viewpoint.

4. The file generating device according to claim 1 , wherein the metadata generating section generates the transition identification information including identification information of a viewpoint group including a plurality of the second viewpoints.

5. 3. The file generating device according to claim 1, wherein the metadata generating section generates the transition identification information including access information to another video specified by a uniform resource locator (URL).

6. 3. The file generating device according to claim 1, wherein the file generating section stores the transition identification information in a ViewpointTrackGroupBox in a TrackBox of an ISOBMFF file.

7. The file generating device according to claim 1 , wherein the file generating unit stores the transition identification information in a VWPT descriptor of a Media Presentation Description (MPD) file.

8. The file generating device according to claim 1 , wherein the metadata generating section generates the transition execution area information including a size of the transition execution area and a display position in the first video.

9. The file generating device according to claim 2 , wherein the file generating section stores the transition execution area information in a ViewpointTrackGroupBox in a TrackBox of an ISOBMFF file.

10. The file generating device according to claim 1 , wherein the metadata generating unit indicates the transition execution region information by an overlay in OMAF.

11. The file generation device according to claim 2 , wherein the metadata generation section generates the transition execution area information including a flag indicating that the transition execution area and the video display area in the overlay are the same area on the first video.

12. The file generating device according to claim 1 , wherein the metadata generating unit includes time information for controlling display of the transition execution region on the first video.

13. the metadata generation unit generates transition trigger information indicating an operation that causes a transition from the first viewpoint to the second viewpoint; The file generation unit generates the file including the transition trigger information.

3. The file generating device according to claim 1 or 2.

14. The file generation device according to claim 13 , wherein the metadata generation section generates the transition trigger information indicating that the transition is to be caused by a click.

15. The file generating device according to claim 13 , wherein the metadata generating section generates the transition trigger information indicating that the transition is caused by zooming.

16. The file generation device according to claim 15 , wherein the metadata generation section generates the transition trigger information including the zoom threshold at which the transition occurs.

17. generating transition identification information for identifying a second viewpoint, which is a viewpoint position in a second image to which a transition can be made from a first image having a first viewpoint as a viewpoint position, and transition execution area information indicating a transition execution area on the first image in which an operation for transitioning from the first viewpoint to the second viewpoint is accepted; generating a file including the data of the first video, the transition identification information, and the transition execution region information; The transition execution area information is stored in the ViewpointTrackGroupBox in the TrackBox of the ISOBMFF file. A method for generating files that causes a computer to execute processing.

18. Generate transition identification information for identifying a second viewpoint which is a viewpoint position in a second image to which a transition can be made from a first image having a first viewpoint as a viewpoint position, and transition execution area information indicating a transition execution area on the first image which accepts an operation for transitioning from the first viewpoint to the second viewpoint; generating a file including the data of the first video, the transition identification information, and the transition execution region information; Showing the transition execution region information by an overlay in OMAF; A method for generating files that causes a computer to execute processing.

19. a file acquisition unit that acquires a file including transition identification information for identifying a second viewpoint that is a viewpoint position in a second image to which a transition can be made from a first image having a first viewpoint as a viewpoint position, and transition execution area information that indicates a transition execution area on the first image that accepts an operation for transitioning from the first viewpoint to the second viewpoint; a file processing unit that acquires the transition identification information from the file acquired by the file acquisition unit, and identifies the second viewpoint to which a transition can be made from the first viewpoint based on the acquired transition identification information; a playback unit that acquires the transition execution area information from the file acquired by the file acquisition unit, specifies the transition execution area, plays back the first video based on the transition identification information and the transition execution area information acquired from the file, and outputs information on the second viewpoint specified by the file processing unit; Equipped with The playback unit acquires the transition execution area information from a ViewpointTrackGroupBox in a TrackBox of an ISOBMFF file. A regeneration processing device.

20. A file acquisition unit that acquires a file including transition identification information for identifying a second viewpoint which is a viewpoint position in a second image to which a transition can be made from a first image having a first viewpoint as a viewpoint position, and transition execution area information that indicates, by an overlay in OMAF, a transition execution area on the first image which accepts an operation for transitioning from the first viewpoint to the second viewpoint; a file processing unit that acquires the transition identification information from the file acquired by the file acquisition unit, and identifies the second viewpoint to which a transition can be made from the first viewpoint based on the acquired transition identification information; a playback unit that acquires the transition execution area information from the file acquired by the file acquisition unit, specifies the transition execution area, plays back the first video based on the transition identification information and the transition execution area information acquired from the file, and outputs information on the second viewpoint specified by the file processing unit; A regeneration processing device comprising:

21. acquiring a file including transition identification information for identifying a second viewpoint, which is a viewpoint position in a second image to which a transition can be made from a first image having a first viewpoint as a viewpoint position, and transition execution area information indicating a transition execution area on the first image that accepts an operation for transitioning from the first viewpoint to the second viewpoint; acquiring the transition identification information from the acquired file, and identifying the second viewpoint to which a transition can be made from the first viewpoint based on the acquired transition identification information; acquiring the transition execution region information from the acquired file to specify the transition execution region, reproducing the first video based on the transition identification information and the transition execution region information acquired from the file, and outputting information on the identified second viewpoint; The transition execution area information is obtained from the ViewpointTrackGroupBox in the TrackBox of the ISOBMFF file. A reproduction processing method for causing a computer to execute processing.

22. Obtaining a file including transition identification information for identifying a second viewpoint which is a viewpoint position in a second image to which a transition can be made from a first image having a first viewpoint as a viewpoint position, and transition execution area information which indicates, by an overlay in OMAF, a transition execution area on the first image which accepts an operation for transitioning from the first viewpoint to the second viewpoint; acquiring the transition identification information from the acquired file, and identifying the second viewpoint to which a transition can be made from the first viewpoint based on the acquired transition identification information; The transition execution area information is acquired from the acquired file to specify the transition execution area, and the first video is played back based on the transition identification information and the transition execution area information acquired from the file, and information on the specified second viewpoint is output. A reproduction processing method for causing a computer to execute processing.

Images