Distribution control system, distribution control device, distribution control method, and program

Abstract

A delivery control system according to an embodiment of the present disclosure has: an arrangement unit for arranging multiple virtual viewpoints around an object represented by three-dimensional data constituting a three-dimensional video content; a calculation unit for calculating, for each of the virtual viewpoints, a first virtual viewpoint obtained when the virtual viewpoint is rotated in a positive direction by a predetermined angle about the object and a second virtual viewpoint obtained when the virtual viewpoint is rotated in a negative direction by the predetermined angle about the object; a generation unit for generating, for each of the virtual viewpoints, single-plane three-dimensional data by reducing the amount of data for a portion which is invisible from the first virtual viewpoint and the second virtual viewpoint; and a delivery unit for delivering single-plane three-dimensional data for one virtual viewpoint to a user's terminal in accordance with a position and a visual field of the user in a virtual space in which the object is placed, this single-plane three-dimensional data being one from among said single-plane three-dimensional data having been generated for each of the virtual viewpoints.

Description

[Technical Field]

[0001] The present disclosure relates to a distribution control system, a distribution control device, a distribution control method, and a program. [Background technology]

[0002] Three-dimensional video content with six degrees of freedom (6DoF), such as volumetric video and holograms, is well known. To deliver such content with high quality over a communication network, not only is it necessary to utilize high-performance data compression technology and communication network / system load balancing technology, but it also requires a mechanism to control the delivery of the content itself. In particular, it is important to have a mechanism that dynamically controls the delivery of content based on the field of view information of the client XR (VR / AR / MR / SR, etc.) device and the user's position information in the virtual space.

[0003] Volumetric video is animation data composed of polygon meshes (hereinafter simply referred to as "meshes") and textures, and is displayed on a display by rendering it together with the virtual environment on the client side, allowing it to be viewed.

[0004] Known volumetric video distribution technologies include those described in Non-Patent Documents 1 and 2. Non-Patent Document 1 proposes a method in which volumetric video is rendered on the server side based on the user's head movement detected by an AR / VR device acting as a client, and transmitted to the client as 2D data. Non-Patent Document 2 also proposes a method in which the level of detail of volumetric video is dynamically changed depending on the bandwidth of a communication network, thereby reducing the amount of data required for playback. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Serhan Gul, Dimitri Podborski, Thomas Buchholz, Thomas Schierl and Cornelius Hellge, "Low-latency cloud-based volumetric video streaming using head motion prediction", NOSSDAV '20: Proceedings of the 30th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, June 2020, Pages 27-33 [Non-patent document 2] Holostream / Arctrus, Internet<URL:https: / / arcturus.studio / holostream> Summary of the Invention [Problem to be solved by the invention]

[0006] However, volumetric video requires a large amount of data and requires a large bandwidth for its distribution over a communication network, so an efficient distribution method is required.

[0007] However, the method proposed in Non-Patent Document 1 requires rendering for each user to be performed on the server side, which places a heavy load on the server. Furthermore, as the number of users increases, the division of server resources may result in a degradation of the quality of the video viewed by each user. Furthermore, location information must be transmitted from the client to the server frequently and with low latency. For example, keeping the Motion to Photon latency, at which VR sickness begins to occur, to 20 ms or less places a heavy burden on both the communication network and the server.

[0008] On the other hand, the method proposed in the above-mentioned non-patent document 2 reduces the image quality and level of detail of the volumetric video being viewed, including the 3D data visible to the user (i.e., the 3D data that corresponds to the front of the user), when the available bandwidth of the communication network is narrow, resulting in a significant degradation in the quality of experience.

[0009] The present disclosure has been made in consideration of the above points, and provides a technology that can reduce the amount of data required to distribute three-dimensional video content while maintaining the user's quality of experience. [Means for solving the problem]

[0010] A distribution control system according to one aspect of the present disclosure includes an arrangement unit that arranges multiple virtual viewpoints centered on an object represented by three-dimensional data constituting three-dimensional video content, the virtual viewpoint being centered on the object; a calculation unit that calculates, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by a predetermined angle in a positive direction around the object, and a second virtual viewpoint obtained by rotating the virtual viewpoint by the predetermined angle in a negative direction around the object; a creation unit that creates, for each of the virtual viewpoints, one-sided three-dimensional data in which the amount of data of portions that cannot be seen from the first virtual viewpoint and the second virtual viewpoint is reduced; and a distribution unit that distributes one-sided three-dimensional data of one of the one-sided three-dimensional data for each of the virtual viewpoints to a user's terminal in accordance with the user's position and field of view in a virtual space in which the object is placed. [Effects of the Invention]

[0011] A technology is provided that can reduce the amount of data required to distribute three-dimensional video content while maintaining the user's quality of experience. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of a distribution control system according to an embodiment of the present invention. [Figure 2] 10 is a flowchart showing an example of a single-plane 3D data creation process according to the present embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of the arrangement of virtual viewpoints. [Figure 4] FIG. 2 is a diagram showing the visible range of a virtual viewpoint Vi. [Figure 5] FIG. 2 is a diagram showing the visible range of a virtual viewpoint Vi1. [Figure 6] FIG. 10 is a diagram showing the visible range of a virtual viewpoint Vi2. [Figure 7] FIG. 10 is a diagram showing the visible range and outside the visible range from −π / N to +π / N of a virtual viewpoint Vi. [Figure 8] 10 is a flowchart illustrating an example of a correspondence table creation process according to the present embodiment. [Figure 9] FIG. 10 is a diagram showing an example of a viewing angle range correspondence table. [Figure 10] 10 is a flowchart illustrating an example of a distribution process according to the present embodiment. [Figure 11] FIG. 2 illustrates an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0013] An embodiment of the present invention will be described below. In the following embodiment, a distribution control system 1 will be described that targets volumetric video as an example of stereoscopic video content and can reduce the amount of data required for its distribution while maintaining the user's quality of experience. Here, volumetric video refers to animation data made up of 3D data (also called three-dimensional data or stereoscopic data) represented by meshes and textures. That is, for example, 3D data of a frame at time t is displayed as d t Then, volumetric video is {d t |t∈[t s ,t e ]}. t s is the start time of the volumetric video, t e is the end time.

[0014] The embodiment described below is not limited to volumetric video, but can also be applied to, for example, stereoscopic video content with six degrees of freedom, such as a hologram.

[0015] <Overall configuration example> First, the overall configuration of a distribution control system 1 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the overall configuration of a distribution control system 1 according to this embodiment.

[0016] 1, the delivery control system 1 according to this embodiment includes a delivery control server 10, a content server 20, a client 30, and a service site server 40. The delivery control server 10 and the client 30 are connected to each other so as to be able to communicate with each other via a communication network 50 such as the Internet. Similarly, the client 30 and the service site server 40 are connected to each other so as to be able to communicate with each other via the communication network 50.

[0017] It should be noted that the distribution control server 10 and the content server 20 are assumed to exist within the same local network and to be communicably connected within the local network, but this is not limited thereto and they may be communicably connected via a communication network 50, for example.

[0018] The distribution control server 10 creates multiple pieces of single-panel 3D data from the 3D data constituting the given volumetric video, and stores these multiple pieces of single-panel 3D data in the content server 20. Here, the single-panel 3D data is three-dimensional data of an object represented by the 3D data constituting the volumetric video when viewed from a single viewpoint, and is obtained by reducing the amount of data compared to the original 3D data while maintaining the user's quality of experience. Note that the object refers to the subject of the volumetric video, and means any object that can be represented by a mesh and texture, such as a person, animal, plant, structure, building, machine, celestial body, natural phenomenon, etc.

[0019] In addition, in response to a viewing request from the client 30, the distribution control server 10 determines appropriate single-plane 3D data based on the user's viewpoint, spatial position, field of view (line-of-sight direction and field-of-view range), etc., and distributes this single-plane 3D data to the client 30. The user's viewpoint and spatial position refer to the user's position in the virtual space where the object is located.

[0020] The content server 20 stores a plurality of single-plane 3D data. In addition, in response to a data request from the distribution control server 10, the content server 20 returns the single-plane 3D data corresponding to this data request to the distribution control server 10.

[0021] The client 30 is a terminal (for example, an XR (VR / AR / MR / SR, etc.) device) used by a user who wants to watch the volumetric video, and renders the one-sided 3D data distributed from the distribution control server 10 and plays the volumetric video. Note that the XR device includes not only an HMD (Head Mount Display), but also a smartphone, tablet terminal, wearable device, etc. equipped with an application program that functions as an XR device.

[0022] When a user views a volumetric video, for example, the following procedure is performed. First, the user accesses the service site server 40 using the client 30 and obtains a list of content (volumetric videos) that the user can view. Next, the user selects the volumetric video that the user wishes to view from this list and obtains a link to the selected volumetric video. Then, when the client 30 accesses the link, a viewing request is sent to the distribution control server 10, and playback of the volumetric video begins when full-screen 3D data is returned in response to this request.

[0023] Additionally, the client 30 appropriately transmits information such as the user's viewpoint, spatial position, field of view, etc. (hereinafter, information representing the user's viewpoint or spatial position and field of view is also referred to as "user viewpoint information") to the distribution control server 10. As a result, one-sided 3D data according to the user's viewpoint, spatial position, field of view, etc. is returned from the distribution control server 10 and played back on the client 30.

[0024] The service site server 40 presents a list of content (volumetric videos) that can be viewed by the user, and provides the client 30 with a link to content selected from this list.

[0025] The distribution control server 10 according to this embodiment includes a single-sided 3D data creation unit 101, a correspondence table creation unit 102, a distribution control unit 103, and a distribution unit 104. Each of these units is realized by, for example, processing executed by a processor such as a CPU (Central Processing Unit) by one or more programs installed in the distribution control server 10.

[0026] The distribution control server 10 according to this embodiment also includes a correspondence table storage unit 105. The correspondence table storage unit 105 is realized by an auxiliary storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), for example.

[0027] The single-plane 3D data creation unit 101 creates multiple single-plane 3D data from the 3D data constituting the volumetric video provided to the distribution control server 10. More specifically, the single-plane 3D data creation unit 101 arranges multiple virtual viewpoints at equal (or unequal) intervals around an object represented by the 3D data constituting the volumetric video, and then, for each virtual viewpoint, performs geometry simplification (level-of-detail reduction processing) on ​​meshes corresponding to portions of the object that cannot be seen when the object is viewed from the virtual viewpoint and a viewpoint obtained by rotating the virtual viewpoint by a certain angle around the object, thereby creating single-plane 3D data. In this way, for each virtual viewpoint, geometry simplification such as vertex deletion is performed on meshes corresponding to portions of the object mesh represented by the 3D data that cannot be seen from the virtual viewpoint and a viewpoint obtained by rotating the virtual viewpoint by a certain angle around the object, thereby creating single-plane 3D data for each virtual viewpoint.

[0028] The single-plane 3D data creation unit 101 then stores the created plurality of single-plane 3D data in the content server 20.

[0029] The correspondence table creation unit 102 creates a correspondence table (hereinafter also referred to as a "viewing angle range correspondence table") that associates each virtual viewpoint arranged when creating multiple single-plane 3D data with the viewing angle range of that virtual viewpoint. The viewing angle range is the range within which, when viewing an object, the same quality of experience can be obtained as when viewing the object from that virtual viewpoint. In other words, it is the range within which the simplified geometric parts of the object cannot be seen when viewing the object, just like from that virtual viewpoint.

[0030] When the delivery control unit 103 receives user viewpoint information from the client 30, it refers to the viewing angle range correspondence table and determines appropriate single-panel 3D data from the user viewpoint information. More specifically, when at least a part of an object is captured within the field of view from a viewpoint or spatial position included in the user viewpoint information, the delivery control unit 103 identifies a virtual viewpoint corresponding to a viewing angle range that includes the viewpoint or spatial position, and determines the single-panel 3D data corresponding to the identified virtual viewpoint as appropriate.

[0031] The distribution unit 104 transmits the single-plane 3D data determined by the distribution control unit 103 to the client 30.

[0032] The correspondence table storage unit 105 stores the viewing angle range correspondence table created by the correspondence table creation unit 102.

[0033] The content server 20 according to this embodiment also includes a single 3D data storage unit 201. The single 3D data storage unit 201 is realized by an auxiliary storage device such as an HDD or SSD, for example.

[0034] The single plane 3D data storage unit 201 stores each single plane 3D data created by the single plane 3D data creation unit 101.

[0035] 1 is merely an example, and other configurations may be used. For example, the distribution control server 10 and the content server 20 may be integrated into one server.

[0036] <Details of various processes> Below, we will explain each of the following processes: a single-plane 3D data creation process for creating multiple single-plane 3D data, a correspondence table creation process for creating a viewing angle range correspondence table, and a distribution process for distributing appropriate single-plane 3D data to client 30. Note that the single-plane 3D data creation process and the correspondence table creation process are pre-processing executed before the distribution process.

[0037] <One-sided 3D data creation process> First, the single-plane 3D data creation process will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of the single-plane 3D data creation process according to this embodiment. In the following, it is assumed that volumetric video has been provided to the distribution control server 10.

[0038] The single-plane 3D data creation unit 101 acquires 3D data for one frame from the 3D data constituting the volumetric video (step S101). Hereinafter, the single-plane 3D data creation unit 101 acquires 3D data d of the frame at time t. t The initial value of t is t=t s This can be done as follows.

[0039] It should be noted that the 3D data is made up of a mesh and a texture, but since no particular processing is performed on the texture below, only the mesh of the 3D data may be acquired in step S101 above.

[0040] Next, the one-plane 3D data creation unit 101 creates the 3D data d obtained in step S102. t N virtual viewpoints are arranged around the object represented by (step S102). However, the line of sight of each virtual viewpoint is assumed to be the object. For example, the single-plane 3D data creation unit 101 arranges N virtual viewpoints, with the line of sight of the object, at equal intervals (or may be at unequal intervals) on the circumference of a circle of a predetermined radius centered on the object. Note that N is a predetermined integer of 2 or greater.

[0041] An example of the arrangement of virtual viewpoints when N=8 is shown in Fig. 3. In the example shown in Fig. 3, virtual viewpoints V1 to V8, with the object O as the line of sight, are arranged counterclockwise at equal intervals (i.e., 2π / N intervals) on the circumference of a circle with radius R, centered on the object O. Also, in the example shown in Fig. 3, the angle of virtual viewpoint V1 is set as the reference angle with respect to the object O (θ1=0), and the rotation angles from virtual viewpoint V1 are set as angles θ2 to θ8 of virtual viewpoints V2 to V8 with respect to the object O.

[0042] Note that placing virtual viewpoints on the circumference of a circle is just one example, and is not limited to this, and for example, virtual viewpoints may be placed on the sides or vertices of a polygon centered on the object. Also, in the example shown in Fig. 3, for simplicity, virtual viewpoints V1 to V8 are placed on the xy plane of the xyz space with the center of object O as the origin, but this is not limiting, and for example, virtual viewpoints may be placed on a sphere (or polyhedron) centered at the origin.

[0043] In the following description, it is assumed that N virtual viewpoints are arranged counterclockwise at equal intervals on the circumference of a circle of a certain radius centered on the object.

[0044] Next, the single-plane 3D data creation unit 101 selects one virtual viewpoint from among the N virtual viewpoints arranged in step S102 (step S103).

[0045] Next, the single plane 3D data creation unit 101 performs geometry simplification (Level Of Detail reduction processing) on ​​meshes corresponding to portions of the object that cannot be seen when the object is viewed from the virtual viewpoint selected in step S103 above and its range of -π / N to +π / N, and creates single plane 3D data corresponding to the virtual viewpoint (step S104). Specifically, the single plane 3D data creation unit 101 creates single plane 3D data corresponding to the virtual viewpoint by the following steps 1 to 6. In the following, the virtual viewpoint selected in step S103 above is referred to as V i , the virtual viewpoint V of the object i The angle is θ i In addition, θ i is the rotation angle from the reference angle (for example, the angle θ1=0 of the virtual viewpoint V1).

[0046] Step 1: The one-plane 3D data creation unit 101 creates a virtual viewpoint V i The visual range is calculated as the angular range of the visible portion of the object when the object is viewed from the virtual viewpoint V. The visual range may be, for example, a range of -π / 2 to +π / 2 from the angle of the viewpoint of the object. iFigure 4 shows the visible range when viewing an object O from a virtual viewpoint V. i When viewing the object O from a virtual viewpoint V i From this, we can see that θ i -π / 2<θ<θ i This indicates that the +π / 2 portion is visible. Note that θ is a variable that takes the angle.

[0047] Step 2: The one-plane 3D data creation unit 101 creates a virtual viewpoint V i The virtual viewpoint V is rotated by +π / N. i1 Set this virtual viewpoint V i1 The visible range when viewing the object from the virtual viewpoint V is calculated. i1 Figure 5 shows the visible range when viewing the object O from the virtual viewpoint V i1 When viewing the object O from a virtual viewpoint V i1 From this, we can see that θ i +(1 / N-1 / 2)π<θ<θ i +(1 / N+1 / 2)π. The angle θ i +π / N is the virtual viewpoint V i and Virtual Viewpoint V i+1 (However, when i=N, V1) is the angle that divides the virtual viewpoint V i and Virtual Viewpoint V i+1 (However, when i=N, it can be regarded as the boundary with V1), so hereinafter it is also called the "positive boundary angle."

[0048] Step 3: The one-plane 3D data creation unit 101 creates a virtual viewpoint V i The virtual viewpoint V is rotated by -π / N. i2 Set this virtual viewpoint V i2 The visible range when viewing the object from the virtual viewpoint V is calculated. i2 Figure 6 shows the visible range when viewing the object O from the virtual viewpoint V i2 When viewing the object O from a virtual viewpoint V i2 From this, we can see that θ i -(1 / N+1 / 2)π<θ<θ iThis means that the part of -(1 / N-1 / 2)π can be seen. i -π / N is the virtual viewpoint V i and Virtual Viewpoint V i-1 (However, when i=1, V N ) and the virtual viewpoint V i and Virtual Viewpoint V i-1 (However, when i=1, V N ), so it is also referred to as the "negative boundary angle" below.

[0049] Step 4: The one-plane 3D data creation unit 101 creates a virtual viewpoint V i The view range of the virtual viewpoint V i1 The view range of the virtual viewpoint V i2 That is, the one-plane 3D data creation unit 101 calculates the union of the visible range of θ i -(1 / N+1 / 2)π<θ<θ i Let +(1 / N+1 / 2)π be the visible range.

[0050] The visible range is virtual viewpoint V i1 Viewing range and virtual viewpoint V i2 Since the visible range of the virtual viewpoint V matches the union of the visible range of the virtual viewpoint V, i Therefore, the above step 1 does not need to be performed. In this case, in the above step 4, the virtual viewpoint V i1 Viewing range and virtual viewpoint V i2 The visible range can be calculated by the union of the visible ranges of the two.

[0051] Step 5: The one-plane 3D data creation unit 101 calculates θ i -π<θ≦θ i +π and the visible range is calculated as the outside of the visible range. i -π<θ≦θ i -(1 / N+1 / 2)π and θ i +(1 / N+1 / 2)π≦θ≦θ i+π is calculated as outside the visible range. The visible range and outside the visible range are shown in Figure 7. The visible range is calculated as θ i -π / N~θ i +π / N represents the angular range of the part of the object O that can be seen when viewing the object O from a viewpoint within the range of θ i -π / N~θ i represents the angular range of the portion of object O that is not visible from viewpoints within +π / N.

[0052] Step 6: The one-plane 3D data creation unit 101 performs geometry simplification (Level Of Detail reduction processing) on ​​the mesh of the part of the object that is outside the visible range. i -π / N~θ i When the object is viewed from a viewpoint within the range of +π / N, the original 3D data d t The same quality of experience as the original 3D data d t The 3D data at time t is called d t Virtual Viewpoint V i The corresponding one-sided 3D data is ti Let's say.

[0053] Next, the single-plane 3D data creation unit 101 determines whether or not all N virtual viewpoints have been selected (step S105).

[0054] If it is determined in step S105 above that there is a virtual viewpoint that has not yet been selected, the single-sided 3D data creation unit 101 returns to step S103 above, selects one virtual viewpoint from among the virtual viewpoints that have not yet been selected, and executes the processing from step S104 onwards.

[0055] On the other hand, if it is determined in step S105 above that all N virtual viewpoints have been selected, the single-plane 3D data creation unit 101 determines whether or not there is a next frame in the given volumetric video (step S106).

[0056] If it is determined in step S106 that there is a next frame, the single plane 3D data creation unit 101 returns to step S101, acquires the 3D data of the next frame, and executes the processes from step S102 onward. That is, in this case, the single plane 3D data creation unit 101 sets t←t+1, returns to step S101, and acquires the 3D data d of the frame at the next time t. t Get.

[0057] On the other hand, if it is determined in step S106 that there is no next frame (i.e., time t=t e If so, the single plane 3D data creation unit 101 stores the single plane 3D data created in step S104 in the single plane 3D data storage unit 200 of the content server 20 (step S107). ti |t∈[t s ,t e ], i∈[1,N]} is stored in the one-plane 3D data storage unit 201.

[0058] In this embodiment, the number N of virtual viewpoints is common to all frames, but it may be different for each frame. Also, in this embodiment, the above steps S102 to S105 are repeatedly executed for each frame. However, for example, it may be different for each frame after a certain time width (time width is t e -t s In this case, 3D data is t If the same, the 3D data d of one frame included in the time span t Steps S102 to S105 may be executed only for the above.

[0059] <Correspondence table creation process> Next, the correspondence table creation process will be described with reference to FIG. 8. FIG. 8 is a flowchart showing an example of the correspondence table creation process according to this embodiment. t (where t∈[t s ,t e ]) will be explained below.t It is expressed as:

[0060] The correspondence table creation unit 102 arranges N virtual viewpoints around the object represented by the 3D data d (step S201), similar to step S102 in Fig. 2. Note that the number of virtual viewpoints to be arranged and the arrangement method (equally spaced or non-equally spaced, arranged on a circle or on a polygon, etc.) are the same as those in step S102 in Fig. 2.

[0061] Next, the correspondence table creation unit 102 selects one virtual viewpoint from among the N virtual viewpoints arranged in the above step S201 (step S202).

[0062] Next, the correspondence table creation unit 102 calculates the boundary angles (boundary angles in the positive direction and boundary angles in the negative direction) for the virtual viewpoint selected in the above step S202 (step S203). i , the virtual viewpoint V of the object i The angle is θ i Then, the correspondence table creation unit 102 calculates the boundary angle θ i +π / N and the boundary angle θ in the negative direction i Calculate -π / N.

[0063] Next, the correspondence table creating unit 102 determines whether or not all N virtual viewpoints have been selected (step S204).

[0064] If it is determined in step S204 above that there is a virtual viewpoint that has not yet been selected, the correspondence table creation unit 102 returns to step S202 above, selects one virtual viewpoint from among the virtual viewpoints that have not yet been selected, and executes the processing from step S203 onwards.

[0065] On the other hand, if it is determined in step S204 that all N virtual viewpoints have been selected, the correspondence table creation unit 102 creates a viewing angle range correspondence table by calculating the viewing angle range of each virtual viewpoint from the boundary angle calculated in step S203, and stores the created viewing angle range correspondence table in the correspondence table storage unit 105 (step S205). i(where i∈[1,N]) has a boundary angle θ in the negative direction. i Boundary angle θ is greater than or equal to -π / N and in the positive direction i The angle range less than +π / N, i.e., θ i -π / N≦θ<θ i +π / N. This gives the virtual viewpoint V i and viewing angle range θ i -π / N≦θ<θ i +π / N, a viewing angle range correspondence table is created. An example of the viewing angle range correspondence table created in this way is shown in FIG.

[0066] In this embodiment, time t∈[t s ,t e ] 3D data d t Although a common viewing angle range correspondence table was created in the above, if the number of virtual viewpoints differs depending on the frame, a viewing angle range correspondence table may be created for each number of virtual viewpoints.

[0067] <Distribution process> Next, the distribution process will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the distribution process according to this embodiment. The following steps S301 to S303 are executed every time user viewpoint information is received, and steps S304 to S305 are executed at each frame interval. However, in the following, it is assumed that at least a part of the object is included in the user's field of view.

[0068] The distribution control unit 103 calculates the user position relative to the object using the viewpoint or spatial position included in the user viewpoint information received from the client 30 (step S301). Here, the user position is the angle of the viewpoint or spatial position relative to the object. The angle standard is the same as when the angle of the virtual viewpoint relative to the object was determined in step S203 of FIG. 8.

[0069] Next, the distribution control unit 103 refers to the viewing angle range correspondence table stored in the correspondence table storage unit 105, and identifies a virtual viewpoint from the user position calculated in the above step S301 (step S302). That is, the distribution control unit 103 identifies, from among the virtual viewpoints, a virtual viewpoint corresponding to the viewing angle range that includes the user position. Specifically, the user position calculated in the above step S301 is compared with θ U When this is the case, the distribution control unit 103 calculates θ i -π / N≦θ U <θ i +π / N i Identify.

[0070] Next, the distribution control unit 103 determines the one-plane 3D data corresponding to the virtual viewpoint identified in the above step S302 as the distribution target (step S303). i If so, the distribution control unit 103 ti} is determined to be the target for distribution.

[0071] The distribution unit 104 acquires the single plane 3D data of the frame at the relevant time from the single plane 3D data to be distributed determined in step S303 above, from the content server 20 (step S304).

[0072] Then, the distribution unit 104 distributes the single 3D data acquired in step S304 to the client 30 (step S305). As a result, the single 3D data is rendered on the client 30 side, and the object is displayed on the display.

[0073] <Hardware configuration> Finally, the hardware configuration of the delivery control server 10 and content server 20 according to this embodiment will be described. The delivery control server 10 and content server 20 according to this embodiment are realized, for example, by the hardware configuration of a computer 500 shown in Fig. 11. The client 30 and service site server 40 may also be realized by a similar hardware configuration.

[0074] 11 includes an input device 501, a display device 502, an external I / F 503, a communication I / F 504, a processor 505, and a memory device 506. Each of these pieces of hardware is connected to each other via a bus 507 so as to be able to communicate with each other.

[0075] The input device 501 is, for example, a keyboard, a mouse, a touch panel, etc. The display device 502 is, for example, a display, etc. Note that the computer 500 does not necessarily have to have at least one of the input device 501 and the display device 502.

[0076] The external I / F 503 is an interface with an external device such as a recording medium 503a. Examples of the recording medium 503a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0077] The communication I / F 504 is an interface for performing data communication with other devices, equipment, systems, etc. The processor 505 is, for example, various types of arithmetic devices such as a CPU, etc. The memory device 506 is, for example, various types of storage devices such as an HDD, SSD, RAM (Random Access Memory), ROM (Read Only Memory), flash memory, etc.

[0078] The distribution control server 10 and the content server 20 according to this embodiment can realize the various processes described above by having the hardware configuration of a computer 500 shown in Fig. 11. Note that the hardware configuration of the computer 500 shown in Fig. 11 is an example, and the computer 500 may have other hardware configurations. For example, the computer 500 may have multiple processors 505 or multiple memory devices 506.

[0079] <Summary> As described above, the distribution control system 1 according to this embodiment arranges multiple virtual viewpoints with respect to an object represented by 3D data constituting stereoscopic video content, and then, for each of these virtual viewpoints, geometrically simplifies the polygon mesh of the portion that cannot be seen from viewpoints within a range from the negative boundary angle to the positive boundary angle of the virtual viewpoint. Because the geometrically simplified portion cannot be seen from viewpoints within a range from the negative boundary angle to the positive boundary angle of each virtual viewpoint, it is possible to create single-plane 3D data that does not degrade the user's quality of experience and has a reduced data volume compared to the original 3D data.

[0080] Therefore, it is possible to reduce the amount of data required to deliver stereoscopic video content while maintaining the quality of user experience by delivering appropriate single-plane 3D data according to the user's position and field of view at client 30. In addition, the rendering load on the client 30 side is also reduced, which makes it possible to reduce the processing load on the client 30 side.

[0081] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, and combinations with known technologies are possible without departing from the scope of the claims. [Explanation of symbols]

[0082] 1. Distribution Control System 10. Distribution Control Server 20 Content Server 30 clients 40 Service Site Server 50 Communication Network 101 One-sided 3D data creation section 102 Correspondence table creation section 103 Distribution control section 104 Distribution Department 105 Correspondence table storage unit 201 One-sided 3D data storage unit 500 computers 501 Input Device 502 Display device 503 External I / F 503a Recording media 504 Communication I / F 505 processor 506 Memory Device 507 Bus

Claims

1. a placement unit that places a plurality of virtual viewpoints centered on an object represented by three-dimensional data constituting three-dimensional video content; a calculation unit that calculates, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by a predetermined angle in a positive direction around the object, and a second virtual viewpoint obtained by rotating the virtual viewpoint by the predetermined angle in a negative direction around the object; a creating unit that creates, for each of the virtual viewpoints, one-plane three-dimensional data in which the amount of data of a portion that cannot be viewed from the first virtual viewpoint and the second virtual viewpoint is reduced; a distribution unit that distributes one-plane three-dimensional data of one virtual viewpoint among the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and field of view of the user in the virtual space in which the object is placed; A distribution control system having:

2. The placement unit The distribution control system according to claim 1 , wherein the plurality of virtual viewpoints are arranged on circles or spheres at equal or non-equidistant intervals around the object, or on vertices or edges of a polygon or polyhedron.

3. The calculation unit 2. The distribution control system according to claim 1, wherein when the arrangement unit arranges N (where N is a predetermined integer of 2 or greater) virtual viewpoints at equal intervals on a circle, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by π / N in a positive direction and a second virtual viewpoint obtained by rotating the virtual viewpoint by π / N in a negative direction are calculated.

4. The creation unit calculating a range of ±π / 2 of the angle of the first virtual viewpoint with respect to the object as a viewable range from the first virtual viewpoint; calculating a range of ±π / 2 of the angle of the second virtual viewpoint with respect to the object as a viewable range from the second virtual viewpoint; The distribution control system according to claim 3, wherein a range excluding a range represented by the sum of the visible range at the first virtual viewpoint and the visible range at the second virtual viewpoint from a range of ±π of the angle of the virtual viewpoint relative to the object is calculated as the portion that cannot be seen from the first virtual viewpoint and the second virtual viewpoint.

5. The distribution unit 5. The distribution control system according to claim 1, further comprising: identifying a virtual viewpoint where the angle of the position relative to the object is less than the angle of a first virtual viewpoint relative to the object and is greater than or equal to the angle of a second virtual viewpoint relative to the object; and distributing one-sided stereoscopic data corresponding to the identified virtual viewpoint to the user's terminal.

6. a placement unit that places a plurality of virtual viewpoints centered on an object represented by three-dimensional data constituting three-dimensional video content; a calculation unit that calculates, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by a predetermined angle in a positive direction around the object, and a second virtual viewpoint obtained by rotating the virtual viewpoint by the predetermined angle in a negative direction around the object; a creating unit that creates, for each of the virtual viewpoints, one-plane three-dimensional data in which the amount of data of a portion that cannot be viewed from the first virtual viewpoint and the second virtual viewpoint is reduced; a distribution unit that distributes one-plane three-dimensional data of one virtual viewpoint among the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and field of view of the user in the virtual space in which the object is placed; A distribution control device having the above configuration.

7. an arrangement step of arranging a plurality of virtual viewpoints centered on an object represented by three-dimensional data constituting the three-dimensional video content; a calculation step of calculating, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by a predetermined angle in a positive direction around the object as a center, and a second virtual viewpoint obtained by rotating the virtual viewpoint by the predetermined angle in a negative direction around the object as a center; a creating step of creating, for each of the virtual viewpoints, one-plane three-dimensional data in which the amount of data of a portion that cannot be viewed from the first virtual viewpoint and the second virtual viewpoint is reduced; a delivery procedure for delivering one of the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and field of view of the user in the virtual space in which the object is placed; The computer executes the delivery control method.

8. an arrangement step of arranging a plurality of virtual viewpoints centered on an object represented by three-dimensional data constituting the three-dimensional video content; a calculation step of calculating, for each of the virtual viewpoints, a first virtual viewpoint obtained by rotating the virtual viewpoint by a predetermined angle in a positive direction around the object as a center, and a second virtual viewpoint obtained by rotating the virtual viewpoint by the predetermined angle in a negative direction around the object as a center; a creating step of creating, for each of the virtual viewpoints, one-plane three-dimensional data in which the amount of data of a portion that cannot be viewed from the first virtual viewpoint and the second virtual viewpoint is reduced; a delivery procedure for delivering one of the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and field of view of the user in the virtual space in which the object is placed; A program that causes a computer to execute the following.

Images