Distribution control system, distribution control device, distribution control method, and program

The system addresses data and server load issues in stereoscopic video distribution by arranging virtual viewpoints and simplifying geometry, ensuring high-quality user experience despite bandwidth variations.

JP7586300B2Active Publication Date: 2024-11-19NIPPON TELEGRAPH & TELEPHONE CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023517012
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-04-30
Publication Date
2024-11-19
Estimated Expiration
2041-04-30

AI Technical Summary

Technical Problem

Existing methods for distributing stereoscopic video content, such as volumetric videos and holograms, face challenges in managing large data volumes, server load, and variable network bandwidth, leading to reduced quality and user experience.

Method used

A distribution system that arranges multiple virtual viewpoints around the object, creates one-dimensional data by simplifying geometry for non-visible areas, and distributes appropriate one-dimensional data based on user position and field of view, reducing data requirements while maintaining quality.

Benefits of technology

Reduces data volume and processing load while maintaining user experience quality, even with varying network conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007586300000001
    Figure 0007586300000001
  • Figure 0007586300000002
    Figure 0007586300000002
  • Figure 0007586300000003
    Figure 0007586300000003
Patent Text Reader

Abstract

A distribution system according to an embodiment has: a disposition unit for disposing, with respect to a subject represented by three-dimensional data constituting three-dimensional video image content, a plurality of virtual viewpoints centered on the subject; a first creation unit for creating, for each of the virtual viewpoints, one-sided three-dimensional data from which data for portions of the subject that cannot be viewed from the virtual viewpoint has been removed; and a distribution unit for distributing, in accordance with the position and the field-of-view of a user in the virtual space in which the subject is disposed, the one-sided three-dimensional data for one of the virtual viewpoints from among the one-sided three-dimensional data for each respective virtual viewpoint, to the user's terminal.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a distribution control system, a distribution control device, a distribution control method, and a program. [Background technology]

[0002] Three-dimensional video content with six degrees of freedom (6DoF), such as volumetric video and holograms, is well known. In order to distribute such content with high quality over a communication network, in addition to using high-performance data compression technology and communication network / system load balancing technology, a mechanism for controlling the distribution of the content itself is required. In particular, a mechanism for dynamically controlling the distribution of content according to the visibility information of the client XR (VR / AR / MR / SR, etc.) device and the user's position information in the virtual space is important.

[0003] Volumetric video is animation data composed of polygon meshes (hereinafter simply referred to as "meshes") and textures, and is displayed on the display by rendering it together with the virtual environment on the client side, allowing it to be viewed.

[0004] As volumetric video distribution technologies, the technologies described in Non-Patent Documents 1 to 3 are known. Non-Patent Document 1 proposes a method in which volumetric video is rendered on the server side based on the head movement of a user detected by an AR / VR device, which is a client, and transmitted to the client as 2D data. Non-Patent Document 2 proposes a method in which volumetric video generated in real time is delivered to a client and rendered on the client side. Furthermore, Non-Patent Document 3 proposes a method in which the amount of data required for playback is reduced by dynamically changing the Level Of Detail of the volumetric video according to the bandwidth of a communication network. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Serhan Gul, Dimitri Podborski, Thomas Buchholz, Thomas Schierl and Cornelius Hellge, "Low-latency cloud-based volumetric video streaming using head motion prediction", NOSSDAV '20: Proceedings of the 30th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, June 2020, Pages 27-33 [Non-Patent Document 2] Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh ​​Kowdle, Yury Degtyarev, et al. "Holoportation: Virtual 3D teleportation in real-time", UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, October 2016, Pages 741-754 [Non-Patent Document 3] Holostream / Arctrus, Internet<URL:https: / / arcturus.studio / holostream> Summary of the Invention [Problem to be solved by the invention]

[0006] However, volumetric video requires a large amount of data and therefore requires a large bandwidth of the communication network to distribute it, so there is a demand for an efficient distribution method.

[0007] However, in the method proposed in the above-mentioned non-patent document 1, rendering for each user needs to be performed on the server side, which places a heavy load on the server. In addition, when the number of users increases, the quality of the video viewed by each user may deteriorate due to the division of server resources. Furthermore, position information needs to be transmitted from the client to the server frequently and with low latency. For example, suppressing the Motion to Photon latency, at which VR sickness begins to occur, to 20 ms or less places a heavy burden on both the communication network and the server.

[0008] On the other hand, the method proposed in the above Non-Patent Document 2 requires a communication bandwidth of 4 Gbps, but it is difficult for a user to always secure a 4 Gbps communication bandwidth stably. Also, since the load on the communication network is large, the available bandwidth for other users who use the same communication network is narrowed, degrading the quality of experience for other users.

[0009] Furthermore, in the method proposed in the above-mentioned non-patent document 3, when the available bandwidth of the communication network is narrow, the image quality and level of detail are reduced, including the 3D data visible to the user in the volumetric video being viewed (i.e., the 3D data corresponding to the front of the user), resulting in a significant deterioration in the quality of experience.

[0010] An embodiment of the present invention has been made in consideration of the above points, and has an object to reduce the amount of data required to deliver stereoscopic video content while maintaining the user's quality of experience. [Means for solving the problem]

[0011] In order to achieve the above object, a distribution system according to one embodiment includes an arrangement unit that arranges a plurality of virtual viewpoints centered on an object represented by three-dimensional data constituting three-dimensional video content, a first creation unit that creates one-sided three-dimensional data for each of the virtual viewpoints by reducing the amount of data for portions of the object that cannot be seen from the virtual viewpoints, and a distribution unit that distributes the one-sided three-dimensional data of one of the one-sided three-dimensional data for each of the virtual viewpoints to a terminal of the user in accordance with the user's position and field of view in a virtual space in which the object is arranged. Effect of the Invention

[0012] It is possible to reduce the amount of data required to deliver stereoscopic video content while maintaining the user's quality of experience. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of an overall configuration of a distribution control system according to an embodiment of the present invention. [Diagram 2] 10 is a flowchart showing an example of a single-plane 3D data creation process according to the present embodiment. [Diagram 3] FIG. 13 is a diagram showing an example of the arrangement of virtual viewpoints. [Figure 4] 10 is a flowchart illustrating an example of a correspondence table creation process according to the embodiment. [Diagram 5] FIG. 4 is a diagram showing an example of a viewing angle range. [Figure 6] FIG. 11 is a diagram showing an example of a viewing angle range correspondence table. [Figure 7] 10 is a flowchart illustrating an example of a distribution process according to the embodiment. [Figure 8] FIG. 13 is a diagram illustrating an example of identifying a virtual viewpoint. [Figure 9] FIG. 2 illustrates an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] An embodiment of the present invention will be described below. In this embodiment, a distribution control system 1 that can reduce the amount of data required for distribution of volumetric video, as an example of stereoscopic video content, while maintaining the user's quality of experience, will be described. Here, volumetric video refers to animation data composed of 3D data (also called three-dimensional data or stereoscopic data) represented by meshes and textures. That is, for example, 3D data of a frame at time t is displayed as d t Then, volumetric video is {d t |t∈[t s ,t e ]}. t s is the start time of the volumetric video, t e is the end time.

[0015] It should be noted that the embodiment described below is not limited to volumetric video, but can also be similarly applied to stereoscopic video content having six degrees of freedom, such as a hologram.

[0016] <Overall composition> First, the overall configuration of a distribution control system 1 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the overall configuration of a distribution control system 1 according to this embodiment.

[0017] 1, the delivery control system 1 according to this embodiment includes a delivery control server 10, a content server 20, a client 30, and a service site server 40. The delivery control server 10 and the client 30 are communicatively connected via a communication network N such as the Internet. Similarly, the client 30 and the service site server 40 are communicatively connected via the communication network N.

[0018] It is assumed that the delivery control server 10 and the content server 20 exist within the same local network and are connected to each other so as to be able to communicate with each other within the local network. However, this is not limited to this and they may be connected to each other so as to be able to communicate with each other via a communication network N, for example.

[0019] The distribution control server 10 creates multiple single-plane 3D data from the 3D data constituting the given volumetric video, and stores the multiple single-plane 3D data in the content server 20. Here, the single-plane 3D data is three-dimensional data when an object represented by the 3D data constituting the volumetric video is viewed from one viewpoint, and the amount of data is reduced compared to the original 3D data while maintaining the user's quality of experience. Note that the object is the subject of the volumetric video, and means any object that can be represented by a mesh and texture, such as a person, an animal or plant, a structure, a building, a machine, a celestial body, a natural phenomenon, etc.

[0020] Furthermore, in response to a viewing request from the client 30, the distribution control server 10 determines appropriate one-plane 3D data based on the user's viewpoint, spatial position, field of view (line of sight direction and field of view range), etc., and distributes this one-plane 3D data to the client 30. Note that the user's viewpoint and spatial position refer to the user's position in the virtual space in which the target object is located.

[0021] The content server 20 stores a plurality of single-level 3D data items. In addition, in response to a data request from the delivery control server 10, the content server 20 returns to the delivery control server 10 the single-level 3D data items corresponding to the data request.

[0022] The client 30 is a terminal (e.g., an XR (VR / AR / MR / SR, etc.) device) used by a user who wants to watch a volumetric video, and renders one-sided 3D data distributed from the distribution control server 10 and plays the volumetric video. Note that the XR device includes, in addition to an HMD (Head Mount Display), a smartphone or tablet terminal equipped with an application program that functions as an XR device, a wearable device, etc.

[0023] When a user views a volumetric video, for example, the following procedure is performed. First, the user accesses the service site server 40 with the client 30 and obtains a list of content (volumetric videos) that the user can view. Next, the user selects a volumetric video that the user wishes to view from the list and obtains a link to the selected volumetric video. Then, the client 30 accesses the link, whereby a viewing request is sent to the distribution control server 10, and playback of the volumetric video is started when one-sided 3D data is returned in response to the request.

[0024] Furthermore, the client 30 appropriately transmits information such as the user's viewpoint, spatial position, field of view, etc. (hereinafter, information indicating the user's viewpoint or spatial position and field of view is also referred to as "user viewpoint information.") to the distribution control server 10. As a result, one-sided 3D data according to the user's viewpoint, spatial position, field of view, etc. is returned from the distribution control server 10 and is played back on the client 30.

[0025] The service site server 40 presents a list of content (volumetric videos) that can be viewed by the user, and provides the client 30 with a link to content selected from this list.

[0026] Here, the distribution control server 10 according to the present embodiment includes a single 3D data creation unit 101, a correspondence table creation unit 102, a distribution control unit 103, and a distribution unit 104. Each of these units is realized, for example, by a process in which one or more programs installed in the distribution control server 10 are executed by a processor such as a CPU (Central Processing Unit).

[0027] Moreover, the distribution control server 10 according to the present embodiment includes a correspondence table storage unit 105. The correspondence table storage unit 105 is realized by an auxiliary storage device such as a hard disk drive (HDD) or a solid state drive (SSD), for example.

[0028] The single-plane 3D data creation unit 101 creates multiple single-plane 3D data from the 3D data constituting the volumetric video provided to the distribution control server 10. More specifically, the single-plane 3D data creation unit 101 arranges multiple virtual viewpoints at equal (or non-equidistant) intervals around an object represented by the 3D data constituting the volumetric video, and creates single-plane 3D data for each virtual viewpoint by performing geometry simplification (Level Of Detail reduction processing) on ​​a mesh that corresponds to the back surface of the object when the object is viewed from the virtual viewpoint. In this way, the single-plane 3D data for each virtual viewpoint is created by performing geometry simplification on a mesh on the back surface of the object represented by the 3D data (i.e., a portion that cannot be seen from the virtual viewpoint).

[0029] Then, the single level 3D data creation unit 101 stores the created multiple single level 3D data in the content server 20.

[0030] The correspondence table creation unit 102 creates a correspondence table (hereinafter also referred to as a "viewing angle range correspondence table") that associates each virtual viewpoint arranged when creating multiple single-plane 3D data with the viewing angle range of the virtual viewpoint. The viewing angle range is a range in which, when viewing an object, the same quality of experience can be obtained as when viewing the object from the virtual viewpoint. In other words, it is a range in which the back of an object with simplified geometry cannot be seen, just like the virtual viewpoint.

[0031] When the delivery control unit 103 receives user viewpoint information from the client 30, the delivery control unit 103 refers to the viewing angle range correspondence table and determines appropriate single-sided 3D data from the user viewpoint information. More specifically, when at least a part of an object is captured within the field of view at a viewpoint or spatial position included in the user viewpoint information, the delivery control unit 103 identifies a virtual viewpoint corresponding to a viewing angle range that includes the viewpoint or spatial position, and determines the single-sided 3D data corresponding to the identified virtual viewpoint as appropriate.

[0032] The distribution unit 104 transmits the single plane 3D data determined by the distribution control unit 103 to the client 30.

[0033] The correspondence table storage unit 105 stores the viewing angle range correspondence table created by the correspondence table creation unit 102 .

[0034] Furthermore, the content server 20 according to this embodiment includes a single 3D data storage unit 201. The single 3D data storage unit 201 is realized by an auxiliary storage device such as an HDD or SSD, for example.

[0035] The single plane 3D data storage unit 201 stores each single plane 3D data created by the single plane 3D data creation unit 101.

[0036] 1 is merely an example, and other configurations may be used. For example, the delivery control server 10 and the content server 20 may be integrated into one server.

[0037] <Details of each process> Below, we will explain each of the single-surface 3D data creation process for creating multiple single-surface 3D data, the correspondence table creation process for creating a viewing angle range correspondence table, and the distribution process for distributing appropriate single-surface 3D data to the client 30. Note that the single-surface 3D data creation process and the correspondence table creation process are pre-processing executed before the distribution process.

[0038] <One-sided 3D data creation process> First, the single-sided 3D data creation process will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of the single-sided 3D data creation process according to the present embodiment. In the following, it is assumed that the volumetric video is provided to the distribution control server 10.

[0039] The single-plane 3D data creation unit 101 acquires one frame of 3D data from among the 3D data constituting the volumetric video (step S101). In the following, the single-plane 3D data creation unit 101 acquires 3D data d of a frame at time t. t The initial value of t is t = t s This can be done as follows.

[0040] It should be noted that the 3D data is made up of a mesh and a texture, but since no particular processing is performed on the texture below, in the above step S101, only the mesh of the 3D data may be acquired.

[0041] Next, the one-plane 3D data creation unit 101 creates the 3D data d obtained in step S102. t N virtual viewpoints are placed around the object represented by (step S102). Note that the line of sight of each virtual viewpoint is the object. For example, the single-surface 3D data creation unit 101 places N virtual viewpoints with the object as the line of sight at equal intervals (or may place them at unequal intervals) on the circumference of a circle of a predetermined radius centered around the object. Note that N is a predetermined integer of 2 or more.

[0042] An example of the arrangement of virtual viewpoints when N = 8 is shown in Fig. 3. In the example shown in Fig. 3, virtual viewpoints V1 to V8, with the object O as the line of sight, are arranged at equal intervals on the circumference of a circle of radius R with the object O as the center.

[0043] Note that placing virtual viewpoints on the circumference of a circle is just an example, and is not limited thereto, and for example, virtual viewpoints may be placed on the sides or vertices of a polygon with the object at its center. Also, in the example shown in Fig. 3, for simplicity, virtual viewpoints V1 to V8 are placed on the xy plane of the xyz space with the center of the object O as the origin, but this is not limited thereto, and for example, virtual viewpoints may be placed on a sphere (or a polyhedron) with the origin as the center.

[0044] Next, the single-plane 3D data creation unit 101 selects one virtual viewpoint from among the N virtual viewpoints arranged in the above step S102 (step S103).

[0045] Next, the single-plane 3D data creation unit 101 performs geometry simplification (Level Of Detail reduction processing) on ​​the mesh that corresponds to the back surface of the object when the object is viewed from the virtual viewpoint selected in step S103 (step S104). In other words, the single-plane 3D data creation unit 101 performs geometry simplification processing on the meshes of the parts that cannot be seen (or are not visible) from the virtual viewpoint among the meshes that make up the object. As a result, when the object is viewed from the virtual viewpoint, the original 3D data d t The quality of experience is equivalent to that of the original 3D data. t This creates single-sided 3D data with less data volume than the conventional method.

[0046] Next, the single plane 3D data creation unit 101 determines whether or not all N virtual viewpoints have been selected (step S105).

[0047] If it is determined in step S105 above that there is a virtual viewpoint that has not yet been selected, the single-sided 3D data creation unit 101 returns to step S103 above, selects one virtual viewpoint from among the virtual viewpoints that have not yet been selected, and executes the processing from step S104 onwards.

[0048] On the other hand, if it is determined in step S105 above that all N virtual viewpoints have been selected, the single plane 3D data creation unit 101 determines whether or not there is a next frame in the given volumetric video (step S106).

[0049] If it is determined in step S106 that there is a next frame, the single plane 3D data creation unit 101 returns to step S101, obtains the 3D data of the next frame, and executes the processes in step S102 and after. That is, in this case, the single plane 3D data creation unit 101 returns to step S101 as t←t+1, and obtains the 3D data d of the frame at the next time t. t Get the.

[0050] On the other hand, if it is determined in step S106 that there is no next frame (i.e., time t=t e If so, the single-level 3D data generator 101 stores the single-level 3D data generated in step S104 in the single-level 3D data storage unit 200 of the content server 20 (step S107). i (where i ∈ [1,N]) is the one-sided 3D data corresponding to d ti Then, the set of one-sided 3D data {d ti |t∈[t s ,t e ], i ∈ [1, N]} is stored in the one-plane 3D data storage unit 201.

[0051] In this embodiment, the number N of virtual viewpoints is common to all frames, but it may be different for each frame. Also, in this embodiment, the above steps S102 to S105 are repeatedly executed for each frame. e -ts In this case, the 3D data d t If they are the same, the 3D data d of one frame included in the time span is t The above steps S102 to S105 may be executed only for the above.

[0052] <Correspondence table creation process> Next, the correspondence table creation process will be described with reference to FIG. 4. FIG. 4 is a flowchart showing an example of the correspondence table creation process according to the present embodiment. t (where t∈[t s ,t e In the following, we will explain how to create a viewing angle range correspondence table for d = d t This is expressed as:

[0053] The correspondence table creation unit 102 arranges N virtual viewpoints around the object represented by the 3D data d (step S201), similar to step S102 in Fig. 2. The number of virtual viewpoints to be arranged and the arrangement method (equally spaced or non-equally spaced, arranged on a circumference or on a polygon, etc.) are the same as those in step S102 in Fig. 2.

[0054] Next, the correspondence table creation unit 102 selects one virtual viewpoint from among the N virtual viewpoints arranged in the above step S201 (step S202).

[0055] Next, the correspondence table creation unit 102 calculates a boundary angle between the virtual viewpoint selected in the above step S202 and an adjacent virtual viewpoint (hereinafter, referred to as an "adjacent virtual viewpoint") (step S203). The boundary angle is the angle between the line of sight of the virtual viewpoint and the boundary between the virtual viewpoint and the adjacent virtual viewpoint.

[0056] Here, as an example, a case will be described where the boundary angle of the virtual viewpoint V2 is calculated for the arrangement of virtual viewpoints shown in Fig. 3. As shown in Fig. 5, the virtual viewpoint V2 is assumed to be at an angle θ2 with respect to the object O. Note that in the example shown in Fig. 5, the positive direction of the x-axis on the xy plane with the center of the object O as the origin is used as the reference for the angle.

[0057] In this case, the adjacent virtual viewpoints of the virtual viewpoint V2 are V1 and V3, and the angle between the line of sight of the virtual viewpoint V2 and the line of sight of the adjacent virtual viewpoint V1 is φ 21 , the angle between the line of sight of the virtual viewpoint V2 and the line of sight of the adjacent virtual viewpoint V3 is φ 22 Then, the angle φ 21 The line that divides the point into two is the boundary with the adjacent virtual viewpoint V1, and the angle is φ 22 The line dividing the virtual viewpoint V1 into two is taken as the boundary with the adjacent virtual viewpoint V3, and the angle to this boundary is taken as the boundary angle.

[0058] That is, the boundary angle with the adjacent virtual viewpoint V1 is θ 21 , the boundary angle with the adjacent virtual viewpoint V3 is θ 22 Then, θ 21 =φ 21 / 2, θ 22 =φ 22 / 2.

[0059] In addition, other virtual viewpoints V i Boundary angle θ for (i=1,3,...,8) i1 and θ i2 can be calculated in the same manner.

[0060] Next, the correspondence table creation unit 102 determines whether or not all N virtual viewpoints have been selected (step S204).

[0061] If it is determined in step S204 above that there is a virtual viewpoint that has not yet been selected, the correspondence table creation unit 102 returns to step S202 above, selects one virtual viewpoint from among the virtual viewpoints that have not yet been selected, and executes the processing from step S203 onwards.

[0062] On the other hand, if it is determined in step S204 that all N virtual viewpoints have been selected, the correspondence table creation unit 102 creates a viewing angle range correspondence table by calculating the viewing angle range of each virtual viewpoint from the boundary angle calculated in step S203, and stores the created viewing angle range correspondence table in the correspondence table storage unit 105 (step S205). i (where i∈[1,N]) is the angle θ i The virtual viewpoint V i The boundary angle is θ i1 ,θ i2 Then, the virtual viewpoint V i The viewing angle range of is θ i -θ i1 ≦θ i <θ i +θ i2 This calculates the virtual viewpoint V i and viewing angle range θ i -θ i1 ≦θ i <θ i +θ i2 An example of the viewing angle range correspondence table created in this way is shown in FIG.

[0063] In this embodiment, time t ∈ [t s ,t e ] 3D data t Although a common viewing angle range correspondence table was created in the above, for example, if the number of virtual viewpoints differs depending on the frame, a viewing angle range correspondence table may be created for each number of virtual viewpoints.

[0064] <Distribution process> Next, the distribution process will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the distribution process according to this embodiment. The following steps S301 to S303 are executed every time user viewpoint information is received, and steps S304 to S305 are executed at each frame interval. However, in the following, it is assumed that at least a part of the object is included within the user's field of view.

[0065] The delivery control unit 103 calculates the user position relative to the object using the viewpoint or spatial position included in the user viewpoint information received from the client 30 (step S301). Here, the user position is the angle of the viewpoint or spatial position relative to the object. Note that the angle standard is the same as when the position of the virtual viewpoint relative to the object was determined in step S203 of FIG. 4.

[0066] Next, the delivery control unit 103 refers to the viewing angle range correspondence table stored in the correspondence table storage unit 105, and identifies a virtual viewpoint from the user position calculated in the above step S301 (step S302). That is, the delivery control unit 103 identifies, from among the virtual viewpoints, a virtual viewpoint that corresponds to the viewing angle range that includes the user position.

[0067] For example, as shown in FIG. 8, the user position of a certain user A is θ A As a result, θ2-θ 21 ≦θ A <θ2+θ 22 In this case, the virtual viewpoint V2 is specified in step S302.

[0068] For example, as shown in FIG. 8, the user position of a certain user B is set to θ B As a result, θ3-θ 31 ≦θ B <θ3+θ 32 In this case, the virtual viewpoint V3 is specified in step S302.

[0069] In the example shown in Figure 8, user A is in a case where object O is present in the line of sight, while user B is in a case where object O is not present in the line of sight but at least a part of object O is included within the field of view.

[0070] Next, the distribution control unit 103 determines the one-sided 3D data corresponding to the virtual viewpoint identified in the above step S302 as the distribution target (step S303). i If so, the distribution control unit 103 distributes the single-sided 3D data {dti} is determined to be the delivery target.

[0071] The distribution unit 104 acquires, from the content server 20, the single plane 3D data of the frame at the relevant time among the single plane 3D data to be distributed determined in step S303 above (step S304).

[0072] Then, the distribution unit 104 distributes the single 3D data acquired in step S304 to the client 30 (step S305). As a result, the single 3D data is rendered on the client 30 side, and the target object is displayed on the display.

[0073] <Hardware configuration> Finally, the hardware configuration of the delivery control server 10 and the content server 20 according to the present embodiment will be described. The delivery control server 10 and the content server 20 according to the present embodiment are realized, for example, by the hardware configuration of a computer 500 shown in Fig. 9. The client 30 and the service site server 40 may also be realized by a similar hardware configuration.

[0074] 9 includes an input device 501, a display device 502, an external I / F 503, a communication I / F 504, a processor 505, and a memory device 506. Each of these pieces of hardware is connected to each other via a bus 507 so as to be able to communicate with each other.

[0075] The input device 501 is, for example, a keyboard, a mouse, a touch panel, etc. The display device 502 is, for example, a display, etc. Note that the computer 500 does not necessarily have to have at least one of the input device 501 and the display device 502.

[0076] The external I / F 503 is an interface with an external device such as a recording medium 503a. Examples of the recording medium 503a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0077] The communication I / F 504 is an interface for performing data communication with other devices, equipment, systems, etc. The processor 505 is, for example, various types of arithmetic devices such as a CPU, etc. The memory device 506 is, for example, various types of storage devices such as an HDD, SSD, RAM (Random Access Memory), ROM (Read Only Memory), and flash memory.

[0078] The delivery control server 10 and the content server 20 according to this embodiment can realize the above-mentioned various processes by having the hardware configuration of a computer 500 shown in Fig. 9. Note that the hardware configuration of the computer 500 shown in Fig. 9 is an example, and the computer 500 may have other hardware configurations. For example, the computer 500 may have multiple processors 505, or multiple memory devices 506.

[0079] <Summary> As described above, the distribution control system 1 according to this embodiment places multiple virtual viewpoints on the object represented by the 3D data constituting the stereoscopic video content, and then simplifies the geometry of the polygon mesh of the part that cannot be seen from each virtual viewpoint. This creates one-sided 3D data for each virtual viewpoint with a reduced amount of data compared to the original 3D data.

[0080] Therefore, it is possible to reduce the amount of data required to deliver stereoscopic video content while maintaining the user's quality of experience by delivering appropriate single-sided 3D data according to the user's position and field of view at client 30. In addition, the rendering load on the client 30 side is also reduced, so that the processing load on the client 30 side can also be reduced.

[0081] The present invention is not limited to the above-described embodiments specifically disclosed, and various modifications, changes, combinations with known technologies, etc. are possible without departing from the scope of the claims. [Explanation of symbols]

[0082] 1. Distribution Control System 10. Distribution Control Server 20 Content Server 30 clients 40 Service Site Server 50 Communication Network 101 One-sided 3D data creation section 102 Correspondence table creation department 103 Distribution control section 104 Distribution Department 105 Correspondence table storage unit 201 One-sided 3D data storage unit 500 Computers 501 Input Device 502 Display device 503 External I / F 503a Recording media 504 Communication I / F 505 Processor 506 Memory Device 507 Bus

Claims

1. a placement unit that places a plurality of virtual viewpoints centered on an object represented by stereoscopic data constituting a stereoscopic video content; a first creating unit that creates, for each of the virtual viewpoints, one-plane three-dimensional data that is three-dimensional data obtained by reducing a data amount of a portion of the object that cannot be viewed from the virtual viewpoint; a distribution unit that distributes one-plane three-dimensional data of one virtual viewpoint among the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and field of view of the user in the virtual space in which the object is placed; A distribution control system having the above configuration.

2. The first creation unit is The distribution control system according to claim 1 , further comprising: performing geometry simplification processing on a polygon mesh representing a portion that cannot be seen from the virtual viewpoint, thereby creating one-sided three-dimensional data that is three-dimensional data in which the amount of data of the portion has been reduced.

3. 3. The distribution control system according to claim 1, wherein the portion that cannot be viewed from the virtual viewpoint is a portion that corresponds to a rear surface of the object when the object is viewed from the virtual viewpoint.

4. a second creation unit that creates a correspondence table that associates the virtual viewpoint with a range in which the same quality of experience can be obtained when the object is viewed from the virtual viewpoint, The distribution unit is When at least a portion of the object is included in the field of view, referring to the correspondence table, one-plane stereoscopic data of a virtual viewpoint corresponding to a range in which the position is included is identified; 4. The distribution control system according to claim 1, further comprising: a control unit for controlling a distribution of the one-plane stereoscopic data to the terminal.

5. a placement unit that places a plurality of virtual viewpoints centered on an object represented by stereoscopic data constituting a stereoscopic video content; a first creating unit that creates, for each of the virtual viewpoints, one-plane three-dimensional data that is three-dimensional data obtained by reducing a data amount of a portion of the object that cannot be viewed from the virtual viewpoint; a distribution unit that distributes one-plane three-dimensional data of one virtual viewpoint among the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and field of view of the user in the virtual space in which the object is placed; A distribution control device having the above configuration.

6. a placement step of placing a plurality of virtual viewpoints centered on an object represented by stereoscopic data constituting a stereoscopic video content; a first generation step of generating, for each of the virtual viewpoints, one-plane three-dimensional data that is three-dimensional data obtained by reducing a data amount of a portion of the object that cannot be viewed from the virtual viewpoint; a delivery step of delivering one-plane three-dimensional data of one virtual viewpoint among the one-plane three-dimensional data for each virtual viewpoint to a terminal of the user in accordance with a position and a field of view of the user in the virtual space in which the object is placed; The computer executes the delivery control method.

7. A program for causing a computer to function as the distribution control system according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Information processing device and method

    WO2020116154A1

  • Generation device, three-dimensional data transmission device, and three-dimensional data reproduction device

    WO2020137876A1