Optimal viewpoint detection device and its program

Abstract

To provide an optimum viewpoint detecting device capable of detecting an optimum viewpoint from a CG scene containing a plurality of objects.SOLUTION: An optimum viewpoint detecting device 1 comprises: optimum degree map generating means 30 for generating a spherical optimum degree map showing an optimum degree when each point of a sphere centering on an object is an object viewpoint for each of objects included in a CG scene; optimum degree map synthesizing means 50 for synthesizing an optimum degree map of each object into one synthetic map by mixed Gaussian distribution or a circumscribed sphere transfer processing; and optimum viewpoint calculating means 60 for calculating an optimum viewpoint from the synthetic map.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an optimum viewpoint detection device and a program for detecting an optimum viewpoint from a CG scene. [Background technology]

[0002] Volumetric capture technology makes it possible to capture a scene in real space as a CG scene of 3D data. By placing a virtual camera in the captured CG scene, it is possible to generate any image after the fact, but it is time-consuming to select the optimal viewpoint for placing the virtual camera on a computer screen. For this reason, detecting the optimal viewpoint for capturing a CG scene is important for efficient production of 3D content. Therefore, conventional technologies have been proposed to detect the optimal viewpoint from a CG scene composed of a single object (Non-Patent Documents 1 and 2). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Jingwu He, Linbo Wang, Wenzhe Zhou, Hongjie Zhang, Xiufen Cui, and Yanwen Guo, “Viewpoint Assessment and Recommendation for Photographing Architectures” IEEE Visualization 2019, VOL.25, NO.8, pp.2636-2649 [Non-Patent Document 2] Polonsky, Oleg, Patane, Giuseppe, Biasotti, Silvia, Gotsman, Craig, and Spagnuolo, Michela, “What's in an image? Towards the Computation of the “Best” View of an Object,” The Visual Computer, Vol.21, pp.840-847, (2005) Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional techniques are limited to CG scenes composed of a single object. Currently, there is no known technique for detecting an optimal viewpoint from a CG scene composed of multiple objects.

[0005] Therefore, an object of the present invention is to provide an optimum viewpoint detection device and a program therefor that detect an optimum viewpoint from a CG scene including a plurality of objects. [Means for solving the problem]

[0006] In order to solve the above problems, the optimal viewpoint detection device of the present invention is an optimal viewpoint detection device that detects an optimal viewpoint from a CG scene including a plurality of objects, and is configured to include an optimality map generation means, an optimality map synthesis means, and an optimal viewpoint calculation means.

[0007] According to this configuration, the optimum map generating means generates, for each object included in the CG scene, a spherical optimum map indicating optimum when each point on a sphere centered on the object is set as the viewpoint of the object. The optimum map synthesizing means synthesizes the optimum maps of each object into one synthesized map by a mixed Gaussian distribution or a circumscribed sphere transfer process. The optimum viewpoint calculating means then calculates the optimum viewpoint from the synthesized map.

[0008] In this way, the optimal viewpoint detection device synthesizes a synthesis map that reflects the optimality of each object, and therefore can detect the optimal viewpoint from a CG scene that includes a plurality of objects.

[0009] The present invention can also be realized by a program for causing a computer to function as the optimum viewpoint detection device described above. Effect of the Invention

[0010] According to the present invention, an optimal viewpoint can be detected from a CG scene containing multiple objects. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of an optimum viewpoint detection device according to an embodiment. [Diagram 2] FIG. 2 is an explanatory diagram for explaining an image of an object in the embodiment. [Diagram 3] In an embodiment, (a) and (b) are diagrams showing optimality maps when the same object is viewed from the front and back, and (c) is an optimality legend. [Figure 4] 1 is an image showing an example of an optimum degree map in an embodiment. [Diagram 5] In an embodiment, (a) and (b) are diagrams showing optimality maps of each object. [Figure 6] 11A and 11B are explanatory diagrams for explaining a circumscribed sphere transfer process in the embodiment. [Figure 7] 4 is a flowchart showing the operation of the optimum viewpoint detecting device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is for embodying the technical idea of ​​the present invention, and unless otherwise specified, the present invention is not limited to the following. In addition, the same symbols are used for the same means, and the description may be omitted.

[0013] (Embodiment) [Configuration of the optimal viewpoint detection device] The configuration of an optimal viewpoint detecting device 1 according to the embodiment will be described with reference to FIG. The optimal viewpoint detection device 1 detects an optimal viewpoint from a CG scene including a plurality of objects, and generates an image of the detected optimal viewpoint (optimum image). As shown in Fig. 1, the optimal viewpoint detection device 1 includes a CG scene input means 10, a CG scene analysis means 20, an optimum map generation means 30, an optimum map recording means 40, an optimum map synthesis means 50, an optimal viewpoint calculation means 60, and an optimum image generation means 70.

[0014] The CG scene input means 10 inputs a CG scene including a plurality of objects. For example, the CG scene is three-dimensional data obtained by photographing a scene in real space using volumetric capture technology, but the method of generating the CG scene is not particularly limited. The CG scene is composed of two or more objects (3D models). Furthermore, the contents of the CG scene and the types of objects are not particularly limited. The CG scene input means 10 outputs the input CG scene to the CG scene analysis means 20 .

[0015] The CG scene analysis means 20 analyzes the CG scene input from the CG scene input means 10 and breaks it down into each object. Since a CG scene is usually described in object units, the CG scene analysis means 20 can break down the CG scene into objects by analyzing the 3D data of the CG scene. The CG scene analysis means 20 then outputs each of the broken down objects to the optimum degree map generation means 30.

[0016] The optimum map generating means 30 generates a spherical optimum map for each object included in a CG scene. In this embodiment, the optimum map generating means 30 generates an optimum map by a general method for each object input from the CG scene analyzing means 20. Then, the optimum map generating means 30 writes the generated optimum map in the optimum map recording means 40.

[0017] <Optimal map> The optimality map of an object will now be described with reference to FIGS. FIG. 2 shows images V of the same object T (e.g., a house) viewed from various viewpoints, such as the front view, an oblique view from the front side, or an oblique view from the back side. The desirability of the image V captured (rendered) by the virtual camera varies depending on the viewpoint, but there is commonality between many people. The optimality map M in FIG. 3(a) and FIG. 3(b) shows the degree of desirability when the object T is observed from all directions. With this optimality map, it is possible to determine the most desirable viewpoint for the object T by finding the viewpoint that maximizes desirability.

[0018] As shown in FIG. 3(a) and FIG. 3(b), the optimality map M is a spherical map indicating the optimality when each point on a sphere centered on the object T is set as the viewpoint of the object T, and is also called a heat map. The central coordinates of the optimality map M correspond to the central position (center of gravity position) of the object T. Also, in the optimality map M, the shade of the dot at each point on the sphere indicates the optimality when the object T is viewed from that point. In other words, each point on the sphere of the optimality map M is the viewpoint of the object T. As shown in FIG. 3(c), in the optimality map M, the darker the dot, the higher the optimality, and the lighter the dot, the lower the optimality.

[0019] Fig. 3(a) illustrates the optimum map M when the object T is viewed from the front side, and Fig. 3(b) illustrates the optimum map M when the object T is viewed from the back side. In this way, the optimum map M shows that the optimum degree differs depending on the viewpoint, even for the same object T.

[0020] In Fig. 3, for ease of explanation, only a portion of the horizontal band of the entire optimum map M is dotted with light and dark shading. However, in reality, the optimum map M is a sphere dotted with light and dark shading drawn outside the object T. In other words, the optimum map M is a sphere with the object T placed inside, and each point on the sphere represents the optimum with a dot shade.

[0021] Here, the optimum degree map M may indicate the degree of optimum degree by color. Fig. 4 shows an example of an image of the optimum degree map M in which the degree of optimum degree is expressed by color. In Fig. 4, the closer to red the color is, the higher the optimum degree is, and the closer to blue the color is, the lower the optimum degree is.

[0022] The optimality map M can be generated by a known image analysis method such as a viewpoint entropy method. For example, a method for generating the optimality map M is described in Non-Patent Documents 1 and 2 or Reference 1, and therefore further description will be omitted.

[0023] Reference 1: Shigeo Takahashi, Issei Fujishiro, Yuriko Takeshima, Tomoyuki Nishita, “A Feature-Driven Approach to Locating Optimal Viewpoints for Volume Visualization Shigeo” IEEE Visualization 2005, Proceedings, 63, 2005

[0024] Returning to FIG. 1, the description of the configuration of the optimum viewpoint detecting device 1 will continue. The optimum map recording means 40 is a recording device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) that records the optimum map M of each object T and a composite map, which will be described later.

[0025] The optimum map synthesis means 50 synthesizes the optimum maps M of each object T recorded in the optimum map recording means 40 into one synthetic map by using a mixed Gaussian distribution or a circumscribed sphere transfer process. In other words, the synthetic map is an optimum map that reflects the optimum of each object T. Then, the optimum map synthesis means 50 writes the synthetic map into the optimum map recording means 40. Below, a first example and a second example of a synthesis method for the optimum map M will be described in order.

[0026] <Example 1: Gaussian mixture distribution> First, the synthesis of the optimality map M using a Gaussian mixture distribution will be described. Here, the optimality map M is expressed in a spherical coordinate system. In this case, the optimality map M of the viewpoint in the spherical coordinates (θ, φ) is n is expressed by the following formula (1). Note that n is the number of objects T n It is an identification (ID) unique to each n (θ,φ) represents a Gaussian mixture distribution. The radius of the optimality map M is the radius of the object T n represents the distance when looking at object T n It can be set to any value greater than the radius of the circumscribing sphere.

[0027]

number

[0028] For example, consider a case where a house object T1 as shown in Fig. 5(a) and a castle object T2 as shown in Fig. 5(b) are included in the same CG scene. In this case, the optimality map M1 of the first object T1 is expressed by the following formula (2), and the optimality map M2 of the second object T2 is expressed by the following formula (3).

[0029]

number

[0030] For each object T n The optimality map M n A mixture of Gaussian distributions F n (θ,φ) are expressed by the following formula (4). Note that μ indicates the mean and Σ indicates the variance-covariance matrix. In other words, the optimality map synthesis means 50 uses the mixed Gaussian distribution F n The optimality map M such that (θ,φ) is the right-hand side of equation (1) n can be generated as a composite map.

[0031]

number

[0032] In addition, the Gaussian mixture distribution F n Each parameter (eg, μ, Σ) constituting (θ, φ) may be set in advance based on the results of a separate evaluation experiment by a human.

[0033] <Second example: Circumscribed sphere transfer processing> Next, a description will be given of synthesis of the optimality map M by the circumscribed sphere transfer process. In the following, for the sake of simplicity, it is assumed that there are two objects T1 and T2. 6, the center coordinates c1 of the optimization map M1 of the first object T1 are (x1, y1, z1), and the center coordinates c2 of the optimization map M2 of the second object T2 are (x2, y2, z2). In this case, the spherical coordinates of each point (each viewpoint) in the optimization map M1 of the first object T1 are expressed by the following formula (5), and the spherical coordinates of each point in the optimization map M2 of the second object T2 are expressed by the following formula (6). Here, r indicates the spherical radius of the optimization map M.

[0034]

number

[0035] As shown in FIG. 6, the circumscribed sphere G that circumscribes the two optimality maps M1 and M2 is expressed by the following formula (7). In other words, the optimality map synthesis means 50 uses formula (7) to synthesize each object T n The optimality map M n 6, X indicates the horizontal direction, Y indicates the vertical direction, and Z indicates the depth direction.

[0036]

number

[0037] Next, the optimum map synthesis means 50 n The optimality map M n Center of c n 6, the optimality map synthesis means 50 calculates the intersection point between the line passing through each point and the circumscribed sphere G. As shown in FIG. 6, the optimality map synthesis means 50 calculates the intersection point between the line passing through each point and the circumscribed sphere G. 11, p 12 A line l passing through 11, l 12 Then, the optimum map synthesis means 50 calculates the straight line l 11, l 12 and the intersection point a with the circumscribed sphere G 11, a 12 Furthermore, the optimum map synthesis means 50 calculates each point p 11, p 12 The optimality corresponding to the intersection point a of the circumscribed sphere is 11, a 12 Transfer to. For the sake of simplicity, the two points p 11, p 12 However, in reality, similar processing is performed for each point on the spherical surface.

[0038] As in the above, the optimum map synthesis means 50 extracts a point p on the sphere from the center c2 of the second optimum map M2. 21, p 22 A line l passing through 21, l 22 and the intersection point a with the circumscribed sphere G21, a 22 Then, the optimum map synthesis means 50 calculates each point p 21, p 22 The optimality corresponding to the intersection point a of the circumscribed sphere G is 21, a 22 In Fig. 6, the intersection point a 22 and point p 22 is the point of contact between the optimality map M2 and the circumscribed sphere G, and therefore is in the same position.

[0039] Here, as shown in Figure 6, the intersection point a of the circumscribed sphere G 12 and intersection point a 21 are in the same position. Therefore, the same intersection point a of the circumscribed sphere G 12 =a 21 The first optimality map M1 has a point p 12 The optimality corresponding to and the point p of the second optimality map M2 21 In other words, the optimality corresponding to each point of the two optimality maps M1 and M2 is reflected on each point of the circumscribed sphere G. In this way, the optimality map synthesis means 50 transfers the optimality of each object T n The circumscribed sphere G onto which the optimality corresponding to is transferred is synthesized as a synthesis map.

[0040] In addition, whether to use the mixed Gaussian distribution or the circumscribed sphere transfer process is manually set in the optimality map synthesis means 50. While the mixed Gaussian distribution in the first example has high speed calculation processing, the circumscribed sphere transfer process in the second example has the advantage that the placement coordinates of the object can be reflected in the optimality of each viewpoint, so that the optimal viewpoint can be detected more accurately.

[0041] In the first example of the Gaussian mixture distribution and the second example of the circumscribed sphere transfer process, the case where there are two objects T has been exemplified, but the case where there are three or more objects T can also be handled. In the second example of the circumscribing sphere transfer process, there may be a case where it is not possible to generate a circumscribing sphere G that circumscribes the optimality maps M of all objects T. In this case, the optimality map synthesis means 50 may generate a circumscribing sphere G that includes the optimality maps M of all objects T and has the smallest radius.

[0042] Returning to FIG. 1, the description of the configuration of the optimum viewpoint detecting device 1 will continue. The optimal viewpoint calculation means 60 calculates an optimal viewpoint from the composite map recorded in the optimum map recording means 40. In this embodiment, the optimal viewpoint calculation means 60 calculates an optimal viewpoint from the composite map by optimization calculation.

[0043] For example, assume that a composite map (mixed Gaussian distribution) of a CG scene including two objects T1 and T2 is expressed by the following formula (8): In this case, the optimal viewpoint in this CG scene is expressed by the following formula (9).

[0044]

number

[0045] The above-mentioned formula (9) cannot be solved analytically. Therefore, the optimal viewpoint calculation means 60 solves the formula (9) by a numerical optimization calculation (for example, the steepest descent method) to obtain the optimal viewpoint. The optimal viewpoint calculation means 60 then outputs the calculated optimal viewpoint to the optimal image generation means 70.

[0046] The optimal image generating means 70 places a virtual camera at the optimal viewpoint input from the optimal viewpoint calculating means 60, and generates an optimal image by capturing a CG scene with the virtual camera. In this embodiment, the optimal image generating means 70 generates the optimal image using a general method. For example, the optimal image generating means 70 renders an optimal image that looks like a CG scene captured with a virtual camera at the optimal viewpoint. Then, the optimal image generating means 70 outputs the generated optimal image to the outside.

[0047] [Operation of the optimal viewpoint detection device] The operation of the optimum viewpoint detecting device 1 will be described with reference to FIG. As shown in FIG. 7, in step S1, the CG scene input means 10 inputs a CG scene including a plurality of objects. In step S2, the CG scene analysis means 20 analyzes the CG scene input in step S1 and breaks it down into individual objects.

[0048] In step S3, the optimum map generating means 30 generates a spherical optimum map for each object included in the CG scene. For example, the optimum map generating means 30 generates the optimum map by using a known image analysis method such as a viewpoint entropy method. In step S4, the optimum map generating means 30 causes the optimum map recording means 40 to record the optimum map for each object.

[0049] In step S5, the optimality map synthesis means 50 synthesizes the optimality maps of each object into one synthetic map by using a mixed Gaussian distribution or a circumscribing sphere transfer process. When using a circumscribing sphere transfer process, the optimality map synthesis means 50 generates a circumscribing sphere that circumscribes the optimality map of each object, calculates the intersection between the circumscribing sphere and a straight line passing through each point from the center of the optimality map, and transfers the optimality corresponding to each point of the optimality map to the intersection of the circumscribing sphere. In this way, the optimality map synthesis means 50 generates a synthetic map that reflects the optimality of each object T.

[0050] In step S6, the optimum viewpoint calculation means 60 calculates an optimum viewpoint from the composite map of each object. For example, the optimum viewpoint calculation means 60 calculates an optimum viewpoint from the composite map by optimization calculation.

[0051] In step S7, the optimum image generating means 70 places a virtual camera at the optimum viewpoint calculated in step S6, and generates an optimum image by capturing the CG scene with the virtual camera. For example, the optimum image generating means 70 renders an optimum image as if the CG scene were captured by a virtual camera at the optimum viewpoint.

[0052] [Actions and Effects] As described above, the optimal viewpoint detection device 1 according to the embodiment synthesizes a synthesis map that reflects the optimality of each object, and therefore can detect the optimal viewpoint from a CG scene that includes multiple objects. In recent years, technology that measures the entire space in 3D, such as volumetric photography studios, has been attracting attention. By using the optimal viewpoint detection device 1, it becomes possible to automatically extract excellent expressions from the 3D measured CG scene, thereby improving the efficiency of video production.

[0053] (Modification) Although the embodiment has been described in detail above, the present invention is not limited to the above-described embodiment, and includes design modifications and the like within the scope of the present invention. In the above embodiment, an example including two objects has been described, but even when three or more objects are included, an optimum degree map can be generated in a similar manner to detect the optimum viewpoint.

[0054] In the above embodiment, the optimum viewpoint detection device is described as having an optimum map recording means, but this is not limited to this. For example, the optimum viewpoint detection device does not need to have an optimum map recording means. In this case, the optimum map generation means outputs the optimum map directly to the optimum map synthesis means, and the optimum map synthesis means outputs the synthesis map directly to the optimum viewpoint calculation means.

[0055] In the above embodiment, the optimum viewpoint detection device is described as having an optimum image generation means and outputting an optimum image, but this is not limited to this. In other words, the optimum viewpoint detection device may output the calculated optimum viewpoint without having an optimum image generation means. In this case, the optimum viewpoint of the CG scene may be converted into coordinates of the real space, and an aerial photography drone or robot camera may be moved to the optimum viewpoint in the real space to perform live photography.

[0056] In the above-described embodiment, the optimum viewpoint detecting device is described as an independent piece of hardware, but the present invention is not limited to this. For example, the present invention can be realized by a program that causes hardware resources such as a CPU, memory, and hard disk of a computer to operate as the optimum viewpoint detecting device. These programs may be distributed via a communication line, or may be written on a recording medium such as a CD-ROM or a flash memory and distributed. [Explanation of symbols]

[0057] 1. Optimal viewpoint detection device 10 CG scene input method 20 CG scene analysis methods 30 Optimal map generation means 40 Optimal map recording means 50 Optimal map synthesis means 60 Optimal viewpoint calculation method 70 Optimal Image Generation Method

Claims

1. An optimal viewpoint detection device for detecting an optimal viewpoint from a CG scene including a plurality of objects, comprising: an optimum map generating means for generating, for each object included in the CG scene, a spherical optimum map indicating optimum when each point on a sphere centered on the object is set as a viewpoint of the object; an optimality map synthesis means for synthesizing the optimality maps of the objects into one synthesized map by a mixed Gaussian distribution or a circumscribed sphere transfer process; an optimal viewpoint calculation means for calculating the optimal viewpoint from the synthesis map; An optimal viewpoint detection device comprising:

2. The optimality map synthesis means performs the circumscribed sphere transfer process as follows: The optimal viewpoint detection device of claim 1, characterized in that it performs a process of generating a circumscribing sphere that circumscribes the optimality map of each object, calculating the intersection between the circumscribing sphere and a straight line passing through each point from the center of the optimality map, and transferring the optimality corresponding to each point of the optimality map to the intersection of the circumscribing sphere, thereby synthesizing the circumscribing sphere to which the optimality has been transferred as the composite map.

3. 3. The optimum viewpoint detection device according to claim 1, wherein the optimum viewpoint calculation means calculates the optimum viewpoint from the synthesis map by an optimization calculation.

4. an optimum image generating means for generating an optimum image by placing a virtual camera at the optimum viewpoint and capturing the CG scene with the virtual camera; 4. The optimum viewpoint detection device according to claim 1, further comprising:

5. A program for causing a computer to function as the optimum viewpoint detection device according to any one of claims 1 to 4.

Images