Image processing device, method, and program

Abstract

To render a virtual object that is a permeation object at a high speed while reducing discomfort in its appearance.SOLUTION: An image processing system comprises attribute determination means for determining the permeation attribute of a virtual object, setting means for setting a virtual viewpoint to observe the virtual object, relations determination means for determining the physical relationship among each of a plurality of polygons representing the virtual object and the virtual viewpoint, determination means for determining if a surface of the polygon is to be rendered for each polygon of the plurality of polygons based on the physical relationship among the permeation attribute of the virtual object, the virtual viewpoint, and the polygon, and rendering means for rendering the surface of the polygon that is determined to be rendered by the determination means.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a technique for displaying a virtual object. [Background technology]

[0002] In recent years, image display using 3D computer graphics (CG) technology has been used for a variety of purposes. 3D CG technology is required to express objects more realistically, and to display CG images quickly. In order to approximate the appearance of real objects, it is necessary to faithfully calculate all information such as light, materials, reflections, and the context of the objects. However, since there is often a trade-off between the processing required to approximate the appearance of real objects and the rendering speed, many innovations have been made to date, such as omitting some of the processing.

[0003] There are also several methods for using CG technology to render solid objects that transmit light, such as glass (hereinafter referred to as "transparent objects"). One method for rendering transparent objects, for example, is to render the entire scene, and if semi-transparent objects overlap, to render them in order from the deepest value. As such, rendering that is faithful to reality requires computational resources. Patent Document 1 proposes a method called depth peeling that requires less computational resources, but the current situation is that a high-performance GPU is still required.

[0004] A relatively computationally inefficient method for rendering transparent objects is to determine the front-to-back relationship of each transparent object and calculate the blending of colors due to transparency. In this case, calculation of the front-to-back relationship of faces within a single object is omitted, so if multiple faces of a single object, such as a cylinder or sphere, are visible overlapping, the front-to-back order of the faces may be drawn differently from the actual order during rendering. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] U.S. Patent No. 8,217,934 Summary of the Invention [Problem to be solved by the invention]

[0006] The above-mentioned method of calculating the blending of drawing colors due to transparency does not take into account the foreground and background of a single object, which can result in an unnatural appearance. In such cases, back-face culling, which omits the drawing of the back faces of transparent objects, is expected to reduce the unnatural appearance caused by unexpected patterns.

[0007] However, while back-face culling of all objects to be drawn contributes to faster processing, it can sometimes result in a visually flawed image. For example, if a virtual viewpoint is located inside a CG cube, and the virtual viewpoint is directed toward the outside of the cube, all of the faces of the cube will be back-faced from the inside of the cube, and will not be rendered. In this way, unconditionally culling all surfaces can cause unexpected side effects. [Means for solving the problem]

[0008] In order to solve the above problem, according to one aspect of the present invention, an image processing device includes attribute determination means for determining a transparency attribute of a virtual object, setting means for setting a virtual viewpoint for observing the virtual object, relationship determination means for determining a positional relationship between each of a plurality of polygons representing the virtual object and the virtual viewpoint, determination means for determining, for each of the plurality of polygons, whether a face of the polygon is to be drawn based on the transparency attribute of the virtual object and the positional relationship between the virtual viewpoint and the polygon, and drawing means for drawing the face of the polygon determined by the determination means to be the drawing target. a ratio determination means for determining the ratio of the virtual object to the display area; The determination means If the ratio is determined to be less than the threshold, all polygon faces of the virtual object are rendered. It is judged that: [Effects of the Invention]

[0009] According to the present invention, when rendering a virtual object that is a transparent object, high-speed rendering is achieved while reducing the sense of incongruity in appearance. [Brief explanation of the drawings]

[0010] [Figure 1] 10A and 10B are schematic diagrams illustrating how a semi-transparent object appears when culling is not performed and when culling is performed. [Figure 2] 1 is a block diagram illustrating an example of a functional configuration of an image processing apparatus according to a first embodiment. [Figure 3] 4 is a flowchart illustrating an example of a processing procedure according to the first embodiment. [Figure 4] 4 is a flowchart illustrating an example of a processing procedure according to the first embodiment. [Figure 5] 10 is a flowchart illustrating an example of a processing procedure in a modified example. [Figure 6] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an image processing apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0011] Preferred embodiments of the present invention will be described in detail below with reference to the drawings. Note that the configurations described in the following embodiments are representative examples, and the scope of the present invention is not limited to these specific configurations.

[0012] (Embodiment 1) In this embodiment, back surface culling is applied when rendering a CG model of a transparent object, thereby achieving light processing and appropriate CG rendering.

[0013] Figure 1 shows an example of a case where the rendering of a transparent object differs from the actual appearance. Figure 1 shows CG model 101 of a transparent object rendered with culling, and CG model 102 of a transparent object rendered without culling. In CG model 102, the bottom surface of a cylinder, which is actually the back surface, is rendered to look the same as the top surface, which differs from the actual appearance. Model 101 shows that the visual discomfort can be reduced by performing back surface culling on transparent objects.

[0014] 2 is a block diagram showing an example of the functional configuration of an image processing apparatus that realizes the above. An apparatus 201 is an image processing apparatus that realizes the first embodiment.

[0015] The model data acquisition unit 203 acquires model data from the CG model database 202. The transparent model determination unit 204 performs attribute determination on the transparency attribute of the acquired model data and checks whether it is a transparent object (semi-transparent). In this embodiment, the rendering of the model data of a transparent object involves back-face culling, which does not render polygons that are positioned so that their back faces face the virtual viewpoint.

[0016] The polygon face direction calculation unit 205 acquires the surface direction of polygon faces in order to perform back-face culling. The procedure for calculating the surface direction of polygon faces is determined, for example, by the arrangement of the polygon vertices. The rotation direction of the vertex arrangement order is taken into account, and settings are made such that if it is "clockwise," it is the "front side," and if it is "counterclockwise," it is the "back side." The normal direction of the polygon surface can be calculated by calculating the cross product of two vectors according to the order of the vertices.

[0017] A viewpoint setting unit 207 sets the position, posture, and direction of a virtual viewpoint from which model data is observed. This viewpoint setting may be obtained by acquiring a numerical value specified by the user via a UI, or may be acquired from an external device such as a camera or HMD (head mounted display) connected to the image processing device 201. A line of sight direction acquisition unit 208 acquires the line of sight direction of the virtual viewpoint.

[0018] The rendering determination unit 206 determines the relationship between the normal direction of the polygon face and the line of sight direction based on the direction of the polygon face acquired from the polygon face direction calculation unit 205 and the line of sight direction acquired from the line of sight direction acquisition unit 208, and determines the CG rendering method. The rendering processing unit 209 executes CG rendering processing based on the determination result of the image determination unit 206. The display unit 210 displays data of the model rendered by the rendering processing unit 209 on an external device.

[0019] Next, the hardware configuration of the image processing device 201 will be described. Fig. 6 is a block diagram showing the hardware configuration of the image processing device 201 in this embodiment. In Fig. 6, a CPU 6200 comprehensively controls each device connected via a bus. The CPU 6200 also reads and executes processing steps and programs stored in a read-only memory (ROM) 6300. The operating system (OS), as well as each processing program, device driver, etc. according to this embodiment are stored in the ROM 6300, temporarily stored in a random access memory (RAM) 6400, and executed appropriately by the CPU 6200.

[0020] The input I / F 6500 receives an input signal from an external device (image capture device) or the like in a format that can be processed by the image processing device 201. The output I / F 6600 outputs an output signal to an external device (display device) or the like in a format that can be processed.

[0021] Next, the processing procedure of this embodiment, which is realized by the above-mentioned device, will be described. Figures 3 and 4 are flowcharts explaining the procedure of the rendering process of this embodiment. Figure 3 shows the processing performed by the model data acquisition unit 203 and the transparent model determination unit 204 when model data is read, and determines whether the acquired model data is a transparent model.

[0022] In step S3100, the model data acquisition unit 203 acquires model data from the CG model database 202. In step S3200, the transmission model determination unit 204 determines whether the acquired model is a CG model of a transparent object. Whether the acquired model is a CG model of a transparent object is determined, for example, by whether the acquired model has semi-transparent material information. If the acquired model data has semi-transparent material information, the process proceeds to step S3300. In step S3300, the transmission model determination unit 204 turns on the back surface culling flag of the model data, and the process proceeds to CG rendering processing. The user may be able to set whether the transmission model determination unit 204 will determine the processing of the flag.

[0023] 4, model data is rendered based on the back-face culling flag and the virtual viewpoint direction set by the transparent model determination unit 204. This processing will be described.

[0024] In step S4100, viewpoint direction acquisition unit 208 acquires the direction of the virtual viewpoint. The direction of the virtual viewpoint is calculated, for example, from the direction of a virtual viewpoint coordinate system with the position and orientation of the virtual viewpoint in the world coordinate system as the origin. In step S4200, it is checked whether the back face culling flag of the model data is ON. If the back face culling flag is ON, the process proceeds to step S4300, and if not ON, the process proceeds to step S4700. In step S4300, polygon surface direction calculation unit 205 calculates the polygon surface direction. The normal direction of the polygon surface is calculated by calculating the cross product of two vectors in accordance with the order of the vertices.

[0025] In step S4400, the rendering determination unit 206 calculates the dot product of the virtual viewpoint direction acquired by the viewpoint direction acquisition unit 208 and the polygon surface direction. If the dot product is greater than 0, the process proceeds to step S4500; otherwise, the process proceeds to step S4600. In step S4500, the rendering determination unit 206 determines that the polygon faces backward relative to the virtual viewpoint, excludes it from the rendering target, and the process proceeds to step S4700.

[0026] In step S4600, models determined to be opaque by the transparent model determination unit 204 or polygons determined not to be back-facing polygons by the rendering determination unit 206 are set as rendering targets. In step S4700, a check is made to see if all polygons have been determined to be rendering targets. If there are any unprocessed polygons, the process proceeds to step S4200. If processing has been completed for all polygons, the process proceeds to step S4800.

[0027] As a result, high-speed rendering can be achieved while reducing the visual discomfort in CG rendering of transparent objects.

[0028] (Variation 1) In the first embodiment, whether a virtual object is translucent is determined based on whether the model data is set as a transparent object, but other determination methods may be used. For example, a method may be considered in which the translucency of a virtual object is determined by threshold processing.

[0029] 5 is a flowchart showing the processing procedure of the modified example. The steps up to S3200 are the same as those of the first embodiment, so a description thereof will be omitted. In step S5100, the transmittance of a model determined to be a transparent model is acquired. If the transmittance is greater than a preset threshold, the process proceeds to S3300.

[0030] (Other embodiments) In the first embodiment, whether or not to draw back faces (whether or not to turn on the back face culling flag) is determined based on whether or not the face is semi-transparent. However, other determination conditions may be added. For example, the ratio of the virtual object to the visible area (display area) may be used. If the ratio determination determines that the ratio is small (less than a threshold), the visual impact is small, so the back face culling flag may be turned off (all polygon faces may be drawn). This ratio may be determined using the number of pixels of the virtual object. Furthermore, the shape of the virtual object may be determined, and the back face culling flag may be turned off for objects with complex shapes. In this shape determination, if the surface area is large relative to the volume, it may be determined that the shape is complex.

[0031] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0032] 201 Image processing device 202 CG model database 203 Model Data Acquisition Unit 204 Transmission model judgement unit 205 Polygon surface direction calculation unit 206 Drawing judgment section 207 Viewpoint Setting Section 208 Gaze direction acquisition unit 209 Drawing processing unit 210 Display section

Claims

1. attribute determination means for determining a transparency attribute of a virtual object; a setting means for setting a virtual viewpoint from which the virtual object is observed; a relationship determination means for determining a positional relationship between each of a plurality of polygons representing the virtual object and the virtual viewpoint; a determining means for determining, for each of the plurality of polygons, whether a surface of the polygon is to be drawn based on a transparency attribute of the virtual object and a positional relationship between the virtual viewpoint and the polygon; a rendering means for rendering the faces of the polygons determined by the determining means to be the rendering targets; a ratio determination means for determining the ratio of the virtual object to the display area; The image processing device is characterized in that the determination means determines that all polygon faces of the virtual object are to be rendered when it is determined that the ratio is less than a threshold value.

2. 2. The image processing apparatus according to claim 1, wherein the ratio determining means determines the ratio using the number of pixels of the virtual object.

3. attribute determination means for determining a transparency attribute of a virtual object; a setting means for setting a virtual viewpoint from which the virtual object is observed; a relationship determination means for determining a positional relationship between each of a plurality of polygons representing the virtual object and the virtual viewpoint; a determining means for determining, for each of the plurality of polygons, whether a surface of the polygon is to be drawn based on a transparency attribute of the virtual object and a positional relationship between the virtual viewpoint and the polygon; a rendering means for rendering the faces of the polygons determined by the determining means to be the rendering targets; a shape determination means for determining the shape of a virtual object, The image processing device is characterized in that the determining means determines that all polygon surfaces of the virtual object are to be drawn when the virtual object is determined to have a complex shape.

4. 4. The image processing apparatus according to claim 3, wherein said shape determining means determines the shape of said virtual object using the volume and surface area of ​​said virtual object.

5. an attribute determination step of determining a transparency attribute of a virtual object; a setting step of setting a virtual viewpoint from which the virtual object is observed; a relationship determination step of determining a positional relationship between each of a plurality of polygons representing the virtual object and the virtual viewpoint; a determining step of determining, for each of the plurality of polygons, whether a surface of the polygon is to be rendered based on a transparency attribute of the virtual object and a positional relationship between the virtual viewpoint and the polygon; a drawing step of drawing the faces of the polygons determined to be the drawing targets in the determination step; a ratio determination step of determining a ratio of the virtual object to the display area, The image processing method according to claim 1, wherein, in the determining step, if it is determined that the ratio is less than a threshold value, it is determined that all polygon faces of the virtual object are to be rendered.

6. an attribute determination step of determining a transparency attribute of a virtual object; a setting step of setting a virtual viewpoint from which the virtual object is observed; a relationship determination step of determining a positional relationship between each of a plurality of polygons representing the virtual object and the virtual viewpoint; a determining step of determining, for each of the plurality of polygons, whether a surface of the polygon is to be rendered based on a transparency attribute of the virtual object and a positional relationship between the virtual viewpoint and the polygon; a drawing step of drawing the faces of the polygons determined to be the drawing targets in the determination step; a shape determination step of determining a shape of a virtual object, The image processing method according to claim 1, wherein in the determining step, if the virtual object is determined to have a complex shape, it is determined that all polygon surfaces of the virtual object are to be rendered.

7. 5. A program for causing a computer to function as each of the means of the image processing apparatus according to claim 1.

Images