Image processing device, image processing system, and program for performing image composition in three-dimensional computer graphics

JP2024160668AInactive Publication Date: 2024-11-14北畑庆
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023084209
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-01
Publication Date
2024-11-14
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Benefits of technology

【0009】 本開示の3DCGにおける画像処理装置、画像処理システム及びプログラムによれば、3DCGにおける仮想空間上において対象オブジェクトを平行投影で描画しつつ、透視投影されている周辺(背景画像)を合成することができる。これにより、人の視覚上、自然で違和感のない視覚効果を得ることが可能となり得る。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024160668000001
    Figure 2024160668000001
  • Figure 2024160668000002
    Figure 2024160668000002
  • Figure 2024160668000003
    Figure 2024160668000003
Patent Text Reader

Abstract

To solve the problem that, when the same projection technique is applied to all objects to be drawn on a three-dimensional virtual space, a sense of incongruity is caused in the drawing such as a background and a shadow because a projection technique is different from that of another object even if only a specific object is drawn by another projection technique, and then to draw only the specific object by another projection technique on the three-dimensional virtual space and compose the specific object and an environment such as the background and the shadow without the sense of incongruity.SOLUTION: A device and a program for performing background composition of an object displayed on a three-dimensional virtual space are provided. The device creates, for an arbitrary number of objects, a virtual camera for parallel projection and a virtual camera for perspective projection of a foreground and background corresponding to a target object, allocates objects to be drawn by the virtual camera for parallel projection and the virtual camera for perspective projection of background which are created for an object to be a target, sets a drawing order and projection parameters, and composes the target object and a surrounding environment.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a technique for expressing images in a virtual space using three-dimensional computer graphics (hereinafter referred to as 3DCG). [Background technology]

[0002] 3DCG (3DCG: three-dimensional computer graphics, hereinafter 3DCG refers to 3DCG animation in a virtual space constructed on a computer or network) can be used in all fields such as movies, games, and animations. In one example of this type of technology, the virtual space is rendered using one virtual camera. In addition, a technology is also known in which multiple virtual cameras are used and the projection method and / or rendering method are appropriately changed for each virtual camera.

[0003] For example, Patent Document 1 discloses a technology that uses multiple virtual cameras, sets a shooting range in which parallel projection can be performed and a shooting range in which perspective projection can be performed according to each virtual camera, and switches the projection method according to the distance to the subject (object). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication 2014-096701 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology of the above-mentioned Patent Document 1 is configured to switch the projection method according to the distance from the object. When parallel projection and perspective projection are switched according to the distance, the set projection method can be applied to all objects based only on the distance regardless of the characteristics of the object. Here, when an object is projected by perspective projection, distortion due to perspective projection occurs and a change in the sense of depth occurs. This makes the object look three-dimensional. In other words, in perspective projection, the perspective law is applied to the projection of the object, so that a visual effect similar to that when a real object is viewed with the actual human eyes can be obtained. When objects are parallel projected, there is no distortion or sense of depth as in perspective projection, and therefore the sense of distance between objects is lost. In other words, perspective is not applied in parallel projection. On the other hand, individual objects themselves can be recognized without distortion. Parallel projection can be used, for example, to draw blueprints. There is a demand for a technology that can seamlessly combine a specific object with the surrounding environment, such as the background and shadows, based on the differences between the characteristics of perspective projection and parallel projection. [Means for solving the problem]

[0006] One aspect of the present disclosure is an image processing device that performs compositing of one or more objects of three-dimensional computer graphics. The image processing device includes a parallel projection virtual camera setting unit, a foreground perspective projection virtual camera setting unit, a rearview perspective projection virtual camera setting unit, and a main processing unit. The parallel projection virtual camera setting unit sets a parallel projection virtual camera for a target object, which is at least one arbitrary object among the one or more objects, to parallel project the target object. The foreground perspective projection virtual camera setting unit sets a foreground perspective projection virtual camera that perspectively projects a foreground object that is located relatively forward of the target object in the background corresponding to the target object. The rearview perspective projection virtual camera setting unit sets a rearview perspective projection virtual camera that perspectively projects a rearview object located relatively behind the target object in the background corresponding to the target object. The main processing unit includes an allocation unit, a drawing order setting unit, a parameter setting unit, and a composition unit. The allocation unit allocates, for each object, a first object to be drawn with the parallel projection virtual camera set for the target object, and a second object to be drawn with the foreground perspective projection virtual camera and / or the rear perspective projection virtual camera. The drawing order setting unit sets the drawing order of the objects for each object. The parameter setting unit sets projection parameters for each object. The composition unit composes the individual objects. In the following, three-dimensional computer graphics will also be referred to as 3DCG.

[0007] In addition, in the image processing device of the present disclosure, the main processing unit has a first processing unit that performs processing to dynamically change the drawing settings of the parallel projection virtual camera set for the target object, and / or the drawing settings of the foreground projection perspective projection virtual camera and / or the background projection perspective projection virtual camera set for the background, depending on the distance between an arbitrary position of the target object and a virtual main camera, which is a perspective projection virtual camera set at a position corresponding to a desired viewpoint. In addition, in the image processing device of the present disclosure, the main processing unit has a second processing unit that controls the drawing order setting unit and the compositing unit so that the drawing order setting unit sets the drawing order for each object in order of distance from the virtual main camera, and the compositing unit calculates the size of any parallel projection plane of a projection target based on the angle of view of the virtual main camera and performs compositing processing. In addition, the image processing device of the present disclosure has a third processing unit that sets the parallel projection camera as a virtual main camera set for the target object, calculates the angles of view of the foreground perspective projection virtual camera and the background perspective projection virtual camera from the size of the parallel projection plane, and performs processing to draw the target object by parallel projection on the parallel projection plane and to draw the background corresponding to the target object by perspective projection. Here, the size of the parallel projection plane of the virtual parallel projection camera corresponding to each target object may be changed based on the angle of view of the perspective projection of the virtual main camera, and the images to be drawn may be synthesized using the created virtual camera.

[0008] In addition, in the present disclosure, parallel projection may be used for the virtual main camera, and the angle of view of the perspective projection onto which the background is projected may be calculated from the size of the parallel projection plane. Furthermore, one aspect of the present disclosure may be an image processing system that performs the above-described method, or may be a program that causes a computer to execute the above-described method. Effect of the Invention

[0009] According to the 3DCG image processing device, image processing system, and program of the present disclosure, a target object can be rendered in a virtual space in 3DCG by parallel projection while synthesizing the surroundings (background image) that are perspective projected. This can produce a visual effect that is natural and does not seem strange to the human eye. [Brief description of the drawings] [Figure 1] 1 is a diagram showing an example of an arrangement of a virtual camera and an object to be drawn in a virtual space according to the present disclosure. [Diagram 2] 1 is a block diagram illustrating a configuration of an image processing device according to an embodiment of the present disclosure. [Diagram 3] 1 is an explanatory diagram showing the position of a target object and a created virtual camera in a scene setting unit of an image processing device according to the present disclosure; [Figure 4] 13 is a flowchart illustrating an example of processing in a virtual camera synthesis unit of the image processing device of the present disclosure. [Diagram 5] 1 is an explanatory diagram illustrating an example of an image rendering order in a virtual camera synthesis unit of an image processing device according to the present disclosure. [Figure 6] 11 is an explanatory diagram showing an example of a process for calculating the size of the projection plane of the virtual parallel projection camera from the virtual perspective projection camera in the virtual camera synthesis unit of the image processing device of the present disclosure; FIG. [Figure 7] 11 is an explanatory diagram illustrating an example of a projection result in a virtual camera synthesis unit of the image processing device of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In the present disclosure, animation data refers to animation values ​​of a three-dimensional computer graphics (3DCG) object and a virtual camera. As shown in Fig. 1, a virtual space is taken as an example in which a main perspective projection virtual camera (hereinafter also referred to as a virtual main camera) and objects to be rendered (Fig. 1: A1 Object1, A2 Object2) exist in the virtual space. The virtual space is a virtual space constructed on a computer or network, and is a virtual space on three-dimensional computer graphics (3DCG). The light source in the virtual space is a parallel light source oriented to illuminate the ground obliquely from the position of the virtual main camera in Fig. 1 toward the object.

[0011] 2 is a schematic diagram of an image processing device 1 according to an example of the present disclosure. The image processing device 1 includes a scene setting unit 3 and a virtual camera synthesis unit 5. Each component will be described in detail below.

[0012] The scene setting unit 3 determines a target object, which is an object to be parallel projected and designated by the user, from one or more objects in the virtual space. Then, it creates and initializes a virtual camera that renders the target object. This virtual camera may be a virtual main camera. 3 is a diagram showing the layout of virtual cameras created by the scene setting unit 3. It is assumed that A1 Object1 and A2 Object2 shown in FIG. 3 are the same as those in FIG.

[0013] 3 are projection targets (i.e., target objects), and for each target object, a virtual camera that renders the target object by parallel projection (parallel projection virtual camera), a virtual camera that renders the foreground by perspective projection (foreground perspective projection virtual camera), and a virtual camera that renders the background by perspective projection (background perspective projection virtual camera) are created. Note that, hereinafter, the parallel projection virtual camera, foreground perspective projection virtual camera, and background perspective projection virtual camera may collectively be referred to simply as "virtual cameras." As shown in Figure 3, for A1 Object1 (target object), a parallel projection virtual camera (C1) that parallel projects the target object, a foreground perspective projection virtual camera (B1) that draws the foreground using perspective projection, and a rear perspective projection virtual camera (B2) that draws the background using perspective projection are created. For target object A2 (Object2), a parallel projection virtual camera (C2) that parallel projects the target object, a foreground perspective projection virtual camera (B2) that perspectively projects the foreground, and a rear perspective projection virtual camera (B3) that draws the background are created. In FIG. 3, the position of the virtual main camera is the same as the position of the created perspective projection virtual camera. The parallel projection virtual camera for parallel projection draws only the target object. The foreground perspective projection virtual camera draws the section corresponding to the target object and the virtual main camera. The rear perspective projection virtual camera draws the range that is the background of the target object. In other words, the foreground perspective projection virtual camera draws the foreground of the background that is located relatively forward of the target object. The rear perspective projection virtual camera draws the background of the background that is located relatively behind the target object. The virtual cameras (parallel projection virtual camera, foreground perspective projection virtual camera, and rear perspective projection virtual camera) to be created are set to follow the virtual main camera. In this case, they do not need to be in the same position or orientation as the virtual main camera.

[0014] When there are multiple target objects, it is not necessary to set a foreground perspective projection virtual camera and a rear perspective projection virtual camera that perform perspective projection for all the target objects, and it is sufficient to set only one of them. For example, in FIG. 3, the projection range of the rear perspective projection virtual camera (B2) corresponding to A1 Object1 and the projection range of the foreground perspective projection virtual camera (B2) corresponding to A2 Object2 may be the same (both may be (B2)). For this reason, the setting of one of them may be omitted (in other words, it may be possible to omit the setting).

[0015] Here, a virtual projector that draws only the shadow of the target object may be set separately. The virtual projector draws the shadow corresponding to the target object from an arbitrary position. In this case, the direction, density, etc. of the shadow to be drawn are determined according to the setting of the light source. Specifically, the direction of the shadow is determined according to the irradiation direction of the light source. Also, the density of the shadow is determined according to the light intensity of the light source.

[0016] In the virtual camera synthesis unit 5 in Fig. 3, the settings of the virtual cameras are dynamically changed in accordance with the environment in the virtual space, and images generated by the virtual cameras are synthesized. Fig. 4 is a flowchart showing an example of processing in the virtual camera synthesis unit 5.

[0017] As shown in FIG. 4, the image processing device 1 first executes animation of the virtual main camera (step S101). Next, the distance between the virtual main camera and the target object is calculated (step S102). Then, the object corresponding to the virtual camera is assigned to the value calculated in step S103 (step S103). Next, the drawing order and drawing parameters for the created virtual camera are set (step S104). Then, it is determined whether or not to stop the playback (step S105). If it is determined that the playback should be continued (step S105=NO), the process from step S101 is repeated in the same manner. If it is determined that the playback should be stopped (step S105=YES), the process is terminated.

[0018] More specifically, in step S101, animation of the virtual main camera is played. Values ​​(parameters) related to the playback of animation may include the rotation, position, and angle of view of the perspective projection of the virtual main camera. In other words, the virtual main camera may rotate, and the position and angle of view of the perspective projection of the virtual main camera may change. Note that the virtual camera set by the scene setting unit 3 is set to follow the animation of the virtual main camera.

[0019] In step S102, the distance between the virtual main camera and a reference position (hereinafter also referred to as the reference position) in the target object that is the subject of parallel projection is calculated. The reference position in the target object can be determined arbitrarily by the user. The reference position may be changed for each frame of the animation. The distance between the target object and the virtual main camera does not have to be a straight-line distance, and may be, for example, the depth or the perpendicular distance from the target object to the virtual main camera plane.

[0020] In step S103, the target objects are reassigned to each virtual camera. For the set foreground perspective projection virtual cameras and rearview perspective projection virtual cameras, settings are made so that the target objects are not drawn by unused virtual cameras. In addition, for target objects that are determined to be close to each other (distance is equal to or less than a predetermined distance), settings are made so that the same virtual camera is used to draw them. In this case, unused virtual cameras are made not to draw the target objects.

[0021] In step S104, the drawing order and drawing parameters of the virtual camera are set. The drawing order is set so that the target objects are drawn in descending order of distance from the virtual main camera based on the calculation in step S102. Fig. 5 shows the drawing order set in step S104. First, the perspective projection virtual camera shown at B3 in Fig. 3 draws a background object shown at D3 and a shadow of a target object shown at A2, that is, a shadow of F2. Next, the target object indicated by A2 is rendered using a parallel projection virtual camera indicated by C2 in Figure 3, and then the background object indicated by D2 and the shadow of the target object indicated by A1, which is a shadow of F1, are rendered using a perspective projection virtual camera indicated by B2 in Figure 3. Next, a target object indicated by A1 is rendered using a parallel projection virtual camera indicated by C1 in FIG. 3, and finally, a background object indicated by D1 is rendered using a perspective projection virtual camera indicated by B1 in FIG.

[0022] In the parameter setting in step S104, the size of the projection plane of the virtual camera for parallel projection is calculated from the angle of view of the virtual main camera for perspective projection. Fig. 6 shows an example of the case where the size of the virtual camera for parallel projection is calculated from the angle of view of perspective projection. θ is the angle of view of the virtual main camera, d, which is the length of the line segment OR, is the distance to the target object, and h, which is the length of the line segment QP, is the size of the parallel projection plane to be obtained. h is obtained from θ and d, and set in the parallel projection virtual camera. The process of step S104 is performed for each parallel projection virtual camera. In addition, a process is performed to change the positions of the front and rear faces of the view frustum, which is the rendering range, based on the distance calculated in step S102, and a process is performed to change the rendering range of the virtual camera that performs perspective projection based on depth data, etc. The rendering range of the background may have overlapping ranges for each virtual camera.

[0023] Ultimately, as shown in FIG. 7, the target objects indicated by A1 and A2 are parallel projected, but are drawn as if perspective is being applied to the target objects indicated by A1 and A2, and can be naturally and seamlessly combined with the shadows and background drawn using perspective projection.

[0024] In the processing of the virtual camera synthesis unit 5, even if an animation is not played back, if the animation value can be acquired, processing may be performed in advance based on the animation value, and the processing result may be stored.

[0025] In the above embodiment, the virtual main camera was a camera that performs perspective projection, and the size of the parallel plane of the parallel projection was calculated from the angle of view of the perspective projection to synthesize the object, but in a similar processing process, the virtual main camera may be a camera that performs parallel projection, the angle of view of the perspective projection is calculated from the size of the parallel projection, and the object may be synthesized.

[0026] The scene setting unit 3 and the virtual camera synthesis unit 5 are realized by one or more computers executing a predetermined program. Also included in the embodiments of the present disclosure are a method, a program, and a recording medium having the program recorded thereon, for causing a computer to function as the scene setting unit 3 and the virtual camera synthesis unit 5.

[0027] In the above embodiment, a humanoid object is used in the drawing as an example, but the present disclosure can be applied to any object, not limited to a humanoid object. Specifically, the present disclosure can be applied as long as animation data can be obtained for an object with animation. In addition, in the above embodiment, there are two target objects, but the number of target objects is arbitrary, and may be one, or may be three or more.

[0028] In addition, if the object can be animated in a virtual space, it can be applied not only to displays such as televisions, but also to head-mounted displays such as VR, and to AR on smartphones. In addition, the present disclosure is useful and has the above-mentioned effects on any display device that synthesizes images captured by a webcam in a virtual space. [Industrial Applicability] In conventional virtual space drawing, all objects are drawn using the same projection method. In the present disclosure, the target object is drawn using parallel projection, and the background is drawn using perspective projection. By compositing the parallel projected object with the perspective projected background (surrounding environment), a natural image without any sense of incongruity can be drawn. More specifically, by setting the drawing order in order of distance from the virtual main camera serving as the viewpoint, a visual effect as if the drawing was done using perspective can be obtained. That is, in the present disclosure, it is possible to realize natural drawing without any sense of incongruity as if the perspective method was applied while enjoying the advantages of both parallel projection and perspective projection. The present disclosure is useful and effective on any display device as long as it is an object that can be displayed in a virtual space. [Explanation of symbols] 1. Animation background composite device 3 Scene setting section 5 Virtual camera synthesis section

Claims

1. 1. An image processing apparatus for compositing one or more objects of three-dimensional computer graphics, comprising: a parallel projection virtual camera setting unit that sets a parallel projection virtual camera for a target object, which is at least one arbitrary object among the one or more objects, to parallel project the target object; a foreground perspective projection virtual camera setting unit that sets a foreground perspective projection virtual camera for perspectively projecting a foreground object that is located relatively forward of the target object in the background corresponding to the target object; a rear perspective projection virtual camera setting unit that sets a rear perspective projection virtual camera for perspectively projecting a background object that is located relatively behind the target object in the background corresponding to the target object; a main processing unit including: an allocation unit that allocates, for each of the objects, a first object to be drawn by the parallel projection virtual camera set for the target object, and a second object to be drawn by the foreground perspective projection virtual camera and / or the rear perspective projection virtual camera; a drawing order setting unit that sets a drawing order for each of the objects; a parameter setting unit that sets projection parameters for each of the objects; and a synthesis unit that synthesizes each of the objects; Image processing device.

2. 2. The image processing device according to claim 1, The main processing unit includes: a first processing unit that performs processing for dynamically changing rendering settings of the parallel projection virtual camera set for the target object and / or the foreground projection perspective projection virtual camera and / or the background projection perspective projection virtual camera set for the background, depending on a distance between an arbitrary position of the target object and a virtual main camera that is a perspective projection virtual camera set at a position corresponding to a desired viewpoint; Image processing device.

3. 3. The image processing device according to claim 2, The main processing unit includes: the drawing order setting unit sets the drawing order for each object in ascending order of distance from the virtual main camera; A second processing unit controls the drawing order setting unit and the compositing unit so that the compositing unit calculates the size of an arbitrary parallel projection plane of a projection target based on an angle of view of the virtual main camera and performs compositing processing. Image processing device.

4. 4. The image processing device according to claim 3, a third processing unit that performs processing to set the parallel projection camera as a virtual main camera that is set for the target object, calculate angles of view of the foreground perspective projection virtual camera and the rear perspective projection virtual camera from the size of the parallel projection plane, and render the target object by parallel projection on the parallel projection plane and render a background corresponding to the target object by perspective projection; Image processing device.

5. 1. An image processing system for compositing one or more objects in three-dimensional computer graphics, comprising: a parallel projection virtual camera setting unit that sets a parallel projection virtual camera for a target object, which is at least one arbitrary object among the one or more objects, to parallel project the target object; a foreground perspective projection virtual camera setting unit that sets a foreground perspective projection virtual camera for perspectively projecting a foreground object that is located relatively forward of the target object in the background corresponding to the target object; a rear perspective projection virtual camera setting unit that sets a rear perspective projection virtual camera for perspectively projecting a background object that is located relatively behind the target object in the background corresponding to the target object; a main processing unit including: an allocation unit that allocates, for each of the objects, a first object to be drawn by the parallel projection virtual camera set for the target object, and a second object to be drawn by the foreground perspective projection virtual camera and / or the rear perspective projection virtual camera; a drawing order setting unit that sets a drawing order for each of the objects; a parameter setting unit that sets projection parameters for each of the objects; and a synthesis unit that synthesizes each of the objects; Image processing system.

6. A program for causing a computer to function as an image processing device for synthesizing one or more objects of three-dimensional computer graphics, comprising: The computer includes: a parallel projection virtual camera setting unit that sets a parallel projection virtual camera for a target object, which is at least one arbitrary object among the one or more objects, to parallel project the target object; a foreground perspective projection virtual camera setting unit that sets a foreground perspective projection virtual camera for perspectively projecting a foreground object that is located relatively forward of the target object in the background corresponding to the target object; a rear perspective projection virtual camera setting unit that sets a rear perspective projection virtual camera for perspectively projecting a background object that is located relatively behind the target object in the background corresponding to the target object; a main processing unit including: an allocation unit that allocates, for each of the objects, a first object to be drawn by the parallel projection virtual camera set for the target object, and a second object to be drawn by the foreground perspective projection virtual camera and / or the rear perspective projection virtual camera; a drawing order setting unit that sets a drawing order for each of the objects; a parameter setting unit that sets projection parameters for each of the objects; and a synthesis unit that synthesizes each of the objects. program.

Citation Information

Patent Citations

  • Telecommunications equipment and method of telecommunications

    JP2014096701A