Method and program for generating image of three-dimensional object

Abstract

[Problem] To allow a computer to more easily draw a three-dimensional object that is to serve as a model for a drawing by a user in a drawing plane using a composition that better corresponds to the intentions of the user in association with the movement of a vanishing point when the user is using drawing software to draw a picture on a virtual canvas by perspective projection. [Solution] The present invention is a method wherein a virtual camera provided in a virtual three-dimensional space is used to project a three-dimensional object in the three-dimensional space onto a drawing plane by perspective projection to generate an image of the three-dimensional object. The method involves updating the position and / or posture of the virtual camera in response to a change in the position of any of a plurality of vanishing points on the drawing plane that correspond to infinity along a coordinate axis on a known rectangular coordinate system in the three-dimensional space and generating the image of the three-dimensional object on the basis of the updated position and / or posture of the virtual camera. The updating involves changing the position and / or posture of the virtual camera by applying processing that returns the updated position and / or posture of the virtual camera to the position and / or posture of the virtual camera before a change in the position of any of the plurality of vanishing points when the changed position of a vanishing point is returned to the position of the vanishing point before the change in the position of any of the plurality of vanishing points.

Description

Method and program for generating an image of a three-dimensional object

[0001] The present invention relates to a method and a program for generating an image of a three-dimensional object.

[0002] Conventionally, when drawing three-dimensional objects such as illustrations using drawing software, a perspective ruler based on perspective (abbreviated as "perspective") is sometimes used.

[0003] A perspective ruler is used to create accurate perspective drawings based on the laws of perspective when drawing an illustration. When a user draws an object, they place the perspective ruler around the object they want to draw. For example, if the object is box-shaped, they place the perspective ruler along the shape of the box. Perspective lines can be drawn along the perspective ruler. When drawing lines, users can use the perspective ruler to draw along the guide lines of the perspective ruler, so they can draw accurate straight lines that follow perspective even when drawing by hand. By repeating the same procedure, they can draw illustrations that follow perspective on a two-dimensional canvas. In typical illustration composition, perspective is generally created using a linear projection method called perspective projection.

[0004] For example, Patent Document 1 discloses a technology that discloses a perspective ruler placed on a planar canvas existing in a virtual three-dimensional space. This perspective ruler is displayed by a computer so that lines are drawn on the canvas along the perspective ruler in response to a user's instruction to draw a fisheye lens representation.

[0005] Furthermore, for example, Non-Patent Document 1 discloses a technology for placing a 3D human body object on a 3D layer that defines a perspective ruler. When the vanishing point or eye level is moved, the angle of the 3D layer changes accordingly, and the appearance of the 3D object also changes accordingly. However, when the vanishing point is moved multiple times, the size of the 3D object gradually changes, resulting in an unintended composition of the picture, making the function provided inconvenient.

[0006] Patent No. 7305241

[0007] "Linking Perspective Rulers and 3D Layers - Using Perspective Rulers Part 2" https: / / tips.clip-studio.com / ja-jp / articles / 1116#1dff0c63 (Accessed July 24, 2023)

[0008] The disclosed technology aims to enable a computer to easily draw a three-dimensional object that serves as a model for the user's drawing on a drawing plane in a composition that more closely matches the user's intentions, in conjunction with the movement of a vanishing point, when a user uses drawing software to draw a picture on a virtual canvas using perspective projection.

[0009] The disclosed technology provides a method for generating an image of a three-dimensional object by using a virtual camera placed in a virtual three-dimensional space to project the three-dimensional object onto a drawing plane by perspective projection, the method comprising: updating a position and / or orientation of the virtual camera in response to a change in the position of at least one of a plurality of vanishing points that correspond to infinity on the coordinate axes of a known Cartesian coordinate system that exists in the three-dimensional space; and generating an image of the three-dimensional object based on the virtual camera whose position and / or orientation has been updated, wherein the updating includes changing the position and / or orientation of the virtual camera by applying a process that has the property that, when the position of the changed vanishing point is returned to the position of the vanishing point before the change in the position of at least one of the plurality of vanishing points, the position and / or orientation of the updated virtual camera is returned to the position and / or orientation of the virtual camera before the change in the position of at least one of the plurality of vanishing points.

[0010] According to the disclosed technology, when a user uses drawing software to draw a picture on a virtual canvas by perspective projection, a computer can easily draw a three-dimensional object that serves as a model for the user's drawing on the drawing plane in a composition that more closely matches the user's intentions, in conjunction with the movement of the vanishing point.

[0011] FIG. 1A shows the camera, rendering plane, reference coordinate system, and sample points viewed from a direction parallel to the rendering plane. FIG. 1B shows the arrangement of points Po, Ps, and vanishing point Vs, where point Ps and vanishing point Vs are located on opposite sides of point Po. FIG. 1C shows an example of arranging the rendering plane in a position different from that shown in FIG. 1A. FIG. 2A shows points Po, Ps, and vanishing point Vs on the rendering plane. FIG. 2B shows an example in which the foot of a perpendicular line drawn from point Ps_old to half-line Vs_newPo is set to Ps_new. FIG. 2C shows an example in which point Ps gradually approaches point Po by repeatedly moving the vanishing point. FIG. 2D shows an example in which the position of Ps_new is determined so that PoPs_old = PoPs_new. FIG. 2E shows an example in which Vs_new is too close to Po in the example of FIG. 2D. FIG. 3 shows an example of determining a new position O_new of the reference coordinate system origin O in the camera coordinate system using point Po and point Ps_new. FIG. 4A shows an example of setting point H_new as the foot of a perpendicular line cast from point H_old to line segment V1_newV2_new on the rendering plane 100. FIG. 4B shows an example of point H approaching V1 when the position of V2 changes successively without changing the position of V1. FIG. 4C shows an example of ... FIG. 5 is a diagram showing an example in which a new reference coordinate system 301 is defined and used instead of the origin O of the reference coordinate system 300. FIG. 6A is a diagram showing the positional relationship between vanishing points V1, V2, and V3 on the rendering plane 100, the camera Pc, and the three coordinate axes of the reference coordinate system 300. FIG. 6B is a diagram showing the characteristics of the vanishing point and camera in two-point perspective. FIG. 6C is a diagram showing a situation in which the position of the point obtained by projecting the origin of the reference coordinate system onto the rendering plane 100 does not change before and after changing the vanishing point position. FIG. 7A is a diagram showing an example in which PcO / PcPo does not change before and after changing the vanishing point position.FIG. 7B shows an example in which PcO remains unchanged before and after changing the vanishing point position. FIG. 7C shows an example in which the magnitude of the PcO component in the camera optical axis direction remains unchanged before and after changing the vanishing point position. FIG. 8A shows an example in which the position of point Po on the rendering plane 100 also changes as the vanishing point position is changed. FIG. 8B shows an example in which the foot of a perpendicular line drawn from point Ps_old to half-line Vs_newPo_new is set to Ps_new. FIG. 8C shows an example in which the position of Ps_new is determined so that Po_oldPs_old = Po_newPs_new. FIG. 8D shows an example in which PcO / PcPo remains unchanged before and after changing the vanishing point position. FIG. 8E shows an example in which PcO remains unchanged before and after changing the vanishing point position. FIG. 8F shows an example in which the magnitude of the PcO component in the camera optical axis direction remains unchanged before and after changing the vanishing point position. FIG. 9 is a diagram explaining perspective projection. 10A and 10B are diagrams illustrating an example of how the image 155 of the object changes when the positions of the vanishing points V1 and V3 are changed on the rendering plane 100. Fig. 11 is a diagram showing the hardware configuration of the embodiment.

[0012] A perspective ruler is provided with various functions when a user draws on a virtual canvas on a plane using an input interface (for example, a touch pen, a mouse, a touch panel, a tablet, a pointing device, etc.) It should be noted that in this specification, explanations of commonly known functions among conventional functions related to a perspective ruler may be omitted.

[0013] The following description is based on the assumption that the disclosed technology is primarily applied to a perspective ruler. However, it should be noted that the disclosed technology is not limited to application to a perspective ruler, and can be used for creating scenes in three-dimensional space, such as producing videos using 3DCG, and for quickly generating videos that estimate changes in the position and / or posture of objects in real space.

[0014] Furthermore, in this specification, a virtual three-dimensional space may be referred to. However, it should be noted that this virtual three-dimensional space is used to clearly explain the present technology (such as a method for generating a perspective ruler), and that the existence of this virtual three-dimensional space is not necessarily essential when a user draws using the present technology (such as a perspective ruler). The following describes the prerequisites for describing the embodiments.

[0015] (1) About the vanishing point When a straight line in three-dimensional space is drawn on a drawing plane using perspective projection with a camera set up within that three-dimensional space, the line at infinity will converge to a single point on the drawing plane. This convergence point is called the vanishing point of that line. The position of the vanishing point on the drawing plane can be found as the intersection of a line that passes through the camera position and is parallel to the line, and the drawing plane. However, if the line is parallel to the drawing plane, the vanishing point will be at infinity and will not appear on the drawing plane. A group of lines that are parallel to each other have the same vanishing point.

[0016] (2) Reference Coordinate System This invention focuses on one known Cartesian coordinate system that exists in three-dimensional space, and deals with the behavior when the position of the vanishing point of that coordinate axis is manipulated. This known Cartesian coordinate system is called the "reference coordinate system."

[0017] 9 is a diagram illustrating perspective projection (a drawing method based on perspective projection). By viewing an object 150 from a camera Pc, which corresponds to the eye in perspective projection, a computer can project the object 150 onto the drawing plane 100 and draw an image 155 of the object 150.

[0018] The position and orientation of the object 150 are specified by a known reference coordinate system 300, which will be described later. The reference coordinate system 300 has three coordinate axes 305, 306, and 307 that are orthogonal to each other, and an origin O.

[0019] Vanishing points V1, V2, and V3 of three-point perspective exist on the drawing plane 100. It goes without saying that drawing on the drawing plane 100 may be done using two-point perspective or one-point perspective.

[0020] Camera Pc is located on a line that passes through the orthocenter of triangle V1V2V3 formed by the three vanishing points and is perpendicular to the drawing plane, and satisfies ∠V1PcV2 = ∠V2PcV3 = ∠V3PcV1 = π / 2. In addition, PcV1 is parallel to coordinate axis 306 of reference coordinate system 300, PcV2 is parallel to coordinate axis 305 of reference coordinate system 300, and PcV3 is parallel to coordinate axis 307 of reference coordinate system 300. Therefore, once vanishing points V1, V2, and V3 are determined, the relative orientation between the camera and the reference coordinate system is determined.

[0021] Note that the position and orientation of object 150 are determined in reference coordinate system 300, and therefore, if the position of at least one vanishing point is changed, the relative orientation between the camera and the reference coordinate system changes accordingly, resulting in a change in image 155. Details of the method for determining the relative position between the camera and the reference coordinate system will be described later.

[0022] FIG. 10 is a diagram illustrating an example of how the image 155 of the object changes when the positions of the vanishing points V1 and V3 on the drawing plane 100 are changed.

[0023] Note that while this example shows an example in which the positions of two vanishing points are changed, it goes without saying that the same concept can be applied when the position of at least one vanishing point is changed, or when the positions of all three vanishing points are changed.

[0024] FIG. 10A shows the positions of the vanishing points V1 and V3 and an image 155a of the object 150 before the positions of the vanishing points V1 and V3 are changed.

[0025] 10B shows the positions of the vanishing points V1 and V3 and an image 155b of object 150 after the positions of the vanishing points V1 and V3 have been changed. The position of vanishing point V1 has been changed from V1a to V1b. The position of vanishing point V3 has been changed from V3a to V3b. An image 155b of object 150 is shown after the positions of vanishing points V1 and V3 have been changed.

[0026] In this way, the image of the object on the drawing plane 100 changes in response to the operator changing the position of at least one of the three vanishing points. By changing the position of at least one vanishing point, the operator can obtain an image of the object that looks as desired. Using the obtained image of the object as a model, the operator can easily draw a three-dimensional illustration that follows perspective drawing.

[0027] (3) Notation The line segment connecting point A and point B is written as "line segment AB." The length of line segment AB is written as "AB" or "length AB." Note that lines with no thickness and surfaces with no thickness are also included in three-dimensional objects.

[0028] (4) Position of camera Pc in three-point perspective If none of the three coordinate axes of the reference coordinate system 300 is parallel to the drawing plane 100, three vanishing points appear on the drawing plane 100. This state is called three-point perspective.

[0029] (5) Vanishing Points in Three-Point Perspective The following characteristics are known regarding vanishing points in three-point perspective. As shown in Figure 6A, let V1, V2, and V3 be the three vanishing points on the drawing plane 100 defined by the three coordinate axes of the reference coordinate system 300. Let Pc be the camera position. Let L be the line that passes through the orthocenter H of triangle V1V2V3 and is perpendicular to the drawing plane 100. Pc exists on line L. Furthermore, ∠V1PcV2 = ∠V2PcV3 = ∠V3PcV1 = π / 2. Furthermore, triangle V1V2V3 is always an acute triangle, and the orthocenter H is located inside triangle V1V2V3.

[0030] This characteristic can be used to determine the relative positional relationship between the drawing plane 100 and the camera Pc. Note that the relative posture between the drawing plane 100 and the camera Pc is usually set in advance and is not changed.

[0031] Furthermore, the lines PcV1, PcV2, and PcV3 are each parallel to one of the coordinate axes of the reference coordinate system 300. Therefore, it is possible to obtain the relative orientation between the camera Pc and the reference coordinate system 300. However, this condition alone does not determine the relative position between the camera Pc and the reference coordinate system 300.

[0032] (6) Position of camera Pc in two-point perspective If one of the coordinate axes of the reference coordinate system 300 is parallel to the drawing plane 100 and the other two are not parallel to the drawing plane 100, two vanishing points exist on the drawing plane 100 (the remaining one is located at infinity). This state is called two-point perspective.

[0033] (7) Vanishing Points in Two-Point Perspective The following characteristics are known regarding vanishing points in two-point perspective. As shown in Figure 6B, let V1 and V2 be the two vanishing points on the drawing plane 100 defined by two of the coordinate axes of the reference coordinate system 300. Let Pc be the camera position. Let M be the plane that contains the line segment V1V2 and is perpendicular to the drawing plane 100. Pc exists on plane M. Furthermore, ∠V1PcV2=π / 2. In other words, Pc exists on the circumference of a semicircle that exists on plane M and has the line segment V1V2 as its diameter (according to the inscribed angle theorem).

[0034] In this way, the relative positional relationship between the rendering plane 100 and the camera Pc cannot be determined from the vanishing point position alone. Therefore, in order to specify the positional relationship, some additional conditions must be added.

[0035] Once the relative positional relationship between the drawing plane 100 and the camera Pc is obtained, the relative orientation between the camera Pc and the reference coordinate system 300 can be obtained because the lines PcV1 and PcV2 are each parallel to one of the coordinate axes of the reference coordinate system 300. However, as with the three-point perspective, the relative position between the camera Pc and the reference coordinate system 300 is not determined.

[0036] (8) Methods of Each Embodiment Each embodiment deals with a method of changing the position and orientation of the camera Pc when the user performs an operation to change the vanishing point position.

[0037] <Embodiment 1 (Three-Point Perspective)> Assume that three vanishing points already exist on the drawing plane 100, and that the positions and orientations of the camera Pc and the reference coordinate system 300 have been obtained. From this state, assume that the position of at least one vanishing point has been changed by a user operation or the like.

[0038] As described above, the relative orientation between the camera Pc and the reference coordinate system 300 can be found from the position of the vanishing point after the change, but the relative position is not determined. Therefore, the following conditions are set to find the relative position between the camera Pc and the reference coordinate system 300.

[0039] <Method A1: Method using the origin of the reference coordinate system and sample points> A sample point is a point whose position coordinates are known in the reference coordinate system 300. The conditions are set as follows.

[0040] (Condition 1) The position of the point obtained by projecting the origin of the reference coordinate system onto the drawing plane 100 must remain unchanged before and after the vanishing point position is changed.

[0041] The setting of Condition 1 is illustrated in Figure 6C. Pc_old is the camera position before the vanishing point position is changed. Pc_new is the camera position calculated from the vanishing point after the vanishing point position is changed. H_old and H_new are the feet of the perpendicular line cast from camera Pc to the rendering plane 100 before and after the vanishing point position is changed, respectively. Point Po is obtained by projecting the origin O_old of the reference coordinate system before the vanishing point position is changed onto the rendering plane 100 using camera Pc_old. The origin O_new of the reference coordinate system after the vanishing point position is changed exists on the line connecting camera Pc_new and point Po, but its location on that line is unknown. In addition, the direction of each coordinate axis of the reference coordinate system 300 after the vanishing point position is changed is obtained as the direction of the line connecting Pc_new and each vanishing point. Since this alone is not enough to satisfy the conditions, the positions of the sample points projected onto the rendering plane 100 are also used. Specific methods for using this will be described later.

[0042] FIG. 1A shows a camera Pc, a drawing plane 100, a reference coordinate system 300, and a sample point S viewed from a direction parallel to the drawing plane 100. The camera position is Pc. The origin O and sample point S of the reference coordinate system 300 are projected to points Po and Ps on the drawing plane 100, respectively, using the camera Pc located at point Pc. The vanishing point on the drawing plane 100 of the line connecting points O and S is Vs. In other words, the line PcVs is parallel to the line OS. Points Pc, O, Po, S, Ps, and Vs all exist on the same plane. Therefore, points Po, Ps, and Vs exist on the same line.

[0043] Note that the arrangement of points Po, Ps, and Vs may be such that point Ps and vanishing point Vs are located on opposite sides of point Po, as shown in Figure 1B. However, points Po and Ps are always located on the same side of vanishing point Vs.

[0044] The distance from the camera Pc to the drawing plane 100 is arbitrary. Specifically, it is as follows: The content drawn on the drawing plane 100 is presented to the user as an image, and the user performs operations on that image, such as changing the vanishing point position. Meanwhile, the camera Pc and objects exist in a virtual three-dimensional space. The units of length on the image and the units of length in the three-dimensional space are independent of each other and can be set arbitrarily. For example, the length on the image can be expressed in pixels, and the unit of length in the three-dimensional space can be expressed in meters. The number of meters that one pixel corresponds to can be arbitrary, and changing this setting changes the distance from the camera Pc to the drawing plane 100.

[0045] Therefore, for example, it is possible to position the rendering plane 100 as shown in Figure 1C instead of Figure 1A. However, it is desirable to maintain consistency before and after changing the vanishing point position. For example, if the conversion ratio between pixels and meters used in the previous example is 1 pixel = k meters, it is desirable to keep the value of k the same before and after changing the vanishing point position. Figure 6C shows the position and orientation of the camera Pc and the reference coordinate system 300 before and after changing the vanishing point position, relative to the rendering plane 100. For example, if the position of one of the three vanishing points is changed, the positions of the remaining two vanishing points on the rendering plane 100 in Figure 6C will not change. If the value of k is kept the same before and after changing the vanishing point position, then in Figure 6C, 1 meter in the reference coordinate system 300 before and after changing the vanishing point position will be the same length, making explanation easier. Therefore, in the following explanation, it is assumed that the value of k is the same before and after changing the vanishing point position. The value of k can be any value within the range of k > 0. Furthermore, in any of the methods of the present invention described below, the results obtained (the relative position and orientation between Pc_new and the reference coordinate system 300) are the same regardless of the value of k.

[0046] 2A shows points Po, Ps, and vanishing point Vs on the drawing plane 100. The positions of points Ps and Vs before the vanishing point position is changed are Ps_old and Vs_old, respectively. The positions of points Ps and Vs after the vanishing point position is changed are Ps_new and Vs_new, respectively. Due to the condition settings described above, the position of point Po does not change before or after the vanishing point position is changed.

[0047] The points Po, Ps_old, and vanishing point Vs_old are known. The position of the vanishing point Vs_new is calculated as follows. The relative orientation between the camera Pc and the reference coordinate system 300 after the vanishing point position has been changed is calculated from the three vanishing points V1, V2, and V3 on the rendering plane 100. Since the position of the sample point S in the reference coordinate system 300 is known, this is used to calculate the direction of the line OS in the camera coordinate system. As in Figure 1A, the intersection of the line passing through point Pc and parallel to line OS with the rendering plane 100 is defined as Vs_new. In this way, even if the relative positions between the camera Pc and the reference coordinate system 300 are unknown, the relative orientation can be determined, and therefore the vanishing point Vs_new can be calculated. Point Ps_new exists on the half line Vs_newPo (which extends infinitely toward Po). The position of point Ps_new is determined using the following method.

[0048] <Method B1> As shown in Figure 2B, the foot of the perpendicular line dropped from point Ps_old to half line Vs_newPo is defined as Ps_new. In other words, the position of Ps_new is determined so that ∠Ps_oldPs_newPo = π / 2. In this method, point Ps_new is the point on the line PoVs_new that is closest to point Ps_old. Therefore, it is possible to minimize the change in the position of point Ps on the rendering plane 100.

[0049] However, since PoPs_new≦PoPs_old, there is a problem that point Ps approaches point Po each time process 11, process 12, and process 13 are repeated, as shown in FIG. 2C.

[0050] <Method B2> As shown in Figure 2D, the position of Ps_new is determined so that PoPs_old = PoPs_new. This method, because the length PoPs does not change even when the process is repeated, does not encounter the problem of Figure 2C in Method B1. However, as shown in Figure 2E, if Vs_new gets too close to Po, point Ps_new must be located on a semi-linear line extending Vs_newPo toward Po, resulting in the positional relationship shown in Figure 1B. While this arrangement is geometrically correct, changing the arrangement from Figure 1A to Figure 1B causes a sudden change in the orientation of the reference coordinate system 300 relative to the camera Pc. This means that the orientation of the camera Pc changes abruptly when based on the reference coordinate system 300, resulting in poor user experience.

[0051] This technique involves a transformation in which, when a user changes the vanishing point position on an image and then returns the vanishing point position to its original state, the position and / or orientation of the camera Pc returns to the state it was in before the user changed the vanishing point position on the image.

[0052] To put it generally, this method can be said to be a method involving a transformation in which the change in the state of the camera Pc before and after the movement of the vanishing point is reversible. Such a method involving a reversible transformation is a method that does not cause any inconvenience in practical use and can be said to be a fully usable method, so long as it uses a reversible transformation. It goes without saying that such a method involving a reversible transformation is a technology that falls within the technical scope defined in the claims.

[0053] <Method B3> The position of Ps_new is determined so that PoPs_old / PoVs_old=PoPs_new / PoVs_new.

[0054] As described above, the position of Ps_new is determined using one of methods B1, B2, or B3. Next, the new position O_new of the reference coordinate system origin O in the camera coordinate system is calculated using point Po and point Ps_new (Figure 3). Since it is already known that O_new exists on the line Pc_newPo, its location on that line can be calculated as follows.

[0055] After the vanishing point position has been changed, the origin and sample point of the reference coordinate system 300 are assumed to be O_new and S_new, respectively. First, the length of O_newS' in the figure is found. Point S' is a point on the line Pc_newPs_new, and the line segment O_newS' is parallel to the line segment PoPs_new. As described above, the orientation of the reference coordinate system 300 relative to the camera coordinate system is found based on the positions of the three vanishing points. Since the position coordinates of the sample points in the reference coordinate system 300 are known, the direction of the line O_newS_new relative to the camera coordinate system can be found. Furthermore, since the direction of the line segment S'S_new is the same as the direction of the line PcPs_new, the direction of the line segment S'S_new in the camera coordinate system is known. Furthermore, since the line segment O_newS' is parallel to the line segment PoPs_new, the direction of the line segment O_newS' in the camera coordinate system is known. Therefore, when we focus on △O_newS'S_new, the directions of its three sides are known, and the angles of its three angles are also known. Furthermore, since the position coordinates of the sample points in the reference coordinate system 300 are known, the length of side O_newS_new is also known. Generally, if the angles of the three angles of a triangle and the length of one side are known, the lengths of the remaining two sides can be found. Therefore, the length of side O_newS' can be found. Next, the length Pc_newO_new is found. Since △Pc_newPoPs_new and △PcO_newS' are similar, Pc_newPo:PoPs_new=Pc_newO_new:O_newS'. Therefore, Pc_newO_new can be obtained by Pc_newO_new=O_newS'*(Pc_newPo / PoPs_new).

[0056] This provides the position of the reference coordinate system 300 in the camera coordinate system. The position and orientation of the camera coordinate system are determined relative to the reference coordinate system 300. Since the position and orientation of the reference coordinate system 300 are known, the position and orientation of the camera Pc are identified.

[0057] <Methods using only the reference coordinate system origin (methods A2, A3, A4, A5)> In the above-mentioned method A1, the reference coordinate system origin and sample points were used, but the following methods (A2, A3, A4) use only the reference coordinate system origin without using sample points. Like method A1, methods A2, A3, and A4 all set the following condition: (Condition 1) The position of the point obtained by projecting the reference coordinate system origin onto the drawing plane 100 must not change before and after the vanishing point position is changed. In addition, the following condition is set for each: This added condition identifies the position of O_new on the line Pc_newPo.

[0058] <Method A2> As shown in Fig. 7A, PcO / PcPo is set to remain unchanged before and after the vanishing point position is changed. In other words, Pc_oldO_old / Pc_oldPo = Pc_newO_new / Pc_newPo is set.

[0059] <Method A3> As shown in Fig. 7B, PcO is set to remain unchanged before and after the vanishing point position is changed. In other words, Pc_oldO_old = Pc_newO_new.

[0060] <Method A4> As shown in FIG. 7C, the magnitude of the component of PcO in the camera optical axis direction (in other words, the component in the direction perpendicular to the rendering plane 100) is kept constant before and after the vanishing point position is changed.

[0061] <Method A5> Ensure that the area of ​​a specified three-dimensional object projected onto the drawing plane 100 remains unchanged before and after changing the vanishing point position. When method A2 is used, the size of an object to be drawn that exists near the origin of the reference coordinate system and is drawn on the drawing plane 100 remains almost unchanged before and after changing the vanishing point position.

[0062] When using methods A3 and A4, for example, if the vanishing point is moved to enlarge the triangle formed by the three vanishing points, the size of the drawing object located near the origin of the reference coordinate system drawn on the drawing plane 100 also increases.

[0063] Method A3 has a side effect that the size of the object to be drawn increases when the position of the object to be drawn as seen from the camera Pc moves away from the front of the camera Pc due to movement of the camera Pc, etc. Method A4 does not have this side effect.

[0064] <Setting the reference coordinate system 300> For the orientation of the reference coordinate system 300, the following can be used, for example (but not limited to): (1) World coordinate system (2) Object coordinate system For the origin position of the reference coordinate system 300, the following can be used, for example (but not limited to): (1) World coordinate system origin (2) Object coordinate system origin The world coordinate system is the most basic coordinate system that is set in advance in the three-dimensional space to be rendered. The object coordinate system is a coordinate system that is set in advance for "a certain object that exists in three-dimensional space."

[0065] There are no particular limitations on the combination of the position and orientation of the reference coordinate system 300. For example, the origin of the object may be used as the origin of the reference coordinate system 300, and the orientation of the world coordinate system may be used as the orientation of the reference coordinate system 300.

[0066] <Positions of sample points in reference coordinate system 300> The positions of the sample points may be set on the coordinate axes of the reference coordinate system 300. In this way, the vanishing point Vs coincides with one of the vanishing points V1, V2, or V3, eliminating the need for calculations to determine the position of Vs.

[0067] Furthermore, the coordinates of the sample points in the reference coordinate system 300 may always be the same, such as (100,0,0), or the coordinate values ​​may be changed for each processing. For example, the object origin may be used as the origin of the reference coordinate system 300, and the coordinates of the sample points may be changed depending on the distance between the camera Pc and the object. If the distance between the camera Pc and the object is short, the distance between the sample points and the origin may be made small, and if the distance is far, the distance between the sample points and the origin may be made large.

[0068] However, if the coordinates of the sample point in the reference coordinate system 300 are set to a fixed value, such as (100,0,0), and the position of the vanishing point corresponding to the direction (100,0,0) is changed significantly on the rendering plane 100, undesirable behavior may occur. For example, when using the above-mentioned method B1 or method B2, if the position of the vanishing point Vs changes significantly and the length of PoVs_new becomes shorter, the position of point Ps_new cannot be taken on the half-line Vs_newPo. The following method may be used as a countermeasure to correct the above-mentioned problem.

[0069] <Method C1> The coordinates of the sample point in the reference coordinate system 300 are selected from three points: (100,0,0), (0,100,0), and (0,0,100). The directions of these three points correspond to vanishing points V1, V2, and V3 on the rendering plane 100, respectively. The sample point corresponding to the vanishing point with the least positional change on the rendering plane 100 is used.

[0070] <Method C2> The coordinates of sample point S in the reference coordinate system 300 are found under the following conditions: (Condition C2-1) It is at a predetermined distance d from the origin of the reference coordinate system (for example, d=100). (Condition C2-2) The direction from Po to Ps on the rendering plane 100 does not change before and after the vanishing point position is changed. The coordinates of a sample point that meets these conditions can be found as follows.

[0071] Before the vanishing point position is changed, the matrix representing the orientation of the reference coordinate system 300 in the camera coordinate system is called M_old. The elements of the matrix are arranged such that each row of M_old is a vector representing each coordinate axis unit vector of the reference coordinate system 300 in the camera coordinate system. Similarly, after the vanishing point position is changed, the matrix representing the orientation of the reference coordinate system 300 in the camera coordinate system is called M_new. The matrix M_rot representing the rotation from M_old to M_new is found using the following equation: M_rot=M_old -1 M_new

[0072] M_rot can be considered as a rotation around a certain rotation axis. This rotation axis vector is called v_axis. There are several ways to calculate v_axis from M_rot, but for example, you can convert M_rot to a quaternion and normalize the imaginary part of that quaternion. The obtained v_axis is a vector expressed in the camera coordinate system, so you can convert v_axis to a row vector and use M_old -1 (or M_new -1 ) from the right side to convert it into a vector v_ref expressed in the reference coordinate system 300. v_ref=v_axisM_old -1 or v_ref=v_axisM_new -1 The obtained v_ref is multiplied by d to obtain the coordinates of the sample point S in the reference coordinate system 300 .

[0073] <Method for changing the position of point Po projected onto drawing plane 100> In the methods described so far, the position and / or orientation of camera Pc was determined by setting the condition that "the position of point Po projected onto drawing plane 100 from origin O of reference coordinate system 300 does not change before and after the vanishing point position is changed."

[0074] As an alternative shown in Figure 8A, a method using different condition settings will be described. In this method, as the vanishing point position is changed, the position of point Po on the rendering plane 100 also changes. The position of point Po after the vanishing point position change is found based on the position of the vanishing point after the vanishing point position change.

[0075] Let V1, V2, and V3 be the three vanishing points on the drawing plane 100. Let Va be the vector going from V1 to V2. Let Vb be the vector going from V1 to V3. As mentioned above, triangle V1V2V3 is an acute triangle, so Va and Vb are not parallel. The position of any point P on the drawing plane 100 can be expressed as a linear combination of vectors Va and Vb as in the following equation: P=V1+KaVa+KbVb

[0076] If Ka and Kb are given, the position of point P is uniquely determined. Conversely, if the position of point P is given, the values ​​of Ka and Kb are uniquely determined. The latter is derived as follows:

[0077] Let Na be the unit vector on the drawing plane 100 that is perpendicular to Va. Let Nb be the unit vector on the drawing plane 100 that is perpendicular to Vb. Utilizing the fact that Va・Na=0 and Vb・Nb=0, Ka and Kb can be found as follows. Note that "・" represents the dot product. Ka=((P-V1)・Nb) / (Va・Nb) Kb=((P-V1)・Na) / (Vb・Na) Using this, the position of point Po after the vanishing point position has been changed is determined using the following method. Let Po, V1, Va, and Vb before the vanishing point position has been changed be Po_old, V1_old, Va_old, and Vb_old, respectively. Let Po, V1, Va, and Vb after the vanishing point position has been changed be Po_new, V1_new, Va_new, and Vb_new, respectively. Find Ka and Kb that satisfy Po_old=V1_old+KaVa_old+KbVb_old. Use the following formula to calculate Po_new: Po_new=V1_new+KaVa_new+KbVb_new

[0078] The origin O_new of the reference coordinate system after the vanishing point position has been changed exists on the line connecting the camera Pc_new and the point Po_new, but where on that line it exists is unknown.

[0079] The process for determining the reference coordinate system origin O_new, which will be explained below, is almost identical to the process when the condition "the position of point Po does not change before and after the vanishing point position is changed" is set, as already mentioned. Therefore, it should be noted that only the processes that differ will be described. In method B1, as shown in Figure 8B, the foot of the perpendicular line drawn from point Ps_old to half-line Vs_newPo_new is set to Ps_new. In method B2, as shown in Figure 8C, the position of Ps_new is determined so that Po_oldPs_old = Po_newPs_new.

[0080] Method B3 determines the position of Ps_new so that Po_oldPs_old / Po_oldVs_old=Po_newPs_new / Po_newVs_new. Po in FIG. 3 corresponds to Po_new.

[0081] Method A2 prevents PcO / PcPo from changing before and after the vanishing point position change, as shown in Fig. 8D. That is, Pc_oldO_old / Pc_oldPo_old = Pc_newO_new / Pc_newPo_new.

[0082] Method A3 keeps PcO unchanged before and after the vanishing point position change, as shown in Fig. 8E. That is, Pc_oldO_old = Pc_newO_new.

[0083] As shown in FIG. 8F, technique A4 prevents the size of the component of PcO in the camera optical axis direction (in other words, the component perpendicular to the rendering plane 100) from changing before and after the vanishing point position is changed. <Vanishing Point Placement in Three-Point Perspective> As mentioned above, it is known that the triangle V1V2V3, whose three vertices are vanishing points V1, V2, and V3 in three-point perspective, is always an acute triangle. In other words, it is theoretically impossible for triangle V1V2V3 to be an obtuse triangle. Therefore, the following countermeasures can be considered. (Variation 1) In a user interface that allows triangle V1V2V3 to be an obtuse triangle, if triangle V1V2V3 becomes an obtuse triangle, the position and / or orientation of camera Pc is not updated to prevent the calculation for determining the camera position and / or orientation from failing. Thereafter, when triangle V1V2V3 returns to an acute triangle at the operator's instruction, the update of the position and / or orientation of camera Pc is resumed. (Variation 2) Another variation is to provide a user interface that limits the range of movement of the three vanishing points so that triangle V1V2V3 does not become an obtuse triangle. Note that such a user interface setting limits the range of movement of the vanishing points, which may cause the user to feel uncomfortable, so it is desirable to adopt Variation 1 above.

[0084] <Embodiment 2 (Two-Point Perspective)> A method for dealing with the case where the state after the vanishing point position has been changed is two-point perspective will be described. As mentioned above, in a two-point perspective state, the relative positional relationship between the rendering plane 100 and the camera Pc cannot be determined from the vanishing point position alone. Therefore, this problem is solved by utilizing the state of the camera Pc before the vanishing point position has been changed. Note that the state before the vanishing point position has been changed can be either two-point perspective or three-point perspective.

[0085] Let point H be the foot of the perpendicular line dropped from camera Pc to the rendering plane 100. Point H is a point on the line segment V1V2. Let point H before and after the change in vanishing point position be point H_old and point H_new, respectively. Let the vanishing points on the rendering plane 100 after the change in vanishing point position be V1_new and V2_new. Point H_new exists on the line segment V1_newV2_new. Therefore, point H_new is found using the following method.

[0086] 4A, the foot of a perpendicular line drawn from point H_old to line segment V1_newV2_new is set to point H_new on the drawing plane 100. However, this method has the following problem.

[0087] (a) Point H approaches the vanishing point. For example, as shown in Figure 4B, when the position of V1 remains unchanged and the position of V2 changes successively through processes 21, 22, and 23, point H approaches V1. This causes camera Pc to approach the drawing plane 100, increasing the angle of view of camera Pc and distorting the drawn image, which is undesirable.

[0088] (b) In cases such as those shown in Figures 4C and 4D, it is not possible to draw a perpendicular line from point H_old to line segment V1_newV2_new.

[0089] <Method D2> As shown in Figure 4E, the distance between the vanishing point and point H is maintained. Of V1 and V2, the one with the least change in position on the drawing plane 100 is designated Vm, and the other is designated Vn. Vm before and after the change in vanishing point position is designated Vm_old and Vm_new, respectively. Vn after the change in vanishing point position is designated Vn_new. H_new is determined so that Vm_oldH_old = Vm_newH_new. However, this method has the following problems.

[0090] (c) As shown in FIG. 4F, if the length V1_newV2_new becomes too short, it becomes impossible to take a point H_new on the line segment V1_newV2_new.

[0091] <Method D3> Maintain the distance ratio between the vanishing point and point H as follows. Let the positions of V1 and V2 before the position change be V1_old and V2_old, respectively, and the positions of V1 and V2 after the position change be V1_new and V2_new, respectively. Define H_new so that V1_oldH_old:H_oldV2_old=V1_newH_new:H_newV2_new.

[0092] As described above, Pc exists on the circumference of a semicircle whose diameter is V1V2 on a plane that includes line segment V1V2 and is perpendicular to the rendering plane 100. Therefore, once H_new is determined, the relative positional relationship between the rendering plane 100 and the camera Pc can be calculated. Thereafter, the position and orientation of the camera coordinate system with respect to the reference coordinate system 300 are calculated using a method similar to that used in three-point perspective.

[0093] <Axial direction reversal> The following points can be noted as characteristics of when a vanishing point exists at infinity. When the coordinate axes of the reference coordinate system 300 are parallel to the rendering plane 100, the vanishing points of those coordinate axes exist at infinity in two directions corresponding to the positive and negative directions of those coordinate axes. Therefore, depending on the specifications of the user interface for the vanishing point movement operation, when the vanishing point of a certain coordinate axis is moved to infinity, the vanishing point of that coordinate axis will appear on the opposite side.

[0094] In this case, from the viewpoint of geometric continuity, the vanishing vanishing point and the new vanishing point correspond to opposite sides of the coordinate axis. For example, if a vanishing point corresponding to the positive X-axis direction of the reference coordinate system 300 is moved to infinity and made to disappear, a vanishing point corresponding to the negative X-axis direction will appear on the opposite side.

[0095] However, this behavior is not necessarily desirable for users, so it is necessary to make a vanishing point appear on the same side as the vanished vanishing point. Therefore, the following method is used.

[0096] <Method E1> When you start an operation to move a vanishing point (such as dragging the mouse), record the positive and negative axial directions of each vanishing point, and while the operation continues (for example, while dragging the mouse), the recorded positive and negative directions are used when a vanishing point goes to infinity, disappears, and then reappears.

[0097] <Method of substituting another point for the origin O> Instead of the origin O of the reference coordinate system 300, a point Q whose position coordinates in the reference coordinate system 300 are known may be used. In this case, a new reference coordinate system 301 that satisfies the following two conditions can be defined and used (FIG. 5). (Condition q1) Point Q is the origin. (Condition q2) The orientation of each coordinate axis is the same as the orientation of each coordinate axis of the original reference coordinate system 300.

[0098] As mentioned above, a group of parallel straight lines have the same vanishing point. Therefore, the position of the vanishing point of each coordinate axis of the new reference coordinate system 301 is the same as the position of the vanishing point of each coordinate axis of the original reference coordinate system 300. Therefore, by using point Q and the new reference coordinate system 301, the method using the origin of the reference coordinate system 300 described above can be applied as is.

[0099] FIG. 11 is a diagram showing the hardware configuration of the embodiment.

[0100] The hardware configuration of the embodiment includes a CPU 4001, a ROM 4002 in which the program and data of the embodiment can be stored, a RAM 4003, a network interface 4005, an input interface 4006, a display interface 4007, and an external memory interface 4008. These pieces of hardware are connected to each other by a bus 4004.

[0101] The network interface 4005 is connected to a network 4015. The network 4015 may be a wired LAN, a wireless LAN, the Internet, a telephone network, or the like. The input interface 4006 is connected to an input unit 4016. The display interface 4007 is connected to a display unit 4017. The display unit 4017 may be realized by a plurality of display devices. The external memory interface 4008 is connected to a storage medium 4018. The storage medium 4018 may be a RAM, a ROM, a CD-ROM, a DVD-ROM, a hard disk, a memory card, a USB memory, or the like.

[0102] The order of steps in the methods or programs of the illustrated embodiments may be changed as long as there is no contradiction. Furthermore, one illustrated step may be executed multiple times at different times as long as there is no contradiction. Furthermore, multiple steps may be executed simultaneously as long as there is no contradiction. Furthermore, not all steps are essential, and some steps may not exist or may not be executed as long as there is no contradiction.

[0103] The same applies to the elements of the methods defined in the claims. That is, the order of the elements can be changed as long as there is no contradiction. Furthermore, multiple elements can be implemented simultaneously as long as there is no contradiction. The implementation of these elements also falls within the technical scope defined in the claims.

[0104] Each procedure may be executed by an operating system or hardware. The program may be distributed in a state stored in a non-transitory medium.

[0105] The programs and methods for realizing the above-described embodiments can be executed by a computer having the hardware configuration shown in Fig. 11. That is, the programs of the embodiments can be implemented as a method executed by a computer. The programs can be stored in the storage medium 4018, the ROM 4002, or the RAM 4003. Each embodiment can be implemented as a hardware device on which the programs are installed.

[0106] 4004 Bus 4005 Network interface 4006 Input interface 4007 Display interface 4008 External memory interface 4015 Network 4016 Input unit 4017 Display unit 4018 Storage medium

Claims

1. A method for a computer to generate an image of a three-dimensional object by using a virtual camera arranged in a virtual three-dimensional space to project a three-dimensional object existing in the three-dimensional space onto a drawing plane by perspective projection, the method comprising: updating a position and / or orientation of the virtual camera in response to a change in the position of at least one of a plurality of vanishing points that exist on the drawing plane and that correspond to infinity on the coordinate axes of a known Cartesian coordinate system existing in the three-dimensional space; and generating an image of the three-dimensional object based on the virtual camera whose position and / or orientation has been updated, the updating including changing the position and / or orientation of the virtual camera by applying a process having a property that, when the position of the changed vanishing point is returned to the position of the vanishing point before the change in the position of at least one of the plurality of vanishing points, the position and / or orientation of the updated virtual camera returns to the position and / or orientation of the virtual camera before the change in the position of at least one of the plurality of vanishing points.

2. The method of claim 1, further comprising updating the position and / or orientation of the virtual camera based on a position of at least one predetermined point in the Cartesian coordinate system projected onto the drawing plane by the virtual camera.

3. The method according to claim 2, further comprising: preventing a position of a point where one of the predetermined points is projected onto the drawing plane by the virtual camera from changing before and after changing the position of at least one of the plurality of vanishing points.

4. The method according to claim 2, further comprising: determining a position of a point at which one of the predetermined points is projected onto the drawing plane by the virtual camera after changing the position of at least one of the plurality of vanishing points, based on the positions of the plurality of vanishing points after changing the position of at least one of the plurality of vanishing points.

5. The method of claim 2, wherein there are two of the specified points in the orthogonal coordinate system, and the updating further comprises: making equal a ratio of a length from a point Po_old, obtained by projecting one of the two specified points onto the drawing plane, to a point Ps_old, obtained by projecting the other of the two specified points onto the drawing plane, before the change in the position of at least one of the multiple vanishing points, to a length from the point Po_old to a vanishing point Vs_old on the drawing plane corresponding to infinity on a straight line connecting the two specified points, and a ratio of a length from a point Po_new, obtained by projecting one of the two specified points onto the drawing plane, to a point Ps_new on the drawing plane on which the other of the two specified points is projected, after the change in the position of at least one of the multiple vanishing points, to a vanishing point Vs_new on the drawing plane corresponding to infinity on a straight line connecting the two specified points.

6. The method according to claim 2, wherein the two predetermined points exist in the orthogonal coordinate system, and the updating further comprises: making equal a length from a point Po_old, at which one of the two predetermined points is projected onto the drawing plane, to a point Ps_old, at which the other of the two predetermined points is projected onto the drawing plane, before the position of at least one of the plurality of vanishing points is changed, and a length from a point Po_new, at which one of the two predetermined points is projected onto the drawing plane, to a point Ps_new, at which the other of the two predetermined points is projected onto the drawing plane, after the position of at least one of the plurality of vanishing points is changed.

7. The method of claim 2, wherein the Cartesian coordinate system includes one predetermined point O, and the updating further includes: determining the position Pc_new so that a length from the position Pc_old of the virtual camera to the point O before changing the position of at least one of the plurality of vanishing points is equal to a length from the position Pc_new of the virtual camera to the point O after changing the position of at least one of the plurality of vanishing points.

8. The method of claim 2, wherein the Cartesian coordinate system includes one predetermined point O, and the updating further includes: determining the position Pc_new such that a component of a length from the position Pc_new of the virtual camera to the point O before changing the position of at least one of the plurality of vanishing points in the optical axis direction of the virtual camera is equal to a component of a length from the position Pc_new of the virtual camera to the point O after changing the position of at least one of the plurality of vanishing points.

9. The method of claim 2, wherein the Cartesian coordinate system includes one predetermined point O, and the updating further includes: determining the position Pc_new so that a ratio of a length from the position Pc_old of the virtual camera to the point O before changing the position of at least one of the plurality of vanishing points to a length from the position Pc_old to a point Po_old at which the point O is projected onto the drawing plane by the virtual camera is equal to a ratio of a length from the position Pc_new of the virtual camera to the point O after changing the position of at least one of the plurality of vanishing points to a length from the position Pc_new to the point O after changing the position of at least one of the plurality of vanishing points to a point Po_new at which the point O is projected onto the drawing plane by the virtual camera.

10. The method according to claim 2, wherein there are two said predetermined points in said Cartesian coordinate system, one of said two predetermined points being the origin of said Cartesian coordinate system, and the other of said two predetermined points being a point lying on any of the three orthogonal axes of said Cartesian coordinate system.

11. The method according to claim 2, wherein there are two predetermined points in the Cartesian coordinate system, and the two predetermined points are two points whose direction from one to the other in the coordinate system of the virtual camera does not change before and after a change in the position of at least one of the vanishing points.

12. The method of claim 1, wherein the updating further comprises: if the number of vanishing points on the drawing plane after the change in the position of at least one of the multiple vanishing points is two, the virtual camera after the change in the position of at least one of the multiple vanishing points is positioned on a perpendicular line erected on the drawing plane from the foot of a perpendicular line drawn from the virtual camera to the drawing plane before the change in the position of at least one of the multiple vanishing points, to a line segment connecting the two vanishing points after the change in the position of at least one of the multiple vanishing points.

13. The method of claim 1, further comprising: when the number of vanishing points on the drawing plane after the change in position of at least one of the multiple vanishing points is two, the one of the two vanishing points whose position has changed less on the drawing plane is designated as Vm, and the other is designated as Vn, Vm before and after the change in position of at least one of the multiple vanishing points is designated as Vm_old and Vm_new, respectively, Vn after the change in position of at least one of the multiple vanishing points is designated as Vn_new, the foot of a perpendicular line drawn from the virtual camera to the drawing plane before the change in position of at least one of the multiple vanishing points is designated as H_old, and the virtual camera after the change in position of at least one of the multiple vanishing points is positioned on a perpendicular line erected on the drawing plane from H_new, which is determined to be a point H_new on a line segment Vm_newVn_new such that Vm_newH_new=Vm_oldH_old.

14. The method of claim 1, further comprising: when the number of vanishing points on the drawing plane after the change in the position of at least one of the multiple vanishing points is two, the two vanishing points are designated as V1 and V2; the positions of the two vanishing points V1 and V2 before the change in the position of at least one of the multiple vanishing points are designated as V1_old and V2_old, respectively; the positions of the two vanishing points V1 and V2 after the change in the position of at least one of the multiple vanishing points are designated as V1_new and V2_new, respectively; the foot of a perpendicular line drawn from the virtual camera to the drawing plane before the change in the position of at least one of the multiple vanishing points is designated as H_old; and a point H_new on a line segment V1_newV2_new, where H_new is determined such that V1_oldH_old:H_oldV2_old=V1_newH_new:H_newV2_new, and the virtual camera after the change in the position of at least one of the multiple vanishing points is positioned on a perpendicular line erected on the drawing plane from H_new to the drawing plane.

15. The method of claim 1, wherein the updating further includes: when one of the multiple vanishing points moves to infinity and the one vanishing point once disappears and then reappears, a vanishing point corresponding to the vanishing point on the same side of the coordinate axis of the Cartesian coordinate system as the vanished side of the vanishing point reappears.

16. The method of claim 1, wherein the updating step does not update the position and / or orientation of the virtual camera if a triangle formed by three vanishing points in three-point perspective becomes an obtuse triangle.

17. A program for causing a computer to execute the method according to any one of claims 1 to 16.

Images