Image generation method and image generation program
By configuring a virtual camera in the virtual space and combining fine and simple image generation methods, the problem of difficult to determine the gate connection relationship in the virtual space is solved, reducing computing resource consumption, and improving user navigation experience and image generation efficiency.
Patent Information
- Application Number
- CN202280085416.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-12-22
- Filing Date
- 2022-08-09
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-08-09
AI Technical Summary
In virtual space, it is difficult for users to determine the door connection relationship between different rooms, resulting in lost problems, and the prior art consumes too much computing resources when generating images across the door, resulting in response delays.
By configuring the virtual camera in the user's virtual location, a fine image generation method is used to generate images in the virtual space to which the user belongs, and a simpler indoor mapping method is used to generate images in the virtual space that does not belong, reducing computing resource consumption, while on the door Set transparent glass to display images of the destination room.
It effectively reduces the storage and computing resources required to generate two-dimensional images in the three-dimensional virtual space, improves the user's navigation experience in the virtual space, reduces the possibility of getting lost, and improves the efficiency of image generation.
Smart Images

Figure CN118451473B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an image generation method and an image generation program. Background Art
[0002] There is a technology that allows a user to perceive images in a way that the user is in a virtual space, and allows the user to experience virtual reality through the user's vision, etc.
[0003] For example, in the prior art, an image processing system for an object having a three-dimensional shape arranged in a real environment that can be perceived by a user provides image recognition information and an image processing program, wherein the image recognition information is associated with an image corresponding to the object having a three-dimensional shape and is arranged on the object, and the image processing program recognizes the image recognition information and overlaps the image on the object. Furthermore, the prior art also includes a recognition means for recognizing the image recognition information, and the image processing program changes the degree of transparency of the image recognition information of the object according to the distance between the recognition means and the image recognition information when overlapping the image on the object (for example, refer to Patent Document 1).
[0004] In addition, in other prior art, there is provided an image of a virtual space in which a scene in front of a door can be seen through a virtual camera through a door. After the virtual camera passes through the door, the virtual camera captures images of other virtual spaces (for example, refer to Non-Patent Document 1).
[0005] Indoor mapping is known as a method of rendering the inside of a building (the floor, the roof, and the walls) without modeling (for example, refer to Non-Patent Document 2).
[0006] Here, if a plurality of doors for users to move around are provided in the virtual space, there is a problem that the user cannot determine which door connects to which room.
[0007] In particular, in the Metaverse-related technologies using the recent XR technology, the trend is to build a virtual world over a wide range, and the problem of users getting lost is becoming increasingly prominent.
[0008] Furthermore, for example, in a case where a user can move between multiple rooms via doors, it is desirable to visualize the direction of the doors in order to confirm which door is connected to which room. However, if adjacent rooms are rendered each time through the doors, the processing becomes an extremely large load, resulting in response delays, etc., and the environment is no longer enjoyable for the user.
[0009] Prior art literature
[0010] Patent Literature
[0011] Patent Document 1: Japanese Patent Application Publication No. 2021-60789
[0012] Non-patent literature
[0013] Non-patent document 1: "Interesting: [Free resource] The view on the other side can be clearly seen in front of the portal! A portal system called "World Portal System" that can teleport to other places through the door like a "anywhere door"" https: / / www.asset-sale.net / entry / World_Portal_System (accessed on December 20, 2021)
[0014] Non-patent document 2: Joost van Dongen "Interior Mapping-A new technique for rendering realistic buildings"
[0015] https: / / www.proun-game.com / Oogst3D / CODING / InteriorMapping / InteriorMapping.pdf (accessed on December 20, 2021). Summary of the invention
[0016] Technical problem to be solved by the invention
[0017] The disclosed technology aims to save resources such as storage capacity and processing unit calculation required when obtaining a two-dimensional image based on a three-dimensional virtual space. It should be noted that the processing unit includes a CPU and a GPU.
[0018] Technical means for solving technical problems
[0019] According to the disclosed technology, there is provided an image generation method, which generates an image of a three-dimensional virtual space captured from a user's virtual position or a virtual camera arranged around the user's virtual position, comprising:
[0020] determining a first virtual space to which the user belongs; and
[0021] Determine a single or multiple second virtual spaces that are the shooting objects in the virtual space to which the user does not belong; generate an image obtained by the virtual camera shooting the objects existing in the first virtual space through a first image generation method; and generate an image equivalent to the image obtained by the virtual camera shooting the objects existing in the second virtual space through a second image generation method.
[0022] In addition, according to the disclosed technology, there is provided an image generation method for generating an image of a three-dimensional virtual space captured by a virtual camera arranged at a virtual position of a user or around the virtual position of the user, the method comprising:
[0023] determining a first virtual space to which the user belongs,
[0024] Determine a single or multiple second virtual spaces that are the subject of the shooting among the virtual spaces to which the user does not belong;
[0025] generating, by a first image generating method, a first partial image obtained by photographing an object existing in the first virtual space by the virtual camera, that is, a first partial image of the image; and
[0026] generating a second partial image corresponding to an image obtained by photographing an object existing in the second virtual space by the virtual camera, that is, a second partial image of the image, by a second image generating method;
[0027] Compared with the first image generation method, the second image generation method is an image generation method that consumes less computing resources in the calculation required for generating an image corresponding to the object.
[0028] The image generation method also has:
[0029] When the user moves from the first virtual space to the second virtual space, so that the second virtual space becomes the new first virtual space and the first virtual space becomes the new second virtual space, a third partial image is generated by the first image generation method, which is obtained by photographing the object existing in the virtual space that becomes the new first virtual space with the virtual camera, and a fourth partial image equivalent to the image obtained by photographing the object existing in the virtual space that becomes the new second virtual space with the virtual camera is generated by the second image generation method, thereby generating a new image having the third partial image and the fourth partial image.
[0030] Furthermore, according to the disclosed technology, a program for causing a computer to execute the above-mentioned image generation method and a medium recording the program are provided.
[0031] Effects of the Invention
[0032] According to the disclosed technology, it is possible to save resources such as the memory capacity and the amount of calculation required by the processing device when converting a three-dimensional virtual space into a two-dimensional image. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1AThe video 100 is a video in which a plurality of objects 12 existing in the room 10A which is a three-dimensional virtual space are photographed through a glass 115 by a virtual camera 11 installed behind a user 20A existing in a room 10B which is a virtual space different from the room 10A. Figure 1B An image 100 obtained by photographing three-dimensional virtual spaces 10A and 10B using a virtual camera 11 is shown.
[0034] Figure 2 This is a diagram for explaining the technology of indoor mapping.
[0035] Figure 3 A to Figure 3 D is a diagram showing one example of an image generation method using the virtual camera 11A.
[0036] Figure 4 The diagram shows that a door 55 for an exit and a door 50 for an entrance are set in each of the room R1 , the room R2 , and the room R3 in different three-dimensional virtual spaces.
[0037] Figure 5 This is a diagram showing that although users enter the same room 400 via the same door 321 , in the case where different users enter, each user can return to a different room even via the same door 321 .
[0038] Figure 6 It is a hardware configuration diagram of an implementation method.
[0039] Figure 7 An overview of the processing flow of the embodiment is shown.
[0040] Figure 8 The following is a flow for simply presenting the user with the doors that they can pass through.
[0041] Fig. 9 An example is shown in which the transparency of the glass 115 is made different depending on the angle between the surface of a transparent material such as the glass 115 and the imaging direction of the virtual camera.
[0042] Fig.10 It is shown Fig. 9 The diagram shows the flow of image generation corresponding to the angle formed by the glass 115 and the shooting direction of the camera.
[0043] Fig.11 The following is a flow for increasing the sense of immersion in the three-dimensional virtual space for a user utilizing the three-dimensional virtual space.
[0044] Fig.12 A and Fig.12 B shows a processing flow for simply notifying the user of a second virtual space that is highly relevant to the user.
[0045] Fig.13 The flow of a method for generating switching images when the user moves in the virtual space is shown.
[0046] Fig.14 It is a diagram showing another embodiment of indoor mapping.
[0047] Fig.15 This is a diagram showing a drawing method of an embodiment using a cube map (CubeMap).
[0048] Fig.16 A is a diagram showing a half of the block 1604A. 1604B is a diagram showing an example of a block with an oblique orientation.
[0049] Fig.17 A and Fig.17 B is a diagram showing an example of using an environment mapping method other than cubemap.
[0050] Fig.18 This is a diagram showing an embodiment of obtaining a rendering result similar to that obtained when rendering using interior mapping by using polygons and textures.
[0051] Fig.19 Is to use the adoption of Fig.14 The cube map of the block CU1 described in the above-mentioned EMBODIMENTS projected from the center 1851 of the block to the five-sided polygon 1820 to create a texture data map for the polygon 1820.
[0052] Fig. 20 Shows an example of processing polygon baking textures without cubemaps.
[0053] Fig.21 This is a diagram in which the virtual plane VP1 corresponding to the back wall surface is moved toward the near virtual plane VP2 and arranged at the position of the virtual plane VP2.
[0054] Fig. 22 A and Fig. 22 B is a diagram showing the difference in rendering results of a rectangular object due to the position of the camera.
[0055] Fig.23 A is a diagram obtained by removing the virtual plane 2305 corresponding to the left and right walls and extending the virtual plane 2310 corresponding to the inner wall. Fig.23 B is a diagram showing a drawing 2360 of a rectangular object that eliminates the fold 2221 of the image.
[0056] Fig.24 This is a diagram of an implementation method in which a virtual cylindrical surface (or virtual spherical surface) 2410 is used instead of the virtual plane 2405.
[0057] Fig.25 A and Fig.25 B is a diagram showing a movement pattern of the three-dimensional virtual space and a view seen from the entrance.
[0058] Fig.26 The flow of configuring a virtual camera at a user's virtual position in a virtual space is shown.
[0059] Fig. 27 The flow of disposing a virtual camera at a position behind the user's virtual position in the virtual space is shown. DETAILED DESCRIPTION
[0060] In order to visualize objects existing in a three-dimensional virtual space, there is a technology for obtaining an image obtained by photographing the three-dimensional space with a virtual camera. For example, for each pixel on the photographing surface, the color of light (including the intensity of light, which is the same in this specification) that passes through the lens of the virtual camera from the object to the photographing surface is calculated, and the three-dimensional virtual space can be made into an image. As such technology, there are various rendering technologies.
[0061] If a computer calculates the color of light reaching the shooting surface one by one based on the three-dimensional virtual space and performs rendering, there are problems such as insufficient memory in ordinary computers and rendering delays.
[0062] In particular, in recent computer graphics, the number of polygons used in three-dimensional objects has increased significantly, and the problem of rendering delay has become prominent. In addition, the computing processing power and memory capacity are limited, and efforts to reduce the rendering load are particularly necessary in game consoles and mobile phones, which are hardware used for a relatively long period of time.
[0063] In addition, in an HMD (head mounted display) installed on the head to allow a user to experience virtual reality, two images are generated for the right eye and the left eye for stereoscopic viewing. In order to obtain multiple images that allow a user to experience virtual reality, it is required to generate multiple, more detailed images. In response to the refinement of the display device, the required image refinement has also increased and the dynamic range has also increased, and the number of pixels and information required to form the image has also increased.
[0064] Therefore, the disclosed technology described below is to appropriately allocate limited hardware resources for drawing objects when generating images.
[0065] Furthermore, the disclosed technology aims to reduce consumption of hardware resources in order to obtain an image corresponding to a rendering of an object existing in a three-dimensional virtual space.
[0066] By making flexible use of the saved resources and effectively drawing images such as those behind doors, users will not easily get lost when moving in a three-dimensional virtual space.
[0067] Embodiments are described below using the drawings. The embodiments are examples of the technology defined in the scope of protection and are not intended to be limiting.
[0068] Figure 1A A plurality of objects 12 existing in the room 10A which is a three-dimensional virtual space are photographed through the glass 115 by a virtual camera 11 installed behind a user 20 who is present in a room 10B which is a virtual space different from the room 10A.
[0069] Figure 1B The image 100 is obtained by photographing the three-dimensional virtual spaces 10A and 10B by the virtual camera 11. The image 100 includes the inner wall and door 114 of the room 10B, the periphery of the door 114, and the back image of the character 30 corresponding to the user 20.
[0070] In the image 100 , an image 113 obtained by rendering the room 10A can be seen through the glass 115 of the door 114 .
[0071] For example, a three-dimensional model of a room 10A is formed by a plurality of polygons, and a plurality of objects 12 are present in the room 10A. At this time, in order to generate an image 100 projected by a virtual camera 11, the color of light that passes through the lens of the virtual camera and reaches each pixel of the imaging element of the virtual camera from the plurality of objects 12 formed by the plurality of polygons is calculated. The operation of generating the image 100 is called rendering, for example.
[0072] In order to generate an image visible through the glass 115 of the door 114 by rendering, for example, light from multiple objects 12 in the room 10A passes through the glass 115 existing in the door 114 and reaches the imaging element via the lens of the virtual camera 11. For this reason, it is necessary to calculate whether it is light that passes through the glass 115 or light that fails to pass through the glass 115 and is blocked by the wall between the room 10A and the room 10B. Such calculations are performed for all objects 12 to generate an image. Therefore, when the number of objects 12 is large or complex, more computing resources are required even to calculate one image. In order to generate an animation, for example, 30 images need to be calculated per second, which consumes a lot of computing resources.
[0073] Furthermore, “a virtual space in which a user exists” or “a virtual space in which a user exists near a virtual space and which is most related to the virtual space” is referred to as “a virtual space to which a user belongs”.
[0074] Figure 2 2 is a diagram for explaining the technology of indoor mapping. As already illustrated, there is the technology of indoor mapping disclosed in Non-Patent Document 2.
[0075] In the technology of indoor mapping, the straight line from the virtual camera 11 to the point A of the glass 115 is extended to obtain the intersection point B with the virtual plane 113A (for example, the surface corresponding to the inner wall of the room 10A). On the virtual plane 113A, a texture imitating the scenery of the room is pre-set. The texture coordinates are derived from the coordinates of the point B. The color is obtained based on the texture information and the texture coordinates. The processing device that calculates the image of the virtual camera 11 associates the color with the position of the point A as the shooting data at the position of the pixel corresponding to the shooting surface of the virtual camera 11.
[0076] Furthermore, in this method, in order to draw objects or furniture existing in the room, it is preferable to arrange an additional virtual plane at their positions.
[0077] Moreover, if Figure 1A As shown, the user can be virtually placed in a room 10B which is a three-dimensional virtual space. That is, a virtual camera 11 is arranged at the position of the user's eyes or around the user (for example, the back), and the virtual camera 11 is used to capture the scene when the user moves in the room 10B, so that the user can be provided with an image as if the user is moving in the room 10B.
[0078] Furthermore, when the user enters the room 10A from the room 10B, it is preferable to use a detailed image generation method to generate images of each of the objects 12 in the room 10A captured by the virtual camera 11, thereby capturing the state of the room 10A in detail. Furthermore, when the user is in the room 10A, it is preferable to capture the objects in the room 10B (for example, objects in the room 10B) that are in the room 10B adjacent to the room 10A and that can be seen from the room 10A through the glass 115 of the door 114. Figure 1A 12B), using indoor mapping to generate images in a simple way.
[0079] According to the above setting, in the virtual space to which the user belongs, a fine image generation method is adopted, and for the virtual space to which the user does not belong, an image generation method (for example, indoor mapping) that does not consume computing resources compared to the fine image generation method is adopted, thereby generating an image that is realistic and does not feel uncomfortable to the user, and reducing the overall computing resource consumption required for generating the image. Thus, for example, a desired image can be generated more quickly.
[0080] Alternatively, when the user 20 is sufficiently close to the door 114 in the room 10B and all or most of the image 100 is occupied by the image 113 of the room 10A seen through the glass, it is determined that the user 20 does not belong to the room 10B but to the room 10A, and the image 113 through the glass is generated by a fine image generation method, while the object 12B existing in the room 10B is generated by a simple image generation method.
[0081] Figure 3 A is a diagram showing one example of an image generation method using the virtual camera 11A.
[0082] The room R1 constitutes one virtual space. Furthermore, the room R2 constitutes a virtual space different from the room R1.
[0083] In the room R1, there are a user 20A and a plurality of objects 12A. Behind the user 20A, there is a virtual camera 11A that captures images. At this time, the user's character may be placed at the position of the user 20A, and the virtual camera 11A may also capture the character of the user 20A.
[0084] exist Figure 3 In case A, the virtual camera 11A exists in the area of room R1, but Figure 3 The position of the camera shown in B may also be the room R2. In this case, it is preferable to assume that the wall 320 does not exist and generate an object of the virtual camera 11A in the room R1 by a sophisticated image generation method.
[0085] In addition, when capturing an image that can be seen from the position of the user 20A, the position of the virtual camera 11A may be made to overlap the position of the user 20A. In this case, the head of the character of the user 20A will not be reflected in the captured image. Figure 3 C and Figure 3 The same is true for D.
[0086] Figure 3 C is a diagram showing an example of an image generation method using a virtual camera when the object 12A exists in a room R1 different from the room R2 to which the user 20A belongs.
[0087] A door 114 having glass that transmits light exists between the room R1 and the room R2. A plurality of objects 12A exist in the room R1. Furthermore, a user 20A exists in the room R2 which is a virtual space different from the room R1.
[0088] In this case, since a plurality of objects 12A exist in room R1 different from room R2 to which user 20A belongs, it is preferable to use an image generation method with less calculation amount, such as the above-mentioned indoor mapping, for the image generation method related to room R1.
[0089] Figure 3 D is a diagram illustrating that a plurality of objects 12B exist in the room R2 to which the user 20A belongs, a plurality of objects 12A exist in the room R1 to which the user 20A does not belong, and an image is generated when the user crosses the door 114 and can see the objects 12A.
[0090] Figure 3 The plurality of objects 12B in the room D are imaged by a sophisticated image generation method, and for the plurality of objects 12A in the room R1 visible across the door 114, indoor mapping or the like, which is a simple image generation method, is preferably used.
[0091] By setting in this way, a sophisticated image generation method is used for a plurality of objects 12B in the room R2 to which the user 20A belongs, and a simple image generation method is used for a plurality of objects 12A in the room R1 to which the user 20A does not belong. In this way, by using different image generation methods, images can be generated with less computing resources.
[0092] Figure 4 The figure shows that in each of the three-dimensional virtual spaces R1, R2, and R3, which are different from each other, a door 55 for exit and a door 50 for entrance are set. The door 50 for entrance is a door that allows entry from the outside of the corresponding room. In addition, the door 55 for exit is a door that allows exit from the corresponding room.
[0093] Since each room is a different three-dimensional virtual space, the exit-only door 55 of each room can be arbitrarily connected to the entrance-only door 50 of another room. This connection may be predetermined or may be changed by the user.
[0094] As one example, there can be the following implementations.
[0095] [Implementation Method 1] Figure 4 )
[0096] Users can set only one entrance in their room.
[0097] Users can set any number of exits in their rooms.
[0098] For one exit, the destination room of the exit can be set to only one.
[0099] By making such setting, the user can limit the entrance of other users into his room to one. In addition, the exit for oneself to move to other users' room is provided with the same number of exits as the number of destination rooms that oneself wants to go.
[0100] The feasible implementation is not limited to the above-mentioned implementation. For example, in room R3 where the art work of a specific user R is displayed, the user may set up more doors leading to room R3. In this way, the user can easily reach room R3 where his / her art work is displayed. Alternatively, the door setting of a room where an information desk is set up may be pre-arranged in a way that is easy for any user to enter and exit. In this way, an environment in which the user can easily use a virtual three-dimensional virtual space can be provided.
[0101] Furthermore, as an embodiment obtained by expanding the above-mentioned Embodiment 1, there are the following embodiments.
[0102] [Implementation Method 2] Fig.25 A and Fig.25 B)
[0103] Even if it is a dedicated door for an entrance, it is limited to the case where you enter the room from that entrance, making it possible to leave the room from that entrance and return to the original room.
[0104] When returning from the entrance-only door, enter from the entrance of the room at the return destination.
[0105] This setting prevents anyone from entering the room through the exit door.
[0106] Furthermore, it is preferred that transparent glass is provided on the door so that the interior of the destination room that can be moved to can be seen through the glass. In this way, the user knows the appearance of the room existing in front of the door, thereby preventing getting lost in the vast three-dimensional virtual space.
[0107] The above-mentioned embodiment 1 ( Figure 4 ), the appearance of the room of the destination of movement as seen from the entrance of the room is displayed on the glass of the exit door. Since it is not possible to move from the entrance door, nothing is displayed on the glass of the entrance door.
[0108] In the above-mentioned embodiment 2 ( Fig.25 A and Fig.25 In the case of B), after returning to the original room from the entrance door, the appearance of the other side of the entrance door will change as shown in the table as you move.
[0109] Figure 5 This is a diagram showing that even if users enter the same room 400 via the same door 321 , in the case where different users enter, each user can return to a different room although via the same door 321 .
[0110] exist Figure 5, it is shown that when the user 20C in the room 410 enters the room 400 via the door 324, the user enters the room 400 from the door 321. In addition, when the user 20B in the room 420 enters the room 400 via the door 325, the user enters the room 400 from the door 321.
[0111] The user 20C in the room 400 may return to the original room 410 via the door 321 and the door 322 of the room 410. The user 20B in the room 400 may return to the original room 420 via the door 321 and the door 323 of the room 420.
[0112] Furthermore, the setting may be such that the user 20C in the room 400 can see the object in the original room 410 from the door 322 through the door 321. Furthermore, the setting may be such that the user 20B in the room 400 can see the object in the original room 420 from the door 323 through the door 321.
[0113] In this way, it is possible for each user to move in the room as a virtual space by tracing back in the reverse order of the room they have passed through, so that the user will not get lost in a complex path.
[0114] Figure 6 It is a hardware configuration diagram of an implementation method.
[0115] The hardware configuration of the embodiment includes a CPU 501 , a ROM 502 capable of storing programs and data of the embodiment, a RAM 503 , a network interface 505 , an input interface 506 , a display interface 507 , and an external memory interface 508 . These hardware components are connected to each other via a bus 504 .
[0116] The network interface 505 is connected to a network 515. The network 515 includes a wired LAN, a wireless LAN, the Internet, a telephone network, etc. The input interface 506 is connected to an input unit 516. The display interface 507 is connected to a display unit 517. The display unit 517 can also be implemented by multiple display devices. The external memory interface 508 is connected to a storage medium 518. The storage medium 518 can also be a RAM, a ROM, a CD-ROM, a DVD-ROM, a hard disk, a memory card, a USB memory, etc.
[0117] Next, the processing flow of the implementation method is described.
[0118] Figure 7 The following is an overview of the processing flow of the embodiment. Each step is described.
[0119] [Step S702] Determine the first virtual space to which the user belongs. Here, the term "belonging" refers to, for example, not only the case where the user exists in the first virtual space, but also the case where the user is close to the first virtual space, is about to enter the first virtual space, the case where the user's artwork decorates the first virtual space, and the case where the user is close to the first virtual space. In addition, the case where there is a special relationship between the user and the first virtual space is cited.
[0120] [Step S704] A single or multiple second virtual spaces to which the user does not belong and which are to be photographed are determined. The photographing is performed by a virtual camera.
[0121] [Step S706] An image is generated by photographing an object existing in the first virtual space by a virtual camera using a first image generation method. The first image generation method is preferably a fine image generation method.
[0122] [Step S708] An image corresponding to an image obtained by photographing an object existing in the second virtual space with a virtual camera is generated by a second image generation method. The second image generation method preferably uses a method such as indoor mapping that can save computing resources for generating images.
[0123] According to the above processing, for objects that are closely related to the user, a fine image generation method is used, and for other objects, a simpler image generation method is used. In this way, the computing resources required for image generation are saved, and appropriate images that can satisfy the user are generated.
[0124] Figure 8 The flow of simply presenting a virtual space that may be a movement destination to the user is shown. The steps are described below.
[0125] [Step S802 ] A door through which the user can move from the first virtual space to the second virtual space is displayed.
[0126] [Step S804] An image equivalent to an image obtained by photographing an object existing in the second virtual space through the door with a virtual camera is generated by a second image generation method, thereby generating an image in which the door has transparency, indicating that the user can pass through the door.
[0127] By performing such processing, the user can know in advance the appearance of the virtual space to which he can move, and a three-dimensional virtual space with higher utilization value can be provided to the user.
[0128] Fig. 9 This is a diagram showing an example in which the transparency of the glass 115 is made different depending on the angle between the surface of a transparent material such as the glass 115 and the imaging direction of the virtual camera.
[0129] Fig. 9 is with Figure 2 Similar figures are marked with the same figure numbers. When the angle θ formed by the shooting direction of the virtual camera 11 and the glass 115 is less than a specified angle, the transparency of the glass 115 can be reduced, for example. When the angle θ is less than the specified angle, the reflection of ambient light in the glass 115 increases, making it difficult to identify objects in the room 10A through the glass 115. In the real world, such an experience occurs in many cases. Therefore, under such conditions, a more natural image is presented by reducing the transparency of the glass 115. Alternatively, the transparency of the glass 115 can be set to opaque. When set to opaque, the process of generating an image of an object in the room 10A visible from the glass 115 can be omitted. Therefore, the consumption of computing resources can be further reduced.
[0130] Fig.10 is shown with Fig. 9 The diagram shows the flow of image generation corresponding to the angle formed by the glass 115 and the shooting direction of the camera.
[0131] [Step S1002] The transparency of the transparent object is changed according to the angle formed by the shooting direction of the virtual camera and the surface of the transparent object set in the virtual space.
[0132] In this way, a more natural image can be generated. Alternatively, by making the glass 115 opaque, the consumption of computing resources can be reduced.
[0133] Fig.11 The following is a flow for increasing the sense of immersion in the three-dimensional virtual space for a user utilizing the three-dimensional virtual space.
[0134] [Step S1102] A character corresponding to the user is arranged at the user's virtual position in the virtual space. A character similar to the user or a character selected by the user from among existing characters is arranged in the three-dimensional virtual space.
[0135] Fig.26 The flow of configuring a virtual camera at a user's virtual position in a virtual space is shown.
[0136] [Step S1602] A virtual camera is configured at a virtual position of the user in the virtual space. In this way, an image related to the three-dimensional virtual space is generated that is seen from the eyes of the virtual character corresponding to the user. By configuring in this way, the three-dimensional virtual space seen from the position of the user's eyes is drawn, which can provide a more realistic experience to the user.
[0137] Fig. 27The flow of disposing a virtual camera at a position behind the user's virtual position in the virtual space is shown.
[0138] [Step S1702] A virtual camera is arranged behind the virtual position of the user in the virtual space. By arranging in this way, a character that is the user's avatar existing in the three-dimensional virtual space can be included in the image.
[0139] Fig.12 A and Fig.12 B shows a processing flow for simply notifying the user of the second virtual space that is highly relevant to the user.
[0140] The following explains Fig.12 Steps in A.
[0141] [Step S1202] A second virtual space is determined based on whether the user has entered the current virtual space from any of the multiple virtual spaces. Through this process, the user can easily know the virtual space the user has passed through in the entire virtual space. In this way, the user can easily return to the virtual space to which he has moved, and the user can be prevented from getting lost in the virtual space.
[0142] [Step S1204] The virtual space of the destination to which the user can move is determined as the second virtual space. This process can easily inform the user of which virtual space the user is associated with.
[0143] Fig.13 The flow of a method for generating switching images when the user moves in a virtual space is shown.
[0144] [Step S1302] The user moves from the first virtual space to the second virtual space, so that the second virtual space becomes the new first virtual space and the first virtual space becomes the new second virtual space. In this case, an image obtained by photographing an object existing in the virtual space that becomes the new first virtual space is generated by the first image generation method, and an image equivalent to the image obtained by photographing an object existing in the virtual space that becomes the new second virtual space is generated by the second image generation method.
[0145] This process enables switching between the detailed image generation method and the simple image generation method. For the user, the detailed image generation method is dynamically switched to the object of higher interest, thereby generating an image that does not make the user feel uncomfortable with less computing resources.
[0146] <Modification of indoor mapping>
[0147] “Indoor mapping” is known as a method of rendering the inside of a building (floor, roof, walls) without modeling.
[0148] Next, modifications of indoor mapping will be described with reference to the drawings.
[0149] return Figure 2 .exist Figure 2 In the method, instead of modeling with polygons, a texture image is used to establish a correspondence between the inner wall surface (floor, ceiling, wall) of the room and the virtual plane 113A corresponding to the inner wall surface of the room. The straight line from the viewpoint of the virtual camera 11 to the point A of the window 115 is extended. The point B that intersects with the virtual plane 113A, which is the inner wall surface of the room, is obtained. The texture coordinates are derived from the coordinates of point B. The color is obtained from the texture image and the texture coordinates. According to this method, an image as to whether the inner wall surface of the room exists can be obtained through the window 115.
[0150] exist Figure 2 In the method shown, in order to draw objects such as placed objects or furniture in the room, it is preferable to configure an additional virtual plane at the location.
[0151] Fig.14 It is a diagram showing another embodiment of indoor mapping.
[0152] In the method of this embodiment, an image corresponding to objects such as placed objects or furniture in the room can be drawn without configuring an additional virtual plane. Therefore, environmental mapping such as block mapping can be used.
[0153] Next, the processing of this embodiment will be described.
[0154] 1) Preparation of environmental maps
[0155] In the method of the present invention, an environment map such as a cube map is prepared in advance.
[0156] Fig.14 The method for making a cube map is shown in FIG. First, a wall surface 1404 made of polygons and an object 12C such as a placed object or furniture are arranged. The wall surface and the placed object or furniture can be made in detail using multiple polygons. In addition, it is preferred to attach a texture to the wall surface and the placed object or furniture. It is preferred to place a cube CU1 for cube mapping as shown in FIG. Fig.14 As shown, the block CU1 is arranged at a distance L from the window 1402 as the outer wall and in the center of the width of the room. The position where the block CU1 is arranged is not limited to this example, and an arrangement different from this example is also possible.
[0157] In adopting the aforementioned Figure 1A , Figure 1B In the description, it is described as follows.
[0158] Alternatively, when the user 20 is sufficiently close to the door 114 in the room 10B and all or most of the image 100 is occupied by the image 113 obtained by observing the room 10A through the glass, it is assumed that the user 20 does not belong to the room 10B but to the room 10A, and the image 113 through the glass is generated by a fine image generation method, and the object 12B existing in the room 10B is generated by a simple image generation method.
[0159] The distance L can be used as the distance for determining “sufficiently close.” By setting in this way, it is possible to reduce the discontinuity of the image that occurs when switching from the simple image generation method to the detailed image generation method.
[0160] A straight line is used to connect point P on the surface of the object 12C where an object or furniture is placed and the center 1406 of the block to obtain the intersection point P1 of the straight line and the block CU1. Similarly, a straight line is used to connect point Q on the surface of the wall 1404 and the center 1406 of the block to obtain the intersection point Q1 of the straight line and the block CU1. Fig.14 In the example of , point P on the surface of object 12C such as a placed object or furniture is mapped to point P1, and the color of point P1 becomes the color of point P. Similarly, point Q on the wall is mapped to point Q1. In this way, block CU1 is created.
[0161] 2) Using environment map drawing
[0162] Fig.15 1402 is a diagram showing a drawing method in an embodiment using a cube map. The color when drawing point A2 on the window 1402 is obtained as follows.
[0163] A virtual plane 1504 is arranged at a position corresponding to the wall surface. The straight line connecting the virtual camera 11 and point A2 is extended to obtain point Q as the intersection point with the virtual plane 1504. Point Q and the center 1406 of the block CU1 are connected by a straight line to obtain point Q1 as the intersection point of the straight line and the block. The color of point Q1 is used as the color of point A2 to draw the window 1402.
[0164] By adopting the above-mentioned methods 1) and 2), a rendering result is obtained as if a wall (inner wall of the room) and placed objects or furniture exist on the inner side of the window 1402.
[0165] <Other Modifications>
[0166] Fig.16 A is a diagram showing half of block 1604A.
[0167] The cube map can also be further used by mapping it onto a spherical surface. Conventional cube maps (1604A and 1604B) map the surrounding environment equivalent to the entire sphere (1602A and 1602B), so the entire cube map is mapped onto the entire sphere for use. Fig.16 As shown in FIG. 1A , the surrounding environment corresponding to the hemisphere 1602A is mapped on the hemispherical surface using the block 1604A which is only half of the room side (left side of the figure) among the blocks corresponding to the surrounding environment of the entire sphere (1602A and 1602B). At this time, only the objects existing in the surrounding environment corresponding to the hemisphere 1602A are mapped on the hemispherical surface.
[0168] Fig.16 B is a diagram showing an example of a block with an inclined orientation.
[0169] like Fig.16 As shown in B, the orientation of the cube 1606 used for the cube map is not limited to being parallel to the window 1402 of the room, but may also be inclined.
[0170] Fig.17 A and Fig.17 B is a diagram showing an example of using an environment mapping method other than cubemap.
[0171] Fig.17 A shows an example of mapping the surrounding environment of half of the three-dimensional virtual space to the environment map texture 1760, which is a circular area on a plane, via the hemispherical surface 1702A. For example, the color of the surrounding environment visible in the line of sight direction O1-S1 is obtained, and the end point of the vertical line from point S1 downward is set to T1 in the plane of the environment map texture 1760, and the color of the surrounding environment obtained above is used as the color of T1, thereby projecting the color of the object visible on the hemisphere 1702A side with O1 as the viewpoint onto the environment map texture 1760 and fixing it.
[0172] By using the environment map texture 1760 , the color in the O1 - S1 direction in the three-dimensional virtual space can be easily acquired.
[0173] Fig.17B is a diagram showing an example of mapping the surrounding environment in all directions of the three-dimensional virtual space onto the environment map texture 1770 via the full spherical surface 1750 and the hemispherical surface 1702B. For example, first, the color of the surrounding environment visible in the line of sight of O2-S2 is obtained. The point S3 where the extension line of O1-S2 intersects the hemisphere 1702B is obtained. The end point of the vertical line drawn from point S3 in the plane of the environment map texture 1760 is set to T2. By using the color of the surrounding environment obtained as described above as the color of T2, the color of the surrounding environment in the entire direction with O2 as the viewpoint can be projected and fixed to the environment map texture 1770.
[0174] By using the environment map texture 1770 , the color in the O2 - S2 direction in the three-dimensional virtual space can be easily acquired.
[0175] Fig.18 This is a diagram showing an embodiment of obtaining the same rendering result as that obtained when rendering using indoor mapping by using polygons and textures.
[0176] <1-1 Preliminary Preparation>
[0177] 1) In Fig.14 In the method, first make a cube map.
[0178] 2) Fig.18 This is a diagram of a room viewed from above. As shown in the figure, five faces corresponding to the five wall faces (1810) of the ground, roof, inner wall, and left and right walls are created using polygons 1820. From the perspective of reducing the consumption of computing resources, the fewer the number of polygons used, the better. For example, the number of polygons used is five for quadrilateral polygons and ten for triangular polygons.
[0179] 3) Remove the original objects 1810 and 1830 (the wall surface and the placed objects or furniture, etc.), and arrange the five-sided polygon 1820.
[0180] like Fig.19 As shown, using the Fig.14 The cube map of the block CU1 described in the above-mentioned illustrative examples projects the cube map onto the five-sided polygon 1820 from the center 1851 of the block to create texture data for the polygon 1820.
[0181] This process is called baking the texture into polygons. Point P on the surface of the object 1830 such as a placed object or furniture is projected from point P1 of the cube map to point P2 on the polygon. Point Q on the wall is projected from point Q1 of the cube map to point Q2 on the polygon. The texture image thus obtained not only draws the wall of the room, but also the placed objects or furniture in the room.
[0182] <1-2 Drawing>
[0183] The texture baked into the five-sided polygon is drawn using a polygon drawing method of texture mapping, which is widely used as a polygon drawing method.
[0184] Fig. 20 Shows an example of a process of baking textures into polygons without going through a cubemap.
[0185] In this method, if Fig. 20 As shown in FIG. 1 , the points (points P and Q) of the objects such as the wall surface of the room and the objects or furniture placed in the room are projected from point 1851 corresponding to the center of the cube map to the five-sided polygon 1820 to create texture data. In other words, the texture is baked for the polygon 1820. Point P is projected to point P2. Point Q is projected to point Q2.
[0186] The subsequent processing of generating an image can be realized using the virtual camera described above.
[0187] exist Figures 18 to 20 In order to make the figure easier to read, the position of polygon 1820 is slightly offset from the position of wall surface 1810, and the figure is easily understood, but the two can also be at the same position. When the two are at the same position, the positions of point Q and point Q2 are consistent.
[0188] <Other variations of rendering using indoor mapping>
[0189] For example, as in Fig. 20 As described in, for example, the existing rendering method using polygons and baked textures can easily obtain the same rendering results as indoor mapping, which can reduce computing resources.
[0190] However, for Fig.15 Indoor mapping such as the above has the advantage of rendering that the image becomes more natural through the following modification examples, which is not available in the conventional methods.
[0191] <2-1 Implementation method of moving and drawing a virtual plane>
[0192] like Fig.21 As shown, point P on the surface of a placed object or furniture is mapped to point P1 of the cube map. Fig.21 When observing from the position of the camera, as the original correct observation method, point P on the placed object should be drawn at the position of point S4 on the outer wall (window).
[0193] However, when rendering using the aforementioned virtual plane 1820, the position of the point S5 rendered is offset from the correct position of the point S4. This offset also occurs in conventional rendering methods using polygons and baked textures.
[0194] Therefore, the equivalent of Fig.21 The virtual plane VP1 of the inner wall is moved to the near virtual plane VP2 and arranged at the position of the virtual plane VP2. Thus, the color of P1 of the cube map is drawn at point S4, and a correct drawing result is obtained.
[0195] In addition, the drawn position is shifted and drawn by moving the point existing on the inner wall from the virtual plane VP1 to the virtual plane VP2. Therefore, this modified embodiment is preferably used when a more important (or more) object is arranged on a specific inner side than the inner wall.
[0196] exist Fig.21 In the example, the virtual plane VP1 corresponding to the inner wall is moved toward the virtual plane VP2, but as an implementation method other than this, the virtual planes corresponding to the left and right walls and the virtual planes corresponding to the ground or the roof may be moved in the normal direction of each plane.
[0197] <2-2 Modification example in which part of the virtual plane is removed>
[0198] Fig. 22 A and Fig. 22 B is a diagram showing the difference in rendering results of a rectangular object due to the position of the camera.
[0199] As an example, a rectangular object is placed closer to the virtual camera than to the inner wall. When the camera position coincides with the center of the cube map (not shown), we get Fig. 22 A is a correct rendering result 2210. However, the indoor mapping of this embodiment adopts the same viewing method as the painting drawn on the wall, so if the position of the camera is changed, for example, Fig. 22 As shown in B, at the boundary 2221 of the virtual plane, the image 2220 is bent toward the object, resulting in a crease 2221. In addition, such bending also occurs in the existing rendering method using polygons and baked textures.
[0200] Fig.23 A is a diagram in which the virtual plane 2305 corresponding to the left and right walls is removed and the virtual plane 2310 corresponding to the inner wall is extended.
[0201] The color of point P1 of the cube map is applied to the position of point P7 on the window 2320 via point P6 on the virtual plane 2310 and is photographed by the virtual camera 11.
[0202] Fig.23 B is a diagram showing a drawing 2360 of a rectangular object in which the fold 2221 of the image has disappeared. This method does not result in an image in a correct viewing manner, but has the advantage of reducing unnaturalness because the fold 2221 is eliminated.
[0203] exist Fig.23 A and Fig.23 In the example of BA, the virtual planes corresponding to the left and right walls are removed, but as another embodiment, the virtual plane corresponding to the ground may be removed, or the virtual plane corresponding to the roof may be removed.
[0204] <2-3 Modification examples using virtual cylindrical surfaces and virtual spherical surfaces>
[0205] Fig.24 An embodiment using a virtual cylindrical surface (or virtual spherical surface) 2410 instead of the virtual plane 2405 is shown. The color of P1 on the cube map CU1 is applied to the point S8 on the window 2420 via the point Q8 on the virtual cylindrical surface 2410, and is photographed by the virtual camera 11. The image obtained in this embodiment is deformed as a whole. However, in the case of using multiple virtual planes, the bend 2221 generated at the boundary of the virtual plane does not occur in this embodiment, so there is an advantage of reducing the degree of unnaturalness.
[0206] exist Figure 3 C and Figure 3 In D, the schematic diagram shows that room R1 and room R2 are adjacent to each other and are connected by a door. However, originally, room R1 and room R2 are three-dimensional virtual spaces different from each other, and therefore, they are not necessarily adjacent to each other.
[0207] Therefore, in the above series of embodiments, in order to generate images related to the first virtual space and the second virtual space based on a virtual camera, at least one of the first virtual space and the second virtual space is moved and / or rotated, and the first virtual space and the second virtual space are virtually configured in a manner such that they are connected to perform image generation.
[0208] On the other hand, in contrast to this, a method for performing image generation based on two virtual cameras may be performed. In this method, a first virtual camera is used for image generation related to a first virtual space, and a second virtual camera is used for image generation related to a second virtual space. The second virtual camera is configured in the second virtual space in a manner that matches image generation related to the first virtual space based on the first virtual camera.
[0209] However, the difference between the two methods lies in performing coordinate transformation (movement and / or rotation) on the virtual space, or using coordinate transformation (movement and / or rotation) to derive the second virtual camera from the first virtual camera. In essence, the two methods are equivalent.
[0210] It goes without saying that each embodiment disclosed in this specification is not limited to the invention described in the claims, but is treated as an example.
[0211] The embodiments are not exclusive and can be combined appropriately. Alternatively, some elements of one embodiment may be replaced with elements of other embodiments.
[0212] In addition, the order of each process of the illustrated flowchart can be replaced within the scope of non-contradiction. In addition, within the scope of non-contradiction, an illustrated process can be executed multiple times at different timings. In addition, within the scope of non-contradiction, multiple processes can be executed simultaneously. In addition, all steps are not required, and within the scope of non-contradiction, some steps may not exist or be not executed. In addition, each step can also be executed by an operating system or hardware. In addition, the program can also be configured in a state of being stored in a non-temporary medium.
[0213] The program and method for implementing the above-mentioned implementation mode can be provided by Figure 7 That is, the program of the embodiment can also be implemented as a method for causing a computer to execute it.
[0214] The program may also be stored in the storage medium 718 , the ROM 702 , or the RAM 703 .
[0215] Each embodiment may be implemented as a hardware device of a program.
[0216] Glass is an example of a transparent object. A room is an example of a virtual space.
[0217] The following additional notes are described with respect to the above-mentioned embodiment.
[0218] [Note 1]
[0219] A method for generating an image of a three-dimensional virtual space captured by a virtual camera arranged at a virtual position of a user or around the virtual position of the user, comprising:
[0220] determining a first virtual space to which the user belongs,
[0221] Determine a single or multiple second virtual spaces that are the subject of the shooting among the virtual spaces to which the user does not belong;
[0222] generating an image obtained by photographing an object existing in the first virtual space by the virtual camera using a first image generating method; and
[0223] A second partial image corresponding to an image obtained by photographing an object existing in the second virtual space by the virtual camera is generated by a second image generating method.
[0224] [Note 2]
[0225] The image generation method as described in Appendix 1,
[0226] Compared with the first image generation method, the second image generation method is an image generation method that consumes less computing resources in the calculation required for generating an image corresponding to the object.
[0227] [Note 3]
[0228] The image generation method as described in Appendix 1 or 2,
[0229] The second image generation method is indoor mapping.
[0230] [Note 4]
[0231] The image generation method according to any one of Supplementary Notes 1 to 3, further comprising:
[0232] displaying a door through which the user can move from the first virtual space to the second virtual space; and
[0233] An image equivalent to an image obtained by photographing an object existing in the second virtual space through the door with the virtual camera is generated by a second image generation method, thereby generating an image in which the door has transparency, indicating that the user can pass through the door.
[0234] [Note 5]
[0235] The image generation method as described in Supplementary Note 4 further comprises:
[0236] The transparency of the transparent object is made different according to the angle formed by the shooting direction of the virtual camera and the surface of the transparent object set in the virtual space.
[0237] [Note 6]
[0238] The image generation method according to any one of Supplementary Notes 1 to 5, further comprising:
[0239] A character corresponding to the user is arranged at the virtual position of the user in the virtual space.
[0240] [Note 7]
[0241] The image generation method as described in any one of Supplementary Notes 1 to 6,
[0242] The virtual camera is configured at a virtual position of the user in the virtual space.
[0243] [Note 8]
[0244] The image generation method as described in any one of Supplementary Notes 1 to 6,
[0245] The virtual camera is arranged at a position behind the virtual position of the user in the virtual space.
[0246] [Note 9]
[0247] The image generation method as described in any one of Supplementary Notes 1 to 8,
[0248] Determining the second virtual space includes:
[0249] The second virtual space is determined according to whether the user enters the current virtual space from any virtual space among the plurality of virtual spaces.
[0250] [Note 10]
[0251] The image generation method as described in any one of Supplementary Notes 1 to 8,
[0252] Determining the second virtual space includes determining a virtual space of a destination to which the user can move as the second virtual space.
[0253] [Note 11]
[0254] The image generation method according to any one of supplementary notes 1 to 10, further comprising:
[0255] When the user moves from the first virtual space to the second virtual space, so that the second virtual space becomes the new first virtual space and the first virtual space becomes the new second virtual space, an image obtained by photographing an object existing in the virtual space that becomes the new first virtual space by the virtual camera is generated by a first image generation method, and an image equivalent to an image obtained by photographing an object existing in the virtual space that becomes the new second virtual space by the virtual camera is generated by a second image generation method.
[0256] [Note 12]
[0257] A procedure,
[0258] Cause a computer to execute the image generation method described in any one of Supplementary Notes 1 to 11.
[0259] Description of Reference Numerals
[0260] 504 Bus
[0261] 505 Network Interface
[0262] 506 Input Interface
[0263] 507 Display Interface
[0264] 508 External Memory Interface
[0265] 515 Network
[0266] 516 Input
[0267] 517 Display
[0268] 518 Storage Media
[0269] 718 Storage Media
Claims
1. A method for generating an image, which generates an image of a three-dimensional virtual space captured by a virtual camera arranged at a virtual position of a user or around the virtual position of the user, comprising: determining a first virtual space to which the user belongs, determining a single or multiple second virtual spaces to which the user does not belong and which are the subject of the shooting, generating a first partial image obtained by photographing an object existing in the first virtual space by the virtual camera, that is, a first partial image of the image, by a first image generating method; and generating, by a second image generating method, a second partial image corresponding to an image obtained by photographing an object existing in the second virtual space by the virtual camera, that is, a second partial image of the image; Compared with the first image generation method, the second image generation method is an image generation method that consumes less computing resources in the calculation required for generating an image corresponding to the object. The image generation method also has: When the user moves from the first virtual space to the second virtual space, so that the second virtual space becomes the new first virtual space, and the first virtual space becomes the new second virtual space, a third partial image is generated by the first image generation method, which is obtained by photographing the object existing in the virtual space that becomes the new first virtual space by the virtual camera, and a fourth partial image equivalent to the image obtained by photographing the object existing in the virtual space that becomes the new second virtual space by the virtual camera is generated by the second image generation method, thereby generating a new image having the third partial image and the fourth partial image.
2. The image generation method according to claim 1, The second image generation method is an indoor mapping method.
3. The image generation method according to claim 1 or 2, further comprising: displaying a door through which the user can move from the first virtual space to the second virtual space; and An image equivalent to an image obtained by photographing an object existing in the second virtual space through the door with the virtual camera is generated by a second image generation method, thereby generating an image in which the door has transparency, indicating that the user can pass through the door.
4. The image generation method according to claim 3, further comprising: The transparency of the transparent object is made different according to the angle formed by the shooting direction of the virtual camera and the surface of the transparent object set in the virtual space.
5. The image generation method according to claim 1 or 2, further comprising: A character corresponding to the user is arranged at the virtual position of the user in the virtual space.
6. The image generation method according to claim 1 or 2, further comprising: The virtual camera is configured at a virtual position of the user in the virtual space.
7. The image generation method according to claim 1 or 2, further comprising: The virtual camera is arranged at a position behind the virtual position of the user in the virtual space.
8. The image generation method according to claim 1 or 2, determining the second virtual space, include: The second virtual space is determined according to whether the user enters the current virtual space from any virtual space among the plurality of virtual spaces.
9. The image generation method according to claim 1 or 2, determining the second virtual space, include: A virtual space of a destination to which the user can move is determined as the second virtual space.
10. A program product, A computer is caused to execute the image generation method according to any one of claims 1 to 9.
