Virtual reality simulator and computer-readable recording medium

Abstract

The present disclosure relates to a virtual reality simulator and a computer-readable recording medium. Discomfort experienced by a user is suppressed in a naked-eye VR simulation using a curved screen. The VR simulator captures a virtual space from a virtual user position corresponding to an actual user position, and acquires a virtual space image. Then, the VR simulator depicts the virtual space image on a curved screen disposed at a position away from the user. A virtual screen position is a position within the virtual space corresponding to a disposition position of the curved screen. A reverse curved screen is a screen configured at the virtual screen position, curved in a direction opposite to the curved screen. The VR simulator captures the virtual space from the virtual user position, acquires a first virtual space image, and depicts the first virtual space image in the reverse curved screen. Further, the VR simulator captures the first virtual space image on the reverse curved screen from the virtual user position, acquires a second virtual space image. The VR simulator depicts the second virtual space image on the curved screen.

Description

Technical Field

[0001] This disclosure relates to virtual reality (VR) simulation. In particular, this disclosure relates to glasses-free VR simulation. Background Technology

[0002] Patent Document 1 discloses a three-dimensional image display device that displays a three-dimensional image on a non-planar display surface. The three-dimensional image display device corrects the viewpoint according to the specifications of the non-planar display surface, so as to allow for stereoscopic observation similar to the case where a three-dimensional image is displayed on a planar virtual display surface. Furthermore, the three-dimensional image display device displays the corrected image on the non-planar display surface.

[0003] Patent Document 2 and Patent Document 3 can also be cited as documents representing the level of technology at the time of the application in the technical field of this disclosure.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2014-030152

[0007] Patent Document 2: Japanese Patent Application Publication No. 2020-120236

[0008] Patent Document 3: Japanese Patent Application Publication No. 2006-229768 Summary of the Invention

[0009] Consider "glasses-free VR simulation," where users can experience VR without wearing VR devices such as head-mounted displays. In glasses-free VR simulation, an image depicting a virtual world (virtual space) is placed at a location far from the user. In such glasses-free VR simulation, it is best to suppress the sense of incongruity perceived by the user as much as possible. In particular, it is best to suppress the sense of incongruity even when the image is distorted.

[0010] One objective of this disclosure is to provide a technique that can suppress the sense of incongruity felt by the user in a naked-eye VR simulation using curved images.

[0011] The first point concerns virtual reality simulators.

[0012] Virtual reality simulators have the following features:

[0013] One or more processors; and

[0014] One or more storage devices for storing virtual space structure information representing the structure of the virtual space.

[0015] One or more processors are configured to perform:

[0016] The processing of information to obtain the actual user's location, which is the position of the user's head in the actual space;

[0017] The process of obtaining the virtual user location, which is a location in a virtual space corresponding to the actual user location;

[0018] The image processing involves capturing images of the virtual space based on its structural information, using cameras positioned at virtual user locations within the virtual space.

[0019] The process of projecting or displaying virtual space images onto a curved screen, which is positioned far from the user in the actual space and does not move in tandem with the user.

[0020] The virtual screen position is the position within virtual space that corresponds to the set position of the curved screen.

[0021] The above-mentioned shooting processing includes:

[0022] The first shooting process involves capturing images of the virtual space using a first camera positioned at the virtual user's location, thereby obtaining an image of the first virtual space.

[0023] The process of projecting or displaying a first virtual space image in a reverse curved image that is positioned at the virtual screen location and curved in the opposite direction to the curved screen;

[0024] The second image capture process involves capturing the first virtual space image on the reverse-curved screen using a second camera positioned at the virtual user's location, thereby obtaining the second virtual space image; and

[0025] The processing of setting the second virtual space image as a projection or a virtual space image displayed on a curved screen.

[0026] The second point concerns virtual reality simulation programs that are executed by computers to perform virtual reality simulations.

[0027] Virtual reality simulation programs enable computers to perform:

[0028] The processing of information to obtain the actual user's location, which is the position of the user's head in the actual space;

[0029] The process of obtaining the virtual user location, which is a location in a virtual space corresponding to the actual user location;

[0030] The image processing involves capturing images of the virtual space using cameras positioned at virtual user locations within the virtual space, based on the virtual space structure information representing the structure of the virtual space; and...

[0031] The process of projecting or displaying virtual space images onto a curved screen, which is positioned far from the user in the actual space and does not move in tandem with the user.

[0032] The virtual screen position is the position within virtual space that corresponds to the set position of the curved screen.

[0033] The above-mentioned shooting processing includes:

[0034] The first shooting process involves capturing images of the virtual space using a first camera positioned at the virtual user's location, thereby obtaining an image of the first virtual space.

[0035] The process of projecting or displaying a first virtual space image in a reverse curved image that is positioned at the virtual screen location and curved in the opposite direction to the curved screen;

[0036] The second image capture process involves capturing the first virtual space image on the reverse-curved screen using a second camera positioned at the virtual user's location, thereby obtaining the second virtual space image; and

[0037] The processing of setting the second virtual space image as a projection or a virtual space image displayed on a curved screen.

[0038] According to this disclosure, it is possible to suppress the sense of incongruity felt by the user in naked-eye VR simulation. In particular, the sense of incongruity can be suppressed even when the image is distorted. Attached Figure Description

[0039] Figure 1 This is a conceptual diagram illustrating the outline of naked-eye VR simulation involved in the embodiments of this disclosure.

[0040] Figure 2 This is a conceptual diagram illustrating the outline of naked-eye VR simulation involved in the embodiments of this disclosure.

[0041] Figure 3 This is a block diagram illustrating a structural example of a VR simulator according to an embodiment of the present disclosure.

[0042] Figure 4 This is a block diagram illustrating a functional structure example of a VR simulator according to an embodiment of the present disclosure.

[0043] Figure 5 This is a flowchart illustrating the processes implemented by a VR simulator as described in embodiments of this disclosure.

[0044] Figure 6 This is a conceptual diagram illustrating the virtual space shooting process (step S130) involved in the embodiments of this disclosure.

[0045] Figure 7 This is a conceptual diagram used to illustrate the first feature in the virtual space shooting process (step S130) involved in the embodiments of this disclosure.

[0046] Figure 8 This is a conceptual diagram used to illustrate the second feature in the virtual space shooting process (step S130) involved in the embodiments of this disclosure.

[0047] Figure 9 This is a conceptual diagram used to illustrate the third feature in the virtual space shooting process (step S130) involved in the embodiments of this disclosure.

[0048] Figure 10 This is a block diagram used to illustrate variations of embodiments of the present disclosure.

[0049] Figure 11 This is a conceptual diagram illustrating an example of a curved screen according to an embodiment of the present disclosure.

[0050] Figure 12 This is a flowchart illustrating the virtual space shooting process (step S130) in the case of a curved image according to the embodiments of this disclosure.

[0051] Figure 13 This is a conceptual diagram illustrating the virtual space shooting process (step S130) in the case of a curved image as described in the embodiments of this disclosure.

[0052] Figure 14 This is a conceptual diagram illustrating the virtual space shooting process (step S130) in the case of a curved image as described in the embodiments of this disclosure.

[0053] Figure 15 This is a conceptual diagram illustrating the virtual space shooting process (step S130) in the case of a curved image as described in the embodiments of this disclosure.

[0054] Figure 16 This is a conceptual diagram illustrating a virtual space image depicted on a curved screen, which is used to explain the embodiments of this disclosure.

[0055] Figure 17 This is a conceptual diagram illustrating a modified example of a reverse-bent screen according to an embodiment of the present disclosure.

[0056] (Symbol Explanation)

[0057] 1: User; 4: Screen; 10: VR simulator; 20: Sensor; 30: Information processing device; 31: Processor; 32: Storage device; 40: Drawing device; 100: VR simulation program; 110: Actual user position acquisition unit; 120: Virtual user position acquisition unit; 130: Virtual space shooting unit; 140: Drawing unit; 200: Sensor detection information; 210: Coordinate transformation information; 220: Virtual space structure information; 230: Screen information; IMG: Virtual space image; IMG1: First virtual space image; IMG2: Second virtual space image; OBJ: Object; Sv: Virtual screen position; Ur: Actual user position; Uv: Virtual user position; VC: Virtual camera; VC1: First virtual camera; VC2: Second virtual camera. Detailed Implementation

[0058] Embodiments of this disclosure will be described with reference to the accompanying drawings.

[0059] 1. Overview of naked-eye VR simulation

[0060] Figure 1 This is a conceptual diagram illustrating the outline of the naked-eye VR (Virtual Reality) simulation involved in this embodiment. In the naked-eye VR simulation, user 1 can experience VR without wearing VR devices such as a head-mounted display.

[0061] More specifically, at least one screen 4 is positioned away from user 1. Unlike a head-mounted display, screen 4 is independent of user 1 and does not move in conjunction with user 1. The position of screen 4 can also be fixed. Screen 4 can also be positioned perpendicular to the ground. Multiple screens 4 can also be positioned around user 1. Multiple screens 4 can also be configured seamlessly and continuously. Figure 1 In the example shown, multiple screens 4 are positioned perpendicular to the ground and arranged seamlessly and continuously in an orthogonal manner. Screens 4 can also be larger than user 1. Screens 4 are, for example, positioned in a VR experience room.

[0062] In this way, an image of the virtual world (virtual space) is depicted on screen 4, which is located away from user 1. For example, a projector is placed at a location away from screen 4, and an image is projected from the projector onto screen 4. As another example, screen 4 can also be the screen of a display device such as a liquid crystal display or an organic EL display. In this case, the image is displayed on screen 4 of the display device. In the following description, "depicting an image on screen 4" means projecting or displaying an image on screen 4.

[0063] In such a glasses-free VR simulation, it is best to suppress the sense of incongruity felt by user 1 as much as possible. For example, there is an object OBJ in the virtual world. It is best to depict the object OBJ in screen 4 in a way that allows user 1 to see the object OBJ without any sense of incongruity from his position. Less sense of incongruity means, for example, that the image of the object OBJ depicted in screen 4 has less distortion. As another example, less sense of incongruity means that the perceived distance of the object OBJ depicted in screen 4 is close to that of reality. As described below, this embodiment provides a technique for suppressing the sense of incongruity felt by user 1 in a glasses-free VR simulation.

[0064] Reference Figure 2 This section explains the main terms used in this embodiment.

[0065] "Actual Space SPr" is the actual space where User 1 and Screen 4 actually exist. In Actual Space SPr, at least one Screen 4 is set away from User 1.

[0066] The "actual spatial coordinate system (Xr, Yr, Zr)" is a coordinate system defined for actual space SPr. The origin of the actual spatial coordinate system is set at any position within actual space SPr. The Xr, Yr, and Zr axes are orthogonal to each other. The Xr and Yr axes represent the horizontal direction, and the Zr axis represents the vertical direction.

[0067] "Actual User Position Ur" is the position of User 1 in the actual space SPr (i.e., the actual space coordinate system). More specifically, the actual user position Ur is the position of User 1's head in the actual space SPr. For example, the actual user position Ur is the position of User 1's eyes in the actual space SPr. The actual user position Ur can also be the center position of User 1's two eyes in the actual space SPr.

[0068] "Virtual Space (SPv)" is a virtual world space that serves as an object in VR simulation. The virtual world is arbitrary. For example, a virtual world can be a street. As another example, a virtual world can also be the world of a game.

[0069] The "virtual space coordinate system (Xv, Yv, Zv)" defines the coordinate system for the virtual space SPv. The origin of the virtual space coordinate system is set at any position within the virtual space SPv. The Xv, Yv, and Zv axes are orthogonal to each other. The Xv and Yv axes represent the horizontal direction, and the Zv axis represents the vertical direction.

[0070] "Virtual User Location Uv" is the location of User 1 in the virtual space SPv (i.e., the virtual space coordinate system).

[0071] The actual spatial coordinate system (Xr, Yr, Zr) and the virtual spatial coordinate system (Xv, Yv, Zv) are pre-assigned to each other. Therefore, coordinate transformations can be performed between the actual and virtual spatial coordinate systems. That is, any position in the actual spatial coordinate system can be transformed into its corresponding position in the virtual spatial coordinate system. Conversely, any position in the virtual spatial coordinate system can also be transformed into its corresponding position in the actual spatial coordinate system. For example, the actual user position Ur can be transformed into its corresponding virtual user position Uv. As another example, the position of screen 4 in the actual space SPr can be transformed into its corresponding position in the virtual space SPv. As yet another example, the position of object OBJ in the virtual space SPv can be transformed into its corresponding position in the actual space SPr.

[0072] "VR simulator 10" is a simulator that implements the naked-eye VR simulation involved in this embodiment. VR simulator 10 exists in the physical space SPr. Hereinafter, the VR simulator 10 involved in this embodiment will be described in detail.

[0073] 2. VR simulator

[0074] 2-1. Example of VR simulator structure

[0075] Figure 3 This is a block diagram illustrating a structural example of the VR simulator 10 according to this embodiment. The VR simulator 10 includes a sensor 20, an information processing device 30, and a drawing device 40.

[0076] Sensor 20 detects information used to obtain the actual user location Ur. For example, sensor 20 is a camera that captures images of user 1. As another example, sensor 20 could also be a position sensor worn on user 1's head. Various examples of methods for obtaining the actual user location Ur using sensor 20 will be described in detail later.

[0077] The display device 40 displays an image on the screen 4. For example, the display device 40 is a projector. The projector is positioned away from the screen 4 and projects an image onto the screen 4. As other examples, the display device 40 is a display device such as a liquid crystal display or an organic EL display. The display device has a screen 4 and displays an image on its own screen 4.

[0078] Information processing device 30 is a computer that performs various information processing tasks. Information processing device 30 includes one or more processors 31 (hereinafter referred to as processors 31) and one or more storage devices 32 (hereinafter referred to as storage devices 32). Processor 31 performs various processing tasks. For example, processor 31 includes a CPU (Central Processing Unit). Storage device 32 stores various types of information. Examples of storage devices 32 include volatile memory, non-volatile memory, HDD (Hard Disk Drive), and SSD (Solid State Drive).

[0079] VR simulation program 100 is a computer program for performing naked-eye VR simulation. The naked-eye VR simulation according to this embodiment is realized by executing VR simulation program 100 through information processing device 30 (processor 31). VR simulation program 100 is stored in storage device 32. VR simulation program 100 may also be recorded on a computer-readable recording medium. VR simulation program 100 may also be provided via a network.

[0080] The storage device 32 pre-stores coordinate transformation information 210, virtual space structure information 220, and screen information 230.

[0081] Coordinate transformation information 210 represents the correspondence between the actual spatial coordinate system (Xr, Yr, Zr) and the virtual spatial coordinate system (Xv, Yv, Zv) (see reference). Figure 2 By using the coordinate transformation information 210, coordinate transformation between the actual spatial coordinate system and the virtual spatial coordinate system can be performed.

[0082] Virtual space structure information 220 represents the configuration of the virtual space SPv. More specifically, various objects OBJ exist within the virtual space SPv. Virtual space structure information 220 represents the area occupied by each object OBJ in the virtual space SPv (virtual space coordinate system). For example, virtual space structure information 220 represents the position, size, and shape of each object OBJ in the virtual space SPv (virtual space coordinate system).

[0083] Screen information 230 represents information about each screen 4 set in the actual space SPr. More specifically, screen information 230 represents the area occupied by each screen 4 in the actual space SPr (actual space coordinate system). For example, screen information 230 represents the position, size, and shape of each screen 4 in the actual space SPr (actual space coordinate system).

[0084] 2-2. Processing example using a VR simulator

[0085] Figure 4 This is a block diagram illustrating an example of the functional structure of the VR simulator 10 according to this embodiment. The VR simulator 10 includes, as functional blocks, a real user position acquisition unit 110, a virtual user position acquisition unit 120, a virtual space imaging unit 130, and a rendering unit 140. These functional blocks are implemented through the cooperation of the processor 31 and the storage device 32 that execute the VR simulation program 100.

[0086] Figure 5 This is a flowchart illustrating the process implemented by the VR simulator 10 according to this embodiment. (Refer to...) Figure 4 as well as Figure 5 This describes the processing flow implemented by VR simulator 10.

[0087] 2-2-1. Step S110 (Actual User Location Acquisition Processing)

[0088] In step S110, the actual user position acquisition unit 110 acquires the actual user position Ur in the actual space SPr. As described above, the actual user position Ur is the position of the user 1's head in the actual space SPr. For example, the actual user position Ur is the position of the user 1's eyes in the actual space SPr. The actual user position Ur can also be the center position of the user 1's two eyes in the actual space SPr.

[0089] More specifically, a sensor 20 installed in the actual space SPr detects information used to obtain the actual user location Ur. Sensor detection information 200 represents the detection result obtained by the sensor 20. The actual user location acquisition unit 110 obtains the sensor detection information 200 from the sensor 20. The sensor detection information 200 is stored in the storage device 32. Then, the actual user location acquisition unit 110 obtains the actual user location Ur based on the sensor detection information 200. The information of the actual user location Ur is stored in the storage device 32.

[0090] For example, sensor 20 is at least one camera that captures images of user 1. The camera is, for example, set at a predetermined location in the VR experience room. Information on the camera's location and orientation in the actual spatial coordinate system is provided in advance. Sensor detection information 200 is an image of user 1 captured by the camera. The actual user position acquisition unit 110 identifies user 1's head (e.g., eyes) by analyzing the image of user 1 represented by the sensor detection information 200. Then, the actual user position acquisition unit 110 acquires the actual user position Ur based on the position of user 1's head as observed from the camera, the camera's location, and its orientation.

[0091] As another example, sensor 20 could also be a position sensor worn on the head of user 1. For example, the position sensor is a ranging sensor that measures the relative distance to the walls and ceiling of the VR experience room. In this case, sensor detection information 200 is the relative distance measured by the position sensor. Information about the positions of the walls and ceiling in the actual spatial coordinate system is provided in advance. The actual user position acquisition unit 110 can acquire the actual user position Ur based on the positions of the walls and ceiling and the relative distances to them.

[0092] 2-2-2. Step S120 (Virtual User Location Acquisition Processing)

[0093] In step S120, the virtual user location acquisition unit 120 acquires a virtual user location Uv corresponding to the actual user location Ur acquired in step S110. More specifically, the virtual user location acquisition unit 120 transforms the actual user location Ur into its corresponding virtual user location Uv based on coordinate transformation information 210. The information of the virtual user location Uv is stored in the storage device 32.

[0094] 2-2-3. Step S130 (Virtual Space Shooting and Processing)

[0095] In step S130, the virtual space imaging unit 130 generates and renders an image of the virtual space SPv observed from the virtual user location Uv. Hereinafter, the image of the virtual space SPv observed from the virtual user location Uv will be referred to as the "virtual space image IMG".

[0096] Figure 6 This is a conceptual diagram illustrating step S130. The virtual space imaging unit 130 configures a virtual camera VC within a virtual user location Uv in the virtual space SPv. The virtual camera VC is a camera implemented in software. The parameters of the virtual camera VC (focal length, sensor size, etc.) are set to match human vision. Furthermore, the virtual space imaging unit 130 captures images of the virtual space SPv using the virtual camera VC configured in the virtual user location Uv. The image of the virtual space SPv captured by the virtual camera VC is a virtual space image IMG. The structure of the virtual space SPv, i.e., the position, shape, and size of each object OBJ in the virtual space SPv (virtual space coordinate system), is obtained from the virtual space structure information 220. Therefore, the virtual space imaging unit 130 can capture images of the virtual space SPv based on the virtual space structure information 220 to obtain a virtual space image IMG, which is then capable of rendering.

[0097] The virtual space imaging unit 130 can also acquire a virtual space image (IMG) for each of the frames 4. The position of each frame 4 in the actual space SPr is obtained from the image information 230. The virtual space imaging unit 130 acquires the position of each frame 4 in the virtual space SPv based on the image information 230 and the coordinate transformation information 210. Furthermore, the virtual space imaging unit 130 uses a virtual camera VC to capture images of the virtual space SPv in the direction of each frame 4 from the virtual user position Uv. Alternatively, the virtual space imaging unit 130 can also capture images of the virtual space SPv in the entire circumference from the virtual user position Uv.

[0098] Furthermore, in this embodiment, information about the user 1's gaze direction is not required. The virtual camera VC captures images of the virtual space SPv in a predetermined direction regardless of the user 1's gaze direction. The predetermined direction includes the direction of each frame 4 as viewed from the virtual user's position Uv. The predetermined direction can also be a full-circle direction. The virtual space capturing unit 130 does not crop out a virtual space image IMG in the user 1's gaze direction, but rather acquires a virtual space image IMG that can be observed from the virtual user's position Uv. As a comparative example, according to the technology disclosed in Patent Document 1, the shooting direction of the virtual camera changes accordingly when the user's orientation changes.

[0099] In the virtual space shooting process (step S130), efforts are made to suppress disharmony when depicting the virtual space image IMG in frame 4. The efforts to suppress disharmony are explained in detail in section 3 later.

[0100] 2-2-4. Step S140 (Painting Processing)

[0101] In step S140, the drawing unit 140 draws a virtual space image IMG on the screen 4 by controlling the drawing device 40. At this time, the drawing unit 140 can also draw the virtual space image IMG obtained for each of each screen 4. If the drawing device 40 is a projector, the drawing unit 140 projects the virtual space image IMG onto the screen 4 by controlling the projector. As another example, if the drawing device 40 is a display device, the drawing unit 140 displays the virtual space image IMG on the screen 4 by controlling the display device.

[0102] 3. Features of virtual space shooting processing (step S130)

[0103] In the virtual space shooting process (step S130), efforts are made to suppress the sense of disharmony when depicting the virtual space image IMG in screen 4. Hereinafter, the features of the virtual space shooting process according to this embodiment will be described using three types of features: "fixing the vanishing point," "lens shifting," and "adjusting the perspective." By performing feature processing of at least one of these three types, at least the sense of disharmony perceived by the user 1 can be suppressed. Of course, it is also possible to perform feature processing of two or more of the three types.

[0104] 3-1. Fixed Vanishing Point

[0105] Figure 7 This is a conceptual diagram illustrating the first feature of the virtual space capture process (step S130). As the first feature, the vanishing point of the virtual space image IMG is fixed in a straight horizontal direction when viewed from the virtual user position Uv. In other words, the virtual space capture unit 130 acquires the virtual space image IMG such that the vanishing point exists in the horizontal direction when viewed from the virtual user position Uv.

[0106] Regardless of the user 1's gaze direction, the vanishing point is fixed in the horizontal direction when viewed from the virtual user position (UV). Even if the user 1's gaze direction changes vertically, the virtual camera (VC) does not rotate vertically. If the virtual camera (VC) were to rotate vertically according to the user 1's gaze direction, the vertical lines of the virtual space image (IMG) depicted on screen 4 would converge, causing the IMG to appear distorted. According to this embodiment, since the vanishing point is fixed in the horizontal direction regardless of the user 1's gaze direction, distortion of the virtual space image (IMG) depicted on screen 4 is prevented. In other words, the sense of disharmony perceived by the user 1 can be suppressed.

[0107] Of course, if user 1 moves their gaze in the actual space SPr, then user 1's field of view changes accordingly. At this time, the virtual space image IMG of screen 4 depicting user 1's surroundings also remains unchanged, so that the objects existing in the actual space SPr themselves do not change.

[0108] 3-2. Lens Shift

[0109] Figure 8This is a conceptual diagram illustrating the second feature of the virtual space shooting process (step S130). As the second feature, the virtual space shooting unit 130 performs a "lens shifting process" by shifting the lens of the virtual camera VC in a manner that converges the entire image 4 within the field of view of the virtual camera VC. In the lens shifting process, the virtual space shooting unit 130 does not change the focal length, but shifts the lens of the virtual camera VC in the vertical and / or horizontal directions. Furthermore, the position of the vanishing point does not change due to the lens shift.

[0110] The amount of lens shift required to bring the entire image 4 into the field of view of the virtual camera VC can be determined based on the size of image 4 and the positional relationship between the actual user position Ur and image 4. The position and size of image 4 in the actual space SPr are obtained from image information 230. Therefore, the virtual space shooting unit 130 can determine the amount of lens shift required to bring the entire image 4 into the field of view of the virtual camera VC based on the actual user position Ur and image information 230.

[0111] By performing this camera shift, areas where nothing is reflected in frame 4 are suppressed. This enhances the user's sense of immersion in VR.

[0112] Here, it is also important to avoid vertical rotation of the virtual camera VC. If the virtual camera VC is rotated vertically to bring the entire image 4 into the field of view, the vertical lines of the virtual space image IMG depicted on image 4 will converge, causing the virtual space image IMG to appear distorted. According to this embodiment, a lens shift is performed instead of rotating the virtual camera VC vertically. This prevents distortion of the virtual space image IMG depicted on image 4. In other words, it suppresses the sense of disharmony perceived by the user 1.

[0113] 3-3. Adjusting the sense of distance

[0114] Figure 9 This is a conceptual diagram used to illustrate the third feature in the virtual space shooting process (step S130). An image of an object OBJ is depicted in a certain frame 4. Distance DA is the distance between user 1 (virtual user position Uv) and the object OBJ. Distance DB is the distance between user 1 (actual user position Ur) and the frame 4.

[0115] The virtual camera VC captures an image of an object OBJ located at a distance DA from the virtual user position Uv, generating a virtual spatial image IMG that includes the object OBJ. If, assuming the image of the object OBJ at a distance DB is depicted exactly as is in frame 4, the object OBJ appears to exist at a distance DA+DB when viewed from the actual user position Ur. For example, suppose distance DA is 10m and distance DB is 2m. If the image of the object OBJ 10m in front of the user is depicted exactly as is in frame 4 2m in front of the user, the object OBJ appears to exist 12m in front of the user. That is, in addition to the sense of distance in the virtual space SPv, the sense of distance in the actual space SPr is unnecessarily added. As a result, the object OBJ appears smaller than it actually is. This could also be a cause of disharmony.

[0116] Therefore, as a third feature, the virtual space shooting unit 130 eliminates the sense of distance that is equivalent to the distance DB between the actual user position Ur and the image 4. More specifically, the virtual space shooting unit 130 adjusts the focal length of the virtual camera VC to the telephoto side to eliminate the sense of distance that is equivalent to the distance DB between the actual user position Ur and the image 4. As a result, unnecessary sense of distance is eliminated, the original sense of distance in the virtual space SPv is reproduced, and the sense of incongruity felt by the user 1 is suppressed.

[0117] Based on the distance DB between the actual user position Ur and the frame 4, the amount of focus adjustment required to eliminate unnecessary perception of distance is determined. The position of the frame 4 in the actual space SPr is obtained from the frame information 230. The virtual space shooting unit 130 can determine the amount of focus adjustment required to eliminate perception of distance equivalent to the distance DB based on the actual user position Ur and the frame information 230. Furthermore, the virtual space shooting unit 130 adjusts the perception of distance for each frame 4.

[0118] 4. Variations

[0119] As described above, various processes are performed from obtaining the actual user location Ur to depicting the virtual space image IMG. The appearance of the virtual space image IMG depicted on screen 4 may experience a delay corresponding to this processing time. To suppress such delay, in a variant example, a prediction (speculation) of the future actual user location Ur is performed.

[0120] Figure 10 This is a block diagram used to illustrate the variations. The details mentioned above are appropriately omitted. Figure 4 Repeating explanation. Historical information HST represents the past history of the actual user location Ur and is stored in storage device 32.

[0121] In step S110, the actual user location acquisition unit 110 acquires the actual user location Ur. The actual user location acquisition unit 110 registers the acquired actual user location Ur in the historical information HST and updates the historical information HST. Furthermore, the actual user location acquisition unit 110 predicts (predicts) the future actual user location Ur based on the past history of the actual user location Ur represented by the historical information HST. For example, the actual user location acquisition unit 110 calculates the acceleration of the actual user location Ur based on its past history, and predicts the future actual user location Ur based on this acceleration. A Kalman filter or similar method can also be used in the prediction of the actual user location Ur.

[0122] In step S120, the virtual user location acquisition unit 120 acquires a virtual user location Uv that corresponds to the future actual user location Ur predicted in step S110. That is, the virtual user location acquisition unit 120 acquires the future virtual user location Uv. Then, in step S130, the virtual space imaging unit 130 configures a virtual camera VC at the future virtual user location Uv and performs virtual space imaging processing.

[0123] Thus, according to the variation, the future actual user location Ur is predicted, and virtual space imaging processing is performed based on the future virtual user location Uv corresponding to the future actual user location Ur. As a result, the delay in the appearance of the virtual space image IMG depicted on screen 4 is suppressed. This also helps to suppress the sense of incongruity perceived by user 1.

[0124] 5. Summary

[0125] As explained above, this embodiment enables naked-eye VR simulation. Specifically, a virtual camera VC is configured at a virtual user location Uv that corresponds to the actual user location Ur, and the virtual camera VC is used to capture images of the virtual space SPv to obtain a virtual space image IMG. The virtual space image IMG is displayed on a screen 4 located at a position far from the user 1 and not moving in conjunction with the user 1.

[0126] As an effort to suppress disharmony when depicting the virtual space image IMG in screen 4, three types of feature processing are explained: "fixed vanishing point," "camera shift," and "adjustment of perspective" (see Chapter 3). By performing feature processing of at least one of these three types, the disharmony perceived by user 1 can be suppressed. Of course, it is also possible to perform feature processing of two or more of the three types.

[0127] The naked-eye VR simulation described in this embodiment does not depend on the position, orientation, or shape of screen 4. It can also create a six-sided VR experience room with a ceiling, floor, front and back, and left and right sides. Furthermore, screen 4 can be further segmented to achieve more complex screen structures. It can also create a VR experience room with a global surface.

[0128] 3. Application of curved images

[0129] Next, we will explain the case where screen 4 is curved. For convenience, we will refer to the curved screen 4 as "curved screen 4C". Except for the curvature, curved screen 4C is the same as screen 4 described above. Figure 11 This shows an example of a curved screen (4C). Figure 11 In the example shown, the curved screen 4C is parallel to the vertical direction (Zr axis) and bends around the vertical direction (Zr axis). User 1 is located in front of this curved screen 4C.

[0130] The naked-eye VR simulation described in this embodiment can also be applied to a curved screen 4C. However, when the virtual space image IMG is depicted on the curved screen 4C without any effort, there is a possibility of appearance distortion and a sense of disharmony experienced by the user. Even in naked-eye VR simulation using a curved screen 4C, it is desirable to suppress the sense of disharmony experienced by the user. Hereinafter, efforts to suppress the sense of disharmony in the case of a curved screen 4C will be explained.

[0131] 3-1. Features of Virtual Space Photography Processing (Step S130)

[0132] Figure 12 This is a flowchart illustrating the virtual space shooting process (step S130) in the case of a curved image 4C. The virtual space shooting process in the case of a curved image 4C will be explained in detail below.

[0133] 3-1-1. Step S131 (First Shot Processing)

[0134] In step S131, the virtual space imaging unit 130 generates an image of the virtual space SPv observed from the virtual user location Uv, i.e., it performs rendering.

[0135] Figure 13This is a conceptual diagram illustrating step S131. The virtual space imaging unit 130 configures a first virtual camera VC1 within a virtual user location Uv in the virtual space SPv. The first virtual camera VC1 is the same as the virtual camera VC described above. The virtual space imaging unit 130 captures images of the virtual space SPv using the first virtual camera VC1 configured in the virtual user location Uv. The first virtual space image IMG1 is an image of the virtual space SPv captured by the first virtual camera VC1. The virtual space imaging unit 130 can capture images of the virtual space SPv based on the virtual space structure information 220, thereby obtaining the first virtual space image IMG1.

[0136] In step S131, the virtual space imaging unit 130 may also perform the feature processing described in sections 2-3 above. That is, the virtual space imaging unit 130 may also acquire the first virtual space image IMG1 in such a way that the vanishing point exists in the horizontal direction as seen from the virtual user position Uv. The virtual space imaging unit 130 may also perform lens shifting processing to shift the lens of the first virtual camera VC1 so that the entire curved image 4C converges within the field of view of the first virtual camera VC1. The virtual space imaging unit 130 may also adjust the focal length of the first virtual camera VC1 in a way that eliminates the sense of distance between the actual user position Ur and the curved image 4C, making it comparable to the distance DB between them.

[0137] As described above, image information 230 is used in lens shift processing and perspective adjustment processing. Image information 230 represents the position, size, and shape of the curved image 4C in actual space SPr (actual space coordinate system). Furthermore, the position of the curved image 4C can also be represented by the position of an "approximate plane A" that approximates the curved image 4C using a plane.

[0138] 3-1-2. Step S132 (Temporary Drawing Processing)

[0139] Figure 14 This is a conceptual diagram used to explain step S132. First, let's explain the image position. The "actual image position" is the setting position of the curved image 4C in the actual space SPr (actual space coordinate system). Alternatively, the position of the approximate plane A that approximates the curved image 4C using a plane can also be used as the actual image position. The actual image position is obtained from the image information 230. The "virtual image position Sv" is the position within the virtual space SPv that corresponds to the actual image position. The virtual space capturing unit 130 transforms the actual image position into the virtual image position Sv according to the coordinate transformation information 210, thereby obtaining the virtual image position Sv.

[0140] "Reverse Curved Frame 4R" is a virtual frame that curves in the opposite direction to the curved frame 4C. For example, if the curved frame 4C is convex when viewed from user 1, then the reverse curved frame 4R is concave when viewed from user 1. Conversely, if the curved frame 4C is concave when viewed from user 1, then the reverse curved frame 4R is convex when viewed from user 1. Figure 14 As shown, when the approximate plane A of the approximately curved image 4C and the approximate plane A of the approximately reverse curved image 4R overlap, the curved image 4C and the reverse curved image 4R face each other, sandwiching the approximate plane A. The curved image 4C and the reverse curved image 4R can also be described as "symmetrical" sandwiching the approximate plane A.

[0141] In step S132, the virtual space imaging unit 130 virtually configures a reverse-curved image 4R at a virtual image position Sv within the virtual space SPv. For example, as... Figure 14 As shown, the reverse curved screen 4R is configured such that the approximate plane A of the approximate curved screen 4C and the approximate plane A of the approximate reverse curved screen 4R are aligned. That is, the reverse curved screen 4R is configured to overlap with the setting position of the curved screen 4C and be symmetrical to the curved screen 4C.

[0142] Furthermore, the virtual space imaging unit 130 depicts (projects or displays) the first virtual space image IMG1 obtained in step S131 on the inverted curved screen 4R positioned at the virtual screen position Sv. For example... Figure 14 As shown, the first virtual space image IMG1 depicted in the reverse curved image 4R appears distorted.

[0143] 3-1-3. Step S133 (Second Shooting Processing)

[0144] Figure 15 This is a conceptual diagram illustrating step S133. The virtual space imaging unit 130 configures a second virtual camera VC2 within the virtual user location Uv in the virtual space SPv. The second virtual camera VC2 is the same as the aforementioned virtual camera VC. The virtual space imaging unit 130 uses the second virtual camera VC2 configured in the virtual user location Uv to capture the first virtual space image IMG1 on the reverse curved screen 4R. The second virtual space image IMG2 is the image captured by the second virtual camera VC2. That is, the virtual space imaging unit 130 captures the first virtual space image IMG1 observed from the virtual user location Uv as the virtual space SPv, obtaining the second virtual space image IMG2.

[0145] In step S133, the virtual space imaging unit 130 may also perform the feature processing described in sections 2-3 above. That is, the virtual space imaging unit 130 may also acquire the second virtual space image IMG2 in such a way that the vanishing point exists in the horizontal direction as seen from the virtual user position Uv. The virtual space imaging unit 130 may also perform lens shifting processing to shift the lens of the second virtual camera VC2 so that the entire curved image 4C converges within the field of view of the second virtual camera VC2. The virtual space imaging unit 130 may also adjust the focal length of the second virtual camera VC2 in such a way that the sense of distance is equivalent to the distance DB between the actual user position Ur and the curved image 4C. Furthermore, in the lens shifting processing and the sense of distance adjustment processing, the curved image 4C is used as an image equivalent to the reverse curved image 4R. The position, size, and shape of the curved image 4C are obtained from the image information 230.

[0146] 3-1-4. Step S134

[0147] In step S134, the virtual space imaging unit 130 sets the second virtual space image IMG2 as the virtual space image IMG that is drawn (projected or displayed) on the curved screen 4C in step S140.

[0148] Figure 16 This is a conceptual diagram illustrating the virtual space image IMG drawn (projected or displayed) on the curved screen 4C in step S140. As described above, the second virtual space image IMG2 has distortion corresponding to the curvature of the reverse curved screen 4R. When the second virtual space image IMG2 is drawn on the curved screen 4C, distortion occurs in the opposite direction to the distortion caused by the reverse curved screen 4R. That is, the distortion caused by the reverse curved screen 4R and the distortion caused by the curved screen 4C cancel each other out. As a result, when viewed from the user 1, the virtual space image IMG (= the second virtual space image IMG2) drawn on the curved screen 4C looks natural and has less distortion. That is, the sense of disharmony felt by the user 1 is suppressed.

[0149] Furthermore, the curved image 4C and the reverse curved image 4R do not necessarily need to be perfectly symmetrical. Even if the distortion caused by the reverse curved image 4R and the distortion caused by the curved image 4C partially cancel each other out, a certain degree of effect can still be achieved. That is, as long as the reverse curved image 4R bends in the opposite direction to the curved image 4C, an effect can be obtained to some extent.

[0150] 3-2. Examples of distortion in reverse-curved images

[0151] Figure 17This is a conceptual diagram illustrating a variation of the reverse curved image 4R. By slightly deforming the reverse curved image 4R, which corresponds to the curved image 4C, image distortion is further suppressed. Specifically, compared to the original reverse curved image 4R, which corresponds to the curved image 4C, the height H is slightly increased and the width W is slightly decreased. Furthermore, the left and right ends of the reverse curved image 4R can also be slightly narrowed.

[0152] 3-3. Effects

[0153] As explained above, according to this embodiment, the sense of disharmony felt by the user 1 can be suppressed in naked-eye VR simulation. In particular, the sense of disharmony can be suppressed even when the screen is curved 4C.

Claims

1. A virtual reality simulator, comprising: One or more processors; and One or more storage devices store virtual space structure information representing the structure of the virtual space. The one or more processors are configured to perform: The processing of information to obtain the actual user's location, which is the position of the user's head in the actual space; The process of obtaining the virtual user location, which is the location within the virtual space corresponding to the actual user location; The image processing involves capturing images of the virtual space based on the virtual space structure information, using a camera positioned at the virtual user's location within the virtual space to obtain virtual space images; and... The process of projecting or displaying the virtual space image onto a curved screen, wherein the curved screen is positioned away from the user in the actual space and does not move in tandem with the user. The virtual image position is the position within the virtual space that corresponds to the setting position of the curved image. The image processing includes: The first shooting process involves capturing images of the virtual space using a first camera positioned at the virtual user's location, thereby obtaining a first virtual space image. The process of projecting or displaying the first virtual space image in a reverse curved image that is configured at the virtual image position and curved in the opposite direction to the curved image; The second shooting process involves capturing the first virtual space image on the reverse-curved screen using a second camera positioned at the virtual user's location, thereby obtaining a second virtual space image; and The process of setting the second virtual space image as a projection or display of the virtual space image on the curved screen.

2. The virtual reality simulator according to claim 1, wherein, The first image capture process further includes obtaining the first virtual space image in a manner that the vanishing point, as viewed from the virtual user's position, exists in the horizontal direction. The second shooting process also includes processing to obtain the second virtual space image in such a way that the vanishing point exists in the horizontal direction as seen from the virtual user's position.

3. The virtual reality simulator according to claim 1 or 2, wherein, The first shooting process also includes a process of shifting the lens of the first camera to bring the entire curved image into the field of view of the first camera. The second shooting process also includes a process of shifting the lens of the second camera to bring the entire curved image into the field of view of the second camera.

4. The virtual reality simulator according to claim 1 or 2, wherein, The first shooting process also includes adjusting the focal length of the first camera in a manner that eliminates the sense of distance between the actual user position and the curved image. The second shooting process also includes adjusting the focal length of the second camera in a manner that eliminates the sense of distance between the actual user position and the curved image.

5. The virtual reality simulator according to claim 1 or 2, wherein, The one or more processors Based on the past history of the actual user's location, predict the future location of the actual user. The shooting process is performed based on the future virtual user location, which corresponds to the future actual user location.

6. A computer-readable recording medium containing a virtual reality simulation program, wherein, When the virtual reality simulation program is executed by a computer, it causes the computer to perform: The processing of information to obtain the actual user's location, which is the position of the user's head in the actual space; The process of obtaining the virtual user location, which is a location in a virtual space corresponding to the actual user location; The image processing involves capturing images of the virtual space using cameras positioned at the virtual user's location within the virtual space, based on virtual space structure information representing the structure of the virtual space; and... The process of projecting or displaying the virtual space image onto a curved screen, wherein the curved screen is positioned away from the user in the actual space and does not move in tandem with the user. The virtual image position is the position within the virtual space that corresponds to the setting position of the curved image. The image processing includes: The first shooting process involves capturing images of the virtual space using a first camera positioned at the virtual user's location, thereby obtaining a first virtual space image. The process of projecting or displaying the first virtual space image in a reverse curved image that is configured at the virtual image position and curved in the opposite direction to the curved image; The second shooting process involves capturing the first virtual space image on the reverse-curved screen using a second camera positioned at the virtual user's location, thereby obtaining a second virtual space image; and The process of setting the second virtual space image as a projection or display of the virtual space image on the curved screen.

Images