Information processing device and information processing method

Abstract

An information processing device 10 according to one embodiment of the present invention comprises: a current location storage unit 112 that stores, in association with the time of acquisition, current location information indicating the current location of a user in a virtual space; a map creation unit 113 that creates a map image indicating the current location of the user in the virtual space, on the basis of the current location information; a movement determination unit 114 that determines movement of the user in the virtual space, on the basis of the current location information; a calculation unit 115 that creates virtual space data which includes a 3D model of the interior of the virtual space, and a viewpoint position and a gaze direction of the user based on the current location information, and if the movement determination unit determines that the user is moving, then superimposes the map image data on the virtual space data such that the map image is displayed in a field of view of the user in the virtual space; and a rendering function unit 116 that creates an image indicating the field of view of the user on the basis of the virtual space data.

Description

Information Processing Apparatus and Information Processing Method

[0001] One aspect of the present invention relates to an information processing apparatus and an information processing method.

[0002] In recent years, information processing apparatuses that construct virtual spaces in a computer, typified by VR (Virtual Reality), have been widely used. Generally, such information processing apparatuses construct an unrealistic virtual space that many people can log in to by using CG (Computer Graphics) technology or the like. However, recently, in addition to this, there is known an information processing apparatus that creates a 3D virtual space that faithfully reproduces the real world based on data measured in the real world.

[0003] As one method for such an information processing apparatus to measure the real world and construct a virtual space that mimics the real world, there is a method that uses 3D point cloud data (see, for example, Non-Patent Documents 1 and 2). 3D point cloud data is generally data composed of attribute information such as three-dimensional coordinate values (X, Y, Z) and color information (R, G, B), and can be acquired by an acquisition method such as the LiDAR (Light Detection and Ranging) method (see, for example, Non-Patent Documents 3 and 4). The LiDAR method irradiates a target object with laser light and measures the distance, direction, position, shape, etc. to the object by measuring the time until the laser light hits the object and bounces back. The LiDAR method represents the scan result of the laser light as points and acquires countless collected points (point cloud) as 3D point cloud data.

[0004] Since 3D point cloud data is three-dimensional information at the time of measurement, it has the advantage that a three-dimensional virtual space with a wide range can be constructed at a low cost compared to constructing a three-dimensional space from CG or photographs. In addition, 3D point cloud data also has the advantage that it can reproduce the real space more accurately than constructing a three-dimensional space from CG or photographs. That is, the information processing apparatus can construct a 360-degree virtual space (point cloud metaverse) based on the 3D point cloud data.

[0005] However, it is known that when users move around for extended periods in a virtual space composed of 3D point cloud data while wearing VR goggles such as an HMD (Head Mounted Display) connected to an information processing device, they may experience what is known as "VR sickness," such as feeling unwell. VR sickness is thought to be caused by a mismatch between the information sent to the brain regarding movement via vision and the information regarding stillness via the sense of balance.

[0006] The following methods (a) to (c) are known to reduce VR sickness: (a) is a method that uses teleportation mode to instantly move to the desired location in the virtual space (see, for example, Non-Patent Document 5). (b) is a method that applies a vignette effect (or tunnel effect) to the surrounding area of ​​the screen when the user moves in the virtual space (see, for example, Non-Patent Documents 6 and 7). (c) is a method that provides the user with sound and vibration linked to vision to compensate for the lack of sensory information linked to vision (see, for example, Non-Patent Document 8).

[0007] "TENGUN Ogijima Project Launched to Promote Regional Co-creation," [online], November 15, 2022, Nippon Telegraph and Telephone Corporation, [Retrieved September 9, 2024], Internet<URL: https: / / group.ntt / jp / newsrelease / 2022 / 11 / 15 / 221115b.html> Takahiro Matsumoto, Kei Komazaki, Yuichi Maki, Hiroshi Chiaki, Takayoshi Mochizuki, "TENGUN Ogijima: Research on High-Presence VR through Visual, Auditory, and Tactile Sensations Based on Measurement Data," [online], March 1, 2023, Information Processing Society of Japan Interaction 2023, [Retrieved September 9, 2024], Internet<URL: https: / / www.interaction-ipsj.org / proceedings / 2023 / data / pdf / 2B-17.pdf> "What is LiDAR? An explanation of the principles and applications of this optical sensor attracting attention in autonomous driving," [online], September 25, 2023, [Retrieved September 9, 2024], Internet<URL:https: / / engineer.fabcross.jp / archeive / 211109_lidar.html> "What is Point Cloud Data? | Acquisition Methods and Application Examples," [online], May 14, 2024, [Retrieved September 9, 2024], Internet<URL: https: / / club.informatix.co.jp / ?p=1125> "The Mechanism of 'VR Sickness' in Virtual Spaces and Solutions," [online], August 8, 2021, [Retrieved September 9, 2024], Internet<URL: https: / / wired.jp / 2021 / 08 / 08 / how-to-reduce-motion-sickness-virtual-reality / > Daiki Kayaba, Yoshiaki Miyashita, "Investigation of Speed ​​Perception and Motion Sickness through Dynamic Tunneling in VR Racing Games," [online], August 2021, Entertainment Computing Symposium (EC2021), [Retrieved September 9, 2024], Internet <URL: https: / / ipsj.ixsq.nii.ac.jp / ej / ?action=repository_uri&item_id=212575&file_id=1&file_no=1>Daiki Kayaba, Yoshiaki Miyashita, "A method to reduce VR sickness by restricting non-circular field of view according to optical flow", [online], March 6, 2023, Information Processing Society of Japan, [Retrieved September 9, 2024], Internet<URL: https: / / www.miyashita.com / researches / 66qeAOtF6mhWEbr8asH6uM> "Reducing VR sickness by utilizing engine sound and vibration - demonstrated by a psychological experiment using a VR motorcycle simulator," [online], May 12, 2020, [Retrieved September 9, 2024], Internet<URL: https: / / www.shizuoka.ac.jp / news / detail.html?CN=6399> Naoki Miura, Hiroyuki Ujiie, Noriko Okura, and Masaomi Kimura, "The effect of maintaining gaze in a specific direction on VR sickness," [online], September 2019, The Virtual Reality Society of Japan, [Retrieved September 9, 2024], Internet<URL: https: / / conference.vrsj.org / ac2019 / program / common / doc / pdf / 4C-03.pdf> .

[0008] While the methods (a) to (c) described above are effective in reducing VR sickness, the following challenges exist. For example, in method (a), if the scenery during movement is meaningful, such as in a point cloud metaverse, it becomes impossible to see it, reducing the quality of the experience and the sense of realism. For example, method (b) forcibly hides part of the screen, causing discomfort in how the screen looks. Also, in method (b), the user's feet become invisible while moving. For example, method (c) requires a separate device from the HMD or other display device to generate vibrations, etc., which increases the cost.

[0009] This invention was made in view of the above circumstances, and aims to provide an information processing device and information processing method that can reduce VR sickness without degrading the quality of the user's experience when moving within a virtual space, and without incurring additional costs.

[0010] An information processing device according to one aspect of this invention includes: a current location storage unit that stores current location information indicating the user's current location in a virtual space in association with the time at which it was acquired; a map creation unit that creates a map image indicating the user's current location in the virtual space based on the current location information; a movement determination unit that determines whether the user is moving in the virtual space based on the current location information; a calculation unit that creates virtual space data including a 3D model in the virtual space and the user's viewpoint position and line of sight based on the current location information, and when the movement determination unit determines that the user is moving, it superimposes map image data onto the virtual space data so that the map image is displayed in the user's field of view in the virtual space; and a rendering function unit that creates an image indicating the user's field of view based on the virtual space data.

[0011] According to one aspect of this invention, it is possible to provide an information processing device and an information processing method that can reduce VR sickness without degrading the quality of the user's experience and without incurring costs when moving within a virtual space.

[0012] Figure 1 is a schematic diagram showing one example configuration of an information processing device according to an embodiment. Figure 2 is a block diagram showing an example of the functional configuration of an information processing device according to an embodiment. Figure 3 is a flowchart showing an example of the operation of an information processing device according to an embodiment. Figure 4 is a diagram showing an example of a map image created by an information processing device according to an embodiment. Figure 5 is a diagram showing an example of an image created by an information processing device according to an embodiment. Figure 6 is a diagram showing an example of an image created by an information processing device according to an embodiment. Figure 7 is a diagram showing an example of an image created by an information processing device according to another embodiment.

[0013] Embodiments of this invention will be described below with reference to the drawings.

[0014] Figure 1 is a schematic diagram showing one example configuration of an information processing device according to the embodiment. Figure 2 is a block diagram showing an example of the functional configuration of the information processing device according to the embodiment. The information processing device 10 is a computer capable of outputting an image of a virtual space that mimics the real world based on acquired 3D point cloud data. The information processing device 10 is, for example, a personal computer. The information processing device 10 may be configured as a server computer installed on the Web or the cloud.

[0015] In this embodiment, the information processing device 10 is connected to the external storage device 20 and the HMD 30 by wire or wireless connection. Note that in Figure 1, the information processing device 10 is connected to the external storage device 20 by wire and to the HMD 30 by wireless connection, but this is not limited to this configuration. The connection configuration between the information processing device 10 and the external storage device 20 and HMD 30 can be appropriately selected.

[0016] The external storage device 20 is a storage device that stores map data and 3D point cloud data. The map data includes image data representing a map image that provides an overview of a virtual space created based on the 3D point cloud data, and supplementary information related to the map image. The supplementary information may include, for example, coordinate information (X, Y, Z) in the virtual space corresponding to coordinate information (X, Y, Z) on the map image. The 3D point cloud data is data used to construct a 3D point cloud model that mimics the real space in a virtual space, and is composed of a collection of multiple point data. The point data that makes up the 3D point cloud data consists of coordinate information (X, Y, Z) in the virtual space and color information (R, G, B). The 3D point cloud data is acquired, for example, using LiDAR (Light Detection And Ranging) and stored in the external storage device 20.

[0017] The HMD 30 is a VR goggle worn by a user and includes, for example, memory, a processor, a display, sensors, an input interface, and a communication interface (not shown). The HMD 30 displays images from the virtual space output from the information processing device 10 on its display by operating the processor according to a program stored in the memory. The HMD 30 also generates the user's current location information based on the user's movements detected by sensors such as a gyro sensor, and operation information input by the user via an input interface such as a controller. The current location information indicates the user's current location (latest position) in the virtual space and includes coordinate information in the virtual space indicating the user's viewpoint position and information on the direction of the line of sight from the user's viewpoint position. The HMD 30 outputs the generated current location information to the information processing device 10 via the communication interface. In the following description, "user" refers to the operator of the HMD 30. Here, the administrator of the information processing device 10 and the operator (user) of the HMD 30 may be the same person or different people.

[0018] In this embodiment, the information processing device 10 comprises a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, and an output unit 15. The storage unit 12 comprises, for example, a main memory unit and an auxiliary memory unit. The main memory unit may include, for example, ROM (read-only memory) and RAM (random-access memory). ROM is a non-volatile memory used exclusively for reading data, and can store data and various setting values ​​used by the control unit 11 in performing various processes. RAM can be used as a so-called work area to temporarily store data when the control unit 11 performs various processes. In this embodiment, the main memory unit is, for example, RAM and is used as memory.

[0019] The auxiliary storage unit is a non-temporary computer-readable storage medium for a computer centered around the control unit 11. Examples of auxiliary storage units include EEPROM® (electric erasable programmable read-only memory), HDD (hard disk drive), or SSD (solid state drive). The auxiliary storage unit can store data used by the control unit 11 in performing various processes, data generated by processing in the control unit 11, or various setting values.

[0020] The communication unit 13 can transmit information received from the outside to the configuration within the information processing device 10, and can also transmit information received from the configuration within the information processing device 10 to the outside. The communication unit 13 is configured to communicate with the external storage device 20 and the HMD 30 via a wired or wireless connection, for example, via a network such as a wired LAN, wireless LAN, Bluetooth®, or the Internet.

[0021] The input unit 14 acquires various types of information supplied from outside the information processing device 10. The input unit 14 may include, for example, a user interface such as a mouse or keyboard, which is operated by the administrator of the information processing device 10 and inputs various types of information (for example, change information such as numerical values ​​of various settings of the information processing device 10, or information indicating operations by the administrator (operation information)), as well as various sensors such as a microphone or touch panel.

[0022] The output unit 15 outputs, for example, visual or auditory information presented to the administrator of the information processing device 10. The output unit 15 may include, for example, a monitor that visually supplies information to the administrator of the information processing device 10. The output unit 15 may also include audio output means such as a speaker that audibly supplies information to the administrator of the information processing device 10.

[0023] The control unit 11 is typically a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), but may also include a processor such as a microcontroller, FPGA (Field Programmable Gate Array), or DSP (Digital Signal Processor). The control unit 11 can realize various functions of the information processing device 10 by executing programs such as system software, application software, or firmware stored in the storage unit 12.

[0024] In this embodiment, the control unit 11 includes an automatic map display function unit 110 during movement. The automatic map display function unit 110 generates an image of the virtual space based on 3D point cloud data acquired from the external storage device 20 and outputs it to the HMD 30. The automatic map display function unit 110 also determines the user's movement in the virtual space based on the current location information acquired from the HMD 30, and displays the map image in the user's field of view when the user is moving.

[0025] Specifically, as shown in Figure 2, the automatic map display function unit 110 during movement consists of a reading unit 111, a current location storage unit 112, a movement determination unit 114, a map creation unit 113, a calculation unit 115, a rendering function unit 116, and an output control unit 117.

[0026] The reading unit 111 reads map data and 3D point cloud data from the external storage device 20. The reading unit 111 acquires map data and 3D point cloud data from the external storage device 20 via the communication unit 13. The reading unit 111 outputs the acquired map data to the map creation unit 113. The reading unit 111 outputs the acquired 3D point cloud data to the calculation unit 115.

[0027] The current location storage unit 112 stores current location information. The current location storage unit 112 stores the current location information acquired from the HMD 30 in a predetermined data storage area of ​​the storage unit 12. For example, the current location storage unit 112 stores the current location information acquired from the HMD 30 in the storage unit 12 in association with the time of acquisition. That is, the storage unit 12 stores the user's current location history. Alternatively, the current location storage unit may be configured to store only the user's viewpoint position included in the acquired current location information in the storage unit 12.

[0028] The map creation unit 113 creates a map image. The map creation unit 113 creates a map image based on the map data input from the reading unit 111 and the user's current location information stored in the current location storage unit 112. Details of the map image created by the map creation unit 113 will be described later.

[0029] The movement determination unit 114 determines the user's movement in the virtual space based on the current location information. The movement determination unit 114 determines the user's movement based on the current location information obtained from the HMD 30 and past current location information stored in the storage unit 12. The movement determination unit 114 outputs the determination result to the calculation unit 115.

[0030] The calculation unit 115 creates virtual space data based on the loaded 3D point cloud data and the current location information. The calculation unit 115 creates a 3D model based on the 3D point cloud data and creates virtual space data that includes the created 3D model and the user's viewpoint position and line of sight based on the current location information. Furthermore, if the movement determination unit 114 determines that the user is moving, the calculation unit 115 superimposes the map image data created by the map creation unit 113 onto the created virtual space data. The calculation unit 115 superimposes the map image data onto the virtual space data so that the map image is displayed in the user's field of view in the virtual space. The calculation unit 115 outputs the created virtual space data to the rendering function unit 116.

[0031] The rendering function unit 116 creates an image for display on the HMD 30 based on the transmitted virtual space data. For example, the rendering function unit 116 performs rendering to create an image of a 3D model viewed from the viewpoint along the line of sight, based on the input virtual space data, and creates an image (still image or video) of the field of view in the virtual space along the line of sight from the viewpoint. The rendering function unit 116 outputs the created image to the output control unit 117.

[0032] The output control unit 117 outputs the image input from the rendering function unit 116 to the HMD 30 via the communication unit 13, and displays the image on the HMD 30.

[0033] Next, an example of the operation of the information processing device 10 according to the embodiment will be described.

[0034] Figure 3 is a flowchart showing an example of the operation of an information processing device according to the embodiment. The control unit 11 performs the following operations based on a program stored in the storage unit 12, for example. The control unit 11 starts the following operations in response to an instruction from the user to access the virtual space, for example, transmitted from the HMD 30 or input via the input unit 14.

[0035] First, the reading unit 111 acquires 3D point cloud data and map data (step S1). The reading unit 111 reads 3D point cloud data and map data from the external storage device 20 via the communication unit 13. The reading unit 111 outputs the acquired 3D point cloud data to the calculation unit 115 and the acquired map data to the map creation unit 113.

[0036] The control unit 11 acquires the current location information (step S2). The control unit 11 acquires the current location information transmitted from the HMD 30 via the communication unit 13. The current location storage unit 112 stores the acquired current location information in a predetermined storage area of ​​the storage unit 12.

[0037] The map creation unit 113 creates a map image of the virtual space (step S3). Based on the map data acquired by the reading unit 111 and the current location information acquired from the HMD 30, the map creation unit 113 creates a map image of the virtual space that includes the user's current location.

[0038] Figure 4 shows an example of a map image created by the information processing device according to the embodiment. The map creation unit 113 generates, for example, the overall map image mp1 shown in Figure 4. The overall map image mp1 is an image that provides an overview of the entire virtual space based on 3D point cloud data, created based on map data. The arrow Cl included in the overall map image mp1 indicates the user's current location in the virtual space based on acquired current location information. The position of the arrow Cl in the overall map image mp1 represents the user's current coordinates in the virtual space, and the direction of the arrow Cl indicates the user's line of sight. That is, the position and direction of the arrow Cl on the overall map image mp1 change based on the current location information transmitted from the HMD 30.

[0039] The movement determination unit 114 determines whether the user is moving or not (step S4). The movement determination unit 114 determines the user's movement based on the acquired current location information and past current location information stored in the storage unit 12.

[0040] Specifically, the movement determination unit 114 calculates the user's travel distance and direction of movement based, for example, on coordinate information indicating the user's viewpoint position included in the acquired current location information and coordinate information indicating the user's viewpoint position included in the latest current location information stored in the storage unit 12, excluding the acquired current location information.

[0041] In this embodiment, the movement determination unit 114 determines that the user is moving if the calculated user movement distance is greater than a preset threshold value. On the other hand, if the calculated user movement distance is less than or equal to the preset threshold value, the movement determination unit 114 determines that the user is not moving. In this embodiment, the information processing device 10 can ignore slight movements caused by actions such as the user looking left or right by setting a threshold value and determining the user's movement.

[0042] If the movement determination unit 114 determines that the user is moving (step S4: YES), the calculation unit 115 creates virtual space data including a map image (step S5). Based on the loaded 3D point cloud data and the current location information obtained from the HMD 30, the calculation unit 115 creates virtual space data that includes a 3D model of the virtual space, the user's viewpoint position in the virtual space, and the direction of the line of sight from the viewpoint position. Also in step S5, the calculation unit 115 superimposes the map image data created by the map creation unit 113 onto the created virtual space data so that the map image is displayed in the user's field of view in the virtual space. For example, the calculation unit 115 superimposes the map image data onto the virtual space data based on the user's direction of movement so that the map image is displayed in the user's field of view when the user faces the direction of movement calculated by the movement determination unit 114. Alternatively, the calculation unit 115 superimposes the map image data onto the virtual space data based on the user's direction of sight based on the current location information so that the map image is always displayed in the field of view of the user while they are moving. The calculation unit 115 outputs the virtual space data, including the data of the created map image, to the rendering function unit 116.

[0043] On the other hand, when the movement determination unit 114 determines that the user is not moving (step S4: NO), the calculation unit 115 creates virtual space data that does not include the map image (step S6). The calculation unit 115 creates virtual space data including a 3D model of the virtual space, the viewpoint position of the user in the virtual space, and the viewing direction from the viewpoint position, based on the read 3D point cloud data and the current location information obtained from the HMD 30. The calculation unit 115 outputs the created virtual space data to the rendering function unit 116.

[0044] The rendering function unit 116 generates an image in the virtual space based on the input virtual space data (step S7). The rendering function unit 116 performs rendering to create an image of the 3D model viewed along the viewing direction from the viewpoint position of the user, based on the virtual space data.

[0045] FIG. 5 and FIG. 6 are diagrams showing an example of an image created by the information processing apparatus according to the embodiment. As described above, when the movement determination unit 114 determines that the user is not moving, the calculation unit 115 creates virtual space data that does not include the data of the map image, and the rendering function unit 116 creates an image in the virtual space based on the virtual space data. The image g1 shown in FIG. 5 is an example of an image showing the user's field of view, created based on the virtual space data that does not include the data of the map image. The rendering function unit 116 creates an image g1 of the 3D model (3D point cloud model) based on the 3D point cloud data, viewed along the viewing direction from the viewpoint position.

[0046] On the other hand, as described above, when the movement determination unit 114 determines that the user is moving, the calculation unit 115 creates virtual space data including the data of the map image, and the rendering function unit 116 creates an image in the virtual space based on the virtual space data. The image g2 shown in FIG. 6 is an example of an image showing the user's field of view created based on the virtual space data including the data of the map image. As shown in FIG. 6, the rendering function unit 116 creates an image g2 in which a 3D model (3D point cloud model) based on the 3D point cloud data is viewed along the line-of-sight direction from the viewpoint position and the overall map image mp1 is superimposed. In FIG. 6, it is assumed that the user's movement direction and the user's line-of-sight direction are the same. That is, it is assumed that the user's field of view is facing the movement direction of the user himself / herself.

[0047] Here, when displaying the map image, the calculation unit 115 superimposes the data of the map image on the virtual space data so that the map image is displayed closer to the right or left of the user's field of view. For example, in the example shown in FIG. 6, the overall map image mp1 is displayed closer to the right side within the image g2 which is the user's field of view facing the movement direction. Thereby, when the user moves within the virtual space, it becomes easier to see the feet, and when there are stairs, obstacles, guiding lines on the ground, etc., it is possible to prevent them from obstructing the user's field of view.

[0048] After creating an image based on the acquired virtual space data, the rendering function unit 116 outputs the image to the output control unit 117.

[0049] The output control unit 117 outputs the input image (step S8). The output control unit 117 outputs the input image to the HMD 30 via the communication unit 13. Thereby, the HMD 30 can output the image in the virtual space transmitted from the information processing apparatus 10.

[0050] The control unit 11 receives a termination instruction (step S9). The control unit 11 receives, for example, a termination instruction transmitted from the HMD 30 via the communication unit 13. If the control unit 11 does not receive a termination instruction (step S9: NO), it returns to step S2. That is, the control unit 11 repeats the processing from step S2 to step S8 until it receives a termination instruction in step S9. On the other hand, if the control unit 11 receives a termination instruction (step S9: YES), it terminates the operation shown in Figure 3.

[0051] The information processing device 10 according to this embodiment operates as described above to determine the user's movement in the virtual space, and if the user is moving, it can display a map image in the direction the user is moving.

[0052] Next, the effects of the information processing device 10 according to this embodiment will be described.

[0053] It is generally known that VR sickness can be reduced by maintaining gaze on a specific object or direction when moving in a virtual space (see, for example, Non-Patent Document 9). The information processing device 10 according to this embodiment determines the user's movement in the virtual space and displays a map image in front of the user while the user is moving. The user can move within the virtual space while looking at the map image displayed in front of them. As a result, when the user moves within the virtual space, the user's gaze is concentrated on the map image displayed in front of them, and as a result, the influence of peripheral visual information is reduced, thereby reducing VR sickness.

[0054] Furthermore, in a metaverse that mimics the real world, such as a virtual space created based on 3D point cloud data measured from the real world, if a UI (user interface) such as a map is displayed within the user's field of view, it partially obscures the view of the metaverse, reducing the user's sense of presence and the quality of their experience. The information processing device 10 according to this embodiment determines the user's movement in the virtual space, and if the user is not moving, it does not display the map image in the user's field of view. As a result, the information processing device 10 according to this embodiment does not obstruct the user's field of view when the user stops and looks around in the virtual space, and does not reduce the quality of the user's experience.

[0055] Therefore, the information processing device 10 according to this embodiment can provide an information processing device and information processing method that can reduce VR sickness without degrading the quality of the user's experience when moving within a virtual space and without incurring additional costs.

[0056] In addition, the information processing device 10 according to this embodiment displays a map image of the virtual space, including the user's current location, as an object to focus the user's gaze on when moving within the virtual space. As a result, the user can visually grasp their position within the virtual space when moving around in it, and as a result, they are less likely to get lost even in a large virtual space such as a point cloud metaverse.

[0057] Next, we will describe some configuration examples of other embodiments of this invention.

[0058] In the above-described embodiment, as illustrated in Figure 6, the information processing device 10 displayed only the map within the virtual space when the user moved; however, this is not limited to this. The information processing device 10 may also display other map images in addition to the overall map image showing the entire virtual space, as illustrated in the above-described embodiment, when the user is moving within the virtual space.

[0059] For example, if the virtual space generated based on 3D point cloud data is very large, and the map creation unit 113 generates a map showing the entire virtual space, even if the user's current location is displayed on the map, it may be difficult to determine the user's exact current location. In this embodiment, the information processing device 10 displays multiple (two or more) map images with different scales as map images to be displayed when the user moves. These include a wide-area map image that includes the user's current location in the virtual space, and a detailed map image that magnifies the area around the user's current location in the virtual space (a detailed map image that magnifies the area around the user's current location in the wide-area map image). Note that the device configuration and functional configuration of the information processing device 10 in this embodiment may be the same as those in the above-described embodiment, for example, as shown in Figures 1 and 2, so a detailed explanation of the device configuration and functional configuration is omitted.

[0060] Specifically, for example, in step S3 of Figure 3, the map creation unit 113 creates an overall map image showing the entire virtual space, as illustrated in Figure 4, and a detailed map image showing an enlarged view of the area around the user's current location, based on the acquired map data and the user's current location information. Here, the overall map image is an example of a wide-area map image. The map creation unit 113 creates the detailed map image by, for example, enlarging the area around the user's current location in the created overall map image showing the entire virtual space by a predetermined arbitrary magnification.

[0061] In step S4, the movement determination unit 114 determines whether or not the user is moving in the virtual space. If the movement determination unit 114 determines that the user is moving, in step S5, the calculation unit 115 creates virtual space data that includes a 3D model of the virtual space, the user's viewpoint position in the virtual space, and the direction of line of sight from the viewpoint position, based on the acquired 3D point cloud data and current location information. In this embodiment, the calculation unit 115 also superimposes the data of a wide-area map image containing the user's current location in the virtual space, created by the map creation unit 113, and data of a detailed map image showing an enlarged view of the area around the user's current location onto the created virtual space data.

[0062] In step S8, the rendering function unit 116 creates an image in the virtual space based on the virtual space data created by the calculation unit 115.

[0063] Figure 7 shows an example of an image created by an information processing device according to another embodiment. The rendering function unit 116 performs rendering based on virtual space data and creates an image in the virtual space. Image g3 shown in Figure 7 is an example of an image showing the user's field of view, created based on virtual space data including map image data. As shown in Figure 7, the rendering function unit 116 creates an image g2 which is an image of a 3D model (3D point cloud model) based on 3D point cloud data viewed from the viewpoint position along the line of sight, with the overall map image mp1 and the detailed map image mp2 superimposed. Note that in Figure 7, the user's direction of movement and the user's line of sight are assumed to be the same. That is, the user's field of view is assumed to be facing the direction of their own movement.

[0064] Subsequently, in step S9, the output control unit 117 outputs the image created by the rendering function unit 116 to the HMD 30 via the communication unit 13. This allows the HMD 30 to output the image in the virtual space transmitted from the information processing device 10 in a manner that can be viewed by the user.

[0065] As described above, when the information processing device 10 according to the other embodiment determines that the user is moving in the virtual space, it displays a wide-area map image including the user's current location in the virtual space and a detailed map image showing an enlarged view of the area around the user's current location in the direction of the user's movement. As a result, the information processing device 10 according to the other embodiment, like the information processing device according to the above embodiment, can provide an information processing device and information processing method that can reduce VR sickness without degrading the quality of the user's experience and without incurring additional costs when moving within the virtual space. Furthermore, the user can more accurately grasp their position within the virtual space when moving within it.

[0066] The above describes an example of the configuration of an information processing device according to one aspect of this invention. However, the functional configuration of the information processing device 10 and its peripheral devices, their processing procedures and contents, the types and uses of data and images, etc., can be modified in various ways without departing from the spirit of this invention.

[0067] For example, in this embodiment, the information processing device 10 displays a map image in the user's field of view when it determines that the user is moving within the virtual space, but this is not limited to this. For example, the information processing device 10 may display other UI elements in the user's field of view when it determines that the user is moving within the virtual space. For example, the information processing device 10 may display a menu window screen showing the current time and coordinate values, or a navigation screen if a destination in the virtual space has been set, in the user's field of view when it determines that the user is moving within the virtual space. Even with such a configuration, the information processing device 10 can reduce VR sickness without degrading the quality of the user experience and without incurring additional costs, similar to this embodiment.

[0068] For example, in one embodiment, the information processing device 10 may be configured to display a map image as semi-transparent when the user is moving within the virtual space. For example, if the movement determination unit 114 determines that the user is moving, the calculation unit 115 processes the map image created by the map creation unit 113 to be semi-transparent based on a transparency rate predetermined by the user, and superimposes the data of the image onto the virtual space data. This allows the user to see the back of the map even while moving.

[0069] For example, in the embodiments shown in Figures 6 and 7, the map image displayed when the user moves is given perspective, but this is not limited to this. The information processing device 10 may, for example, control the map image to be displayed in a planar view within the user's field of vision when the user moves. Specifically, for example, the calculation unit 115 may superimpose the map image data onto the virtual space data so that the map is displayed on a plane orthogonal to the user's direction of movement or line of sight.

[0070] For example, in the embodiment, a VR headset such as an HMD 30 worn by the user was used as the output destination for the image created by the information processing device 10, but the embodiment is not limited to this. For example, the output control unit 117 may be configured to output the created image to the output unit 15 of the information processing device 10. In this case, "user" in the above description refers to the administrator of the information processing device 10. For example, if a flat-panel display is used as the configuration for the output unit 15, and the display is a large screen, the large screen will occupy almost the entire field of view, making it easy for the user to experience VR sickness. Taking this into consideration, even if a flat-panel display is used instead of a VR headset such as an HMD, the same effects as in the embodiment described above can be obtained.

[0071] It should be noted that the present invention is not limited to the embodiments described above, and can be modified in various ways during implementation without departing from its essence. Furthermore, each embodiment may be combined as appropriate, and in that case, the combined effects can be obtained. Moreover, the above embodiments include various inventions, and various inventions can be extracted by selecting combinations from the multiple constituent elements disclosed. For example, if the problem can be solved and effects obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiment, then the configuration with these deleted constituent elements can be extracted as an invention.

[0072] 10... Information processing unit 11... Control unit 12... Memory unit 13... Communication unit 14... Input unit 15... Output unit 110... Automatic map display function unit during movement 111... Reading unit 112... Current location memory unit 113... Map creation unit 114... Movement determination unit 115... Calculation unit 116... Rendering function unit 117... Output control unit 20... External storage device 30... HMD

Claims

1. An information processing device comprising: a current location storage unit that stores current location information indicating the user's current location in a virtual space in association with the time at which it was acquired; a map creation unit that creates a map image indicating the user's current location in the virtual space based on the current location information; a movement determination unit that determines whether the user is moving in the virtual space based on the current location information; a calculation unit that creates virtual space data including a 3D model in the virtual space and the user's viewpoint position and line of sight based on the current location information, and, when the movement determination unit determines that the user is moving, superimposes the map image data onto the virtual space data so that the map image is displayed in the user's field of view in the virtual space; and a rendering function unit that creates an image indicating the user's field of view based on the virtual space data.

2. The information processing apparatus according to claim 1, wherein the movement determination unit calculates the user's travel distance based on the acquired current location information and the latest current location information from the stored current location information excluding the acquired current location information, and determines that the user is moving if the value of the travel distance is greater than a preset threshold.

3. The information processing apparatus according to claim 1, wherein the map creation unit creates a plurality of map images of different scales, including the user's current location in the virtual space, as the map image.

4. An information processing method comprising: storing current location information indicating the user's current location in a virtual space in association with the time it was acquired; creating a map image indicating the user's current location in the virtual space based on the current location information; determining the user's movement in the virtual space based on the current location information; creating virtual space data including a 3D model in the virtual space and the user's viewpoint position and line of sight based on the current location information; superimposing the map image data onto the virtual space data so that the map image is displayed in the user's field of view in the virtual space when it is determined that the user is moving; and creating an image indicating the user's field of view based on the virtual space data.

Images