Display device, display device control method and program

Abstract

To provide a user with a virtual space appropriately.SOLUTION: A display device is attached to a user and provides the user with a virtual space SV, and includes a virtual space information acquisition part for acquiring information on a movable area AV2 to which a user can travel in the virtual space SV, an actual space information acquisition part for acquiring information on a movable area AR2 to which a user can travel in an actual space SR where the user really exists, a correspondence relationship acquisition part for acquiring a correspondence relationship between the virtual space SV and the actual space SR set based on a superposed area as an area in which the movable area AV2 and the movable area AR2 are superposed when the virtual space SV and the actual space SR are superposed, and a display control part for displaying an image for the virtual space SV on a display part on the basis of the correspondence relationship and a position of the display device in the actual space SR.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a display device, a control method for a display device, and a program. [Background technology]

[0002] Information devices have evolved significantly in recent years. Not only smartphones, which are a representative example, but also so-called wearable devices have become widespread. A known wearable device is a glasses-type HMD (head mounted display) that directly stimulates the visual sense. By using such an HMD, a virtual space can be provided to a user U by displaying an image according to the user's line of sight. For example, Patent Document 1 describes that the user's viewpoint position is corrected by holding coordinate data from a three-dimensional sensor as reference coordinates in a real three-dimensional space and matching the coordinate data to a reference position in a virtual three-dimensional space. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 9-311618 Summary of the Invention [Problem to be solved by the invention]

[0004] In such a display device, it is required to provide a virtual space to a user appropriately.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a display device, a control method for the display device, and a program that are capable of properly associating coordinates in real space with coordinates in virtual space. [Means for solving the problem]

[0006] A display device according to one aspect of the present invention is a display device worn by a user to provide the user with a virtual space, and includes a virtual space information acquisition unit that acquires information of a virtual movement area in which the user can move in the virtual space, a real space information acquisition unit that acquires information of a real movement area in which the user can move in a real space in which the user actually exists, a correspondence acquisition unit that acquires a correspondence relationship between the virtual space and the real space, which is set based on an overlap area, which is the area in which the virtual movement area and the real movement area overlap when the virtual space and the real space are superimposed, and a display control unit that displays an image for the virtual space on a display unit based on the correspondence relationship and the position of the display device in the real space.

[0007] A control method for a display device according to one aspect of the present invention is a control method for a display device worn by a user to provide the user with a virtual space, and includes the steps of: acquiring information about a virtual movement area in which the user can move in the virtual space; acquiring information about a real movement area in which the user can move in a real space in which the user actually exists; acquiring a correspondence relationship between the virtual space and the real space, the correspondence relationship being the area in which the virtual movement area and the real movement area overlap when the virtual space and the real space are superimposed; and displaying an image for the virtual space on a display unit based on the correspondence relationship and the position of the display device in the real space.

[0008] A program according to one aspect of the present invention is a program that causes a computer to execute a control method for a display device that is worn by a user and provides the user with a virtual space, and causes the computer to execute the following steps: acquiring information about a virtual movement area in which the user can move in the virtual space; acquiring information about a real movement area in which the user can move in a real space in which the user actually exists; acquiring a correspondence relationship between the virtual space and the real space, the correspondence relationship being the area in which the virtual movement area and the real movement area overlap when the virtual space and the real space are superimposed; and displaying an image for the virtual space on a display unit based on the correspondence relationship and the position of the display device in the real space. Effect of the Invention

[0009] According to the present invention, a virtual space can be appropriately provided to a user. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a real space and a virtual space. [Diagram 2] FIG. 2 is a schematic block diagram of the display device according to the present embodiment. [Diagram 3] FIG. 3 is a schematic diagram showing an example of a virtual space. [Figure 4] FIG. 4 is a schematic diagram showing an example of a real space. [Diagram 5] FIG. 5 is a schematic diagram showing an example of superimposition of a virtual space and a real space. [Figure 6] FIG. 6 is a schematic diagram showing another example of superimposition of a virtual space and a real space. [Figure 7] FIG. 7 is a flowchart illustrating a display flow of an image in a virtual space. [Figure 8] FIG. 8 is a schematic diagram illustrating an example of a priority area. [Figure 9] FIG. 9 is a schematic diagram showing another example of superimposition of a virtual space and a real space. [Figure 10]FIG. 10 is a schematic diagram showing an example in which a user is visually recognizing a virtual space. [Figure 11] FIG. 11 is a schematic diagram showing another example of superimposition of a virtual space and a real space. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiment described below.

[0012] (Real and virtual space) FIG. 1 is a schematic diagram for explaining an example of a real space and a virtual space. The display device 10 according to the present embodiment is a display device that displays an image. As shown in FIG. 1, the display device 10 is a so-called HMD (Head Mount Display) that is worn on the head of a user U. The display device 10 displays an image to provide a virtual space to the user U. As shown in FIG. 1, the real space in which the user U actually exists is the real space SR, and the virtual space provided to the user U by the display device 10 is the virtual space SV. In this case, the display device 10 displays an image for the virtual space SV according to the movement (line of sight) of the user U in the real space SR. That is, the image for the virtual space SV is displayed by simulating that the user U is operating in the virtual space SV as an avatar UV. Therefore, the user U can recognize that he or she exists in the virtual space SV. The virtual space SV here is MR (Mixed Reality), which is a space that reproduces a real place away from the place where the user U exists, but is not limited thereto, and may be a virtual space that does not actually exist, that is, VR (Vertial Reality). Hereinafter, in the coordinate system of the real space SR, one direction along the horizontal direction is referred to as the direction XR, the direction along the horizontal direction perpendicular to the direction XR is referred to as the direction YR, and the vertical direction is referred to as the direction ZR. Also, in the coordinate system of the virtual space SV, one direction along the horizontal direction is referred to as the direction XV, the direction along the horizontal direction perpendicular to the direction XV is referred to as the direction YV, and the vertical direction is referred to as the direction ZV.

[0013] (display device) FIG. 2 is a schematic block diagram of the display device according to the present embodiment. The display device 10 can be said to be a computer, and includes an input unit 20, a display unit 22, a storage unit 24, a communication unit 26, a real space detection unit 28, and a control unit 30, as shown in FIG. 2. The input unit 20 is a mechanism for accepting an operation by a user U, and may be, for example, a controller or a microphone provided in an HMD. The display unit 22 is a display for displaying an image. The display unit 22 provides a virtual space SV to the user U by outputting an image. In addition to the display unit 22, the display device 10 may include a device for outputting information, such as a speaker for outputting sound.

[0014] The storage unit 24 is a memory that stores various information such as the contents of calculations and programs of the control unit 30, and includes at least one of a main storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), and an external storage device such as a HDD (Hard Disk Drive). The program for the control unit 30 stored in the storage unit 24 may be stored in a recording medium readable by the display device 10.

[0015] The communication unit 26 is a communication module that communicates with an external device, such as an antenna, etc. The display device 10 communicates with an external device by wireless communication, but may also use wired communication, and any communication method may be used.

[0016] The real space detection unit 28 is a sensor that detects the surroundings of the display device 10 (user U) in the real space SR. The real space detection unit 28 detects objects present around the display device 10 (user U) in the real space SR, and is a camera in this embodiment. However, the real space detection unit 28 is not limited to being a camera as long as it can detect objects present in the real space SR around the display device 10 (user U), and may be, for example, a LIDAR (Light Detection And Ranging) device.

[0017] The control unit 30 is a calculation device and includes a calculation circuit such as a CPU (Central Processing Unit). The control unit 30 includes a virtual space information acquisition unit 40, a real space information acquisition unit 42, a correspondence relationship acquisition unit 44, a display control unit 46, and an avatar information transmission unit 48. The control unit 30 realizes the virtual space information acquisition unit 40, the real space information acquisition unit 42, the correspondence relationship acquisition unit 44, the display control unit 46, and the avatar information transmission unit 48 by reading and executing a program (software) from the storage unit 24, and executes the program. The control unit 30 may execute these processes using one CPU, or may include multiple CPUs and execute the processes using the multiple CPUs. At least a part of the processes performed by the virtual space information acquisition unit 40, the real space information acquisition unit 42, the correspondence relationship acquisition unit 44, the display control unit 46, and the avatar information transmission unit 48 may be realized by a hardware circuit.

[0018] (Virtual space information acquisition section) The virtual space information acquisition unit 40 acquires information on the virtual space SV. The virtual space information acquisition unit 40 acquires information on the virtual space SV from an external device (server) via the communication unit 26, for example. The information on the virtual space SV includes image data of the virtual space SV in the coordinate system of the virtual space SV. The image data of the virtual space SV indicates the coordinates and shape of an object to be displayed as an image for the virtual space SV. Note that in this embodiment, the virtual space SV is not constructed according to the environment around the user U in the real space SR, but is set in advance regardless of the environment around the user U in the real space SR.

[0019] FIG. 3 is a schematic diagram showing an example of a virtual space. FIG. 3 is an example of a plan view of the virtual space SV as viewed from the ZV direction. The virtual space information acquisition unit 40 also acquires information on a movable area AV2 (virtual movement area) in the virtual space SV as information on the virtual space SV. That is, the virtual space information acquisition unit 40 also acquires information indicating the position occupied by the movable area AV2 in the coordinate system of the virtual space SV. The movable area AV2 is an area in the virtual space SV where the avatar UV of the user U can move (or a space in which the avatar UV can move), and may be an area in the virtual space SV excluding the non-movable area AV1 which is an area in which the avatar UV of the user U cannot move (or a space in which the avatar UV cannot move). The movable area AV2 may be, for example, a floor of a room where avatars gather in the virtual space SV, and the non-movable area AV1 may be, for example, an area in the virtual space SV where an obstacle exists that the avatar UV cannot pass through, or a focus area in a meeting in the virtual space SV (for example, a desk, a screen, etc.). In this embodiment, the movable area AV2 may be set in advance, for example, when setting the virtual space SV, or may be set by the virtual space information acquisition unit 40, for example, based on the size of the avatar UV and the size of the unmovable area AV1.

[0020] In Figure 3, the virtual space SV and the unmovable area AV1 are rectangular when viewed from the ZV direction, but this is merely an example, and the shapes and sizes of the virtual space SV, the unmovable area AV1, and the movable area AV2 are not limited to the example in Figure 3 and may be arbitrary.

[0021] (Real space information acquisition unit) The real space information acquisition unit 42 acquires information on the real space SR. The information on the real space SR refers to position information indicating the coordinates and shape of an object present around the display device 10 (user U) in the coordinate system of the real space SR. In this embodiment, the real space information acquisition unit 42 controls the real space detection unit 28 to cause the real space detection unit 28 to detect objects around the display device 10 (user U), and acquires the detection result as information on the real space SR. However, the method of acquiring the information on the real space SR is not limited to detection by the real space detection unit 28. For example, information on the real space SR, such as layout information of the user U's room, may be set in advance, and the real space information acquisition unit 42 may acquire the set information on the real space SR.

[0022] FIG. 4 is a schematic diagram showing an example of a real space. FIG. 4 is an example of a plan view of the real space SR viewed from the ZR direction. The real space information acquisition unit 42 acquires information of a movable area AR2 (real movement area) in the real space SR. That is, the real space information acquisition unit 42 acquires information indicating the position of the movable area AR2 in the coordinate system of the real space SR. The movable area AR2 may be an area in the real space SR where the user U can move (or a space where the user U can move), excluding the non-movable area AR1 which is an area where the user U cannot move (or a space where the user U cannot move). The movable area AR2 may be, for example, the floor of a room where the user U is present, and the non-movable area AR1 may be, for example, an area in the virtual space SV where an obstacle (for example, a desk, a bed, etc.) exists that the user U cannot pass through. In this embodiment, the real space information acquisition unit 42 sets the movable area AR2 and the non-movable area AR1 based on information of the real space SR. The real space information acquisition unit 42 may identify the position of an object that the user U cannot move to based on the information of the real space SR, and set the area (or space) occupied by the object that the user U cannot move to as the non-movable area AR1, and the area (or space) where the object that the user U cannot move to does not exist as the movable area AR2. However, the movable area AR2 and the non-movable area AR1 are not limited to being set based on the information of the real space SR. For example, information on the movable area AR2 and the non-movable area AR1, such as layout information of the user U's room, may be set in advance, and the real space information acquisition unit 42 may acquire information on the set movable area AR2 and non-movable area AR1.

[0023] It should be noted that FIG. 4 is merely an example, and the shapes and sizes of the real space SR, the immovable area AR1, and the movable area AR2 are not limited to those shown in FIG. 4 and may be arbitrary.

[0024] (Correspondence acquisition section) The correspondence relationship acquisition unit 44 sets the correspondence relationship between the coordinate system of the virtual space SV and the coordinate system of the real space SR based on the information of the movable area AV2 acquired by the virtual space information acquisition unit 40 and the information of the movable area AR2 acquired by the real space information acquisition unit 42. The correspondence relationship between the coordinate system of the virtual space SV and the coordinate system of the real space SR can be said to be information indicating the position and posture of the real space SR in the coordinate system of the virtual space SV, and can also be said to be a value for converting the coordinate system of the real space SR to the coordinate system of the virtual space SV. For example, when the user U is at a reference position in the real space SV, the display device 10 displays an image of the virtual space SV when the viewpoint (avatar UV) of the user U is present at a position in the virtual space SV corresponding to the reference position of the real space SV. The process of the correspondence relationship acquisition unit 44 will be specifically described below.

[0025] FIG. 5 is a schematic diagram showing an example of superimposition of a virtual space and a real space. The correspondence relationship acquisition unit 44 superimposes the virtual space SV acquired by the virtual space information acquisition unit 40 and the real space SR acquired by the real space information acquisition unit 42 in a common coordinate system. In other words, the correspondence relationship acquisition unit 44 converts the coordinates of the non-movable area AV1 and the movable area AV2 in the virtual space SV and the coordinates of the non-movable area AR1 and the movable area AR2 in the real space SR into a common coordinate system, thereby superimposing the non-movable area AV1 and the movable area AV2 and the non-movable area AR1 and the movable area AR2 in the common coordinate system. The common coordinate system may be any coordinate system, and in the example of FIG. 5, one direction along the horizontal direction in the common coordinate system is the direction X, the direction perpendicular to the direction X along the horizontal direction is the direction Y, and the vertical direction is the direction Z.

[0026] The correspondence relationship acquisition unit 44 calculates the area of ​​the overlapping region (or the volume of the overlapping space) between the movable area AV2 and the movable area AR2 when the virtual space SV and the real space SR are overlapped in a common coordinate system as the overlapping area. The correspondence relationship acquisition unit 44 calculates the correspondence relationship between the coordinate system of the virtual space SV and the coordinate system of the real space SR based on the calculated overlapping area.

[0027] In this embodiment, the correspondence relationship acquisition unit 44 calculates the overlapping area while moving at least one of the relative position and the relative orientation between the virtual space SV and the real space SR in a common coordinate system, as shown in the example of Fig. 5. In other words, the correspondence relationship acquisition unit 44 calculates the overlapping area of ​​the movable area AV2 and the movable area AR2 for each virtual space SV and real space SR that differ in at least one of the relative position and the relative orientation in a common coordinate system. Note that, in the example of Fig. 5, an example is shown in which the position and orientation of the virtual space SR are moved while the position and orientation of the real space SV are fixed in a common coordinate system, but the present invention is not limited thereto, and the overlapping area may be calculated by moving the position and orientation of the real space SV while fixing the position and orientation of the virtual space SR.

[0028] The correspondence acquisition unit 44 sets the correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR based on the overlapping area for each combination of the virtual space SV and the real space SR that differ in at least one of the relative position and the relative orientation in a common coordinate system. More specifically, the correspondence acquisition unit 44 extracts a combination of the virtual space SV and the real space SR that has the largest overlapping area among the combinations of the virtual space SV and the real space SR that differ in at least one of the relative position and the relative orientation. Then, the correspondence acquisition unit 44 calculates the correspondence between the coordinate system of the extracted virtual space SV and the coordinate system of the real space SR (a value for converting the coordinate system of the extracted real space SR into the coordinate system of the extracted virtual space SV) and sets it as the correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR. In other words, the correspondence acquisition unit 44 extracts the virtual space SV that has the position and orientation that maximizes the overlapping area, and associates the coordinate system of the extracted virtual space SV with the coordinate system of the real space SR.

[0029] FIG. 6 is a schematic diagram showing another example of superimposition of a virtual space and a real space. In the description of FIG. 5, the correspondence relationship acquisition unit 44 superimposes the virtual space SV on the real space SR while changing the position and orientation of the virtual space SV while keeping the size of the virtual space SV fixed. However, as shown in FIG. 6, the virtual space SV may be superimposed on the real space SR while changing the size of the virtual space SV. In this case, as shown in the example of FIG. 6, the correspondence relationship acquisition unit 44 calculates the superimposition area while changing the relative size of the virtual space SV and the real space SR in a common coordinate system. In other words, the correspondence relationship acquisition unit 44 calculates the superimposition area of ​​the movable area AV2 and the movable area AR2 for each combination of the virtual space SV and the real space SR having different relative sizes in a common coordinate system. Note that even when the relative sizes are changed, it is preferable to keep the area ratio of the non-movable area AV1 and the movable area AV2 to the virtual space SV and the area ratio of the non-movable area AR1 and the movable area AR2 to the real space SR fixed. In other words, it is preferable to uniformly scale the entire area of ​​the virtual space SV and the real space SR, without scaling only a part of the virtual space SV or the real space SR. In addition, in the example of Fig. 6, an example is shown in which the size of the virtual space SR is changed while the size of the real space SV is fixed in a common coordinate system, but the present invention is not limited to this, and the size of the virtual space SR may be fixed while the size of the real space SV is changed to calculate the overlapping area.

[0030] In the example of Fig. 6, the correspondence relationship acquisition unit 44 sets the correspondence relationship between the coordinate system of the virtual space SV and the coordinate system of the real space SR based on the overlapping area for each combination of the virtual space SV and the real space SR that have different relative sizes in a common coordinate system. More specifically, the correspondence relationship acquisition unit 44 extracts a combination of the virtual space SV and the real space SR that has the largest overlapping area among the combinations of the virtual space SV and the real space SR that have different relative sizes. Then, the correspondence relationship acquisition unit 44 sets the correspondence relationship between the extracted coordinate system of the virtual space SV and the coordinate system of the real space SR as the correspondence relationship between the coordinate system of the virtual space SV and the coordinate system of the real space SR. In other words, the correspondence relationship acquisition unit 44 extracts the virtual space SV with a reduced scale that has the largest overlapping area, and associates the coordinate system of the virtual space SV with the extracted reduced scale with the coordinate system of the real space SR.

[0031] Note that the example of Fig. 5 and the example of Fig. 6 may be combined. That is, the correspondence relationship acquisition unit 44 calculates the overlapping area while changing the relative position, relative orientation, and relative size of the virtual space SV and the real space SR in a common coordinate system. Then, the correspondence relationship acquisition unit 44 extracts a combination of the virtual space SV and the real space SR that has the largest overlapping area from among combinations of the virtual space SV and the real space SR that differ in at least one of the relative position, relative orientation, and relative size in the common coordinate system. Then, the correspondence relationship acquisition unit 44 sets the extracted correspondence relationship between the coordinate system of the virtual space SV and the coordinate system of the real space SR as the correspondence relationship between the coordinate system of the virtual space SV and the coordinate system of the real space SR.

[0032] In the above description, the correspondence acquisition unit 44 associates the virtual space SV with the real space SR so that the overlapping area between the movable area AV2 indicating the two-dimensional movable area in the virtual space SV and the movable area AR2 indicating the two-dimensional movable area in the real space SR is maximized, but is not limited to maximizing the overlapping area in two dimensions. For example, the correspondence acquisition unit 44 may associate the virtual space SV with the real space SR so that the overlapping volume between the movable area AV2 (virtual movement space) indicating the three-dimensional movable space in the virtual space SV and the movable area AR2 (real movement space) indicating the three-dimensional movable space in the real space SR is maximized.

[0033] In the above description, the correspondence relationship acquisition unit 44 calculates the overlapping area by overlapping the virtual space SV and the real space SR in a common coordinate system, and sets the correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR based on the overlapping area. However, the calculation of the overlapping area and the setting of the correspondence relationship are not limited to being performed by the correspondence relationship acquisition unit 44. For example, an external device may calculate the overlapping area, and the correspondence relationship acquisition unit 44 may acquire information on the overlapping area from the external device and set the correspondence relationship based on the information. Also, for example, an external device may calculate the overlapping area and set the correspondence relationship based on the overlapping area, and the correspondence relationship acquisition unit 44 may acquire information on the correspondence relationship from the external device.

[0034] (Display control unit) The display control unit 46 displays an image for the virtual space SR on the display unit 22 based on the correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR set by the correspondence acquisition unit 44 and the position of the user U (display device 10) in the real space SR. Specifically, the display control unit 46 acquires information on the position and orientation of the user U in the real space SR, and converts the position and orientation of the user U in the real space SR into the position and orientation of the viewpoint (avatar UV) of the user U in the coordinate system of the virtual space SV based on the correspondence. The display control unit 46 displays an image of the virtual space SV when the virtual space SV is viewed from the calculated position and orientation of the viewpoint of the user U on the display unit 22 as an image for the virtual space SV. Note that the information on the position and orientation of the user U (display device 10) in the real space SR may be acquired by any method, but may be calculated using, for example, the detection result of the real space detection unit 28 (i.e., the captured image of the real space SR here).

[0035] In this way, the position and orientation of the user U in the real space SR is reflected in the position and orientation of the user U's viewpoint in the virtual space SV. Therefore, when the user U moves in the real space SR, the position and orientation of the user U's viewpoint in the virtual space SV (i.e., the position and orientation of the avatar UV) also moves. In this case, it is preferable that the movement amount of the user U in the real space SR and the movement amount of the viewpoint of the user U in the virtual space SV are associated with each other. More specifically, when the size of the virtual space SV is changed in a common coordinate system when setting the correspondence, it is preferable to reflect the degree of change in the size of the virtual space SV in the movement amount. Specifically, if the rate at which the size of the virtual space SV is changed in the common coordinate system is the change rate, the display control unit 46 displays an image for the virtual space SV on the display unit 22, assuming that the viewpoint of the user U has moved in the virtual space SV by an amount equal to the reciprocal of the change rate with respect to the amount of movement of the user U in the real space SR. That is, the display control unit 46 causes the display unit 22 to display an image of the virtual space SV from a viewpoint moved by an amount that is the reciprocal of the rate of change relative to the amount of movement of the user U in the real space SR. For example, if the size of the virtual space SV is doubled when setting the correspondence relationship, the display control unit 46 causes the display unit 22 to display an image of the virtual space SV from a viewpoint moved by an amount that is 1 / 2 times the amount of movement of the user U in the real space SR.

[0036] The display control unit 46 may also display an object in the real space SR superimposed on the image of the virtual space SV. In this case, the display unit 22 may provide AR (Augumented Reality) that displays the image of the virtual space SV while transmitting the real space SR, or may display the image of the virtual space SV and an image showing an object in the real space SR superimposed on each other. In addition, the spatial portion of the movable area AV2 in the virtual space SV that does not overlap with the movable area AR2 of the real space SR may be deleted from the image of the virtual space SV or presented as information on an unmovable area. In this case, even if the area remains as a movable area, an area that does not have a spatial width that allows the user U (the avatar UV that reproduces his / her body shape) to pass through may be deleted from the image of the virtual space SV.

[0037] The display device 10 according to the present embodiment can appropriately provide the virtual space SV to the user U by displaying an image for the virtual space SV based on the correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR set in this way. For example, the user U moves through the real space SR while visually recognizing the virtual space SV. That is, the user U tries to move within the movable area AV2 in the virtual space SV, but the area in which the user U can actually move is the movable area AR2 in the real space SR. In this way, the movable area recognized by the user U and the area in which the user U can actually move are different. In contrast, in the present embodiment, the virtual space SV and the real space SR are associated with each other so that the overlapping area between the movable area AV2 in the virtual space SV and the movable area AR2 in the real space SR is large, so that the deviation between the movable area recognized by the user U and the area in which the user U can actually move is reduced, and the area in which the user U can move can be secured as wide as possible. Therefore, according to the display device 10, even when the user U moves, the virtual space SV can be appropriately provided.

[0038] (Avatar information transmission unit) The avatar information transmission unit 48 transmits information on the avatar UV of the user U in the virtual space SV to external information via the communication unit 26. The avatar information transmission unit 48 acquires information on the position and orientation of the user U in the real space SR, and converts the position and orientation of the user U in the real space SR into the position and orientation of the avatar UV in the coordinate system of the virtual space SV based on the correspondence between the coordinate system of the real space SR and the coordinate system of the virtual space SV. The avatar information transmission unit 48 transmits information on the position and orientation of the avatar UV in the coordinate system of the virtual space SV and image data of the avatar UV (data indicating the shape, etc.) to an external device. The external device transmits information on the position and orientation of the avatar UV in the coordinate system of the virtual space SV and the image data of the avatar UV as image data of the virtual space SV to a display device used by another user. The display device displays image data of the virtual space SVG including an image of the avatar UV toward the user wearing the display device. By transmitting image data of the avatar to an external device in this manner, the virtual space SV can be shared by multiple users.

[0039] (Processing flow) The display flow of the image of the virtual space SV described above will be described. FIG. 7 is a flowchart for explaining the display flow of the image of the virtual space. As shown in FIG. 7, the display device 10 acquires information of the virtual space SV by the virtual space information acquisition unit 40 (step S10), and acquires information of the real space SR by the real space information acquisition unit 42 (step S12). Then, the display device 10 superimposes the virtual space SV and the real space SR in a common coordinate system while changing at least one of the relative position, relative orientation, and relative size of the virtual space SV and the real space SR by the correspondence relationship acquisition unit 44, and calculates the superimposed area (step S14). The correspondence relationship acquisition unit 44 extracts a combination of the virtual space SV and the real space SR that has the largest superimposed area among the combinations of the virtual space SV and the real space SR (step S16), and sets the correspondence relationship between the extracted coordinate system of the virtual space SV and the coordinate system of the real space SR (step S18). The display device 10 causes the display control unit 46 to display an image of the virtual space SV on the display unit 22 based on the set correspondence relationship and the position of the display device 10 (user U) in the real space SR (step S20).

[0040] (effect) As described above, the display device 10 according to the present embodiment is worn by the user U to provide the user U with a virtual space SV, and includes a virtual space information acquisition unit 40, a real space information acquisition unit 42, a correspondence acquisition unit 44, and a display control unit 46. The virtual space information acquisition unit 40 acquires information on a movable area AV2 (virtual movement area) in which the user U (avatar UV) can move in the virtual space SV. The real space information acquisition unit 42 acquires information on a movable area AR2 (real movement area) in which the user U can move in the real space SR in which the user U actually exists. The correspondence acquisition unit 44 acquires a correspondence relationship between the coordinate system of the virtual space SV and the coordinate system of the real space SR, which is set based on the overlapping area. The overlapping area is an area where the movable area AV2 (virtual movement area) and the movable area AR2 (real movement area) overlap when the virtual space SV and the real space SR are overlapped in a common coordinate system. The display control unit 46 causes the display unit 22 to display an image for the virtual space SV, based on the correspondence relationship and the position of the display device 10 in the real space SR.

[0041] When the virtual space SV is provided to the user U from the display device 10, the movable area in the virtual space SV recognized by the user U differs from the area in the real space SR in which the user U can actually move. In contrast, the display device 10 according to the present embodiment associates the virtual space SV with the real space SR based on the overlapping area between the movable area AV2 in the virtual space SV and the movable area AR2 in the real space SR, so that the deviation between the movable area recognized by the user U and the area in which the user U can actually move is reduced, and the area in which the user U can move can be secured as wide as possible. Therefore, the display device 10 can appropriately provide the virtual space SV even when the user U moves.

[0042] The correspondence is a correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR that maximizes the overlapping area among the combinations of the virtual space SV and the real space SR obtained by moving at least one of the relative position and the relative orientation of the virtual space SV and the real space SR in a common coordinate system. By moving the virtual space SV with respect to the real space SR in this way and associating the virtual space SV with the real space SR so as to maximize the overlapping area, the area in which the user U can move can be maximized.

[0043] The correspondence relationship is a correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR that maximizes the overlapping area among the combinations of the virtual space SV and the real space SR in which the relative sizes of the virtual space SV and the real space SR are changed in a common coordinate system. By changing the size of the virtual space SV with respect to the real space SR in this way and associating the virtual space SV with the real space SR so as to maximize the overlapping area, the area in which the user U can move can be maximized.

[0044] Furthermore, the display control unit 46 causes the display unit 22 to display an image for the virtual space SV, assuming that the user U has moved in the virtual space SV an amount that is the reciprocal multiple of the rate of change in the size of the virtual space SV in a common coordinate system relative to the amount of movement of the display device 10 (user U) in the real space SR. The display device 10 according to this embodiment sets the amount of movement in the virtual space SV taking into account the scale reduction when superimposing on the actual amount of movement of the user U, and therefore can provide the virtual space SV appropriately in accordance with the movement of the user U.

[0045] (Another example of how to set up correspondences) Another example of the method for setting the correspondence relationship between the coordinate system of the virtual space SV and the coordinate system of the real space SR described in this embodiment will be described.

[0046] For example, as described in Fig. 5 and Fig. 6, when the virtual space SV and the real space SR are superimposed so that the superimposed area is maximized while changing the position, orientation, and size of the virtual space SV, it is assumed that the non-movable area AR1 (e.g., an actual obstacle) in the real space SR is located around the non-movable area AV1 (e.g., an area of ​​interest) in the virtual space SV, and may become an obstacle when approaching the area of ​​interest in the virtual space SV. In preparation for such a case, the correspondence relationship acquisition unit 44 may set a priority area in the movable area AV2 of the virtual space SV, and superimpose the virtual space SV and the real space SR so that the priority area does not overlap with the non-movable area AR1 in the real space SR. This will be described in more detail below.

[0047] FIG. 8 is a schematic diagram for explaining an example of a priority area, and FIG. 9 is a schematic diagram showing another example of superimposition of a virtual space and a real space. The correspondence relationship acquisition unit 44 sets a priority area in the movable area AV2 of the virtual space SV. The priority area is an area that is prioritized to be superimposed on the movable area AR2 without being superimposed on the non-movable area AR1 when the virtual space SV and the real space SR are superimposed. The correspondence relationship acquisition unit 44 may set the priority area by any method, and for example, may set an area of ​​a predetermined size around the non-movable area AV1, which is the attention area, in the entire movable area AV2 as the priority area. In addition, the correspondence relationship acquisition unit 44 may set a plurality of priority areas with different priorities. In the example of FIG. 8, the correspondence relationship acquisition unit 44 sets a priority area AV2a around the non-movable area AV1, and sets a priority area AV2b around the priority area AV2a. In this case, the priority area AV2a, which is closer to the non-movable area AV1, is set to have a higher priority than the priority area AV2b. In the following description, any area of ​​the movable area AV2 other than the priority area will be referred to as a non-priority area, as appropriate. That is, in the example of Fig. 9, the area outside the priority area AV2b is a non-priority area AV2c.

[0048] The correspondence relationship acquisition unit 44 superimposes the virtual space SV in which the priority area is set and the real space SR in a common coordinate system. In this case, as described in Fig. 5 and Fig. 6, the correspondence relationship acquisition unit 44 calculates the overlapping area while changing at least one of the relative position, relative orientation, and relative size between the virtual space SV and the real space SR in the common coordinate system, and sets the correspondence relationship between the coordinate system of the virtual space SV and the coordinate system of the real space SR from the combination of the virtual space SV and the real space SR that maximizes the overlapping area. However, in this example, the overlapping area is calculated so that the size of the priority overlapping area, which is the area where the priority area and the movable area AR2 overlap, has a greater influence on the calculated overlapping area than the size of the non-priority overlapping area, which is the area where the non-priority area and the movable area AR2 overlap. In other words, the overlap area is calculated to be larger as the priority overlap area and non-priority overlap area are larger, but the degree to which the overlap area increases when the priority overlap area increases by a unit amount is greater than the degree to which the overlap area increases when the non-priority overlap area increases by a unit amount.

[0049] For example, the correspondence relationship acquisition unit 44 of this example assigns a weight to the priority overlapping area, and calculates the sum of the priority overlapping area multiplied by the weight and the non-priority overlapping area as the overlapping area. By assigning a weight to the priority overlapping area in this way, the degree of influence of the priority overlapping area (priority area) on the overlapping area becomes greater than the degree of influence of the non-priority overlapping area (non-priority area) on the overlapping area. Therefore, for example, as shown in FIG. 9, it is important that the priority area around the attention area (immovable area AV1) does not overlap the immovable area AR1, and the combination of the virtual space SV and the real space SR that maximizes the overlapping area makes it difficult for the priority overlapping area to overlap the immovable area AR1. Therefore, for example, when approaching the attention area in the virtual space SV, the possibility that the immovable area AR1 becomes an obstacle can be suppressed.

[0050] In this way, in this example, the larger the priority overlapping area where the priority area set in the movable area AV2 overlaps with the movable area AV2, the larger the overlapping area is calculated. The priority area is set to have a greater influence on the size of the overlapping area than non-priority areas (areas in the movable area AV2 other than the priority area). This reduces the possibility that the unmovable area AR1 will be an obstacle when approaching the attention area in the virtual space SV. When multiple priority areas are set, the weight value may be different for each priority area. In this case, in the example of FIG. 8, the weight of the priority area AV2a close to the unmovable area AV1 is set to be greater than the weight of the priority area AV2b.

[0051] Next, as another example of the method for setting the correspondence relationship, a method for setting the correspondence relationship between the position in the height direction of the real space SR and the position in the height direction of the virtual space SV will be described. FIG. 10 is a schematic diagram showing an example in which a user is viewing a virtual space. As shown in FIG. 10, it is assumed that the non-movable area AV1a, which is an attention area in the virtual space SV, overlaps with the non-movable area AR1, which is an obstacle in the real space SV, in the height direction (ZR direction in real space coordinates), and the attention area in the virtual space SV is hidden. In order to avoid such a state, the correspondence relationship acquisition unit 44 may set the correspondence relationship between the position in the height direction of the real space SR and the position in the height direction of the virtual space SV so that the positions in the height direction of the non-movable area AV1 and the non-movable area AR1 do not overlap. Thereby, for example, as shown in the non-movable area AV1b in FIG. 10, the position of the non-movable area can be set at a position that does not overlap with the non-movable area AR1. The correspondence relationship between the position in the height direction of the real space SR and the position in the height direction of the virtual space SV may be set automatically by the correspondence relationship acquisition unit 44 or may be set by input from the user U.

[0052] Next, as another example of the method for setting the correspondence relationship, an example in which the virtual space SV and the real space SR are superimposed so that the degree of similarity between the virtual space SV and the real space SR is increased will be described. FIG. 11 is a schematic diagram showing another example of superimposition between the virtual space and the real space. In this example, the correspondence relationship acquisition unit 44 compares the shapes of the real space SR and the virtual space SV, and extracts a region in the real space SR whose shape is similar to that of the virtual space SV as a similar region SRS having a high degree of similarity. Then, as shown in the example of FIG. 11, the correspondence relationship acquisition unit 44 superimposes the virtual space SV on the real space SR while changing at least one of the relative position, relative orientation, and relative size between the virtual space SV and the real space SR, thereby superimposing the virtual space SV on the similar region. The correspondence relationship acquisition unit 44 calculates the correspondence relationship between the coordinate system of the virtual space SV superimposed on the similar region SRS and the coordinate system of the real space SR. In this way, even if the virtual space SV is superimposed on an area with a high degree of similarity, the difference between the area that the user U recognizes as movable and the area that the user U can actually move in is reduced, and the area that the user U can move in can be secured as wide as possible. Note that, in this example, the overlapping area does not need to be considered when superimposing, but this is not limited thereto, and the overlapping area may also be considered. That is, for example, the correspondence relationship acquisition unit 44 may superimpose the virtual space SV and the real space SR so that the degree of similarity is large and the overlapping area is large.

[0053] Although the embodiments of the present invention have been described above, the embodiments are not limited to the contents of these embodiments. The above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be appropriately combined, and the configurations of each embodiment can also be combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the above-described embodiments. [Explanation of symbols]

[0054] 10 Display device 22 Display section 40 Virtual space information acquisition unit 42 Real space information acquisition unit 44 Correspondence acquisition unit 46 Display control section AR2 Possible movement area (actual movement area) AV2 Movable area (virtual moving area) SR Real Space SV Virtual Space U User

Claims

1. A display device that is worn by a user and provides a virtual space to the user, a virtual space information acquisition unit that acquires information about a virtual movement area in which the user can move in the virtual space; a real space information acquisition unit that acquires information about a real movement area in which the user can move in a real space in which the user actually exists; a correspondence relationship acquisition unit that calculates an overlap area, which is an area where the virtual movement area and the real movement area overlap when the virtual space and the real space are overlapped, and acquires a correspondence relationship between the virtual space and the real space that is set based on the overlap area; a display control unit that causes a display unit to display an image for the virtual space based on the correspondence relationship and a position of the display device in the real space; Including, the correspondence relationship acquisition unit calculates the overlapping area when the virtual space and the real space are combined by moving at least one of a relative position and a relative orientation between the virtual space and the real space, and acquires a correspondence relationship among the overlapping areas that maximizes the overlapping area. Display device.

2. The display device according to claim 1, wherein the correspondence acquisition unit calculates the overlap area when each of the virtual spaces and the real space is combined with the virtual space and the real space while changing the relative size of the virtual space and the real space, and acquires the correspondence relationship that maximizes the overlap area among the overlap areas.

3. The display device according to claim 1 or claim 2, wherein the overlap area is calculated to be larger the larger the area of ​​overlap between a priority area set within the virtual movement area and the actual movement area, and the priority area is set to have a greater influence on the size of the overlap area than areas within the virtual movement area other than the priority area.

4. A method for controlling a display device worn by a user and providing the user with a virtual space, comprising: acquiring information about a virtual movement area in which the user can move in the virtual space; acquiring information about a real movement area in which the user can move in a real space in which the user actually exists; calculating an overlapping area, which is an area where the virtual movement area and the real movement area overlap when the virtual space and the real space are overlapped, and acquiring a correspondence relationship between the virtual space and the real space, which is set based on the overlapping area; displaying an image for the virtual space on a display unit based on the correspondence relationship and a position of the display device in the real space; Including, In the step of acquiring the correspondence relationship, at least one of a relative position and a relative orientation between the virtual space and the real space is moved, and an overlapping area is calculated when the virtual space and the real space are combined, and a correspondence relationship that maximizes the overlapping area is acquired from among the overlapping areas. A method for controlling a display device.

5. A program for causing a computer to execute a control method for a display device worn by a user to provide the user with a virtual space, acquiring information about a virtual movement area in which the user can move in the virtual space; acquiring information about a real movement area in which the user can move in a real space in which the user actually exists; calculating an overlapping area, which is an area where the virtual movement area and the real movement area overlap when the virtual space and the real space are overlapped, and acquiring a correspondence relationship between the virtual space and the real space, which is set based on the overlapping area; displaying an image for the virtual space on a display unit based on the correspondence relationship and a position of the display device in the real space; Run the following on your computer: In the step of acquiring the correspondence relationship, at least one of a relative position and a relative orientation between the virtual space and the real space is moved, and an overlapping area is calculated when the virtual space and the real space are combined, and a correspondence relationship that maximizes the overlapping area is acquired from among the overlapping areas. program.

Images