Display apparatus, method for controlling display apparatus, and storage medium

Abstract

To appropriately provide a virtual space to a user.SOLUTION: The display apparatus includes a virtual-space-information acquiring unit that acquires information of a movable region AV2 in which the user can move in the virtual space, a real-space-information acquiring unit that acquires information of a movable region AR2 in which the user can move in a real space in which the user actually exists, a correspondence-relation acquiring unit that acquires a correspondence relation between the virtual space and the real space, the correspondence relation being set on the basis of an overlapping area that is an area in which the movable region AV2 and the movable region AR2 overlap each other when the virtual space and the real space overlap each other, and a display control unit that causes a display unit to display an image for the virtual space on the basis of the correspondence relation and a position of the display apparatus in the real space.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a display device, a control method for a display device, and a program. [Background technology]

[0002] Information devices have been evolving dramatically in recent years. Smartphones are a prime example of this. In addition, so-called wearable devices have also become popular. There is a known type of HMD (head mounted display) that uses glasses. By using MD, the image is displayed in accordance with the user's line of sight, creating a virtual space for the user U. For example, in Patent Document 1, coordinate data from a three-dimensional sensor is used to It is stored as a reference coordinate in 3D space and is matched to the reference position in virtual 3D space. It is described that the user's viewpoint position is corrected by this. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 9-311618 Summary of the Invention [Problem to be solved by the invention]

[0004] Such display devices are required to provide users with an appropriate virtual space. There are.

[0005] In view of the above-mentioned problems, the present invention provides a method for appropriately associating coordinates in real space with coordinates in virtual space. The present invention aims to provide a display device, a control method for a display device, and a program. [Means for solving the problem]

[0006] A display device according to one aspect of the present invention is worn by a user and provides the user with a virtual space. a display device for displaying information on a virtual movement area in which the user can move in the virtual space; a virtual space information acquisition unit that acquires information about a user's movements in a real space where the user actually exists; a real space information acquisition unit that acquires information on a possible real movement area; and a real space information acquisition unit that acquires information on the virtual space and the real space. Based on an overlapping area, which is the area where the virtual movement area and the actual movement area overlap when they are overlapped, a correspondence relationship acquisition unit that acquires the correspondence relationship between the virtual space and the real space set based on the Based on the correspondence relationship and the position of the display device in the real space, and a display control unit that causes the image to be displayed on the display unit.

[0007] A method for controlling a display device according to one aspect of the present invention includes: A method for controlling a display device that provides a space, the method comprising: a step of acquiring information about a virtual movement area in which the user is actually present; acquiring information on a real movement area in which the user can move; The overlapping area is the area where the virtual movement area and the actual movement area overlap when the two areas are overlapped. acquiring a correspondence relationship between the virtual space and the real space set based on the Based on the correspondence relationship and the position of the display device in the real space, an image for the virtual space is generated. and displaying the image on a display unit.

[0008] A program according to one aspect of the present invention is worn by a user and provides the user with a virtual space. A program for causing a computer to execute a control method for a display device provided by the virtual space acquiring information about a virtual moving area in which the user can move between the two locations; acquiring information about a real movement area in which the user can move in a real space that actually exists; When the virtual space and the real space are superimposed, the virtual movement area and the real movement area are The correspondence between the virtual space and the real space is set based on an overlapping area, which is an area where the two spaces overlap. a step of acquiring a relationship, the correspondence relationship, and a position of the display device in the real space; and a step of displaying the image for the virtual space on a display unit based on the image. Make them do so. [Effects of the Invention]

[0009] According to the present invention, a virtual space can be appropriately provided to a user. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a real space and a virtual space. [Figure 2] FIG. 2 is a schematic block diagram of the display device according to this embodiment. [Figure 3] FIG. 3 is a schematic diagram showing an example of a virtual space. [Figure 4] FIG. 4 is a schematic diagram showing an example of a real space. [Figure 5] FIG. 5 is a schematic diagram showing an example of superimposition of a virtual space and a real space. [Figure 6] FIG. 6 is a schematic diagram showing another example of superimposition of virtual space and real space. [Figure 7] FIG. 7 is a flowchart illustrating a display flow of an image in a virtual space. [Figure 8] FIG. 8 is a schematic diagram illustrating an example of a priority area. [Figure 9] FIG. 9 is a schematic diagram showing another example of superimposition of virtual space and real space. [Figure 10] FIG. 10 is a schematic diagram showing an example in which a user is visually recognizing a virtual space. [Figure 11] FIG. 11 is a schematic diagram showing another example of superimposition of virtual space and real space. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. The present invention is not limited to the embodiments.

[0012] (Real space and virtual space) 1 is a schematic diagram illustrating an example of a real space and a virtual space. 10 is a display device that displays an image. As shown in FIG. 1, the display device 10 is It is a so-called HMD (Head Mount Display) that is worn on the head. The display device 10 displays an image to provide a virtual space to the user U. In this way, the real space where the user U actually exists is defined as the real space SR, and the display device 10 displays the user U The virtual space provided to the user is referred to as a virtual space SV. In this case, the display device 10 displays the real space SR. The image for the virtual space SV is displayed according to the movement (line of sight) of the user U. The U is operating in virtual space SV as an avatar UV, and Therefore, the user U recognizes that he or she is in the virtual space SV. The virtual space SV here is MR (Mixed Reality) ) is a space that reproduces a real place away from where the user U is located, but It is not limited to virtual spaces that do not actually exist, namely VR (Vertial Reality) In the following, a direction along the horizontal direction in the coordinate system of the real space SR is The direction is defined as XR, the direction perpendicular to the direction XR along the horizontal direction is defined as YR, and the vertical direction is defined as The direction is ZR. In addition, the direction along the horizontal direction in the coordinate system of the virtual space SV is the direction X. The direction perpendicular to the XV direction along the horizontal direction is the YV direction, and the vertical direction is the ZV direction. Let's say.

[0013] (display device) FIG. 2 is a schematic block diagram of a display device according to this embodiment. As shown in FIG. 2, the computer is comprised of an input unit 20, a display unit 22, and a memory unit 23. The input unit 20 includes a user interface 4, a communication unit 26, a real space detection unit 28, and a control unit 30. For example, it is a mechanism that accepts operations by the user, such as a controller or microphone attached to the HMD. The display unit 22 is a display that displays an image. The display device 10 provides the user U with a virtual space SV by outputting an image. In addition to the 22, the device may be equipped with a device for outputting information, such as a speaker for outputting audio. stomach.

[0014] The storage unit 24 is a memory that stores various information such as the calculation contents and programs of the control unit 30. For example, RAM (Random Access Memory) and ROM (Re Main memory such as HDD (Hard Disk Only Memory) and The storage unit 24 includes at least one of the following: The program for the control unit 30 is stored in a recording medium that can be read by the display device 10. Good too.

[0015] The communication unit 26 is a communication module that communicates with an external device, and is, for example, an antenna. The display device 10 communicates with an external device via wireless communication, but wired communication is also acceptable. The method may be any method.

[0016] The real space detection unit 28 detects the surroundings of the display device 10 (user U) in the real space SR. The real space detection unit 28 is a sensor that detects the surroundings of the display device 10 (user U) in the real space SR. In this embodiment, it is a camera that detects objects in the real space. The detection unit 28 is capable of detecting an object present in the real space SR around the display device 10 (user U). If so, it is not limited to cameras, but can also be used with LIDAR (Light Detect ion And Ranging), etc.

[0017] The control unit 30 is a computing device, for example, a CPU (Central Processing Unit). The control unit 30 includes a virtual space information acquisition unit 40 and a real space information acquisition unit 42. The information acquisition unit 42, the correspondence relationship acquisition unit 44, the display control unit 46, and the avatar information transmission unit 4 The control unit 30 reads out a program (software) from the storage unit 24 and executes it. By performing this, the virtual space information acquisition unit 40, the real space information acquisition unit 42, and the correspondence relationship acquisition unit 44 The display control unit 46 and the avatar information transmission unit 48 are realized and the processes thereof are executed. The control unit 30 may execute these processes using one CPU or multiple CPUs. The virtual space information acquisition unit may include a plurality of CPUs, and the plurality of CPUs may execute the processing. 40, real space information acquisition unit 42, correspondence relationship acquisition unit 44, display control unit 46, and avatar information transmission unit At least part of the processing by unit 48 may be realized by a hardware circuit.

[0018] (Virtual space information acquisition section) The virtual space information acquisition unit 40 acquires information about the virtual space SV. For example, the information of the virtual space SV is acquired from an external device (server) via the communication unit 26. The information of the virtual space SV is the image data of the virtual space SV in the coordinate system of the virtual space SV. The image data of the virtual space SV is the object to be displayed as an image for the virtual space SV. In this embodiment, the virtual space SV indicates the coordinates and shape of the real space SR. It is not constructed according to the surrounding environment of the user U, but is based on the user U in the real space SR. It is preset regardless of the surrounding environment.

[0019] 3 is a schematic diagram showing an example of a virtual space. The virtual space information acquisition unit 40 acquires the following as information on the virtual space SV: Information on the movable area AV2 (virtual movement area) in the virtual space SV is also acquired. That is, the virtual space information acquisition unit 40 acquires information on the movable area AV2 in the coordinate system of the virtual space SV. The movable area AV2 is the area in the virtual space SV where the user can move. The area where the user's avatar UV can move (or the space where the avatar UV can move), In the virtual space SV, the area where the user U's avatar UV cannot move (or the avatar UV The area may be any area other than the unmovable area AV1, which is a space where movement is not possible. The area AV2 may be, for example, the floor of a room where avatars gather in the virtual space SV. The unmovable area AV1 is, for example, an obstacle that the avatar UV cannot pass through in the virtual space SV. The area of ​​interest in a meeting in the virtual space SV (for example, a desk or a screen) In this embodiment, the movable area AV2 may be, for example, a virtual space SV. For example, the size of the avatar UV and the size of the unmovable area AV1 may be set in advance when The virtual space information acquisition unit 40 may set it based on the size.

[0020] In FIG. 3, the virtual space SV and the immovable area AV1 when viewed from the ZV direction are is a rectangle, but this is just an example. The shape and size of the area AV2 are not limited to the example shown in FIG. 3 and may be any shape and size.

[0021] (Real space information acquisition unit) The real space information acquisition unit 42 acquires information about the real space SR. The coordinates and shapes of objects around the display device 10 (user U) in the SR coordinate system are calculated. In this embodiment, the real space information acquisition unit 42 controls the real space detection unit 28. By controlling the real space detection unit 28, the real space detection unit 28 detects objects around the display device 10 (user U), and The result is acquired as real-space SR information. However, the method for acquiring real-space SR information is as follows: For example, the layout information of the room of the user U is not limited to being detected by the real space detection unit 28. The real space information acquisition unit 42 acquires information about the real space SR, such as the information about the real space SR, in advance. Information on the real space SR may be acquired.

[0022] Fig. 4 is a schematic diagram showing an example of a real space. Fig. 4 shows the real space SR as viewed from the ZR direction. The real space information acquisition unit 42 acquires information on the movable area A in the real space SR. R2 (real movement area). That is, the real space information acquisition unit 42 acquires information on the real space SR The information indicating the position of the movable area AR2 in the coordinate system is acquired. The area AR2 is the area in which the user U can move in the real space SR (or the area in which the user U can move). It is an area where user U cannot move (or a space where user U cannot move) The movable area AR2 may be an area excluding a certain unmovable area AR1. The unmovable area AR1 may be, for example, the floor of the room where the user U is. In this case, there is an obstacle (such as a desk or a bed) that the user U cannot pass through. In this embodiment, the real space information acquisition unit 42 acquires information about the real space SR based on the information about the real space SR. The real space information acquisition unit 42 sets the area AR2 and the immovable area AR1. Based on the information, the location of the object that the user U cannot move is identified, and the object that the user U cannot move is identified. The area (or space) occupied by the body is defined as the unmovable area AR1, and the objects that the user U cannot move are defined as An area (or space) that does not exist may be set as the movable area AR2. The movable area AR2 and the unmovable area AR1 are not limited to being set based on information from the real space SR. For example, the layout information of the user U's room, etc., can be used to distinguish between the movable area AR2 and the non-movable area AR3. The information of the area AR1 is set in advance, and the real space information acquisition unit 42 uses the set movable area Information on the area AR2 and the unmovable area AR1 may be acquired.

[0023] Note that FIG. 4 is merely an example, and the real space SR, the unmovable area AR1, and the movable area The shape and size of AR2 are not limited to the example shown in FIG. 4 and may be any shape or size.

[0024] (Correspondence acquisition section) The correspondence relationship acquisition unit 44 acquires the information of the movable area AV2 acquired by the virtual space information acquisition unit 40. and the information on the movable area AR2 acquired by the real space information acquisition unit 42, the virtual space S Set the correspondence between the coordinate system of V and the coordinate system of the real space SR. The correspondence relationship between the coordinate system of the real space SR and the virtual space SV is This is information that indicates posture, and is a value used to convert the coordinate system of the real space SR into the coordinate system of the virtual space SV. For example, when the user U is at a reference position in the real space SV, the display device 10 is the position of the user U's viewpoint (avatar) in the virtual space SV that corresponds to the reference position in the real space SV. The image of the virtual space SV is displayed when the corresponding relationship acquisition unit 44 (UV) exists. The process will be specifically described.

[0025] 5 is a schematic diagram showing an example of superimposition of a virtual space and a real space. , the virtual space SV acquired by the virtual space information acquisition unit 40 and the real space information acquired by the real space information acquisition unit 42 In other words, the correspondence relationship acquisition unit 44 superimposes the virtual space SR and the real space SR on a common coordinate system. The coordinates of the unmovable area AV1 and the movable area AV2 in the space SV and the coordinates of the imaginary space SR The coordinates of the unmovable area AR1 and the movable area AR2 are converted into a common coordinate system. By doing so, the unmovable area AV1 and the movable area AV2, and the unmovable area AR1 and the movable area AR2 are The movable area AR2 is superimposed on a common coordinate system. The common coordinate system can be any coordinate system. In the example of FIG. 5, one direction along the horizontal direction in the common coordinate system is designated as the direction X The direction perpendicular to the horizontal direction is defined as direction Y, and the vertical direction is defined as direction Z. do.

[0026] The correspondence relationship acquisition unit 44 acquires the following when the virtual space SV and the real space SR are superimposed on a common coordinate system: The area of ​​the overlapping region of the movable area AV2 and the movable area AR2 (or the overlapping space) The correspondence relationship acquisition unit 44 calculates the overlapping area based on the calculated overlapping area. , the correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR is calculated.

[0027] In this embodiment, the correspondence relationship acquisition unit 44 acquires the coordinates in a common coordinate system as shown in the example of FIG. By doing so, at least one of the relative position and the relative orientation between the virtual space SV and the real space SR is changed. In other words, the correspondence relationship acquisition unit 44 calculates the overlapping area in the common coordinate system. In the virtual space SV and the real space SR, at least one of the relative position and the relative orientation is different. Then, the overlapping area of ​​the movable area AV2 and the movable area AR2 is calculated. In the example of Figure 5, the position and orientation of the real space SV are fixed in a common coordinate system, Although an example of moving the position and orientation of the virtual space SR is shown, the present invention is not limited to this. While fixing the position and orientation of the SR, the position and orientation of the real space SV are moved to obtain the overlap area. may be calculated.

[0028] The correspondence relationship acquisition unit 44 acquires at least one of the relative position and the relative orientation in the common coordinate system. Based on the overlapping area for each combination of the virtual space SV and the real space SR, The correspondence relationship between the coordinate system of the real space SR and the coordinate system of the real space SR is set. 44 is a diagram showing a virtual space SV and a real space SR in which at least one of the relative position and the relative orientation is different. Among the combinations, the combination of virtual space SV and real space SR that has the largest overlapping area is selected. , and the correspondence relationship acquisition unit 44 extracts the coordinate system of the extracted virtual space SV and the coordinate system of the real space S. Correspondence with the coordinate system of R (the coordinate system of the extracted real space SR and the coordinate system of the extracted virtual space SV) ) to calculate the correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR. In other words, the correspondence relationship acquisition unit 44 sets the position where the overlapping area is largest. The virtual space SV, which is the position and direction, is extracted, and the coordinate system of the extracted virtual space SV is Map to a coordinate system.

[0029] 6 is a schematic diagram showing another example of superimposition of virtual space and real space. The correspondence relationship acquisition unit 44 acquires the position and While changing its orientation, the virtual space SV was superimposed on the real space SR. The image may be superimposed on the real space SR while changing its size. In this case, the correspondence relationship acquisition unit 44 As shown in the example above, the relative size of the virtual space SV and the real space SR in a common coordinate system is expressed as In other words, the correspondence relationship acquisition unit 44 calculates the overlapping area while changing the common coordinates. The relative size of the virtual space SV and the real space SR in the system is different, and each combination can be moved. The overlapping area of ​​the area AV2 and the movable area AR2 is calculated. Even when changing the virtual space SV, the immovable area AV1 and the movable area AV 2, and the area of ​​the unmovable area AR1 and the movable area AR2 relative to the real space SR. It is preferable to keep the ratio fixed. In other words, the virtual space SV and the real space SR The entire virtual space SV and real space SR are scaled uniformly without scaling only a part of them. In the example of FIG. 6, the size of the real space SV is fixed in the common coordinate system. While the example shows how to change the size of the virtual space SR while setting the The size of the real space SV is changed while the size of the space SR is fixed, and the overlapping area is calculated. Good too.

[0030] In the example of FIG. 6, the correspondence relationship acquisition unit 44 acquires virtual objects with different relative sizes in a common coordinate system. Based on the overlapping area for each combination of space SV and real space SR, the coordinate system of virtual space SV and real space SR are The correspondence relationship with the coordinate system of the space SR is set. Among the combinations of virtual space SV and real space SR with different sizes, the one with the largest overlapping area is Then, the correspondence relationship acquisition unit 44 extracts the combination of the virtual space SV and the real space SR. The correspondence between the extracted virtual space SV coordinate system and the real space SR coordinate system is expressed as the virtual space SV The correspondence between the coordinate system of the real space SR and the coordinate system of the real space SR is set as a correspondence relationship. The obtaining unit 44 extracts the virtual space SV at a reduced scale that maximizes the overlapping area, and The coordinate system of the virtual space SV, which is set to the size of the overlap, is then associated with the coordinate system of the real space SR.

[0031] 5 and 6 may be combined. That is, the correspondence relationship acquisition unit 44 In the common coordinate system, the relative position, relative orientation and relative magnitude between the virtual space SV and the real space SR are The overlapping area is calculated while changing the size. Then, the correspondence relationship acquisition unit 44 calculates the overlapping area by using a common coordinate system The virtual space SV and the real space SV have at least one of the relative position, relative orientation, and relative size different from each other. Among the combinations with the space SR, the virtual space SV and real space SR that have the largest overlapping area are Then, the correspondence relationship acquisition unit 44 extracts the combination of the extracted coordinates of the virtual space SV. The correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR is The correspondence is set as follows.

[0032] In the above description, the correspondence relationship acquisition unit 44 calculates the two-dimensional movable area in the virtual space SV. The movable area AV2 indicates the movable area in two dimensions in the real space SR. The virtual space SV and the real space SR are associated so that the overlapping area with the area AR2 is maximized. However, it is not limited to maximizing the overlap area in two dimensions. For example, the correspondence relationship acquisition unit 4 4 is a movable area AV2 (virtual movement) that indicates the three-dimensional movable space in the virtual space SV. space) and the movable area AR2 (real space SR) that indicates the movable space in three dimensions. The virtual space SV and the real space SR may be associated so that the overlapping volume with the virtual space SV is maximized. .

[0033] In the above description, the correspondence relationship acquisition unit 44 calculates the virtual space SV and the real space SV in the common coordinate system. The overlapping area is calculated by superimposing the virtual space SR and the coordinate system of the virtual space SV based on the overlapping area. Set the correspondence with the coordinate system of the real space SR. However, the calculation of the overlap area and the setting of the correspondence are The above is not limited to being performed by the correspondence relationship acquisition unit 44. For example, The correspondence relationship acquisition unit 44 acquires information on the overlap area from the external device. The correspondence relationship may be set based on the calculated overlap area. The correspondence relationship is set based on the overlap area, and the correspondence relationship acquisition unit 44 receives the correspondence from the external device. Relationship information may also be obtained.

[0034] (Display control unit) The display control unit 46 compares the coordinate system of the virtual space SV set by the correspondence acquisition unit 44 with the coordinate system of the real space SR based on the correspondence relationship with the coordinate system of the real space SR and the position of the user U (display device 10) in the real space SR. Then, the image for the virtual space SR is displayed on the display unit 22. Specifically, the display control unit 46 The position and orientation information of the user U in the real space SR is acquired, and based on the correspondence, The position and orientation of the user U in the SR are calculated based on the viewpoint of the user U in the coordinate system of the virtual space SV. The display control unit 46 converts the calculated view of the user U into the position and orientation of the avatar U. The image of the virtual space SV when the virtual space SV is viewed from the position and orientation of the point is called the virtual space S The image for user U (display device) in the real space SR is displayed on the display unit 22 as an image for user V. The position and orientation information of the device 10 may be acquired by any method. For example, The calculation may be performed using the detection results of 8 (that is, the captured image of the real space SR in this case).

[0035] In this way, the position and posture of the user U in the real space SR is the same as that of the user U in the virtual space SV. Therefore, when user U moves in the real space SR, the virtual The position and orientation of the user U's viewpoint in the space SV (i.e., the position and orientation of the avatar UV) also moves. In this case, the movement amount of the user U in the real space SR and the movement amount of the user U in the virtual space SV are It is preferable that the distance between the camera and the camera is associated with the distance between the camera and the camera. When changing the size of the virtual space SV in the common coordinate system, It is preferable to reflect the degree of change in the size of the space SV in the amount of movement. The ratio (scale ratio) at which the size of the virtual space SV is changed in the common coordinate system is the change ratio. Then, the display control unit 46 calculates the change rate of the amount of movement of the user U in the real space SR. If the user U's viewpoint moves in the virtual space SV, the reciprocal of the movement amount is That is, the display control unit 46 controls the display unit 22 to display the image for the user U in the real space S. The virtual view from the viewpoint moved by the reciprocal of the change rate of the movement amount moved by R An image of the space SV is displayed on the display unit 22. For example, when setting the correspondence relationship, When the size of the SV is doubled, the display control unit 46 displays the user U moving in the real space SR. The image of the virtual space SV from a viewpoint that has moved only half the amount of movement of the object is The display unit 22 displays the information.

[0036] The display control unit 46 also displays an object in the real space SR superimposed on the image in the virtual space SV. In this case, the display unit 22 displays an image of the virtual space SV while transmitting the image of the real space SR. It may also provide AR (Augumented Reality) that shows The image of the virtual space SV and the image showing the object in the real space SR may be displayed superimposed on each other. In addition, the movable area AV2 in the virtual space SV is the movable area AR2 in the real space SR. The space that does not overlap with the SV image can be deleted from the image or set as a non-movable area. In this case, even if the area is still available for movement, The spatial width is secured so that the user U (the avatar UV that reproduces his / her body shape) can pass through. The area that is not included may be deleted from the image of the virtual space SV.

[0037] The display device 10 according to this embodiment uses the coordinate system of the virtual space SV set in this way and the real By displaying images for the virtual space SV based on the correspondence with the coordinate system of the space SR, the user For example, the user U can provide the virtual space SV to the user U. In other words, the user U moves through the real space SR while visually recognizing the virtual space SV. The user U tries to move within the movable area AV2 in is the movable area AR2 in the real space SR. The movable area and the actual movable area are different. In terms of form, the movable area AV2 in the virtual space SV and the movable area in the real space SR The virtual space SV and the real space SR are associated so that the overlapping area with AR2 is large. Therefore, the difference between the area that the user U perceives as available for movement and the area that the user U can actually move is reduced. This allows the user U to move around in a wide area. In this way, even if the user U moves, the virtual space SV can be provided appropriately.

[0038] (Avatar information transmission unit) The avatar information transmission unit 48 transmits information about the avatar UV of the user U in the virtual space SV as follows: The avatar information transmitting unit 48 transmits the information to the outside via the communication unit 26. The position and orientation information of the user U is acquired, and the coordinate system of the real space SR and the coordinate system of the virtual space SV are Based on the correspondence with the system, the position and orientation of the user U in the real space SR are calculated based on the The avatar information transmission unit 48 converts the information into the position and orientation of the avatar UV in the coordinate system. The position and orientation information of the avatar UV in the coordinate system of the virtual space SV, and the image of the avatar UV The external device transmits the image data (data showing the shape, etc.) to the virtual space. Avatar UV position and orientation information in the SV coordinate system and avatar UV image data The data is transmitted as image data of the virtual space SV to a display device used by another user. The display device displays an image of the avatar UV to the user wearing the display device. Display the image data of the virtual space SVG. In this way, the image data of the avatar can be sent to an external device. By sending it to , multiple users can share the virtual space SV.

[0039] (Processing flow) The display flow of the image in the virtual space SV explained above will be described. 7 is a flowchart illustrating a display flow. As shown in FIG. 7, the display device 10 The space information acquisition unit 40 acquires information on the virtual space SV (step S10), and the real space information The information acquisition unit 42 acquires information about the real space SR (step S12). The device 10 acquires the relative position and orientation between the virtual space SV and the real space SR by the correspondence relationship acquisition unit 44. , and the relative size while changing at least one of the virtual space SV and the real space SR. The correspondence relationship acquisition unit 44 then calculates the overlapping area in the coordinate system (step S14). , Among the combinations of the virtual space SV and the real space SR, the virtual space S with the largest overlapping area is A combination of V and the real space SR is extracted (step S16), and the coordinates of the extracted virtual space SV are calculated. The correspondence between the reference system and the coordinate system of the real space SR is set (step S18). The display control unit 46 controls the display device 10 (user U) in the real space SR to correspond to the set correspondence relationship. ) and the position of the virtual space SV, the image of the virtual space SV is displayed on the display unit 22 (step S20). .

[0040] (effect) As described above, the display device 10 according to this embodiment is worn by the user U. A virtual space information acquisition unit 40 and a real space information acquisition unit 50 provide a virtual space SV to the user U. The virtual space information acquisition unit 40 includes a virtual space information acquisition unit 42, a correspondence relationship acquisition unit 44, and a display control unit 46. , a movable area AV2 (a virtual area) in which the user U (avatar UV) can move in the virtual space SV (a virtual area) The real space information acquisition unit 42 acquires information about the real space S where the user U actually exists. At R, information on a movable area AR2 (actual movement area) in which the user U can move is acquired. The correspondence relationship acquisition unit 44 acquires the coordinate system of the virtual space SV and the coordinate system of the real space SV, which are set based on the overlapping area. The overlapping area is the area where the virtual space SV and the real space SR are overlapped. When superimposed in the common coordinate system, the movable area AV2 (virtual movable area) and the movable area AR 2 (actual movement area) overlaps with the real space SR. and a position of the display device 10 in the virtual space SV. do.

[0041] When the virtual space SV is provided to the user U from the display device 10, the user U recognizes The area that can be moved in the virtual space SV is different from the area that can actually be moved in the real space SR. In contrast, the display device 10 according to this embodiment is Based on the overlapping area between the movable area AV2 and the movable area AR2 in the real space SR, Since the space SV and the real space SR are associated with each other, the movable area recognized by the user U and The deviation from the actual movable area is reduced, and the user U can move around in the widest possible area. Therefore, according to the display device 10, even when the user U moves, the virtual space S V can be provided appropriately.

[0042] The correspondence relationship is the relative position and Combination of each virtual space SV and real space SR with at least one of the relative directions moved Among the coordinate systems, the coordinate system of the virtual space SV and the coordinate system of the real space SR that have the largest overlapping area are In this way, the virtual space SV is moved relative to the real space SR, and the superimposition By associating the virtual space SV with the real space SR so that the area is maximized, the user U can This allows for the widest possible area of ​​movement.

[0043] The correspondence relationship is expressed as the relative size of the virtual space SV and the real space SR in a common coordinate system. Among the combinations of the virtual space SV and the real space SR that were changed, the overlapping area was The coordinate system of the virtual space SV, which is the maximum, corresponds to the coordinate system of the real space SR. By changing the size of the virtual space SV relative to the real space SR, the overlapping area is maximized. By associating the virtual space SV with the real space SR, the area in which the user U can move is maximized. It can be secured as widely as possible.

[0044] In addition, the display control unit 46 determines the amount of movement of the display device 10 (user U) in the real space SR. On the other hand, the movement is the reciprocal of the change rate of the size of the virtual space SV in the common coordinate system. The user U moves in the virtual space SV, and the image for the virtual space SV is displayed on the display unit 22. The display device 10 according to this embodiment displays the actual movement amount of the user U. Taking into account the scale reduction, the amount of movement in the virtual space SV is set, so the movement of the user U is In addition, the virtual space SV can be provided appropriately.

[0045] (Another example of how to set up correspondence) The correspondence between the coordinate system of the virtual space SV and the coordinate system of the real space SR described in this embodiment is Another example of the determination method will be described.

[0046] For example, as explained in Fig. 5 and Fig. 6, while changing the position, orientation, and size of the virtual space SV, When the virtual space SV and the real space SR are superimposed so that the overlapping area is maximized, The unmovable area AV1 (e.g., the area of ​​interest) in the SV is surrounded by an unmovable area in the real space SR. AR1 (e.g., a real obstacle) is located and gets in the way when approaching the area of ​​interest in the virtual space SV. In preparation for such a case, the correspondence acquisition unit 44 Set a priority area in the SV's movable area AV2, and the priority area will be the non-movable area in the real space SR. The virtual space SV and the real space SR may be superimposed on each other so as not to overlap with the area AR1. We will explain this in more detail.

[0047] FIG. 8 is a schematic diagram illustrating an example of a priority area, and FIG. 9 is a diagram illustrating an example of an overlapping of a virtual space and a real space. The correspondence relationship acquisition unit 44 acquires the movable area A of the virtual space SV. A priority area is set within V2. The priority area is the area where the virtual space SV and the real space SR are superimposed. When moving the object, priority is given to overlapping with the movable area AR2 without overlapping with the unmovable area AR1. The correspondence relationship acquisition unit 44 may set the priority area by any method, for example. , within the entire movable area AV2, a predetermined area around the unmovable area AV1, which is the area of ​​interest, The correspondence relationship acquisition unit 44 may set an area having a size of the priority area. In the example of FIG. 8, the correspondence relationship acquisition unit 44 A priority area AV2a is set around the immovable area AV1, and a priority area AV2b is set around the priority area AV2a. In this case, the priority area AV2a is closer to the restricted area AV1. The priority of the area AV2a is set higher than that of the priority area AV2b. The area AV2 other than the priority area will be referred to as a non-priority area. In this example, the area outside the priority area AV2b is the non-priority area AV2c.

[0048] The correspondence relationship acquisition unit 44 compares the virtual space SV in which the priority area is set and the real space SR in a common manner. In this case, the correspondence relationship acquisition unit 44 performs the following as described with reference to FIGS. In a common coordinate system, the relative position, relative direction, and relative size between the virtual space SV and the real space SR are The overlapping area is calculated while changing at least one of the dimensions, and the virtual space S with the largest overlapping area is found. From the combination of V and real space SR, the correspondence between the coordinate system of virtual space SV and the coordinate system of real space SR is However, in this example, the area where the priority area and the movable area AR2 overlap is The size of the priority overlap area is the area where the non-priority area and the movable area AR2 overlap. The overlapping area is set so that it has a greater effect on the calculated overlapping area than the size of the non-priority overlapping area. That is, the larger the priority overlap area and the non-priority overlap area, the larger the overlap area. The overlap area is calculated to be larger, but when the priority overlap area increases by a unit amount, The degree to which the area increases is the degree to which the overlap area increases when the non-priority overlap area increases by a unit amount. It is greater than the degree to which it becomes.

[0049] For example, the correspondence relationship acquisition unit 44 in this example assigns a weight to the priority overlap area, The overlapping area is calculated as the sum of the area multiplied by the weight and the non-priority overlapping area. By assigning weights to the priority overlap area, the priority overlap area (priority area) is given to the overlap area. The degree of influence of the non-priority overlap area (non-priority area) on the overlap area is greater than the degree of influence of the non-priority overlap area (non-priority area) on the overlap area. , becomes larger. For example, as shown in FIG. 9, It is important that the surrounding priority area does not overlap with the unmovable area AR1, and the overlapping area is maximized. The combination of the virtual space SV and the real space SR is such that the priority overlap area is the unmovable area AR1. Therefore, for example, when approaching the area of ​​interest in the virtual space SV, This reduces the possibility that the AR1 region will be a nuisance.

[0050] In this example, the priority area set within the movable area AV2 and the movable area A The overlapping area is calculated to be larger as the priority overlapping area with V2 is larger. The priority area is larger than the non-priority area (the area within the movable area AV2 other than the priority area). The influence on the magnitude of the product is set to be large. This allows the focus area in the virtual space SV to be When approaching the target, the possibility that the unmovable area AR1 will be an obstacle can be reduced. When setting the regions, different weight values ​​may be assigned to each priority region. In this case, in the example of Figure 8, Then, the weight of the priority area AV2a, which is close to the unmovable area AV1, is higher than the weight of the priority area AV2b. It is set larger than the

[0051] Next, as another example of how to set the correspondence, the height direction of the real space SR and the virtual space S A method for setting the correspondence between the height direction of V and the position of V will be described. 10 is a schematic diagram showing an example of a case where a virtual space is visually recognized. The unmovable area AV1a, which is the area of ​​interest in V, is an unmovable area that is an obstacle in the real space SV. The position of the area AR1 and the height direction (ZR direction in real space coordinates) overlap, and in the virtual space SV In order to avoid this situation, the corresponding area may be hidden. The acquisition unit 44 determines whether the positions of the unmovable area AV1 and the unmovable area AR1 in the height direction overlap. In order to achieve this, the correspondence between the height position in the real space SR and the height position in the virtual space SV is As a result, for example, as shown in the no-movement area AV1b in FIG. The position of the unmovable area can be set in a position that does not overlap with the movable area AR1. The correspondence relationship between the height direction position and the height direction position of the virtual space SV is set by the correspondence relationship acquisition unit. 44 may be performed automatically or by input from the user U.

[0052] Next, as another example of how to set the correspondence, the degree of similarity between the virtual space SV and the real space SR is An example in which the virtual space SV and the real space SR are superimposed so that the σ is increased will be described. 1 is a schematic diagram showing another example of superimposition of virtual space and real space. The relationship acquisition unit 44 compares the shapes of the real space SR and the virtual space SV, and determines whether the real space SR has a shape that is different from the virtual space SV. The region having a shape similar to that of the virtual space SV is extracted as a similar region SRS having a high degree of similarity. Then, as shown in the example of FIG. 11, the correspondence relationship acquisition unit 44 acquires the correspondence between the virtual space SV and the real space SR. The virtual space SV is converted into the real space while changing at least one of the relative position, relative orientation, and relative size of By superimposing the SR, the virtual space SV and the similar region are superimposed. is the correspondence between the coordinate system of the virtual space SV superimposed on the similarity region SRS and the coordinate system of the real space SR. In this way, by superimposing the virtual space SV on the area with high similarity, Even if the user U moves around, the difference between the area that the user U perceives and the area that the user U can actually move around will be reduced. In this example, the user U can move around in a wide area. The overlapping area may not be taken into consideration, but is not limited to this, and the overlapping area may also be taken into consideration. For example, the correspondence relationship acquisition unit 44 may select a position where the degree of similarity is large and the overlapping area is large. In this way, the virtual space SV and the real space SR may be superimposed.

[0053] Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. In addition, the above-mentioned components include those that a person skilled in the art can easily imagine, and Furthermore, the above-mentioned components are included in the scope of equivalents. They can be combined as desired, and the configurations of the respective embodiments can also be combined. Furthermore, various omissions, substitutions or modifications of the components may be made without departing from the spirit of the above-described embodiments. Changes can be made. [Explanation of symbols]

[0054] 10 Display device 22 Display section 40 Virtual space information acquisition unit 42 Real space information acquisition unit 44 Correspondence acquisition unit 46 Display control unit AR2 Movable area (actual moving area) AV2 Movable area (virtual moving area) SR real space SV Virtual Space U User

Claims

[Claim 1] a virtual space information acquisition unit that acquires information about a virtual movement area in which a user can move in the virtual space; a real space information acquisition unit that acquires information about a real movement area in which the user can move in a real space where the user actually exists; a correspondence relationship acquisition unit that calculates an overlap area, which is an area where the virtual movement area and the real movement area overlap when the virtual space and the real space are overlapped, and acquires a correspondence relationship between the virtual space and the real space that is set based on the overlap area; Including, the correspondence relationship acquisition unit calculates the overlapping area to be larger as the overlapping area between the priority area set in the virtual movement area and the actual movement area becomes larger; Device.

Images