Providing device and providing method

Abstract

The providing device comprises: a real space information acquisition unit for acquiring position information of a real object in a real space in which a user is located in the real space; a virtual space information acquisition unit that acquires position information of a virtual object present in the virtual space in the virtual space; a user action determination unit that determines whether or not the user has performed a predetermined action on the real object in the real space; and a virtual space provision unit that matches the position information of the real space with the position information of the virtual space, and generates a transition image indicating the state between the real space and the virtual space when the user action determination unit determines that the user has performed the predetermined action on the real object in the real space. And providing a transition space containing the transition image before providing the virtual space to the user.

Description

Technical Field

[0001] The present disclosure relates to providing apparatus and providing methods. Background Art

[0002] There are known technologies for providing virtual spaces such as AR (Augmented Reality) space and VR (Virtual Reality) space using wearable devices such as head mounted displays (HMD). For example, Patent Document 1 discloses a technology that can provide a high-quality virtual reality experience.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent document 1: Japanese Patent Application Publication No. 2019-8513. Summary of the invention

[0006] Problems to be solved by the invention

[0007] When a virtual space is provided to a user, the image visually recognized by the user changes suddenly, and thus, the user may feel a sense of discomfort when the image visually recognized by the user changes from the real space to the virtual space.

[0008] An object of the present disclosure is to provide a providing device and a providing method that can reduce the sense of discomfort when providing a virtual space.

[0009] Means used to solve problems

[0010] The providing device disclosed in the present invention includes: a real space information acquisition unit, which acquires position information of real objects in the real space where the user is located in the real space; a virtual space information acquisition unit, which acquires position information of virtual objects existing in the virtual space in the virtual space; a user action determination unit, which determines whether the user has performed a predetermined action on the real object in the real space; and a virtual space providing unit, which matches the position information of the real space with the position information of the virtual space, and when the user action determination unit determines that the user has performed a predetermined action on the real object in the real space, generates a transition image representing the state between the real space and the virtual space, and provides a transition space containing the transition image before providing the virtual space to the user.

[0011] The providing method disclosed herein includes the following steps: obtaining position information of a real object in a real space where a user is located in the real space; obtaining position information of a virtual object existing in a virtual space in the virtual space; determining whether the user has performed a predetermined action on the real object in the real space; and matching the position information of the real space with the position information of the virtual space, and when it is determined that the user has performed a predetermined action on the real object in the real space, generating a transition image representing the state between the real space and the virtual space, and providing a transition space containing the transition image before providing the virtual space to the user.

[0012] Effects of the Invention

[0013] According to the present disclosure, it is possible to reduce the sense of discomfort when providing a virtual space. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a diagram for explaining an overview of a real space and a virtual space according to an embodiment.

[0015] Figure 2 It is a diagram showing a configuration example of a providing system according to an embodiment.

[0016] Figure 3 It is a block diagram showing a configuration example of a providing device according to an embodiment.

[0017] Figure 4 This is a flowchart showing the flow of a virtual space providing process performed by a providing device according to an embodiment.

[0018] Figure 5 This is a flowchart showing the flow of a process for terminating virtual space provision by a provision device according to an embodiment.

[0019] Figure 6 It is a block diagram showing a configuration example of a providing device according to a modification example of the embodiment.

[0020] Figure 7 This is a flowchart showing the flow of a virtual space providing process of a providing device according to a modification of the embodiment. DETAILED DESCRIPTION

[0021] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The present disclosure is not limited to the embodiments, and in the following embodiments, the same reference numerals are given to the same parts to omit repeated descriptions.

[0022] [Implementation Method]

[0023] use Figure 1An overview of the real space and virtual space according to the embodiment will be described. Figure 1 This is a diagram for explaining an overview of a real space and a virtual space according to an embodiment.

[0024] In the present embodiment, the providing device 10 provides a virtual space to the user U. The providing device 10 provides the virtual space to the user U by displaying an image. The providing device 10 is a head mounted display that is worn on the head of the user U, for example.

[0025] exist Figure 1 In the example shown, the real space where the user U actually exists is set as the real space 100, and the virtual space provided by the providing device 10 is set as the virtual space 200. The providing device 10 displays an image for the virtual space 200 according to the action including the line of sight and the movement of the body of the user U in the real space 100. The providing device 10 simulates the situation where the user U acts as the user avatar AU in the virtual space 200, and displays the image for the virtual space 200. In this embodiment, when the user U takes a predetermined action on the real objects such as the window 110 and the table 120 in the real space 100, the providing device 10 displays the image for the virtual space 200 after displaying the transition image between the real space 100 and the virtual space 200. As a result, the user U can recognize that the user U himself exists in the virtual space 200. In this embodiment, the virtual space 200 can be an AR space, a VR space, or an MR (Mixed Reality) space.

[0026] (Provide system)

[0027] use Figure 2 A configuration example of a providing system according to an embodiment of the present invention will be described. Figure 2 It is a diagram showing a configuration example of a providing system according to an embodiment.

[0028] like Figure 2 As shown, the providing system 1 includes a providing device 10 and an information processing device 12. The providing device 10 and the information processing device 12 are connected to each other in a communicable manner via a wired or wireless network N. The network N is, for example, the Internet, but is not limited thereto.

[0029] In this embodiment, the providing system 1 provides the virtual space 200 to the user U. Figure 1In the example shown, the user U wears the providing device 10, and the providing device 10 provides the image of the virtual space 200 in the field of view R of the user avatar AU in the virtual space 200 to the user U as an image for the virtual space 200. In addition, the information processing device 12 can provide the virtual space 200 to a plurality of other users other than the user U. That is, the virtual space 200 can also be a space that can be shared by a plurality of users. The number of users who share the virtual space can be arbitrary.

[0030] The information processing device 12 is a device that performs information processing of information for the virtual space 200. The information processing device 12 can be said to be a server that performs image processing of images for the virtual space 200 by transmitting and receiving information with the providing device 10. For example, the information processing device 12 has a computing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The information processing device 12 is a computer that performs processing by reading a program (software) from the storage device and executing it.

[0031] (Provide device)

[0032] use Figure 3 A configuration example of a providing device according to an embodiment will be described. Figure 3 It is a block diagram showing a configuration example of a providing device according to an embodiment.

[0033] like Figure 3 As shown, the providing device 10 includes an input unit 20 , an output unit 22 , a storage unit 24 , a communication unit 26 , a detection unit 28 , and a control unit 30 .

[0034] The input unit 20 receives various input operations to the providing device 10. The input unit 20 outputs an operation signal corresponding to the received input operation to the control unit 30. The input unit 20 includes, for example, a button, a switch, a microphone, and the like.

[0035] The output unit 22 outputs various information. The output unit 22 includes, for example, a display unit that displays various images. The display unit includes, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, etc. The output unit 22 includes, for example, a speaker that outputs various sounds, an earphone jack, etc.

[0036] The storage unit 24 stores various information. The storage unit 24 stores information such as the calculation contents and programs of the control unit 30. The storage unit 24 includes at least one of a main storage device such as a RAM and a ROM, and an external storage device such as a HDD (Hard Disk Drive).

[0037] The communication unit 26 is a communication module that performs communication between the providing device 10 and an external device. The communication unit 26 performs communication between the providing device 10 and the information processing device 12, for example.

[0038] The detection unit 28 is a sensor that detects various states of the user U in the real space 100 and the surroundings of the providing apparatus 10. The detection unit 28 is a sensor that detects the position of the user U in the real space 100, the posture of the user U, and the surroundings of the providing apparatus 10.

[0039] The detection unit 28 includes, for example, a position sensor, an acceleration sensor, a gyro sensor, a stereo camera, a time of flight sensor, etc. The position sensor detects the position of the user U in the real space. The acceleration sensor detects the movement of the user U in the real space 100. The gyro sensor detects the posture of the user U in the real space 100. The stereo camera and the ToF sensor detect the surroundings of the providing device 10 in the real space. The detection unit 28 is worn on the head, hand, waist, foot, etc. of the user U, for example.

[0040] The detection unit 28 is not limited thereto as long as it can detect the position of the user U in the real space 100, the posture of the user U, and the objects in the real space 100 that exist around the providing device 10. For example, the detection unit 28 can use LiDAR (Light Detection And Ranging) to detect objects that exist in the real space 100. For example, the detection unit 28 can use an image of the real space 100 captured by a camera to detect the position and posture of the user U. The detection unit 28 can also use GPS (Global Positioning System), beacon, etc. as a position sensor.

[0041] The control unit 30 controls each part of the providing device 10. The control unit 30 includes, for example, an information processing device such as a CPU or an MPU and a storage device such as a RAM or a ROM. The control unit 30 executes a program for controlling the operation of the providing device 10 involved in the present disclosure. The control unit 30 may also be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 30 may also be implemented by a combination of hardware and software.

[0042] The control unit 30 includes a real space information acquisition unit 40 , a virtual space information acquisition unit 42 , a position information acquisition unit 44 , a user action determination unit 46 , and a virtual space providing unit 48 .

[0043] The real space information acquisition unit 40 acquires information about real objects existing in the real space 100. The real objects are objects existing in the real space 100. Figure 1 In the example shown, the window 110 and the table 120 are types of real objects. The information related to the real objects includes relative position information indicating the relative position relationship between the user U and the real objects in the real space 100 and absolute position information indicating the absolute position information of the real objects. The information related to the real objects includes information related to the shape of the real objects and information related to the types of the real objects.

[0044] The real space information acquisition unit 40 controls the detection unit 28 to make the detection unit 28 detect objects around the providing device 10. The real space information acquisition unit 40 extracts a plurality of feature points and regions from the objects detected using a known technique based on the detection result of the detection unit 28. The real space information acquisition unit 40 acquires information related to the real object based on the extracted feature points and regions. The method by which the real space information acquisition unit 40 acquires information related to the real object is not limited thereto. For example, information on the position and shape of furniture in the room of the user U is pre-set as information related to the real object, and the real space information acquisition unit 40 can acquire the set information related to the real object.

[0045] The type of the real object that is targeted by the real space information acquisition unit 40 may be any. In the present embodiment, the real space information acquisition unit 40 acquires position information and shape information about an object that does not move in the real space (e.g., the window 110 or the table 120, etc.) as information about the real object.

[0046] The virtual space information acquisition unit 42 acquires information related to virtual objects existing in the virtual space 200. A virtual object is an object existing in the virtual space 200. Figure 1 In the example shown, the virtual window 210 and the virtual table 220 are types of virtual objects. The information related to the virtual objects includes relative position information indicating the relative position relationship between the user avatar AU and the virtual objects in the virtual space 200, and absolute position information indicating the absolute position information of the virtual objects. The information related to the virtual objects includes information related to the shape of the virtual objects and information related to the type of the virtual objects.

[0047] The position information acquisition unit 44 acquires user position information. The user position information is information indicating the position of the user avatar AU in the virtual space 200, which is set based on the information about the real object acquired by the real space information acquisition unit 40 and the information about the virtual object acquired by the virtual space information acquisition unit 42.

[0048] In the present embodiment, the position information acquisition unit 44 sets the user position information by using the position relationship between the virtual object and the user avatar AU in the virtual space 200 calculated based on the position relationship between the real object and the user U. Specifically, the position information acquisition unit 44 sets the user position information so that the position of the virtual object in the virtual space 200 relative to the user avatar AU corresponds to the position of the real object in the real space 100 relative to the user U. In other words, the position of the user avatar AU in the virtual space 200 represented by the user position information is set so that the direction and distance from the virtual object to the user avatar AU correspond to the direction and distance from the real object to the user U in the real space 100.

[0049] The user action determination unit 46 determines whether the user U performs a predetermined action on the real object in the real space 100. For example, the user action determination unit 46 determines the user U's attention to the real object. The attention to the real object is calculated by the time of gazing at the real object, the number of times the real object is repeatedly viewed at a predetermined interval, etc. The longer the time of gazing at the real object and the more times the real object is repeatedly viewed at a predetermined interval, the higher the attention calculated by the user action determination unit 46. The user action determination unit 46 determines whether the user U's attention to the real object exceeds a predetermined threshold. For example, the user action determination unit 46 determines whether the user U covers the real object with his hand. For example, the user action determination unit 46 determines whether the user U covers the real object with his hand when approaching the real object within a predetermined distance. The predetermined action in the real space 100 is not limited to this, and may also include other actions. The predetermined action in the real space 100 can be arbitrarily set in advance by the user U.

[0050] The user action determination unit 46 determines whether the user avatar AU performs a predetermined action on a virtual object in the virtual space 200. The user action determination unit 46 determines, for example, the degree of attention of the user avatar AU to the virtual object. The degree of attention to the virtual object is calculated by the time of gazing at the virtual object, the number of times the virtual object is repeatedly viewed at a predetermined interval, and the like. The longer the time of gazing at the virtual object, and the more times the virtual object is repeatedly viewed at a predetermined interval, the higher the degree of attention calculated by the user action determination unit 46. The user action determination unit 46 determines whether the degree of attention of the user avatar AU to the virtual object exceeds a predetermined threshold. The user action determination unit 46 determines, for example, whether the user avatar AU covers the virtual object with his hand. The user action determination unit 46 determines, for example, whether the user avatar AU covers the virtual object with his hand when approaching the virtual object within a predetermined distance. The predetermined actions in the virtual space 200 are not limited to these, and may also include other actions. The predetermined actions in the virtual space 200 can be arbitrarily set in advance by the user U.

[0051] The virtual space providing unit 48 provides the virtual space 200 to the user U based on the user position information. The virtual space providing unit 48 sets the position indicated by the user position information as the position of the user avatar AU in the virtual space 200, and provides the virtual space 200 to the user U. Specifically, the virtual space providing unit 48 obtains information on the position and orientation of the user U in the real space 100, and converts the position and orientation of the user U in the real space 100 into the viewpoint position and orientation of the user avatar AU indicated by the user position information in the coordinate system of the virtual space 200. The virtual space providing unit 48 causes the output unit 22 to output an image of the virtual space 200 in which the user U visually recognizes the virtual space 200 based on the calculated viewpoint position and posture as an image for the virtual space 200.

[0052] The virtual space providing unit 48 provides the virtual space 200 to the user U based on the determination result of the user action determination unit 46 on the action of the user U. The virtual space providing unit 48 provides the virtual space 200 to the user U when it is determined that the user U has performed a predetermined action. For example, when the user action determination unit 46 determines that the user U's attention to the table 120 exceeds a predetermined threshold, the virtual space providing unit 48 provides the virtual space 200 to the user U. For example, when the user action determination unit 46 determines that the user U's attention to the table 120 exceeds a predetermined threshold and the user U covers the table 120 with his hand, the virtual space providing unit 48 provides the virtual space 200 to the user U.

[0053] In the present embodiment, the position and posture of the user U in the real space 100 are reflected in the viewpoint position and posture of the user avatar AU in the virtual space 200. Therefore, when the user U moves in the real space 100, the viewpoint position and posture of the user avatar AU in the virtual space 200 also move. In this case, it is preferable to establish a correspondence between the movement amount of the user U in the real space 100 and the movement amount of the viewpoint of the user avatar AU in the virtual space 200.

[0054] The virtual space providing unit 48 may display an image representing a real object in the virtual space 200 in a manner superimposed on the virtual object. The virtual space providing unit 48 may display an image representing a real object in the virtual space 200 in a manner superimposed on the virtual object based on the shape information of the virtual object and the shape information of the real object. For example, in a case where the size of the virtual object is larger than the size of the real object, the virtual space providing unit 48 may display an image representing the real object in the virtual space 200 in a manner superimposed on the virtual object. In this case, the output unit 22 may provide an AR space that displays an image of the virtual space 200 while penetrating the real space 100, or may superimpose and display an image of the virtual space 200 and an image representing the real object. In addition, a portion of the virtual object that does not overlap with the real object may be deleted from the image of the virtual space 200, or information may be presented as an area that is not associated with the real object.

[0055] In the present embodiment, the virtual space providing unit 48 provides the user U with a transition space before providing the virtual space 200, and the transition space includes a transition image indicating the state between the real space 100 and the virtual space 200. The virtual space providing unit 48 provides the user U with a transition space including a transition image indicating the state between the real object of the real space 100 and the virtual object of the virtual space 200 before providing the virtual space 200. The transition image is, for example, an image including an image of the real space 100 at a ratio of 50% and an image of the virtual space 200 at a ratio of 50%, but is not limited thereto. For example, the virtual space providing unit 48 superimposes a 3D table having the same size and shape as the table 120 on the table 120 as a transition image through AR and displays it on the output unit 22. In this case, when the user U opens the drawer of the 3D table, the virtual space providing unit 48 gradually enlarges the display area of ​​the virtual space 200, causing an experience of being sucked into the virtual space 200. At this time, the virtual space providing unit 48 not only superimposes the image of the virtual space 200, but also superimposes the sound of the virtual space 200 with the sound of the real space 100 and outputs it through the output unit 22. After the user U moves to the virtual space 200, the virtual space providing unit 48 causes the output unit 22 to output only the image of the virtual space 200 and the sound of the virtual space 200.

[0056] The virtual space providing unit 48 may cause the output unit 22 to display a transition image corresponding to the type of the real object. For example, if the real object is a manhole, the virtual space providing unit 48 may display a transition image that causes an experience that the manhole extends like a soft tube and the body of the user U becomes smaller and enters the tube on the output unit 22. For example, if the real object is a window of a building, the virtual space providing unit 48 may display a transition image that causes an experience that a ladder or an escalator appears and rises toward the window on the output unit 22.

[0057] The virtual space providing unit 48 terminates the provision of the virtual space 200 based on the determination result of the action of the user avatar AU by the user action determination unit 46, and returns the user U to the real space 100. The virtual space providing unit 48 returns the user U to the real space 100 when it is determined that the user avatar AU has performed a predetermined action. The virtual space providing unit 48 provides the user U with a transition image between the virtual space 200 and the real space 100 before returning from the virtual space 200 to the real space 100. For example, when it is determined by the user action determination unit 46 that the degree of attention of the user avatar AU to the virtual table 220 exceeds a predetermined threshold, the virtual space providing unit 48 returns the user U to the real space 100. For example, when it is determined by the user action determination unit 46 that the degree of attention of the user avatar AU to the virtual table 220 exceeds a predetermined threshold and the user avatar AU covers the virtual table 220 with his hand, the virtual space providing unit 48 returns the user U to the real space 100.

[0058] The virtual space providing unit 48 may also display an exit for returning to the real space 100 in the virtual space 200. In this case, for example, when the user action determination unit 46 determines that the user avatar AU's attention to the exit exceeds a predetermined threshold, the virtual space providing unit 48 returns the user U to the real space 100. For example, when the user action determination unit 46 determines that the user avatar AU's attention to the exit exceeds a predetermined threshold and the user avatar AU covers the exit with his hand, the virtual space providing unit 48 returns the user U to the real space 100.

[0059] The virtual space providing unit 48 provides the user U with a transition image between the virtual space 200 and the real space 100 before returning from the virtual space 200 to the real space 100. For example, the virtual space providing unit 48 causes the user U to experience the user U being sucked into the real space 100 by gradually enlarging the display area of ​​the exit to the real space 100. At this time, the virtual space providing unit 48 superimposes not only the image of the real space 100 but also the sound of the real space 100 and the sound of the virtual space 200, and causes the output unit 22 to output the image.

[0060] (Processing of providing device)

[0061] use Figure 4 The processing contents of the providing device according to the embodiment will be described. Figure 4 This is a flowchart showing the flow of a virtual space provision process performed by a provision device according to an embodiment.

[0062] Figure 4 The process shown is a process in which the providing device 10 provides the virtual space 200 to the user. Specifically, Figure 4 The process shows an example in which the user U wears the providing device 10 in a room and watches fireworks in the virtual space 200 .

[0063] The real space information acquisition unit 40 acquires information about real objects existing around the providing device 10 (step S10). Specifically, the real space information acquisition unit 40 acquires information about real objects existing around the providing device 10 detected by the detection unit 28 from the detection unit 28. Then, the process proceeds to step S12.

[0064] The virtual space information acquisition unit 42 acquires information on virtual objects existing in the virtual space (step S12). Specifically, the virtual space information acquisition unit 42 acquires information on virtual objects from the information processing device 12 via the communication unit 26. Then, the process proceeds to step S14.

[0065] The position information acquisition unit 44 acquires user position information indicating the position of the user in the virtual space (step S14 ). Then, the process proceeds to step S16 .

[0066] The user action determination unit 46 determines whether the user U has performed a predetermined action in the real space 100 (step S16). For example, when the user's hand touches the door of the room to enter the room, the user action determination unit 46 determines that the predetermined action has been performed in the real space 100. If it is determined that the user has performed the predetermined action in the real space (step S16: Yes), the process proceeds to step S18. If it is not determined that the user has performed the predetermined action in the real space (step S16: No), the process of step S16 is repeated.

[0067] If the determination result in step S16 is yes, the virtual space providing unit 48 provides the transition space to the user U (step S18). For example, when the user action determining unit 46 determines that the hand of the user U touches the door of the room, the virtual space providing unit 48 causes the output unit 22 to output the sound of fireworks heard from the other side of the door. Then, when the user action determining unit 46 determines that the user U enters the room and turns his / her sight toward the window 110, the virtual space providing unit 48 superimposes a faint image of fireworks on the window 110 and displays it on the output unit 22. Then, the process proceeds to step S20.

[0068] The virtual space providing unit 48 provides the user U with a virtual space (step S20). Specifically, the virtual space providing unit 48 provides the user U with a virtual space in which the position information of the user U in the real space 100 matches the position information of the user avatar AU in the virtual space 200. For example, the virtual space providing unit 48 causes the output unit 22 to output an image of a ladder extending from a virtual window 210 located at a high position in the virtual space 200. For example, when the user action determination unit 46 determines that the user avatar AU holds a ladder, the virtual space providing unit 48 causes the output unit 22 to output an image of the user avatar AU climbing the ladder. At this time, the virtual space providing unit 48 increases the sound of fireworks heard from outside the virtual window 210 as the user climbs the ladder and causes the output unit 22 to output it. For example, when the user action determination unit 46 determines that the user avatar AU has opened the virtual window 210, the virtual space providing unit 48 causes the output unit 22 to output an image of fireworks rising in a clear night scene outside the virtual window 210. At this time, the virtual space providing unit 48 causes the sound of fireworks to be clearly output to the output unit 22. Then, when the user action determination unit 46 determines that the user avatar AU looks away from the virtual window 210 and looks around, for example, the virtual space providing unit 48 causes the output unit 22 to display an image of the user avatar looking around the room from a high position. Figure 4 processing.

[0069] use Figure 5 The processing contents of the providing device according to the embodiment will be described. Figure 5 This is a flowchart showing the flow of processing for ending provision of a virtual space by the provision device according to the embodiment.

[0070] The user action determination unit 46 determines whether the user avatar AU has performed a predetermined action in the virtual space 200 (step S30). For example, when the user avatar AU closes the virtual window 210 and looks at the ladder, the user action determination unit 46 determines that the user avatar AU has performed a predetermined action in the virtual space 200. When it is determined that the user avatar AU has performed a predetermined action in the virtual space 200 (step S30: Yes), the process proceeds to step S32. When it is not determined that the user avatar AU has performed a predetermined action in the virtual space 200 (step S30: No), the process of step S30 is repeated.

[0071] If the determination result in step S30 is yes, the virtual space providing unit 48 provides the transition space to the user U (step S32). For example, when the user action determining unit 46 determines that the user U is holding a ladder, the virtual space providing unit 48 causes the output unit 22 to output an image of the user descending from the ladder. Then, when the user action determining unit 46 determines that the user U has directed his / her sight toward the virtual window 210, the virtual space providing unit 48 hears the sound of fireworks from the virtual window 210, superimposes a faint image of fireworks on the virtual window 210, and displays it on the output unit 22. Then, the process proceeds to step S34.

[0072] The virtual space providing unit 48 ends providing the virtual space 200 to the user (step S34). Specifically, the virtual space providing unit 48 ends outputting the video of the fireworks and the sound of the fireworks, and causes the output unit 22 to stop displaying the virtual space 200. Then, the virtual space providing unit 48 ends. Figure 5 processing.

[0073] As described above, the embodiment provides a transition space indicating the state between the real space and the virtual space before providing the virtual space to the user. Thus, the embodiment can reduce the sense of disharmony caused by the sudden change of the image when switching from the real space to the virtual space when providing the virtual space. In addition, the embodiment can also reduce the sense of disharmony in hearing caused by the sudden change of the sound when switching from the real space to the virtual space when providing the virtual space.

[0074] [Modification of the embodiment]

[0075] use Figure 6 A configuration example of a providing device according to a modification of the embodiment will be described. Figure 6 It is a block diagram showing a configuration example of a providing device according to a modification example of the embodiment.

[0076] like Figure 6 As shown, a device 10A is provided with Figure 3 The illustrated providing device 10 is different in that a control unit 30A includes an event detection unit 50 and a notification unit 52 .

[0077] The event detection unit 50 detects an event occurring in the real space 100 for the user U. The event detection unit 50 detects an event occurring in the real space 100 for the user U while the virtual space providing unit 48 is providing the virtual space 200 to the user U. For example, the event detection unit 50 detects that the user U and a real object may come into contact in the real space 100 as an event.

[0078] For example, the event detection unit 50 detects an event based on the distance between the user U and the real object detected by the detection unit 28. For example, when the distance between the user U and the real object is below a threshold, the event detection unit 50 detects that an event has occurred for the user U. For example, the event detection unit 50 may change the threshold of the distance according to the type of the real object. For example, in the case where the real object is a moving object approaching the user U, the event detection unit 50 may increase the threshold of the distance compared to the case where the real object is a stationary object. For example, in the case where the real object is a moving object, the event detection unit 50 may change the distance threshold based on the speed information of the real object detected by the detection unit 28. For example, the faster the moving speed of the real object, the larger the distance threshold is made by the event detection unit 50.

[0079] The event detection unit 50 may also determine the danger level of the detected event when an event is detected. The danger level refers to the degree of urgency of the notification event. For example, when the user U comes into contact with the detected real object, the event detection unit 50 determines that the danger level increases as the possibility of the user U being injured increases. For example, when the user U comes into contact with the detected real object, the event detection unit 50 determines that the danger level increases as the possibility of the user U being seriously injured increases. The event detection unit 50 may determine the danger level in stages, such as "high", "medium", "low", etc. The event detection unit 50 may determine the danger level by a numerical value, such as "1", "2", "3", etc.

[0080] For example, the event detection unit 50 may determine the danger level according to the type of the detected real object. For example, when the real object is furniture such as a chair, the event detection unit 50 may determine the danger level to be low, and when the real object is a moving vehicle, the event detection unit 50 may determine the danger level to be high.

[0081] For example, the event detection unit 50 may determine the danger level according to the size of the detected real object. For example, the smaller the size of the real object, the lower the danger level determined by the event detection unit 50, and the larger the size of the real object, the higher the danger level determined by the event detection unit 50.

[0082] For example, the event detection unit 50 may determine the danger level according to the detected moving speed of the real object. For example, the lower the moving speed of the real object, the lower the danger level determined by the event detection unit 50, and the higher the moving speed of the real object, the higher the danger level determined by the event detection unit 50.

[0083] When the event detection unit 50 detects an event, the notification unit 52 notifies the user U of the detection of the event. For example, when the event detection unit 50 detects an event, the notification unit 52 controls the output unit 22 to output a notification sound and a notification image, thereby notifying the user U of the detection of the event.

[0084] The notification unit 52 may change the notification method according to the danger level of the event detected by the event detection unit 50. For example, the higher the danger level, the stronger the notification intensity may be. The notification unit 52 may, for example, make the notification sound louder and output it through the output unit 22 as the danger level increases. The notification unit 52 may, for example, make the notification image more conspicuous as the danger level increases.

[0085] (Processing of providing device)

[0086] use Figure 7 The processing contents of the providing device according to the modification of the embodiment will be described. Figure 7 This is a flowchart showing the flow of a virtual space provision process performed by a provision device according to a modification of the embodiment.

[0087] Figure 7 The illustrated processing is a processing executed when the providing apparatus 10A is providing the user U with the virtual space 200 .

[0088] The event detection unit 50 determines whether an event for the user U is detected in the real space 100 (step S40). Specifically, when the distance between the user U and the real object detected by the detection unit 28 is less than a predetermined threshold, the event detection unit 50 detects an event for the user U. When an event for the user U is detected (step S40: Yes), the process proceeds to step S42. When an event for the user U is not detected (step S40: No), the process proceeds to step S46.

[0089] If the result of the determination in step S40 is YES, the event detection unit 50 determines the danger level of the detected event (step S42). Specifically, the event detection unit 50 determines the danger level based on the type and movement speed of the detected real object. Then, the process proceeds to step S44.

[0090] The notification unit 52 performs notification according to the danger level (step S44). Specifically, the notification unit 52 controls the output unit 22 to output a notification sound or a notification image according to the danger level. Then, the process proceeds to step S46.

[0091] The control unit 30A determines whether to terminate the process (step S46). Specifically, when the virtual space providing unit 48 terminates the provision of the virtual space 200 to the user U, the control unit 30A determines that the process is terminated. When it is determined that the process is terminated (step S46: Yes), the process is terminated. Figure 7 If it is not determined to be the end of the process (step S46: No), the process proceeds to step S40.

[0092] In a variation of the embodiment, when a virtual space is being provided to a user, the possibility of contact with a real object in a real space is detected as an event, and when an event is detected, the user is notified of the detection of the event. Thus, the variation of the embodiment can improve the safety of users experiencing the virtual space.

[0093] In addition, in the embodiment and the modification of the embodiment, the providing device 10 and the providing device 10A are described as head mounted displays worn on the head of the user U, but the present disclosure is not limited thereto. The providing device 10 and the providing device 10A may be, for example, a projection device that projects an image onto a real object, a screen, a real space, etc. In this case, the providing device 10 or the providing device 10A may provide a virtual space to the user U by, for example, projecting an image onto the real space, etc.

[0094] The components of the devices shown in the figure are conceptual functions and do not necessarily need to be physically configured as shown in the figure. That is, the specific method of distributing and integrating the devices is not limited to the method shown in the figure, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads, usage conditions, etc. In addition, the structure based on this distribution and integration can also be dynamically performed.

[0095] The embodiments of the present disclosure are described above, but the present disclosure is not limited to the contents of these embodiments. In addition, the above-mentioned constituent elements include elements that can be easily thought of by those skilled in the art, substantially the same elements, and elements of so-called equal scope. Moreover, the above-mentioned constituent elements can be appropriately combined. Moreover, various omissions, substitutions or changes of constituent elements can be made within the scope of the gist of the above-mentioned embodiments.

[0096] Industrial Applicability

[0097] The providing device and providing method of the present disclosure can be utilized in, for example, a head-mounted display.

[0098] Description of symbols

[0099] 1Provide system

[0100] 10.10A provides device

[0101] 12Information processing device

[0102] 20 Input unit

[0103] 22 Output unit

[0104] 24 Storage

[0105] 26 Ministry of Communications

[0106] 28. Inspection Department

[0107] 30, 30A control unit

[0108] 40 Real Space Information Acquisition Department

[0109] 42 Virtual Space Information Acquisition Department

[0110] 44 Position information acquisition unit

[0111] 46 User action determination unit

[0112] 48 Virtual Space Provision Department

[0113] 50 Event Detection Department

[0114] 52 Notification Department.

Claims

1. A providing device, comprising: A real space information acquisition unit, which acquires position information of a real object in the real space where the user is located; A virtual space information acquisition unit, which acquires position information of a virtual object existing in the virtual space in the virtual space; A user action determination unit, for determining whether the user performs a predetermined action on the real object in the real space; as well as The virtual space providing unit matches the position information of the real space with the position information of the virtual space, generates a transition image representing the state between the real space and the virtual space when the user action determination unit determines that the user has performed a predetermined action on the real object in the real space, and provides a transition space including the transition image before providing the virtual space to the user.

2. The providing device according to claim 1, wherein: The virtual space providing unit generates the transition image according to the type of the real object.

3. The providing device according to claim 1 or 2, wherein: The predetermined action includes at least one of an action of looking at the real object and an action of covering the real object with a hand.

4. The providing device according to claim 1 or 2, wherein: The user action determination unit determines whether the user performs a predetermined action on the virtual object in the virtual space. When the user action determination unit determines that the user has performed a predetermined action on the virtual object in the virtual space, the virtual space providing unit provides the transition image before providing the real space to the user.

5. The providing device according to claim 1 or 2, further comprising: a position information acquisition unit, which acquires user position information, wherein the user position information is information indicating the position of the user in the virtual space; The position information acquisition section acquires the user position information so that the position information of the virtual object in the virtual space relative to the user corresponds to the position information of the real object in the real space relative to the user.

6. The providing device according to claim 1 or 2, wherein: The transition image is an image including a portion of the real space and a portion of the image of the virtual space. The virtual space providing unit outputs an image in which the virtual object having the same size and shape as the real object is superimposed on the real object as the transition image.

7. A providing method comprising the following steps: Acquire position information of a real object in the real space where the user is located; Acquiring position information of a virtual object existing in the virtual space in the virtual space; determining whether the user performs a predetermined action on the real object in the real space; as well as The position information of the real space is matched with the position information of the virtual space. When it is determined that the user has performed a predetermined action on the real object in the real space, a transition image representing the state between the real space and the virtual space is generated, and a transition space including the transition image is provided before providing the virtual space to the user.

Images