Virtual reality play area expansion system

Abstract

To provide a system and a method for expanding a play area in virtual reality, capable of expanding the play area in a virtual space without being constrained by the size of a real space and preventing the occurrence of 3D motion sickness.SOLUTION: A head-mounted display 1 includes: a block detection part 2 for detecting a position of a user U on a play area in a real space and interruption information such as blinking and an insert screen; a visual correction part 3 for generating a video obtained by shifting a posture of the user U on the virtual screen by a predetermined amount based on the interruption information detected by the interruption detection part 2 and the position information on the user U in the real space; and an image presentation part 4 for presenting the corrected image of the virtual screen generated by the visual correction part 3 to the user U.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention provides a system for expanding a play area in virtual reality, which allows the expansion of a play area in a virtual space without being restricted by the size of the real space. Mu Regarding. [Background technology]

[0002] In recent years, systems using head mounted displays (HMDs) and virtual reality (VRs) have come into practical use. However, the size of the virtual space seen by a user experiencing virtual reality differs from the size of the real space the user actually exists in. Therefore, in virtual reality, even if the virtual space is large and the user perceives that they can walk in a straight line forever, in reality, the user will bump into the walls of the real room and be blocked. In other words, the range in which the user can actually move within the virtual space is restricted by the real space.

[0003] In addition, the expansion of the play area in real space in HMD is achieved using a technique called Redirected Walking (RDW). RDW utilizes the fact that vision is dominant in spatial perception, and visually presents images that slightly shift the user's position and posture while walking, without the user noticing, to help them find their walking path.

[0004] With regard to expanding the play area in this type of virtual reality, the techniques described in Patent Documents 1 and 2 are known. The display processing device disclosed in Patent Document 1 has a function of displaying a spatial object representing a virtual space to a user on an HMD. For example, in the control unit of this HMD, when the user takes a step from an upright position towards a spatial object and then changes to a forward-leaning position, the display unit performs screen control to move and enlarge the spatial object that was displayed in a reduced size to the viewing position.

[0005] Furthermore, a virtual reality head mounted display (HMD) disclosed in Patent Document 2 includes a screen, a processor, a plurality of inertial sensors, a motion tracker module, and a display adjustment module. The motion tracker tracks movement of the HMD based on inertial data from the plurality of inertial sensors. The display adjuster generates modified display data for image frames scanned onto the screen when movement of the HMD exceeds a movement threshold amount. The display data comprises pixel values ​​scanned sequentially across multiple rows. The modified display data includes adjusted pixel values ​​for a current pixel row of the image frame to compensate for a distance moved by the HMD during the elapsed time between scanning a first pixel row of the image frame and scanning a current pixel row of the image frame. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2021 / 044745 [Patent Document 2] Special Publication No. 2018-503114 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the HMD technology described in the above patent documents cannot expand the play area without the user's movement, so a large space in real space is still required, and there is also the problem that it is prone to causing 3D (three dimensions) sickness when the amount of movement and rotation is changed in real time while the user is walking.

[0008] The present invention has been made in view of the above circumstances, and provides a system for expanding a play area in virtual reality, which allows a play area to be expanded in a virtual space without being restricted by the size of the real space, and which can prevent the occurrence of 3D motion sickness. M provide. [Means for solving the problem]

[0009] In order to solve the above problems, the present invention proposes the following means. In the system for expanding a play area in virtual reality according to a first aspect of the present invention, a head-mounted display includes an interruption detection unit that detects a user's position on the play area in real space and a sneeze by the user; A point where the distance between the user and the play area boundary is maximum within a predetermined angle range to the left and right of the user's direction of travel is detected, and the rotation direction for rotating the image of the virtual space around the user's position is set to the direction in which the user is heading toward the detected point.If multiple points where the distance between the user and the play area boundary is maximum are detected, the point with the smallest angle from the user's direction of travel is used, Sneezing detected by the interruption detection unit 、 and , at the point where it was detected or adopted The system is characterized by having a visual correction unit that generates a corrected image in which the user's posture on the virtual screen is shifted by a predetermined amount based on the visual correction unit, and an image presentation unit that presents the corrected image of the virtual screen generated by the visual correction unit to the user. [Effects of the Invention]

[0011] According to the present invention, an image can be generated and presented in which the user's posture on the virtual screen is shifted by a predetermined amount based on the occlusion information detected by the unit and the user's position information in the real space, thereby enabling the play area to be expanded within the virtual space without being restricted by the size of the real space. [Brief explanation of the drawings]

[0012] [Figure 1] 1 shows the minimum configuration of the virtual reality play area expansion system according to the present invention, where (A) is a schematic diagram, (B) shows the image before correction presented on the head-mounted display, and (C) shows the image after correction presented on the head-mounted display. [Figure 2] 1 is a schematic configuration diagram of a system for expanding a play area in virtual reality according to an embodiment of the present invention. [Figure 3] FIG. 1 is a schematic diagram illustrating the configuration of an HMD-equipped device. [Figure 4] FIG. 2 is a schematic configuration diagram showing a vision correction server. [Figure 5] 10 is a control flowchart when an insertion of an "insert screen" into a virtual screen is detected. [Figure 6] 10 is a control flowchart when the start of a "blink" is detected within the virtual screen. [Figure 7] 7 is a control flowchart specifically showing the process for detecting the start of a "blink" in FIG. 6. [Figure 8] FIG. 10 is an explanatory diagram of a method for calculating a correction angle of a user. [Figure 9] 10 is a control flowchart when a "sneeze" of a user is detected. DETAILED DESCRIPTION OF THE INVENTION

[0013] The minimum configuration of a system 100 for expanding a play space in virtual reality according to the present invention will be described with reference to FIGS. 1(A) to 1(C). FIG. 1(A) shows a head-mounted display 1 according to the present invention, which includes an occlusion detection unit 2, a vision correction unit 3, and an image presentation unit 4.

[0014] The occlusion detection unit 2 detects the position of the user U on the play area in real space, and occlusion information such as blinking and insert screens. The visual correction unit 3 generates a corrected image in which the posture of the user U on the virtual screen is shifted by a predetermined amount based on the occlusion information detected by the occlusion detection unit 2 and the position information of the user U in real space (see Figures 1(B) and (C)). Furthermore, the image presentation unit 4 presents the corrected image of the virtual screen generated by the vision correction unit 3 to the user U. 1B shows the image presented on the head-mounted display 1 before correction, and FIG. 1C shows the image presented on the head-mounted display 1 after correction.

[0015] That is, according to the head-mounted display 1 to which the present invention is applied, the occlusion detection unit 2 can detect the position of the user U on the play area in real space and occlusion information such as blinking and insert screens. Then, the visual correction unit 3 generates an image in which the posture of the user U on the virtual screen is shifted by a predetermined amount based on the occlusion information detected by the occlusion detection unit 2 and the position information of the user U in real space, and the image presentation unit 4 can present the corrected image generated by the visual correction unit 3 to the user U. As a result, the head-mounted display 1 of the present invention can generate and present an image in which the posture of user U on the virtual screen is shifted by a predetermined amount based on the occlusion information detected by the occlusion detection unit 2 and the position information of user U in real space, thereby enabling the play area to be expanded within the virtual space without being restricted by the size of the real space (see Figures 1(B) and (C)). Furthermore, in the present invention, when the screen is blocked by blinking, insertion, or the like, an image is generated in which the posture of the user U on the virtual screen is shifted by a predetermined amount, so that 3D sickness does not occur and the user U can view the virtual screen in good condition.

[0016] (Embodiment) A system 101 for expanding a play area in a real space during a virtual space experience according to an embodiment of the present invention will be described in detail with reference to FIGS. FIG. 2 is a block diagram showing the entire play area expansion system 101, which is composed of an HMD-equipped device 10A and a vision correction server 10B. The HMD-equipped device 10A is provided inside the head-mounted display body, but the location of the vision correction server 10B is not particularly limited. The HMD-equipped device 10A and the vision correction server 10B are connected to each other so that they can communicate with each other.

[0017] The HMD-equipped device 10A includes an insertion detection device 11, a blink detection device 12, a position detection device 13, a control device 14, and a communication device 15, as shown in FIG. The insert detection device 11 transmits the timing at which the insert screen is detected in the virtual space to the control device 14. The blink detection device 12 transmits the timing at which it confirms that the user has blinked to the control device 14 . The position detection device 13 transmits the position and direction of the user in the real space to the control device 14 . The control device 14 transmits the user's position and angle information at the time when visual interruption occurs, such as blinking or screen insertion, to the vision correction server via the communication device 15. The communication device 15 has a function of transmitting the visual occlusion information and position information associated by the control device 14 to the vision correction server 10B, and a function of reflecting the angle adjustment information received from the vision correction server on the screen of the HMD.

[0018] The vision correction server 10B includes a control device 21 and a communication device 22 as shown in FIG. The control device 21 calculates the amount of change in the position and posture of the user using the visual occlusion information and position information received from the HMD-equipped device 10A. The communication device 22 receives visual obstruction information and position information from the HMD-equipped device 10A, and transmits position and posture change amount information to the HMD-equipped device 10A.

[0019] Next, the control content of the play area expansion system 101 according to this embodiment will be described with reference to the flowcharts of FIGS. The processes in these flowcharts are executed by the control device 14 of the HMD-equipped device 10A and the control device 21 of the vision correction server 10B. In the flowcharts of Figures 5 and 6, steps S102 and S202 constitute the "occlusion detection unit" of the present invention, steps S103, S104, S203, and S204 constitute the "vision correction unit" of the present invention, and steps S105 and S205 constitute the "image presentation unit" of the present invention.

[0020] Referring to FIG. 5, a control flowchart when the insertion of an "insert screen" into a virtual screen is detected will be described.

[0021] [Steps S101 to S102] When the experience of the virtual space begins (step S101), the insert detection device 11 determines whether the insertion of an insert screen has begun (step S102), and if YES, the insert data is sent to the control device 14, and then the process proceeds to the next step S103.

[0022] [Step S103] The position detection device 13 detects the position and direction of the user in the play area in real space, and transmits the position data to the control device 14 . Thereafter, the communication device 15 is used to transmit information regarding the timing, angle, and position of the screen insert from the control device 14 to the vision correction server 10B.

[0023] [Step S104] Next, the control device 21 calculates a correction angle adapted to the user's line of sight based on the user's line of sight obtained by a detection device (not shown).

[0024] [Step S105] Next, the communication device 22 of the vision correction server 10B transmits the correction angle obtained in step S104 to the communication device 15 of the HMD-equipped device 10A. Thereafter, the control device 14 of the HMD-equipped device 10A executes the angle correction in the virtual space below the insert screen.

[0025] Note that, in this case, control device 14 of HMD-equipped device 10A immediately performs angle correction if the screen occupancy rate of the insert screen obtained in step S102 is 40% or more, but if it is less than 40%, the screen is changed in stages every 100 ms by a value obtained by dividing the correction angle by 100. The speed of the angle change can be freely set within a range that does not cause discomfort.

[0026] [Steps S106 to S107] After step S105 is executed and the insert screen is finished (step S106), the above-described series of operations continues until the virtual space experience is finished (step S107).

[0027] In the flowchart of Figure 5 above, in step S102, the insert detection device 11 determines whether or not insertion of the insert image has begun. However, instead of this process, it is preferable to perform angle correction at the timing when a blink occurs in step S202 of the flowchart shown in Figure 6. Specifically, the trigger for starting the system is the detection of an insert screen in step S102 of the flowchart shown in FIG. 5, but is changed to the detection of the start of a blink in step S202 of the flowchart shown in FIG.

[0028] Next, the process (step S202) performed by blink detection device 12 in FIG. 3 to detect the start of a blink will be described with reference to the flowchart shown in FIG.

[0029] [Steps S301 to S302] When the experience of the virtual space begins (step S301), the initial pupil size is measured and stored (step S302).

[0030] [Step S303] Based on the measurement result in step S302, it is determined whether the pupil size has become 50% or less, and if YES, the process proceeds to the next step S304.

[0031] [Step S304] The blink detection device 12 detects the start of blinking, and the detected data is supplied to step S202 of the flowchart shown in FIG. 6, and this operation is repeated until the virtual space experience ends (step S305).

[0032] Next, a method for the control device 21 to calculate the user's correction angle shown in steps S104 and S204 of the flowcharts of FIGS. 5 and 6 will be described with reference to FIG. 8 (processing in steps S104 and S204). As shown in Figure 8, an angle of X degrees is taken to the left and right of the user's direction of travel m1, and within that range, the point at which the distance between the user and the play area boundary is greatest is found. The control device 21 determines that the angle and rotation direction that form the maximum distance direction m2 from the current traveling direction m1 is the X value that is the amount of change in the line of sight angle. This X value is set to 45 degrees when visual interruption occurs due to an insert screen, and to 30 degrees when visual interruption occurs due to blinking. However, this X value can be freely set within a range that does not cause discomfort. If there are directions with the same maximum distance, the direction with the smaller angle from the direction of travel will be used.If there are maximum distances at the same angle on both the left and right, the range up to 180 degrees on the left and right will be looked at, and the direction of rotation with the maximum distance will be used.

[0033] According to the head-mounted display of this embodiment described in detail above, the occlusion detection unit (steps S102, S202) can detect the user's position on the play area in real space and occlusion information such as blinking and insert screens. Thereafter, the visual correction unit (steps S103-S104, S203-S204) generates an image in which the user's posture on the virtual screen is shifted by a predetermined amount based on the occlusion information detected by the occlusion detection unit and the user's position information in real space, and the image presentation unit (steps S105, S205) presents the corrected image generated by the visual correction unit to the user.

[0034] As a result, the head-mounted display of this embodiment can generate and present an image in which the user's posture on the virtual screen is shifted by a predetermined amount based on the occlusion information detected by the occlusion detection unit and the user's position information in real space, thereby making it possible to expand the play area within the virtual space without being restricted by the size of the real space. In addition, in this embodiment, by performing angle correction when the field of view is blocked in the virtual space, it is possible to expand the play area in the real space without moving from the location. Furthermore, in this embodiment, when the screen is blocked by blinking, insertion, or the like, an image is generated in which the user's posture on the virtual screen is shifted by a predetermined amount, so that 3D sickness does not occur and the user can view the virtual screen in good condition.

[0035] (Other embodiments) In the above embodiment, an insert screen or blinking is used as a trigger for performing angle correction (see step S102 in FIG. 5 and step S202 in FIG. 6), but the timing of a sneeze may also be used, as shown in step S402 in FIG. 9. When sneezing, the eyes are closed and the neck angle moves significantly, so angle correction can be performed at a larger angle than usual. Therefore, when calculating the angle in step S404, it is recommended to take angles of 60 degrees to the left and right of the user's direction of movement and find the longest distance. By adopting this method, the opportunities for angle correction in virtual space will increase.

[0036] Although the embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of the present invention. [Industrial Applicability]

[0037] The present invention relates to a system and method for expanding a play area in virtual reality, which allows the expansion of a play area within a virtual space without being restricted by the size of the real space. [Explanation of symbols]

[0038] 1. Head-mounted display 2. Interruption detection unit 3. Vision correction unit 4 Image presentation section 10A HMD-equipped device 10B Vision Correction Server 11 Insert detection device 12 Blink detection device 13 Position detection device 14 Control device 15. Communications equipment 21 Control device 22 Communication equipment 100 Expansion System 101 Expansion System U User

Claims

1. an interruption detection unit in the head mounted display that detects a position of a user on a play area in real space and a sneeze by the user; a visual correction unit that detects the point at which the distance between the user and the play area boundary is greatest within a predetermined angle range to the left and right of the user's direction of travel, rotates the image of the virtual space around the user's position in the direction in which the user is heading toward the detected point, and if multiple points at which the distance between the user and the play area boundary is greatest are detected, adopts the point with the smaller angle from the user's direction of travel, and generates a corrected image in which the user's posture on the virtual screen is shifted by a predetermined amount based on the sneeze detected by the occlusion detection unit and the detected or adopted point; A system for expanding a play area in virtual reality, characterized by having an image presentation unit that presents the corrected image of the virtual screen generated by this vision correction unit to the user.

2. The system for expanding a play area in virtual reality as described in claim 1, characterized in that when the occlusion detection unit detects a sneeze, the visual correction unit generates a corrected image by rotating the image of the virtual space around the user's position at a larger angle than when the occlusion detection unit does not detect a sneeze.

3. The system for expanding a play area in virtual reality as described in claim 1 or claim 2, characterized in that when the visual correction unit detects a point at which the distance between the user and the play area boundary is greatest at the same angle to the left and right of the user's direction of travel, it refers to a range of up to 180 degrees to the left and right, and adopts the direction in which the point at which the distance between the user and the play area boundary is greatest as the rotation direction of the image in the virtual space.

Images