Virtual reality control method to avoid motion sickness

Abstract

This case discloses a method for controlling virtual reality. In the present invention, when the VR system receives a displacement control instruction from a VR user, it further detects the shaking direction of the VR user's head-mounted device, and then determines whether the displacement control instruction matches the shaking direction according to an instruction matching rule. If they match, the VR system will execute the displacement control instruction; if they do not match, the VR system will not execute the displacement control instruction. The present invention ensures that before the VR scene changes, the VR user's head has already reacted in advance to stimulate the vestibular system of the inner ear, thereby avoiding the occurrence of motion sickness.

Description

Technical Field

[0001] This case is about a virtual reality control method that allows VR users to perform displacement operations in the VR world. Background Art

[0002] Virtual reality (hereinafter referred to as VR) is a three-dimensional virtual world created by computer modules. As early as 1950, scholars proposed a similar concept. With the advancement of science and technology, VR technology has gradually been developed in recent years, and 2016 is even called the first year of VR.

[0003] Generally speaking, VR users experience VR technology through VR systems. VR systems usually include a head-mounted device and a controller. Some VR systems also include a somatosensory detection device that can be used to detect the body movements of the VR operator. When experiencing VR technology, VR users will wear a head-mounted device to view the VR scenes in the VR world, and hold a controller in their hands to control the VR user's movements in the VR world.

[0004] The head mounted device is equipped with a gyroscope and a g-sensor to detect the displacement of the head of the VR user. The controller can be a game controller, a remote controller, a keyboard, a mouse, a mobile phone or any combination of the above controllers.

[0005] The so-called movement in the VR world refers to the movement of objects in the VR scene seen by the VR user through the head-mounted device. For example, if the VR user issues a "forward" command through the controller, the objects in the VR scene will gradually approach the VR user, and the VR user will feel that he is moving forward; if the VR user issues a "backward" command through the controller, the objects in the VR scene will gradually move away from the VR user, and the VR user will feel that he is moving backward.

[0006] The above principle produces a visual effect on the human body through the movement of objects in the VR scene, making the VR user feel that he is moving in the VR world.

[0007] Although VR technology has gradually matured, some VR users still experience motion sickness symptoms such as dizziness, nausea, fatigue and blurred vision when experiencing VR technology. The reason is that when the movement felt by the human body does not match the visual image, the brain will make errors in judging the body's balance and spatial orientation, which will manifest as motion sickness.

[0008] In order to solve the above problems, some developers use control methods such as hand pulling, arm swinging, and pushing hamster balls to allow VR users to move slowly in the VR world to reduce the symptoms of motion sickness. However, the speed of these movement methods is too slow and does not match the movement of VR users in the real world.

[0009] In addition, some developers use teleportation or instant movement to allow VR users to move in the VR world. By changing the screen instantly, the visual stimulation of VR users is reduced, thereby avoiding nausea. However, the above method does not conform to the movement of VR users in the real world, and therefore reduces the immersive experience of VR users.

[0010] Some developers have VR users move around in the VR world by riding in a vehicle, so that they feel like they are sitting in a car, an animal, or a spaceship to reduce dizziness. However, this control method still produces symptoms similar to motion sickness or seasickness.

[0011] US10890976B2 discloses a VR platform that uses pressure to control motion feedback. The VR platform senses the actual pressure movement direction of the VR user and transmits the direction information to a PC. The PC calculates and transmits a physical motion feedback to the platform, causing the platform to move or vibrate, thereby allowing the VR user to have a more realistic experience in the VR world. Although the invention of US10890976B2 can alleviate the effects of motion sickness, the invention is inconvenient to use. VR users must stand on a gravity sensing device to operate, and the movement range of VR users is also limited to the range of the gravity sensing device, wherein the gravity sensing device must also occupy some space, which also causes the user's home space to be squeezed, thereby reducing popularity.

[0012] US2019 / 0204909A1 discloses a method for providing realistic and anti-motion sickness in a VR world. Paragraphs 0018-0019 and 0024-0079 of the specification specifically disclose that in a VR system in which views are coupled to rotations, when a user rotates using a joystick / game controller button, the view displayed to the user also rotates together, which can cause motion sickness in the user. When a user navigates in a virtual world, motion measurements are acquired in real time through the head HMD detection mechanism, and the acquired motion measurements are used to determine whether the user's activities are intentional. When it is determined that the user's movements are intentional, the measured movements are converted into movements for VR in real time. However, US2019 / 0204909A1 only proposes a method of "how to capture user limb movement signals through multiple motion sensors, analyze them, and then determine whether the user's limb movements are intentional for VR operations or are just unintentional movements" to avoid motion sickness symptoms. However, everyone's body movement habits are different, and this analysis method cannot accurately determine whether the user's movements are intended to engage in VR operations. Summary of the invention

[0013] In view of this, the object of the present invention is to provide a virtual reality control method. The method comprises the following steps: receiving a displacement control instruction from a VR user, the displacement control instruction being selected from a group consisting of forward, backward, left, right, stop, up, down and a combination thereof; detecting a shaking direction of a shaking of the head of the VR user; and determining whether the displacement control instruction matches the shaking direction of the shaking according to an instruction matching rule; wherein, when the displacement control instruction matches the shaking direction of the shaking, the displacement control instruction is executed, and when the displacement control instruction does not match the shaking direction of the shaking, the displacement control instruction is not executed.

[0014] In some embodiments of the aforementioned virtual reality control method, the step of detecting the shaking direction is performed within a preset time after receiving the displacement control instruction.

[0015] In some embodiments of the above-mentioned virtual reality control method, the step of executing the displacement control instruction is to control the object in the VR scene to move according to the direction of the displacement control instruction.

[0016] In some embodiments of the above-mentioned virtual reality control method, the step of executing the displacement control instruction is to control the object in the VR scene to move in a direction opposite to the displacement control instruction.

[0017] In some embodiments of the above-mentioned virtual reality control method, it further includes: determining whether a shaking displacement of the shaking is greater than a displacement threshold; when the shaking displacement of the shaking is greater than the displacement threshold, determining whether the displacement control instruction matches the shaking direction of the shaking according to the instruction matching rule.

[0018] In some embodiments of the above-mentioned virtual reality control method, it further includes: determining whether a shaking acceleration of the shaking is greater than an acceleration threshold; when the shaking acceleration of the shaking is greater than the acceleration threshold, determining whether the displacement control instruction matches the shaking direction of the shaking according to the instruction matching rule.

[0019] In some embodiments of the above-mentioned virtual reality control method, when the shaking acceleration of the shaking is greater than the acceleration threshold, the moving speed of the VR scene increases.

[0020] In some embodiments of the above-mentioned virtual reality control method, it further includes: judging whether a shaking displacement of the shaking is greater than a displacement threshold and judging whether a shaking acceleration of the shaking is greater than an acceleration threshold; when the shaking displacement of the shaking is greater than the displacement threshold and the shaking acceleration of the shaking is greater than the acceleration threshold, judging whether the displacement control instruction matches the shaking direction of the shaking according to the instruction matching rule.

[0021] In some embodiments of the above virtual reality control method, a head mounted device is used to detect the shaking of the VR user's head.

[0022] In some embodiments of the above-mentioned virtual reality control method, the shaking of the head of the VR user is detected by a sense of unity detection device.

[0023] In some embodiments of the aforementioned virtual reality control method, the displacement control instruction is generated by the VR user operating a controller.

[0024] In some embodiments of the aforementioned virtual reality control method, the displacement control instruction is generated by the VR system detecting a body movement of the VR user.

[0025] In some embodiments of the aforementioned virtual reality control method, the body motion is selected from a group consisting of a hand motion, a foot motion, a body posture, and a combination thereof.

[0026] In some embodiments of the aforementioned virtual reality control method, the displacement control instruction is generated by the VR system detecting a brain wave signal of the VR user.

[0027] In some embodiments of the aforementioned virtual reality control method, the displacement control instruction is generated by the VR system detecting a voice instruction of the VR user.

[0028] In some embodiments of the aforementioned virtual reality control method, the displacement control instruction is generated by the VR system detecting a displacement trajectory of a marker held by the VR user.

[0029] In some embodiments of the aforementioned virtual reality control method, the displacement control instruction is generated when the VR system detects a shape change of a marker held by the VR user.

[0030] In summary, one of the key technical means of this case is to determine whether the displacement control command issued by the VR user matches the shaking direction of the VR user's head according to a command matching rule. If the two match each other, the displacement control command is executed, and if they do not match each other, the displacement control command is not executed. The virtual reality control method of this case is based on the principle of the human body's natural vestibular system, combining a head shaking action used to stimulate the vestibular system with the operation of the VR system, thereby perfectly achieving the effect of avoiding motion sickness.

[0031] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of a virtual reality control method in one embodiment of the present invention;

[0033] Figure 2 is a schematic diagram (1) of a VR user performing a forward operation in a VR world in an embodiment of the present invention. At this time, the VR user only presses the "forward" button, and the head of the VR user does not shake. Therefore, the displacement control instruction does not match the shaking direction, and the VR system controls the objects in the VR scene to maintain the original moving operation state;

[0034] Figure 3 is a schematic diagram (2) of a VR user performing a forward movement operation in a VR world in one embodiment of the present invention. At this time, the displacement control instruction does not match the shaking direction, so the VR system only controls the objects in the VR scene to swing, making the VR user visually feel that he is shaking his head in the VR world;

[0035] Figure 4 is a schematic diagram (iii) of a VR user performing a forward operation in a VR world in one embodiment of the present invention, where the displacement control instruction matches the shaking direction, and the VR system controls the objects in the VR scene to gradually approach;

[0036] Figure 5is a schematic diagram (1) of a VR user performing a left movement operation in a VR world in one embodiment of the present invention. At this time, the VR user only presses the "left" button, and the VR user's head does not shake. Therefore, the displacement control instruction does not match the shaking direction, and the VR system controls the objects in the VR scene to maintain the original movement operation state;

[0037] Figure 6 is a schematic diagram (2) of a VR user performing a left shift operation in a VR world in one embodiment of the present invention. At this time, the displacement control instruction does not match the shaking direction, so the VR system only controls the objects in the VR scene to swing, making the VR user visually feel that he is shaking his head in the VR world;

[0038] Figure 7 is a schematic diagram (iii) of a VR user performing a left movement operation in a VR world in one embodiment of the present invention. At this time, the displacement control instruction matches the shaking direction, so the VR system controls the objects in the VR scene to gradually move to the right, so that the VR user visually feels that he is moving to the left in the VR world;

[0039] Figure 8 is a schematic diagram (1) of a VR user performing a jump / up operation in a VR world in one embodiment of the present invention. At this time, the VR user only presses the "jump / up" button, and the VR user's head does not shake. Therefore, the displacement control instruction does not match the shaking direction, and the VR system controls the objects in the VR scene to maintain the original moving operation state;

[0040] Fig. 9 is a schematic diagram (2) of a VR user performing a jump / upward operation in a VR world in one embodiment of the present invention. At this time, the displacement control instruction does not match the shaking direction, so the VR system only controls the objects in the VR scene to swing, making the VR user visually feel that he is shaking his head in the VR world;

[0041] Fig.10 is a schematic diagram (iii) of a VR user performing a jump / upward operation in a VR world in one embodiment of the present invention, where the displacement control instruction matches the shaking direction, and the VR system controls the object in the VR scene to move downward;

[0042] Fig.11 is a schematic diagram (1) of a VR user executing a stop forward movement in a VR world in an embodiment of the present invention. At this time, the VR user only releases the pressed "forward" button, and the head of the VR user does not shake. Therefore, the displacement control instruction does not match the shaking direction, and the VR system controls the objects in the VR scene to maintain the original moving operation state;

[0043] Fig.12is a schematic diagram (2) of a VR user executing a stop forward movement in a VR world in an embodiment of the present invention. At this time, the displacement control instruction does not match the shaking direction, so the VR system only controls the objects in the VR scene to swing, so that the VR user visually feels that he is shaking his head in the VR world;

[0044] Fig.13 is a schematic diagram (iii) of a VR user executing a stop forward movement in a VR world in one embodiment of the present invention, where the displacement control instruction matches the shaking direction, and the VR system controls the objects in the VR scene to stop moving;

[0045] Fig.14 is a schematic diagram of instruction matching rules in one embodiment of the present invention;

[0046] Fig.15A is a schematic diagram of hand movements of a VR user in one embodiment of the present invention (I);

[0047] Fig. 15B is a schematic diagram of hand movements of a VR user in one embodiment of the present invention (II);

[0048] Fig. 15C is a schematic diagram of hand movements of a VR user in one embodiment of the present invention (III);

[0049] Fig.15D is a schematic diagram (IV) of hand movements of a VR user in one embodiment of the present invention;

[0050] Fig.15E is a schematic diagram of hand movements of a VR user in one embodiment of the present invention (V);

[0051] Fig.15F is a schematic diagram (VI) of hand movements of a VR user in one embodiment of the present invention;

[0052] Fig.16A is a schematic diagram of foot movements of a VR user in one embodiment of the present invention (I);

[0053] Fig. 16B is a schematic diagram of foot movements of a VR user in one embodiment of the present invention (II);

[0054] Fig. 16C is a schematic diagram of foot movements of a VR user in one embodiment of the present invention (III);

[0055] Fig.16D is a schematic diagram (IV) of the foot movements of a VR user in one embodiment of the present invention;

[0056] Fig.16Eis a schematic diagram of foot movements of a VR user in one embodiment of the present invention (V);

[0057] Fig.16F is a schematic diagram (VI) of the foot movements of a VR user in one embodiment of the present invention;

[0058] Fig.17A is a schematic diagram (I) of the displacement trajectory of a VR user holding a marker in one embodiment of the present invention;

[0059] Fig. 17B is a schematic diagram (II) of the displacement trajectory of a handheld marker by a VR user in one embodiment of the present invention;

[0060] Fig. 17C is a schematic diagram (III) of the displacement trajectory of a handheld marker of a VR user in one embodiment of the present invention;

[0061] Fig.17D is a schematic diagram (iv) of a displacement trajectory of a handheld marker by a VR user in one embodiment of the present invention;

[0062] Fig.17E is a schematic diagram (V) of the displacement trajectory of a VR user holding a marker in one embodiment of the present invention;

[0063] Fig.17F FIG6 is a schematic diagram of the displacement trajectory of a VR user holding a marker in one embodiment of the present invention. ...

[0064] Among them, the reference numerals

[0065] 1: Head

[0066] 2: Head-mounted device

[0067] 3: Controller

[0068] 31: Button

[0069] 32: Button

[0070] 33: Button

[0071] 4: VR scene

[0072] 41:Object

[0073] 50: Instruction matching rules / instruction matching table

[0074] 6: VR users

[0075] 7: Markers

[0076] S10~S50: Steps DETAILED DESCRIPTION

[0077] The structural principle and working principle of the present invention are described in detail below in conjunction with the accompanying drawings:

[0078] The vestibular system is an important organ of balance in the human body, which includes three semicircular canals (lateral, superior and posterior canals) that can sense the acceleration of the head in three-dimensional space. When the human body is performing variable-speed movements in various directions, the vestibular system will be stimulated and provide information to the brain, so that the visual response of the human body can keep up with the actual movement. However, when the human body is performing constant-speed movements, the vestibular system will not play a role.

[0079] For example, when we ride a bicycle, we will feel the forward movement at the start (i.e., speed change). At this time, the vestibular system of the human body will be stimulated by the speed change, allowing the human body to adapt to the visual movement of the picture. When we continue to move forward at a constant speed in the direction of inertial motion, although the vestibular system is no longer stimulated, the human body can still adapt to the visual movement of the picture without the symptoms of motion sickness.

[0080] Reference Figures 1 to 4 ,in Figure 1 A flowchart of an exemplary virtual reality control method of the present invention is shown. Figures 2 to 4 Schematic diagrams (i) to (iii) are respectively drawn showing a VR user performing a forward operation in a VR world. In this embodiment, a VR user executes a VR program through a VR system, wherein the VR system includes a head-mounted device 2 and a controller 3. The controller may be a game handle, a remote controller, a keyboard, a mouse, a pair of controllers (left / right), a touchpad, a displacement sensor equipped with a gyroscope and a gravity accelerometer sensor or equipped with a camera that can detect spatial image changes and calculate displacement, a mobile phone, or any combination of the above controllers. The processor used to execute the VR program may be directly set in the head-mounted device, or may be separately set in a computer host. When the processor used to execute the VR program is separately set in a computer host, the computer host is also considered to be part of the VR system. The computer host may be a smart phone, a personal computer, a remote server, or a commercially available game console, such as a Nintendo Switch console, a Sony PS console, and a Microsoft Xbox console.

[0081] In some embodiments, the displacement control instruction is generated by the VR user operating the controller 3. Figure 2As shown, the VR user's hand holds the controller 3, the head 1 is equipped with the head-mounted device 2, the eyes see the VR scene 4 through the head-mounted device 2, and the VR scene 4 includes an object 41. When the VR user presses the button 31 on the controller 3 and issues a displacement control instruction of "forward", the VR system will not execute the displacement control instruction when it receives the displacement control instruction of "forward" (step S10), that is, the VR user remains in the same place in the VR world (original moving operation state). The so-called original moving operation state means that if the VR user is in a stationary state before pressing the button 31 on the controller 3, the original moving operation state is the stationary state; if the VR user is in a moving state, such as a forward moving state, before pressing the button 31 on the controller 3, the original moving operation state is the forward moving state.

[0082] Next, within a preset time after the VR user issues the “forward” displacement control command, the head mounted device 2 detects the shaking direction of the VR user’s head 1 (step S20), and the VR system then controls the VR user’s movement direction according to the following steps: Fig.14 The command matching rule 50 (hereinafter referred to as the command matching table 50) shown determines whether the above-mentioned displacement control command matches the above-mentioned shaking direction (step S30), wherein the preset time may be 1 second or less than 1 second. The command matching table 50 may be pre-stored in the head-mounted device 2 or the computer host, or loaded into the head-mounted device 2 or the computer host along with the execution of the VR program. It should be particularly noted that although the head-mounted device 2 is used in this embodiment to detect the shaking direction of the VR user's head 1, the detection of the shaking direction may also be achieved through a somatosensory detection device. In addition, in some embodiments, the head-mounted device 2 may detect the shaking direction of the VR user's head 1 within a duration of the VR user issuing a "forward" displacement control command, wherein the duration is the length of time that the VR user continues to press the button 31 on the controller 3.

[0083] like Figure 3 As shown, when the user issues a "forward" displacement control instruction through the controller 3, but the shaking direction of the head 1 is to the left, the VR system judges it as "mismatch" according to the instruction matching table 50. Therefore, the VR system does not execute the "forward" displacement control instruction issued by the VR user (step S40), and only controls the object 41 in the VR scene 4 to swing to the right (equivalent to the VR user's head 1 shaking to the left in the VR world).

[0084] like Figure 4As shown, when the user issues a "forward" displacement control instruction and the shaking direction of the head 1 is forward, the VR system determines it as a "match" according to the instruction matching table 50, so the VR system will execute the "forward" displacement control instruction issued by the VR user (step S50) to control the object 41 in the VR scene 4 to gradually approach the VR user (equivalent to the VR user moving forward in the VR world).

[0085] In this embodiment, when the VR user simply issues a displacement control instruction through the controller 3, it cannot directly cause a corresponding change in the VR world. Only when the movement operation of the displacement control instruction matches the shaking direction of the VR user's head 1, the VR system will execute the displacement control instruction issued by the controller 3 and make the VR scene 4 change accordingly. In this way, it can be ensured that before the VR scene 4 changes, the VR user's head 1 has already reacted in advance and stimulated the vestibular system of the inner ear, thereby avoiding the occurrence of motion sickness.

[0086] In some embodiments, VR users can add or reduce movement operations (such as climbing, flying, lying down, rolling, etc.) in the command matching table 50 according to the needs of different scenarios, and can customize the matching conditions of each movement operation.

[0087] In some embodiments, in addition to detecting the shaking direction of the VR user's head 1, the head-mounted device 2 can further detect the shaking displacement of the VR user's head 1, and only when the shaking displacement is greater than a displacement threshold, the VR system begins to determine whether the displacement control instruction matches the shaking direction according to the instruction matching table 50. The so-called shaking displacement refers to the shaking amplitude of the VR user's head 1. The larger the shaking amplitude of the VR user's head 1, the larger the shaking displacement; the smaller the shaking amplitude of the VR user's head 1, the smaller the shaking displacement. In some embodiments, the detection of the above-mentioned shaking displacement can also be achieved through a somatosensory detection device.

[0088] The function of the above displacement threshold is to filter out the subtle shaking naturally produced by the human body. Even if a person deliberately keeps his limbs still, there will still be some subtle shaking. The shaking displacement of these shaking can be regarded as background noise and filtered out to avoid misjudgment by the VR system. In some embodiments, the displacement threshold can be a fixed value or can be customized by the VR user. In some embodiments, the displacement threshold can be learned by machine learning to learn the degree of subtle shaking naturally produced by the current VR user, and then adaptively adjusted.

[0089] In some embodiments, in addition to detecting the shaking direction of the VR user's head 1, the head-mounted device 2 can further detect the shaking acceleration of the VR user's head 1, and only when the shaking acceleration is greater than an acceleration threshold, the VR system begins to determine whether the displacement control instruction matches the shaking direction according to the instruction matching table 50. The so-called shaking acceleration refers to the change in the shaking speed of the VR user's head 1. If the shaking speed of the VR user's head 1 changes faster, the shaking acceleration is greater; if the shaking speed of the VR user's head 1 changes slower, the shaking acceleration is smaller. In some embodiments, the detection of the above-mentioned shaking acceleration can also be achieved through a somatosensory detection device.

[0090] The function of the above acceleration threshold is also to filter out the subtle shaking that occurs naturally in the human body. Even if a person deliberately keeps his limbs still, there will still be some subtle shaking, but the speed of these shaking is not fast and there is no obvious speed change. By setting the acceleration threshold, the unintentional shaking of the VR user's head 1 can be regarded as background noise and filtered out to avoid misjudgment by the VR system.

[0091] In some embodiments, another function of the above acceleration threshold is to assist the VR user in performing movement operations in the VR world. That is, when the movement operation of the displacement control instruction issued by the VR user through the controller 3 matches the shaking direction of the VR user's head 1, the VR system can change the movement speed of the VR user in the VR world according to the shaking acceleration signal. Taking the "forward" movement operation as an example, when the shaking acceleration is greater than the acceleration threshold, the VR system controls the speed at which the object 41 in the VR scene 4 approaches the VR user (that is, the VR user's forward movement speed in the VR world increases).

[0092] In some embodiments, the head-mounted device 2 of the VR system simultaneously detects the shaking displacement and shaking acceleration of the VR user's head 1, and determines whether the shaking displacement and shaking acceleration are respectively greater than the displacement threshold and the acceleration threshold, and only when the shaking displacement and shaking acceleration are respectively greater than the displacement threshold and the acceleration threshold, the VR system begins to determine whether the displacement control instruction matches the shaking direction according to the instruction matching table.

[0093] In addition to pressing the button 31, the control method for generating the "forward" displacement control command can also be to push the joystick on the controller 3 "forward", roll the roller "forward", slide the touch panel "forward" or swing the controller 3 itself "forward".

[0094] Affected by the inertia of the human body, when the head 1 of the VR user shakes in one direction, the head 1 will immediately shake in the opposite direction to return to the original limb state, that is, the shaking of the head 1 is a reciprocating displacement action. Therefore, the matching condition of the above-mentioned "forward" movement operation can be whether the shaking direction of the head 1 of the VR user is "forward" or "backward".

[0095] Similarly, the above-mentioned "forward" displacement operation can also be applied to the "backward" displacement control instruction by analogy.

[0096] like Figure 5 As shown in FIG. 1 , when the VR user presses the button 33 on the controller 3 to issue a displacement control command of "move left", the VR system will not execute the displacement control command at this time, that is, the VR user remains in the original position in the VR world (original moving operation state). Next, the head mounted device 2 detects the shaking direction of the VR user's head 1, and then the VR system responds accordingly. Fig.14 The command matching table 50 shown determines whether the above displacement control command matches the above shaking direction.

[0097] like Figure 6 As shown, when the user issues a "move left" displacement control instruction through the controller 3, but the shaking direction of the head 1 is forward, the VR system judges it as "mismatch" according to the instruction matching table 50. Therefore, the VR system does not execute the "move left" displacement control instruction issued by the VR user, and only controls the object 41 in the VR scene 4 to swing toward the VR user (equivalent to the VR user's head 1 shaking forward in the VR world).

[0098] like Figure 7 As shown, when the user issues a displacement control command of "move left" and the shaking direction of the head 1 is to the left, the VR system determines it as a "match" according to the command matching table 50, so the VR system will execute the displacement control command of "move left" issued by the VR user to control the object 41 in the VR scene 4 to gradually move to the right (equivalent to the VR user moving to the left in the VR world).

[0099] In addition to pressing button 33, the control method for generating the displacement control command of "moving left" can also be pushing the joystick on the controller "to the left", rolling the scroll wheel "to the left", sliding the touch panel "to the left" or swinging the controller itself "to the left".

[0100] Affected by the inertia of the human body, when the head 1 of the VR user shakes in one direction, the head 1 will immediately shake in the opposite direction to return to the original limb state, that is, the shaking of the human head is a reciprocating displacement action. Therefore, the matching condition of the above-mentioned "left move" movement operation can be whether the shaking direction of the head 1 of the VR user is "left" or "right".

[0101] Similarly, the above-mentioned "left shift" control method can also be applied to the "right shift" control method by analogy.

[0102] like Figure 8 As shown in FIG. 1 , when the VR user presses the button 32 on the controller 3 to generate the displacement control command of "jump / move up", the VR system does not execute the displacement control command at this time, that is, the VR user remains in the same place (or in the original moving operation state) in the VR world. Next, the head mounted device 2 detects the shaking direction of the VR user's head 1, and then the VR system responds according to the following. Fig.14 The command matching table 50 shown determines whether the above displacement control command matches the above shaking direction.

[0103] like Fig. 9 As shown, when the user issues a "jump / move up" displacement control instruction through the controller 3, but the shaking direction of the head 1 is to the left, the VR system judges it as "mismatch" according to the instruction matching table 50. Therefore, the VR system does not execute the "jump / move up" displacement control instruction issued by the VR user, and only controls the object 41 in the VR scene 4 to swing to the right (equivalent to the VR user's head 1 shaking to the left in the VR world).

[0104] like Fig.10 As shown, when the user issues a "jump / move up" displacement control instruction, and the shaking direction of the head 1 is upward, the VR system determines it as a "match" according to the instruction matching table 50, so the VR system will execute the "jump / move up" displacement control instruction issued by the VR user to control the object 41 in the VR scene 4 to move downward (equivalent to the VR user jumping or moving upward in the VR world). Affected by the inertia of the human body, when the head 1 of the VR user shakes in one direction, the human head will immediately shake in the opposite direction to return to the original limb state, that is, the shaking of the human head is a reciprocating displacement action. Therefore, the matching condition of the above-mentioned "jump" movement operation can be whether the shaking direction of the VR user's head 1 is "up" or "down".

[0105] Similarly, the above-mentioned "jump / move up" control method can also be applied to the "crouch / move down" control method.

[0106] like Fig.11As shown in FIG. 1 , when the VR user continues to perform the forward movement operation, when the VR user releases the button 31 pressed on the controller 3 and issues a "stop" displacement control command, the VR user will continue to move forward in the VR world (original movement operation state). Next, the head mounted device 2 detects the shaking direction of the VR user's head 1, and the VR system then responds accordingly. Fig.14 The command matching table 50 shown determines whether the above displacement control command matches the above shaking direction.

[0107] like Fig.12 As shown, when the user sends a "stop" displacement control instruction through the controller 3 while continuously performing the forward movement operation, but the shaking direction of the head 1 is shaking to the left, the VR system determines that it is "unmatched" according to the instruction matching table 50, so the VR system does not execute the "stop" displacement control instruction issued by the VR user, and continues to perform the forward movement operation, so that the object 41 in the VR scene 4 continues to approach. At this time, the VR system only controls the object 41 in the VR scene 4 to swing to the right (equivalent to the VR user's head 1 shaking to the left in the VR world).

[0108] like Fig.13 As shown, when the user sends a "stop" displacement control command through the controller 3 while continuously performing a forward movement operation, and the shaking direction of the head 1 is "backward" or "forward", the VR system determines it as a "match" according to the command matching table 50. Therefore, the VR system will execute the "stop" displacement control command issued by the VR user and stop performing the forward movement operation, so that the object 41 in the VR scene 4 stops moving.

[0109] The above-mentioned control method for generating the "stop" displacement control instruction, in addition to releasing the pressed button 31, can also be to "relax" the joystick on the controller 3, "stop" rolling the scroll wheel, "stop" sliding the touch panel, "stop" swinging the controller 3 itself, or restore the controller 3 itself to the stop position preset by the VR program (for example, returning from a tilted state relative to the horizontal plane to a horizontal state relative to the horizontal plane).

[0110] The control method for generating the "stop" displacement control command can also be pressing the "backward" button on the controller 3, pushing the joystick "backward", rolling the roller "backward", sliding the touch panel "backward" or swinging the controller 3 itself "backward".

[0111] In some embodiments, for example, when the VR scene is any one of rowing, skiing, rocket control or other movement operations that apply reaction force, the displacement operation of the VR user swinging the controller 3 backward corresponds to the displacement control instruction of "forward", but this is only a special variation of the above embodiment, and can still be controlled by the original instruction matching table 50 or by designing another instruction matching table that conforms to the special scenario operation. For example, in the VR scene of skiing (the instruction matching table at this time is set for the skiing scenario), the VR user performs movement operations through a pair of controllers (left / right). If the VR user wants to turn left, the right controller is slid backward to generate a displacement control instruction of "left movement" and match the shaking direction of the VR user's head 1 ("left" or "right"), such operation can not only simulate the actual skiing action, but also can achieve the effect of preventing the VR user from motion sickness.

[0112] In some embodiments, the displacement control instruction is generated by the VR system detecting a body movement of the VR user 6, wherein the VR system detects the body movement of the VR user 6 through a camera, and then generates a corresponding displacement control instruction according to the body movement. It should be noted that the VR user 6 can define the displacement control instructions corresponding to different body movements, so the same displacement control instruction can be generated by different body movements. In some embodiments, the body movement is selected from a group consisting of a hand movement, a foot movement, a body posture and a combination thereof. The displacement control instructions corresponding to different body movements will be described below with multiple embodiments.

[0113] Please refer to FIG. 15A to FIG. 15F , FIG. 15A to FIG. 15F Schematic diagrams showing the hand movements of the VR user 6 in different embodiments are shown. Fig.15A As shown in FIG. 1 , when the VR user 6 swings his arm from top to bottom, the VR system detects the movement trajectory of the VR user 6's arm by image analysis or optical sensing, and analyzes the movement trajectory of the arm with an algorithm to generate a "forward" displacement control instruction. Fig. 15B As shown in FIG. 1 , when the VR user 6 swings his arm from bottom to top, the VR system generates a displacement control command of “backward” or “stop”; Fig. 15C As shown in FIG. 1 , when the VR user 6 swings his arm from right to left, the VR system generates a displacement control command of “move left”; Fig.15D As shown in FIG. 1 , when the VR user 6 swings his arm from left to right, the VR system generates a displacement control command of “move right”; Fig.15E As shown in FIG. 1 , when the arm of the VR user 6 is extended from bottom to top, the VR system generates a displacement control command of “upward”; Fig.15FAs shown, when the arm of the VR user 6 is extended from top to bottom, the VR system generates a "downward" displacement control command.

[0114] In some embodiments, the VR system detects the finger movement trajectory of the VR user 6 by image analysis or optical sensing, and analyzes the finger movement trajectory with an algorithm to generate various displacement control instructions. In some embodiments, the VR system detects the shape change of the VR user 6's finger by image analysis to generate various displacement control instructions. In other embodiments, the VR system detects the palm / finger movement trajectory of the VR user 6 by image analysis, and corresponds the palm / finger movement trajectory to the position movement of the VR user's palm / finger in the VR world, and then analyzes how the position movement of the palm / finger in the VR world pushes / presses the forward rod / button of the virtual controller in the VR world, thereby generating a "forward" displacement control instruction. Similarly, the VR system can also generate displacement control instructions of "backward" or "stop", "left move" or "right move", "up" or "down" by detecting the palm / finger movement trajectory of the VR user 6 by image analysis.

[0115] Please refer to FIG. 16A to FIG. 16F , FIG. 16A to FIG. 16F Schematic diagrams showing the foot movements of the VR user 6 in different embodiments are shown. Fig.16A As shown in FIG. 1 , when the VR user 6 steps forward, the VR system detects the movement trajectory of the VR user 6's foot by image analysis or optical sensing, and analyzes the movement trajectory of the foot with an algorithm to generate a "forward" displacement control instruction. Fig. 16B As shown in FIG. 1 , when the VR user 6 steps backward, the VR system generates a displacement control command of “backward” or “stop”; Fig. 16C As shown in FIG. 1 , when the VR user 6 steps to the left, the VR system generates a displacement control command of “move left”; Fig.16D As shown in FIG. 1 , when the VR user 6 steps to the right, the VR system generates a displacement control command of “move right”; Fig.16E As shown in FIG. 1 , when the VR user 6 lifts his foot upward, the VR system generates a “move up” displacement control command; Fig.16F As shown, when the VR user 6 steps downward, the VR system generates a "move down" displacement control command.

[0116] In some embodiments, the displacement control instruction is generated by the VR system detecting a brain wave signal of the VR user 6, wherein the VR system detects and analyzes the brain wave signal of the VR user 6 through a brain computer interface (Brain Computer Interface, BCI), and then generates a corresponding displacement control instruction according to the brain wave signal. For example, when the VR user 6 thinks of "moving forward" and sends out a corresponding brain wave, the VR system generates a displacement control instruction of "moving forward" according to the brain wave. When the VR user 6 thinks of "moving backward" and sends out a corresponding brain wave, the VR system generates a displacement control instruction of "moving backward" according to the brain wave. When the VR user 6 thinks of "moving left" and sends out a corresponding brain wave, the VR system generates a displacement control instruction of "moving left" according to the brain wave. When the VR user 6 thinks of "moving right" and sends out a corresponding brain wave, the VR system generates a displacement control instruction of "moving right" according to the brain wave. When the VR user 6 thinks of "stopping" and sends out a corresponding brain wave, the VR system generates a displacement control instruction of "stopping" according to the brain wave. When the VR user 6 thinks of "moving up" and emits a corresponding brain wave, the VR system generates a displacement control command of "moving up" according to the brain wave. When the VR user 6 thinks of "moving down" and emits a corresponding brain wave, the VR system generates a displacement control command of "moving down" according to the brain wave.

[0117] In some embodiments, the displacement control command is generated by the VR system detecting a voice signal of the VR user 6, wherein the VR system receives the voice signal sent by the VR user 6 through a microphone, and then generates the corresponding displacement control command according to the voice signal. For example, when the VR user 6 says "forward", the VR system generates a displacement control command of "forward" according to the voice. When the VR user 6 says "backward", the VR system generates a displacement control command of "backward" according to the voice. When the VR user 6 says "left", the VR system generates a displacement control command of "left" according to the voice. When the VR user 6 says "right", the VR system generates a displacement control command of "right" according to the voice. When the VR user 6 says "stop", the VR system generates a displacement control command of "stop" according to the voice. When the VR user 6 says "up", the VR system generates a displacement control command of "up" according to the voice. When the VR user 6 says "down", the VR system generates a displacement control command of "down" according to the voice.

[0118] Please refer to FIG. 17A to FIG. 17F , FIG. 17A to FIG. 17FSchematic diagrams showing the displacement trajectory of a marker 7 held by a VR user 6 in different embodiments are shown. In some embodiments, the displacement control command is generated by the VR system detecting the displacement trajectory of a marker 7 held by the VR user 6 by image analysis or optical sensing. Taking image analysis as an example, the VR system can detect the displacement trajectory of the marker 7 through a camera, and then generate corresponding displacement control commands according to the displacement trajectory of the marker 7. Fig.17A As shown in FIG. 1 , when the marker 7 moves from the back to the front, the VR system detects the displacement trajectory of the marker 7 and analyzes the detected displacement trajectory of the marker 7 with an algorithm to generate a "forward" displacement control instruction. Fig. 17B As shown, when the VR system detects that the displacement trajectory of the marker 7 is from forward to backward, the VR system generates a "backward" or "stop" displacement control command; Fig. 17C As shown in FIG. 1 , when the VR system detects that the displacement trajectory of the marker 7 is from right to left, the VR system generates a displacement control instruction of “leftward movement”; Fig.17D As shown in FIG. 1 , when the VR system detects that the displacement trajectory of the marker 7 is from left to right, the VR system generates a displacement control instruction of “move right”; Fig.17E As shown, when the VR system detects that the displacement trajectory of the marker 7 is from bottom to top, the VR system generates a "move up" displacement control instruction; Fig.17F As shown, when the VR system detects that the displacement trajectory of the marker 7 is from top to bottom, the VR system generates a "downward movement" displacement control instruction.

[0119] In some embodiments, the displacement control instruction is generated by the VR system detecting the shape of the marker 7 held by the VR user 6, and then generating a corresponding displacement control instruction according to the shape change or deformation degree of the marker 7. For example, when the VR user 6 holds a shape-changeable marker 7 (such as a commercially available fitness ring), the VR system detects the shape change or deformation degree of the marker 7 by image analysis or optical sensing, and analyzes the detected shape change or deformation degree of the marker 7 with an algorithm to generate displacement control instructions such as "forward", "backward", "stop", "left", "right", "up", and "down".

[0120] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A virtual reality control method, characterized in that: Include: Receiving a displacement control instruction from a VR user, the displacement control instruction being selected from the group consisting of forward, backward, left, right, stop, up, down, and combinations thereof; Detecting the shaking direction of the VR user's head; and Determining whether the displacement control instruction matches the shaking direction of the shaking according to an instruction matching rule; When the displacement control instruction matches the shaking direction of the shaking, the displacement control instruction is executed; when the displacement control instruction does not match the shaking direction of the shaking, the displacement control instruction is not executed.

2. The virtual reality control method according to claim 1, wherein: The step of detecting the shaking direction is performed within a preset time after receiving the displacement control instruction.

3. The virtual reality control method according to claim 1, wherein: The step of executing the displacement control instruction is to control an object in a VR scene to move according to the direction of the displacement control instruction.

4. The virtual reality control method according to claim 1, characterized in that: The step of executing the displacement control instruction is to control an object in a VR scene to move in a direction opposite to the displacement control instruction.

5. The virtual reality control method according to claim 1, characterized in that: Also includes: It is determined whether a shaking displacement of the shaking is greater than a displacement threshold. When the shaking displacement of the shaking is greater than the displacement threshold, it is determined whether the displacement control instruction matches the shaking direction of the shaking according to the instruction matching rule.

6. The virtual reality control method according to claim 1, characterized in that: Also includes: It is determined whether a shaking acceleration of the shaking is greater than an acceleration threshold. When the shaking acceleration of the shaking is greater than the acceleration threshold, it is determined whether the displacement control instruction matches the shaking direction of the shaking according to the instruction matching rule.

7. The virtual reality control method according to claim 6, characterized in that: When the shaking acceleration of the shaking is greater than the acceleration threshold, the moving speed of an object in a VR scene increases.

8. The virtual reality control method according to claim 1, characterized in that: Also includes: Determine whether a shaking displacement of the shaking is greater than a displacement threshold and determine whether a shaking acceleration of the shaking is greater than an acceleration threshold. When the shaking displacement of the shaking is greater than the displacement threshold and the shaking acceleration of the shaking is greater than the acceleration threshold, determine whether the displacement control instruction matches the shaking direction of the shaking according to the instruction matching rule.

9. The virtual reality control method according to claim 1, characterized in that: A head mounted device is used to detect the shaking of the head of the VR user.

10. The virtual reality control method according to claim 1, characterized in that: The VR user's head shaking is detected by a sense of unity detection device.

11. The virtual reality control method according to claim 1, characterized in that: The step of detecting the shaking direction is performed within a duration of receiving the displacement control instruction.

12. The virtual reality control method according to any one of claims 1 to 11, characterized in that: The displacement control instruction is generated by the VR user operating a controller.

13. The virtual reality control method according to any one of claims 1 to 11, characterized in that: The displacement control instruction is generated by the VR system detecting a body movement of the VR user.

14. The virtual reality control method according to claim 13, characterized in that: The body movement is selected from a group consisting of a hand movement, a foot movement, a body posture and a combination thereof.

15. The virtual reality control method according to any one of claims 1 to 11, characterized in that: The displacement control instruction is generated by the VR system detecting a brain wave signal of the VR user.

16. The virtual reality control method according to any one of claims 1 to 11, characterized in that: The displacement control instruction is generated by the VR system detecting a voice instruction of the VR user.

17. The virtual reality control method according to any one of claims 1 to 11, characterized in that: The displacement control instruction is generated by the VR system detecting the displacement trajectory of a marker held by the VR user.

18. The virtual reality control method according to any one of claims 1 to 11, characterized in that: The displacement control instruction is generated by the VR system detecting the shape of a marker held by the VR user.

Images