Virtual reality moving method and device for gait self-adaptive translation gain

Abstract

The invention provides a virtual reality moving method and device for gait self-adaptive translation gain. The method comprises the following steps: determining natural disturbance in a gait cycle when a user naturally walks; according to natural disturbance in the gait cycle, the translation gain of the user in the virtual reality environment is dynamically adjusted; wherein the larger the natural disturbance is, the larger the translation gain corresponding to the natural disturbance is, and the translation gain is used for adjusting the proportion of the walking speed of the user in the virtual space to the actual walking speed of the user in the physical space. According to the method, the use of the translation gain is dynamically adjusted according to the gait cycle of the user, the inconsistency between the user body feeling and the visual perception is reduced by using the motion disturbance of the user, the perception threshold value of the user for the difference between the virtual space motion rate and the physical space motion rate is widened, meanwhile, the occurrence probability of motion sickness is not increased, and the user experience is improved. Therefore, the overall comfort and immersion of the user are improved.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to a virtual reality movement method and device with gait adaptive translation gain. Background Art

[0002] Virtual Reality (VR) technology is an emerging technology that has rapidly emerged in recent years. It uses computer simulation to generate a three-dimensional virtual world, allowing users to interact with the generated three-dimensional virtual world through dedicated equipment such as virtual reality head-mounted displays (HMDs), handles, and motion capture devices. This technology has shown great application potential in many fields such as games, education, medical care, and architectural design, bringing users an immersive experience, as if they were actually there.

[0003] In VR experiences, high-fidelity graphics and realistic sounds greatly enhance users' sense of immersion, allowing them to experience interactions and actions in the virtual world similar to those in the real world.

[0004] However, existing VR movement methods always have the problem of limited physical movement: the size of the virtual space is often larger than the physical space where the user is located, or the shape and obstacle distribution of the physical space do not match the virtual space. This requires users to pause or reposition themselves in the physical space when walking in the virtual world to avoid colliding with physical obstacles, reducing the smoothness and comfort of the interaction.

[0005] In order to overcome this problem, researchers have developed a variety of VR walking solutions. Among them, the simplest and most direct method is to use traditional input devices such as handles, keyboards or mice to control walking in the virtual world. This method has low equipment cost, is easy to use, and is suitable for most virtual reality applications. However, this method can easily cause 3D motion sickness in users due to the inconsistency between visual feedback and physical body perception. At the same time, since this method relies on button or joystick input and lacks direct connection with the user's body movement, it is easy for users to feel that the control of the virtual world is too strong, affecting the sense of immersion.

[0006] Another solution is the omnidirectional treadmill, which allows users to stand on a fixed platform and control movement in the virtual world through walking movements. However, omnidirectional treadmills are expensive and bulky, making them difficult to promote in home environments. In addition, although omnidirectional treadmills capture the user's step movements, since the user's steps are actually completed by sliding or stepping, there is a lack of real ground feedback, which may make the user feel unstable or uncoordinated.

[0007] Motion detection technology provides a simplified solution that does not require additional hardware. The technology detects the user's step-like movements (such as leg lifting or arm swinging) to achieve movement in the virtual world. However, this unnatural movement pattern can easily cause user fatigue and reduce the accuracy of movement.

[0008] In order to achieve more natural walking in virtual reality, the ideal solution is to let users walk in real physical space and feed back the physical posture of the virtual reality head-mounted display to the user's virtual posture in real time. However, there are often conflicts between the physical space and the virtual space due to spatial factors such as size and obstacle layout. The virtual world is often an empty and vast space, but users need to walk and explore the virtual world in a small physical space, which is restricted by walls, obstacles, etc., greatly affecting the virtual reality experience.

[0009] Redirected Walking (RDW) technology takes advantage of the errors in the human perception system and subtly manipulates the user's visual feedback to change the movement speed ratio, direction difference, curvature difference, etc. between the virtual and physical spaces without the user's knowledge. This enables the VR system to support longer virtual walking distances within a limited physical space while preventing users from experiencing dizziness and discomfort due to perceived path curvature or motion scaling.

[0010] In RDW technology, translation gain is crucial to optimizing the user experience. Translation gain is responsible for adjusting the ratio of the user's walking speed in the virtual world to the walking speed in the physical world, mapping the movement in the real space to a larger or smaller movement in the virtual space, which not only facilitates users to explore the vast virtual environment, but also effectively avoids collisions. However, when applying translation gain, designers usually need to control the gain within the user's perception threshold so that the difference between the virtual and physical movement rates is not noticeable. When the difference between the movement speed in the virtual space and the movement speed in the physical space is too large and exceeds the user's perception threshold, the user may feel uncomfortable or lose immersion, and realize that they are not in a real environment.

[0011] Currently, translation gain is usually applied in a constant manner, that is, the ratio of the user's virtual movement rate to the physical movement rate is adjusted consistently throughout the user's gait cycle, and the translation gain does not change with the user's gait cycle during walking. Since the translation gain is applied constantly to the user's movement, the user will more easily detect the inconsistency between the virtual and physical rates in a relatively stable state (such as a stationary standing state or a slow moving state), which limits the use of translation gain.

[0012] Therefore, how to dynamically adjust the ratio of virtual and physical rates by combining the user's gait cycle and dynamic changes, and make more full use of the user's perceptual characteristics to enhance immersion, is an important issue that needs to be urgently addressed in the field of virtual reality. Summary of the invention

[0013] The present invention provides a virtual reality movement method and device with gait adaptive translation gain, which are used to dynamically adjust the ratio of virtual and physical speeds by combining the user's gait cycle and dynamic changes, make fuller use of the user's perceptual characteristics to improve the immersion, reduce the user's sensitivity to the difference between physical speed and virtual speed, allow a wider range of virtual space exploration, and do not increase the probability of motion sickness, thereby improving the user's overall comfort and immersion.

[0014] On the one hand, the present invention provides a virtual reality movement method with gait-adaptive translation gain, comprising: determining natural disturbances in a gait cycle when a user walks naturally; dynamically adjusting the translation gain of the user in a virtual reality environment according to the natural disturbances in the gait cycle; wherein, the greater the natural disturbance, the greater the corresponding translation gain, and the translation gain is used to adjust the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space.

[0015] Further, determining the natural disturbance in the gait cycle of the user when walking naturally includes: determining a target time period in which the natural disturbance in the gait cycle of the user when walking naturally is the largest; correspondingly, dynamically adjusting the translation gain of the user in the virtual reality environment according to the natural disturbance in the gait cycle includes: within the target time period, adjusting the translation gain of the user in the virtual reality environment to increase it toward the target translation gain; outside the target time period, adjusting the translation gain of the user in the virtual reality environment to decrease it toward the basic translation gain; wherein, the target translation gain is greater than the basic translation gain.

[0016] Furthermore, determining the target time period in the gait cycle of the user when walking naturally and having the largest natural disturbance includes: obtaining a physical vertical movement speed value of a head-mounted display worn by the user; calculating a physical vertical movement acceleration of the user's head based on the physical vertical movement speed value; and determining the target time period in the gait cycle of the user when walking naturally and having the largest natural disturbance based on the physical vertical movement acceleration.

[0017] Further, the method of obtaining the physical vertical movement speed value of the head-mounted display worn by the user then includes: smoothing and stabilizing the physical vertical movement speed value to obtain the physical vertical movement speed value after the smoothing and stabilization; correspondingly, calculating the physical vertical movement acceleration of the user's head according to the physical vertical movement speed value, including: calculating the physical vertical movement acceleration of the user's head according to the physical vertical movement speed value after the smoothing and stabilization.

[0018] Furthermore, based on the physical vertical movement acceleration, the target time period with the largest natural disturbance in the gait cycle of the user when walking naturally is determined, including: when the physical vertical movement acceleration of the user's head is greater than a first acceleration threshold, determining the target time period with the largest natural disturbance in the gait cycle when the user enters natural walking; when the physical vertical movement acceleration of the user's head is less than or equal to a second acceleration threshold, determining the target time period with the largest natural disturbance in the gait cycle when the user exits natural walking; wherein the first acceleration threshold is greater than or equal to the second acceleration threshold.

[0019] Furthermore, within the target time period, the translation gain of the user in the virtual reality environment is adjusted so that the translation gain increases toward the target translation gain, including: obtaining the user's physical horizontal movement speed vector; determining the user's current target translation gain based on the physical horizontal movement speed vector and the forward direction unit vector of the head-mounted display worn by the user; within the target time period, gradually increasing the current translation gain with the target translation gain as the target; and adjusting the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space based on the gradually increased current translation gain.

[0020] Furthermore, based on the physical horizontal movement velocity vector and the forward direction unit vector of the head-mounted display worn by the user, the user's current target translation gain is determined, including: calculating the inner product of the physical horizontal movement velocity vector and the forward direction unit vector, and using the inner product as the user's physical horizontal movement forward velocity component; determining the target translation gain based on the physical horizontal movement forward velocity component.

[0021] Furthermore, the method of adjusting the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space according to the gradually increasing current translation gain specifically includes: determining the current frame physical position and the previous frame physical position of the user in the physical space; calculating the current frame physical displacement of the user according to the current frame physical position and the previous frame physical position; obtaining the previous frame virtual position of the user in the virtual space; calculating the current frame virtual position of the user in the virtual space according to the current frame physical displacement, the previous frame virtual position and the current translation gain, so as to realize virtual reality movement.

[0022] In a second aspect, the present invention also provides a virtual reality mobile device with gait-adaptive translation gain, comprising: a natural disturbance determination module, used to determine the natural disturbance in the gait cycle of the user when walking naturally; a translation gain adjustment module, used to dynamically adjust the translation gain of the user in the virtual reality environment according to the natural disturbance in the gait cycle; wherein, the larger the natural disturbance, the larger the corresponding translation gain, and the translation gain is used to adjust the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space.

[0023] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the virtual reality movement method with gait adaptive translation gain as described in any one of the above is implemented.

[0024] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the virtual reality movement method with gait adaptive translation gain as described in any one of the above.

[0025] The virtual reality movement method of gait adaptive translation gain provided by the present invention determines the natural disturbance in the gait cycle of the user when walking naturally, and dynamically adjusts the translation gain of the user in the virtual reality environment according to the natural disturbance in the gait cycle; wherein, the larger the natural disturbance, the larger the corresponding translation gain, and the translation gain is used to adjust the ratio of the walking speed of the user in the virtual space to the actual walking speed of the user in the physical space. The method dynamically adjusts the use of translation gain according to the user's gait cycle, uses the user's own motion disturbance to reduce the inconsistency between the user's proprioception and visual perception, broadens the user's perception threshold of the difference in motion rate between the virtual and physical spaces, and does not increase the probability of motion sickness, thereby improving the user's overall comfort and immersion. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 It is a flow chart of a virtual reality movement method with gait adaptive translation gain provided by an embodiment of the present invention.

[0028] Figure 2It is a detailed flow chart of a virtual reality movement method with gait adaptive translation gain provided in an embodiment of the present invention.

[0029] Figure 3 It is a structural schematic diagram of a virtual reality mobile device with gait adaptive translation gain provided by an embodiment of the present invention.

[0030] Figure 4 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] It should be noted that in order to achieve more natural walking in virtual reality, the ideal solution is to let users walk in real physical space and feed back the physical posture of the virtual reality head-mounted display to the user's virtual posture in real time. However, there are often conflicts between the physical space and the virtual space due to spatial factors such as size and obstacle layout. The virtual world is often an empty and vast space, but users need to walk and explore the virtual world in a small physical space, which is restricted by walls, obstacles, etc., greatly affecting the virtual reality experience.

[0033] RDW technology utilizes the errors in the human perception system and subtly manipulates the user's visual feedback to change the movement speed ratio, direction difference, curvature difference, etc. between the virtual and physical spaces without the user's knowledge. This enables the VR system to support longer virtual walking distances within a limited physical space while preventing users from experiencing dizziness and discomfort due to perceived path curvature or motion scaling.

[0034] In RDW technology, translation gain is crucial to optimizing the user experience. Translation gain is responsible for adjusting the ratio of the user's walking speed in the virtual world to the walking speed in the physical world, mapping the movement in the real space to a larger or smaller movement in the virtual space, which not only facilitates users to explore the vast virtual environment, but also effectively avoids collisions. However, when applying translation gain, designers usually need to control the gain within the user's perception threshold so that the difference between the virtual and physical movement rates is not noticeable. When the difference between the movement speed in the virtual space and the movement speed in the physical space is too large and exceeds the user's perception threshold, the user may feel uncomfortable or lose immersion, and realize that they are not in a real environment.

[0035] Currently, the perceptual threshold of panning gain is generally between 0.86 and 1.26, which greatly limits the application scope of panning gain. If these thresholds can be further broadened, the performance and application scope of RDW technology will be greatly improved.

[0036] Currently, translation gain is usually applied in a constant manner, that is, translation gain is applied constantly throughout the walking process without considering the user's walking state. Since translation gain continuously acts on the user's movement, the user will more easily detect the inconsistency between virtual and physical rates in a relatively stable state (such as standing still or moving slowly), which limits the use of translation gain.

[0037] In view of this, the present invention proposes a virtual reality movement method with gait adaptive translation gain, specifically, Figure 1 A schematic flow chart of a virtual reality movement method with gait adaptive translation gain provided in an embodiment of the present invention is shown.

[0038] like Figure 1 As shown, the method includes: S110, determining the natural disturbance in the gait cycle of the user when walking naturally; S120, dynamically adjusting the translation gain of the user in the virtual reality environment according to the natural disturbance in the gait cycle; wherein, the larger the natural disturbance is, the larger the corresponding translation gain is, and the translation gain is used to adjust the ratio of the walking speed of the user in the virtual space to the actual walking speed of the user in the physical space.

[0039] The following will describe steps S110 - S120 and related steps in detail.

[0040] S110, determining natural disturbances in a gait cycle of a user walking naturally.

[0041] The gait cycle refers to the process that a person goes through when walking, starting from the heel of one foot touching the ground to the heel of the same foot touching the ground again. A complete gait cycle includes the support phase and the swing phase. The support phase accounts for about 60% of the entire gait cycle, during which at least one foot is in contact with the ground. The swing phase accounts for about 40% of the entire gait cycle, during which the foot is not in contact with the ground, but swings forward to prepare for the next landing.

[0042] It is easy to understand that in the gait cycle of the user when walking naturally, the vertical movement acceleration of the head is constantly changing. Therefore, in this embodiment, the physical vertical movement acceleration of the head-mounted display worn by the user is used as a standard for measuring the size of the natural disturbance in the gait cycle of the user when walking naturally.

[0043] Based on the determination of the natural disturbance in the gait cycle of the user when walking naturally in step S110, step S120 is further performed.

[0044] S120, dynamically adjusting the translation gain of the user in the virtual reality environment according to the natural disturbance in the gait cycle; wherein, the larger the natural disturbance, the larger the corresponding translation gain, and the translation gain is used to adjust the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space.

[0045] It is easy to understand that in redirected walking technology, translation gain is crucial to optimizing the user experience. It is responsible for adjusting the ratio of the user's walking speed in the virtual world to the walking speed in the physical world, and mapping the movement in the real physical space into larger or smaller movements in the virtual space, which not only facilitates users to explore the grand virtual environment, but also effectively avoids collisions.

[0046] In this embodiment, the translation gain adjustment of the user in the virtual reality environment is positively correlated with the change in the size of the natural disturbance in the gait cycle. That is, the greater the natural disturbance in the gait cycle when the user walks naturally, the greater the corresponding adjustment of the translation gain of the user in the virtual reality environment.

[0047] It is worth mentioning that, since the natural disturbance in the gait cycle changes in real time, the adjustment process of the translation gain of the user in the virtual reality environment in this embodiment is also dynamic.

[0048] It should be noted that step S110 and step S120 will be described in more detail in the following embodiments and will not be described in detail here.

[0049] In this embodiment, the natural disturbance in the gait cycle of the user when walking naturally is determined, and the translation gain of the user in the virtual reality environment is dynamically adjusted according to the natural disturbance in the gait cycle; wherein, the greater the natural disturbance, the greater the corresponding translation gain, and the translation gain is used to adjust the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space. This method dynamically adjusts the use of translation gain according to the user's gait cycle, uses the user's own motion disturbance to reduce the inconsistency between the user's proprioception and visual perception, broadens the user's perception threshold of the difference in motion rate between virtual and physical spaces, and does not increase the probability of motion sickness, thereby improving the user's overall comfort and immersion.

[0050] Figure 2 A detailed flow chart of a virtual reality movement method with gait adaptive translation gain provided in an embodiment of the present invention is shown.

[0051] like Figure 2As shown, the method includes: S210, determining a target time period in which the natural disturbance in the gait cycle of the user is the largest when walking naturally; S220, adjusting the translation gain of the user in the virtual reality environment within the target time period so that the translation gain increases toward the target translation gain; S230, outside the target time period, adjusting the translation gain of the user in the virtual reality environment so that the translation gain decreases toward the basic translation gain; wherein the translation gain is used to adjust the ratio of the walking speed of the user in the virtual space to the actual walking speed of the user in the physical space, and the target translation gain is greater than the basic translation gain.

[0052] The following will describe steps S210 - S230 and related steps in detail.

[0053] S210, determining a target time period in a gait cycle of a user walking naturally with a maximum natural disturbance.

[0054] It is easy to understand that the vertical acceleration of the head is constantly changing during the gait cycle of a user walking naturally. When the user is walking naturally and is in the period of maximum natural disturbance in the gait cycle (usually the stage when the front heel of the user touches the ground during a step), the vertical acceleration of the head will have an obvious peak.

[0055] Therefore, this embodiment uses the physical vertical movement acceleration of the head-mounted display worn by the user as a standard for measuring the size of natural disturbances in the gait cycle of the user when walking naturally, and uses this to determine the time period with the largest natural disturbance in the gait cycle of the user when walking naturally, that is, the target time period.

[0056] Specifically, firstly, the physical vertical movement speed value of the head mounted display worn by the user is obtained, and then the physical vertical movement acceleration of the user's head is calculated according to the physical vertical movement speed value, and then the physical vertical movement acceleration is compared with a preset acceleration threshold. If the physical vertical movement acceleration is higher than the preset acceleration threshold, it is determined that the user is in the target period of the gait cycle when walking naturally with the maximum natural disturbance, that is, within the target period; otherwise, it is determined that the user is not in the target period of the gait cycle when walking naturally with the maximum natural disturbance, that is, outside the target period.

[0057] The preset acceleration threshold can be set according to actual needs and is not specifically limited here.

[0058] After determining the target time period in the gait cycle of the user when walking naturally in step S210, the step S220 or step S230 is further performed.

[0059] S220: During the target time period, adjust the translation gain of the user in the virtual reality environment so that the translation gain increases toward the target translation gain.

[0060] It should be noted that in redirected walking technology, translation gain is crucial to optimizing the user experience. It is responsible for adjusting the ratio of the user's walking speed in the virtual world to the walking speed in the physical world, and mapping the movement in the real physical space into larger or smaller movements in the virtual space, which not only facilitates users to explore the grand virtual environment, but also effectively avoids collisions.

[0061] It is easy to understand that after determining the target period in which the natural disturbance is the largest in the gait cycle when the user walks naturally, a larger translation gain, that is, the ratio of the movement rate in the virtual space to the movement rate in the actual physical space, is applied to the target period than to other periods.

[0062] In this embodiment, the target translation gain is used as the target in the target period, and the translation gain of the user in the virtual reality environment is gradually increased. The specific value of the target translation gain can be adjusted according to the actual situation and is not specifically limited here.

[0063] S230: Outside the target time period, adjust the translation gain of the user in the virtual reality environment to decrease toward the basic translation gain.

[0064] It is easy to understand that after determining the target period in which the natural disturbance is the largest in the gait cycle when the user walks naturally, a translation gain that is smaller than the target translation gain is applied in other periods outside the target period, that is, the ratio of the movement rate in the virtual space to the movement rate in the actual physical space.

[0065] In this embodiment, in other time periods other than the target time period, the basic translation gain is used as the target to gradually reduce the translation gain of the user in the virtual reality environment. The specific value of the basic translation gain can also be adjusted according to actual conditions and is not specifically limited here.

[0066] However, it should be noted that the target value for gradually increasing the translation gain within the target period must be greater than the target value for gradually decreasing the translation gain outside the target period, that is, the target translation gain must be greater than the basic translation gain.

[0067] It should also be noted that step S220 and step S230 in this embodiment are two parallel steps. In the same time period, only step S220 or step S230 will be selected for execution, depending on the current time period.

[0068] Based on the above, it can be seen that this embodiment applies a larger translation gain during the period of time in the gait cycle when the natural body disturbance is the largest when the user is walking naturally, and applies a smaller translation gain during the period of time in the gait cycle when the body disturbance is relatively small, thereby utilizing the natural disturbance when the user is walking to mask the application of the translation gain, making the use of the translation gain less noticeable.

[0069] In this embodiment, the target period of time in which the natural disturbance in the gait cycle of the user is the largest when walking naturally is determined, and the translation gain of the user in the virtual reality environment is adjusted within the target period so that the translation gain increases toward the target translation gain, and the translation gain of the user in the virtual reality environment is adjusted outside the target period so that the translation gain decreases toward the base translation gain, wherein the translation gain is used to adjust the ratio of the walking speed of the user in the virtual space to the actual walking speed of the user in the physical space, and the target translation gain is greater than the base translation gain. This method dynamically adjusts the use of translation gain according to the user's gait cycle, uses the user's own motion disturbance to reduce the inconsistency between the user's proprioception and visual perception, broadens the user's perception threshold of the difference in motion rate between the virtual and physical spaces, and does not increase the probability of motion sickness, thereby improving the user's overall comfort and immersion.

[0070] In addition, the virtual reality movement method with gait adaptive translation gain provided in the embodiment of the present invention is a universal virtual reality movement solution. It does not require any additional equipment except the user's head-mounted display, has no special restrictions on virtual scenes and tasks, and can be applied to various types of virtual reality applications such as single-player VR applications, multi-player VR applications, and dynamic VR scenes. It can also be combined with existing virtual reality movement technologies such as redirected walking to jointly improve the performance of virtual reality movement technologies.

[0071] On the basis of the above-mentioned embodiment, the process of determining the target time period in which the natural disturbance in the gait cycle of the user is the largest when walking naturally will be described in detail below.

[0072] Determining a target time period in a gait cycle of a user when walking naturally, during which natural disturbance is the largest, includes: obtaining a physical vertical movement speed value of a head-mounted display worn by the user; calculating a physical vertical movement acceleration of the user's head according to the physical vertical movement speed value; and determining a target time period in a gait cycle of the user when walking naturally, during which natural disturbance is the largest according to the physical vertical movement acceleration.

[0073] It is easy to understand that first, you can get The physical vertical movement speed value of the head mounted display at the moment is set to , and use the double exponential smoothing method to calculate the physical vertical movement speed value Smoothing is performed. Definition is the smoothing factor and takes the value of 0.05, then The physical vertical movement speed value of the head-mounted display worn by the user at the moment The processes of single exponential smoothing and double exponential smoothing are as follows: (1)-(2).

[0074] (1).

[0075] (2).

[0076] In formulas (1)-(2), and Respectively The physical vertical movement speed value of the head mounted display at all times Values ​​after single exponential smoothing and double exponential smoothing. and They are the last moment ( The physical vertical movement speed value of the head mounted display at that moment The values ​​after single exponential smoothing and double exponential smoothing are used to smooth The physical vertical movement speed of the head mounted display at the moment. ) due to and does not exist, and It can be replaced by 0.

[0077] The physical vertical moving speed value after double exponential smoothing in formula (2) is The physical vertical movement speed value of the head mounted display after smoothing and stabilization.

[0078] It should be noted that smoothing and stabilizing the physical vertical movement speed value of the head-mounted display worn by the user can remove noise in the data, thereby helping to more accurately distinguish the various stages of the user's steps and apply gait synchronization translation gain in the subsequent process.

[0079] Then, the physical vertical movement acceleration of the user's head is calculated according to the physical vertical movement speed value after smoothing and stabilization.

[0080] Specifically, is the physical vertical moving speed value of the head mounted display after smoothing and stabilization at the current moment, is the physical vertical moving speed value of the head mounted display after smoothing and stabilization at the previous moment, and the time interval between the previous moment and the current moment is , then the physical vertical movement acceleration of the user's head at the current moment The calculation equation is as follows (3).

[0081] (3).

[0082] Further, according to the physical vertical movement acceleration, the target period of time in the gait cycle of the user when walking naturally with the maximum natural disturbance is determined. Specifically, when the physical vertical movement acceleration of the user's head is greater than the first acceleration threshold, it is determined that the user is in the target period of time in the gait cycle of the user when walking naturally with the maximum natural disturbance; when the physical vertical movement acceleration of the user's head is less than or equal to the second acceleration threshold, it is determined that the user is not in the target period of time in the gait cycle of the user when walking naturally with the maximum natural disturbance.

[0083] Specifically, the vertical acceleration of the head changes during the gait cycle of natural human walking. When the natural disturbance of the gait cycle is the largest (usually when the heel of the front leg touches the ground during the step), the vertical acceleration of the head will have a significant peak.

[0084] It should be noted that the first acceleration threshold and the second acceleration threshold can be set to the same value or different values, which is not specifically limited here.

[0085] For example, in one specific embodiment, the peak value is greater than , so the first acceleration threshold and the second acceleration threshold are both set to .

[0086] set up For users The physical vertical movement acceleration of the user's head at the moment, if , then determine the period of time when the user is in the natural walking gait cycle with the largest natural disturbance, that is, the target period; if , it is determined that the user is not in the period with the largest natural disturbance in the gait cycle when walking naturally, that is, other periods other than the target period.

[0087] In another specific embodiment, the first acceleration threshold is set to , set the second acceleration threshold to , such a setting can make the switching criterion more directional.

[0088] set up For users The physical vertical movement acceleration of the user's head at the moment, in terms of the overall process: at the beginning, , determine the period of time when the user is not in natural walking and the natural disturbance is the largest in the gait cycle, that is, other periods other than the target period; then, , determine the time period with the largest natural disturbance in the gait cycle when the user is walking naturally, that is, the target time period; then, , still determine the period of time when the user is in the natural walking gait cycle with the largest natural disturbance, that is, the target period; finally, , determine the period of time in the gait cycle when the user is not in natural walking with the largest natural disturbance, that is, other periods other than the target period.

[0089] In this embodiment, by obtaining the physical vertical movement speed value of the head mounted display worn by the user, and calculating the physical vertical movement acceleration of the user's head according to the physical vertical movement speed value, and then determining the target period of time in the gait cycle of the user when walking naturally according to the physical vertical movement acceleration, the translation gain of the user in the virtual reality environment is adjusted within the target period to increase the target translation gain, and the translation gain of the user in the virtual reality environment is adjusted outside the target period to decrease the base translation gain. The translation gain is used to adjust the ratio of the walking speed of the user in the virtual space to the actual walking speed of the user in the physical space, and the target translation gain is greater than the base translation gain. The method dynamically adjusts the use of the translation gain according to the user's gait cycle, uses the user's own motion disturbance to reduce the inconsistency between the user's proprioception and visual perception, broadens the user's perception threshold of the difference in motion rate between the virtual and physical spaces, and does not increase the probability of motion sickness, thereby improving the overall comfort and immersion of the user.

[0090] On the basis of the above embodiments, the application process of the translation gain in different time periods will be described in detail below.

[0091] In one embodiment, during a target time period, the translation gain of a user in a virtual reality environment is adjusted so that the translation gain increases toward a target translation gain, including: obtaining a physical horizontal movement velocity vector of the user; determining the user's current target translation gain based on the physical horizontal movement velocity vector and a forward direction unit vector of a head mounted display worn by the user; during the target time period, gradually increasing the current translation gain with the target translation gain as a target; and adjusting the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space based on the gradually increased current translation gain.

[0092] It is easy to understand that it can be obtained through VR tracker The physical horizontal moving velocity vector of the head mounted display at time is set to , and use the double exponential smoothing method to smooth the velocity vector Smoothing is performed. Definition is the smoothing factor and takes the value of 0.05, then The physical horizontal movement velocity vector of the head-mounted display worn by the user at the moment The processes of single exponential smoothing and double exponential smoothing are as follows: (4)-(5).

[0093] (4).

[0094] (5).

[0095] In equations (4)-(5), and Respectively Physical horizontal movement velocity vector of the head mounted display at all times Values ​​after single exponential smoothing and double exponential smoothing. and They are the last moment ( The physical horizontal movement velocity vector of the head mounted display (at the moment) The values ​​after single exponential smoothing and double exponential smoothing are used to smooth The physical horizontal movement speed value of the head mounted display at the moment. ) due to and does not exist, and A zero vector can be used instead.

[0096] The physical horizontal motion vector after double exponential smoothing in equation (5) is The physical horizontal velocity vector of the HMD after smoothing and stabilization.

[0097] It is also worth noting that smoothing and stabilizing the physical horizontal movement velocity vector of the head-mounted display worn by the user can remove noise in the data, thereby maintaining the user's visual consistency and stability when applying gait synchronization translation gain, giving the user a more comfortable experience.

[0098] Subsequently, according to the physical horizontal moving speed vector and the forward direction unit vector of the head mounted display worn by the user, an inner product of the physical horizontal moving speed vector and the forward direction unit vector is calculated; Expanded, set for Physical horizontal movement velocity vector after smoothing and stabilization at all times, obtained through VR tracker The forward direction unit vector of the head mounted display at this moment ( Length of mold is 1.0). and The angle between two vectors , then the inner product of the smoothed and stabilized physical horizontal moving velocity vector and the forward direction unit vector of the head mounted display worn by the user is As shown in formula (6).

[0099] (6).

[0100] in, In physical terms, it represents the component of the physical horizontal moving velocity vector in the forward direction of the head-mounted display worn by the user after smoothing and stabilization. When it is a positive number and large, it means that the user is moving forward; when When the absolute value of is small or negative, it means that the user is stationary, moving sideways, or moving backward.

[0101] Next, the user's current target translation gain is determined.

[0102] Specifically, the target translation gain may be a set fixed value, or an uncertain value that changes in real time with the horizontal forward speed of the user, which is not specifically limited here.

[0103] For example, in one specific embodiment, the target translation gain The value is 2.0, at which point the user's movement speed in the virtual space is 2.0 times the movement speed in the actual physical space.

[0104] Preferably, in another specific embodiment, the target translation gain The forward velocity component can be shifted according to the user's physical level The specific calculation process can be found in the following formula (7).

[0105] (7).

[0106] In equation (7), the forward velocity component of physical horizontal movement is is the inner product of the physical horizontal moving velocity vector (after smoothing and stabilization) and the forward unit vector of the head mounted display worn by the user, that is, the component of the horizontal moving velocity of the head mounted display worn by the user in the forward direction. The specific calculation formula can be found in the above formula (6). is the adjustment coefficient, which can be 1.05 in this embodiment.

[0107] It should be noted that, in the target period, by associating the target translation gain with the user's horizontal forward speed, the target translation gain can be controlled within the user's perception threshold range, so that the difference between the virtual and physical movement rates is not perceived by the user. After determining the target translation gain, further, in the target period, the ratio of the movement rate in the virtual space to the movement rate in the actual physical space is gradually increased with the target translation gain as the target, so as to achieve virtual reality movement.

[0108] Specifically, the current frame physical position and the previous frame physical position of the user in the physical space are determined; the current frame physical displacement of the user is calculated according to the current frame physical position and the previous frame physical position; the previous frame virtual position of the user in the virtual space is obtained; the current frame virtual position of the user in the virtual space is calculated according to the current frame physical displacement, the previous frame virtual position and the current translation gain that is gradually increased with the target translation gain as the target, so as to realize virtual reality movement.

[0109] Specifically, first, the VR tracker can be used to obtain and record the user's current frame physical position and previous frame physical position in the physical space, and the current frame physical position is recorded as , the physical position of the previous frame is recorded as , from which the user's physical displacement in the current frame and the physical displacement of the current frame can be calculated .

[0110] Then, get the user's previous frame virtual position in the virtual space The target translation gain is calculated based on the target period of the gait cycle when the natural disturbance is the largest when the user walks naturally. As the goal, gradually increase the current translation gain (i.e., the ratio of the movement rate in the virtual space to the movement rate in the actual physical space), and calculate the user's current frame virtual position in the virtual space. For details, see the following equations (8)-(9).

[0111] (8).

[0112] (9).

[0113] Equation (8) shows the calculation of the gradually increasing current translation gain. In equation (8), Indicates the translation gain of the previous frame. The translation gain target is the target translation gain. , and , The coefficient that controls the speed of gain gradient change. To set the control parameters, The forward velocity component of the user's physical horizontal movement. The value of is related to the user's physical horizontal forward velocity component The larger the forward velocity component of the user's physical horizontal movement, During the target period, The value is 0.5.

[0114] In formula (8), It not only covers the target translation gain, but also covers the current translation gain which is gradually increased with the target translation gain as the goal.

[0115] Then, the virtual camera position used to obtain the user's field of view in the virtual space is moved to , to achieve virtual reality movement.

[0116] In another embodiment, outside the target time period, the translation gain of the user in the virtual reality environment is adjusted to decrease toward the base translation gain.

[0117] Specifically, during the period other than the target period when the natural disturbance in the gait cycle is the largest when the user walks naturally, the basic translation gain As the goal, gradually reduce the current translation gain (that is, the ratio of the movement rate in the virtual space to the movement rate in the actual physical space).

[0118] Specifically, the VR tracker can be used to obtain and record the user's current frame physical position and the previous frame physical position in the physical space, that is, the current frame physical position is recorded as , the physical position of the previous frame is recorded as , the user's current frame physical displacement can be calculated .

[0119] Determine the user's previous frame virtual position in the virtual space as , then the user's current frame virtual position in the virtual space can be seen in the following formula (10).

[0120] (10).

[0121] In formula (10), . Since it is outside the target period, It not only covers the basic translation gain, but also covers the current translation gain that is gradually reduced with the basic translation gain as the target. The translation gain target is the basic translation gain. .

[0122] Likewise, , but outside the target period, The value is 0.05.

[0123] Move the virtual camera position used to obtain the user's field of view in the virtual space to the virtual position of the current frame , to achieve virtual reality movement.

[0124] Optionally, the base translation gain Take 1.0, at which point the user's movement speed in the virtual space is equal to the movement speed in the actual physical space.

[0125] In this embodiment, the translation gain of the user in the virtual reality environment is adjusted within the target period so that the translation gain increases toward the target translation gain, and the translation gain of the user in the virtual reality environment is adjusted outside the target period so that the translation gain decreases toward the base translation gain, wherein the translation gain is used to adjust the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space, and the target translation gain is greater than the base translation gain. This method dynamically adjusts the use of translation gain according to the user's gait cycle, uses the user's own motion disturbance to reduce the inconsistency between the user's proprioception and visual perception, broadens the user's perception threshold of the difference in motion rate between the virtual and physical spaces, and does not increase the probability of motion sickness, thereby improving the user's overall comfort and immersion.

[0126] Corresponding to the virtual reality movement method with gait adaptive translation gain described in the above embodiments, the present invention also proposes a virtual reality movement device with gait adaptive translation gain.

[0127] Specifically, Figure 3 A schematic structural diagram of a virtual reality mobile device with gait adaptive translation gain provided by an embodiment of the present invention is shown.

[0128] like Figure 3 As shown, the device includes: a natural disturbance determination module 310, which is used to determine the natural disturbance in the gait cycle of the user when walking naturally; a translation gain adjustment module 320, which is used to dynamically adjust the translation gain of the user in the virtual reality environment according to the natural disturbance in the gait cycle; wherein, the larger the natural disturbance, the larger the corresponding translation gain, and the translation gain is used to adjust the ratio of the walking speed of the user in the virtual space to the actual walking speed of the user in the physical space.

[0129] In this embodiment, the natural disturbance in the gait cycle of the user when walking naturally is determined by the natural disturbance determination module 310, and the translation gain of the user in the virtual reality environment is dynamically adjusted according to the natural disturbance in the gait cycle by the translation gain adjustment module 320; wherein, the greater the natural disturbance, the greater the corresponding translation gain, and the translation gain is used to adjust the ratio of the walking speed of the user in the virtual space to the actual walking speed of the user in the physical space. The device dynamically adjusts the use of translation gain according to the gait cycle of the user, uses the user's own motion disturbance to reduce the inconsistency between the user's proprioception and visual perception, broadens the user's perception threshold of the difference in motion rate between the virtual and physical spaces, and does not increase the probability of motion sickness, thereby improving the overall comfort and immersion of the user.

[0130] It should be noted that the virtual reality mobile device with gait adaptive translation gain provided in the embodiment of the present invention and the virtual reality mobile method with gait adaptive translation gain described in the above embodiments can correspond to each other and will not be described in detail here.

[0131] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the virtual reality movement method of gait adaptive translation gain, the method comprising: a natural disturbance determination module, used to determine the natural disturbance in the gait cycle when the user walks naturally; a translation gain adjustment module, used to dynamically adjust the translation gain of the user in the virtual reality environment according to the natural disturbance in the gait cycle; wherein, the greater the natural disturbance, the greater the corresponding translation gain, and the translation gain is used to adjust the ratio of the walking speed of the user in the virtual space to the actual walking speed of the user in the physical space.

[0132] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0133] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the virtual reality movement method with gait adaptive translation gain provided by the above-mentioned methods. The method includes: a natural disturbance determination module, used to determine the natural disturbance in the gait cycle of the user when walking naturally; a translation gain adjustment module, used to dynamically adjust the translation gain of the user in the virtual reality environment according to the natural disturbance in the gait cycle; wherein, the larger the natural disturbance, the larger the corresponding translation gain, and the translation gain is used to adjust the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space.

[0134] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0135] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual reality movement method with gait adaptive translation gain, characterized in that: include: Determine natural disturbances in the gait cycle of a user as they walk naturally; Dynamically adjust the translation gain of a user in a virtual reality environment based on natural perturbations in the gait cycle; Among them, the larger the natural disturbance is, the larger the corresponding translation gain is, and the translation gain is used to adjust the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space.

2. The virtual reality movement method with gait adaptive translation gain according to claim 1, characterized in that: Determining the natural disturbance in the gait cycle of the user when walking naturally includes: Determine a target period of time in the user's gait cycle where natural disturbances are greatest when the user is walking naturally; Accordingly, dynamically adjusting the translation gain of the user in the virtual reality environment according to the natural disturbance in the gait cycle includes: During the target time period, adjusting the translation gain of the user in the virtual reality environment so that the translation gain increases toward the target; Outside the target time period, adjusting the translation gain of the user in the virtual reality environment to decrease toward the base translation gain; Wherein, the target translation gain is greater than the basic translation gain.

3. The virtual reality movement method with gait adaptive translation gain according to claim 2, characterized in that: The step of determining a target time period in a gait cycle in which natural disturbance is the largest when the user walks naturally comprises: Obtain the physical vertical movement speed value of the head mounted display worn by the user; Calculating the physical vertical movement acceleration of the user's head according to the physical vertical movement speed value; A target time period in which natural disturbance is maximum in a gait cycle of a user walking naturally is determined according to the physical vertical movement acceleration.

4. The virtual reality movement method with gait adaptive translation gain according to claim 3, characterized in that: The step of obtaining the physical vertical movement speed value of the head mounted display worn by the user comprises: Performing smoothing and stabilization processing on the physical vertical moving speed value to obtain a smoothed and stabilized physical vertical moving speed value; Correspondingly, calculating the physical vertical movement acceleration of the user's head according to the physical vertical movement speed value includes: The physical vertical movement acceleration of the user's head is calculated according to the physical vertical movement speed value after smoothing and stabilization.

5. The virtual reality movement method with gait adaptive translation gain according to claim 3, characterized in that: Determining, according to the physical vertical movement acceleration, a target period of time in a gait cycle of a user walking naturally with a maximum natural disturbance, comprising: When the physical vertical movement acceleration of the user's head is greater than a first acceleration threshold, determining a target period of time in a gait cycle when the user enters natural walking and has the largest natural disturbance; When the physical vertical movement acceleration of the user's head is less than or equal to the second acceleration threshold, a target time period in the gait cycle when the user exits natural walking and the natural disturbance is the largest is determined.

6. The virtual reality movement method with gait adaptive translation gain according to claim 2, characterized in that: During the target time period, adjusting the translation gain of the user in the virtual reality environment so that the translation gain increases toward the target, including: Get the user's physical horizontal moving speed vector; Determine the current target translation gain of the user according to the physical horizontal movement speed vector and a forward direction unit vector of a head mounted display worn by the user; In the target time period, taking the target translation gain as a target, gradually increasing the current translation gain; According to the gradually increasing current translation gain, the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space is adjusted.

7. The virtual reality movement method with gait adaptive translation gain according to claim 6, characterized in that: Determining the current target translation gain of the user according to the physical horizontal movement speed vector and a forward direction unit vector of a head mounted display worn by the user includes: Calculate the inner product of the physical horizontal movement velocity vector and the forward direction unit vector, and use the inner product as the physical horizontal movement forward velocity component of the user; The target translation gain is determined according to the forward velocity component of the physical horizontal movement.

8. The virtual reality movement method with gait adaptive translation gain according to claim 6, characterized in that: The adjusting the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space according to the gradually increasing current translation gain specifically includes: Determine the user's current frame physical position and previous frame physical position in the physical space; Calculate the user's current frame physical displacement based on the current frame physical position and the previous frame physical position; Get the user's previous frame virtual position in the virtual space; According to the current frame physical displacement, the previous frame virtual position and the current translation gain, the current frame virtual position of the user in the virtual space is calculated to achieve virtual reality movement.

9. A virtual reality mobile device with gait adaptive translation gain, characterized in that: include: A natural disturbance determination module, used to determine natural disturbances in a gait cycle when a user walks naturally; A translation gain adjustment module for dynamically adjusting the translation gain of a user in a virtual reality environment based on natural disturbances in a gait cycle; Among them, the larger the natural disturbance is, the larger the corresponding translation gain is, and the translation gain is used to adjust the ratio of the user's walking speed in the virtual space to the user's actual walking speed in the physical space.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the virtual reality movement method with gait adaptive translation gain according to any one of claims 1 to 8 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the virtual reality movement method with gait adaptive translation gain according to any one of claims 1 to 8 is implemented.

Images