Virtual ground plane determination method, apparatus, electronic device, and readable storage medium

Abstract

The application discloses a virtual ground plane determination method and device, electronic equipment and a readable storage medium, and relates to the technical field of virtual reality, and is applied to a VR device. The VR device comprises an inertial sensor. The virtual ground plane determination method comprises the following steps: acquiring motion parameters of the VR device based on the inertial sensor; determining the direction of a gravity axis of the VR device according to the motion parameters; and determining a virtual ground plane of the VR device according to the direction of the gravity axis, wherein the direction of the gravity axis is the normal vector direction of the virtual ground plane. The application solves the problem of poor simulation effect of the ground plane of the existing virtual scene.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, and in particular to a method, apparatus, electronic device, and readable storage medium for determining a virtual ground plane. Background Technology

[0002] When using VR (Virtual Reality) devices, it is necessary to first construct a virtual scene that simulates the real scene. However, if the ground plane of the constructed virtual scene does not coincide with the ground plane of the real scene, the user will encounter the problem of walking below the ground or in the air when using VR devices to walk in the simulated scene. The simulation effect of the ground plane of the existing virtual scene is poor. Summary of the Invention

[0003] The main objective of this application is to provide a method, apparatus, electronic device, and readable storage medium for determining a virtual ground plane, aiming to solve the technical problem of poor simulation effect of the ground plane in existing virtual scenes.

[0004] To achieve the above objectives, this application provides a virtual ground plane determination method applied to a VR device, wherein the VR device includes an inertial sensor, and the virtual ground plane determination method includes:

[0005] The motion parameters of the VR device are collected based on the inertial sensor.

[0006] The direction of the gravity axis of the VR device is determined based on the motion parameters.

[0007] The virtual ground plane of the VR device is determined based on the direction of the gravity axis, wherein the direction of the gravity axis is the normal vector direction of the virtual ground plane.

[0008] Optionally, the motion parameters include movement acceleration, and the step of determining the gravity axis direction of the VR device based on the motion parameters includes:

[0009] The VR device is detected as being stationary based on the motion parameters.

[0010] If the VR device is stationary, the direction of the acceleration is taken as the initial direction of the VR device's gravity axis, and the direction of the VR device's gravity axis is determined based on the initial direction.

[0011] Optionally, the motion parameters include rotational angular velocity, and the step of determining the gravity axis direction of the VR device based on the initial direction includes:

[0012] Integrating the rotational angular velocity yields the rotation angle;

[0013] The initial direction is adjusted according to the rotation angle, and the adjusted initial direction is taken as the direction of the gravity axis.

[0014] Optionally, the motion parameters include rotational angular velocity, and the step of detecting whether the VR device is stationary based on the motion parameters includes:

[0015] Determine the acceleration difference between the moving acceleration and the gravitational acceleration;

[0016] Verify whether the acceleration difference is less than a preset acceleration difference and whether the rotational angular velocity is less than a preset angular velocity;

[0017] If the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, then the VR device is determined to be in a stationary state.

[0018] Optionally, the step of determining that the VR device is in a stationary state if the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity includes:

[0019] If the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, then the VR device is determined to be in a stationary state, and it is verified whether the duration of the VR device being in a stationary state has reached the preset duration.

[0020] If the duration reaches the preset duration, the VR device is determined to be in a static state.

[0021] Optionally, the step of verifying whether the duration of the VR device remaining in a stationary state reaches a preset duration includes:

[0022] Increment the number of determinations that the VR device is in a stationary state by one, and verify whether the number of determinations is greater than the preset number of determinations;

[0023] If the number of determinations is greater than the preset number of determinations, then it is determined that the duration of the VR device remaining in a stationary state has reached the preset duration.

[0024] If the number of determinations is less than or equal to the preset number of determinations, then it is determined that the duration has not reached the preset duration.

[0025] Optionally, after the step of detecting whether the VR device is in a stationary state based on the motion parameters, the method further includes:

[0026] If the VR device is not stationary, the linear acceleration on each axis is estimated based on the angular velocity of each axis in the inertial sensor.

[0027] The gravitational acceleration of the VR device is obtained by subtracting the linear acceleration from the accelerometer data in the motion parameters. The gravitational acceleration data is normalized to obtain a unit vector representing the gravity vector, and the direction of the unit vector is taken as the direction of the gravity axis.

[0028] This application also provides a virtual ground plane determination device for use in VR devices, wherein the VR devices include inertial sensors, and the virtual ground plane determination device includes:

[0029] The acquisition module is used to acquire motion parameters of the VR device based on the inertial sensor;

[0030] The first determining module is used to determine the direction of the gravity axis of the VR device based on the motion parameters;

[0031] The second determining module is used to determine the virtual ground plane of the VR device based on the direction of the gravity axis, wherein the direction of the gravity axis is the normal vector direction of the virtual ground plane.

[0032] This application also provides an electronic device, which is a physical device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the virtual ground plane determination method as described above.

[0033] This application also provides a readable storage medium, which is a computer-readable storage medium, storing a program that implements the virtual ground plane determination method. The program that implements the virtual ground plane determination method is executed by a processor to implement the steps of the virtual ground plane determination method as described above.

[0034] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the virtual ground plane determination method described above.

[0035] This application provides a method for determining a virtual ground plane. First, the motion parameters of the VR device are collected based on the inertial sensor; then, the gravity axis direction of the VR device is determined based on the motion parameters; finally, the virtual ground plane of the VR device is determined based on the gravity axis direction, wherein the gravity axis direction is the normal vector direction of the virtual ground plane.

[0036] The gravity axis typically points perpendicularly to the ground plane. This application ensures that the virtual ground plane is perpendicular to the gravity axis by aligning its normal vector with the gravity axis, thus aligning the virtual ground plane with the real ground plane. This solves the technical problem of poor ground plane simulation in existing virtual scenes and improves the simulation effect of the ground plane in virtual scenes. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating an embodiment of the virtual ground plane determination method of this application.

[0040] Figure 2 This is a flowchart illustrating Embodiment 2 of the virtual ground plane determination method of this application.

[0041] Figure 3 A simplified flowchart illustrating the process of determining a virtual ground plane as provided in Embodiment 2 of this application;

[0042] Figure 4 This is a schematic diagram of the module structure of the virtual ground plane determination device according to an embodiment of this application;

[0043] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the virtual ground plane determination method in the embodiments of this application.

[0044] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0045] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example 1

[0047] When using VR (Virtual Reality) devices, it is necessary to first construct a virtual scene that simulates the real scene. However, if the ground plane of the constructed virtual scene does not coincide with the ground plane of the real scene, the user will encounter the problem of walking below the ground or in the air when using VR devices to walk in the simulated scene. The simulation effect of the ground plane of the existing virtual scene is poor.

[0048] To address the aforementioned issues, this application proposes a method that uses an IMU (Inertial Measurement Unit) to determine the direction of the gravity axis, and then uses the direction of the gravity axis as the direction of the normal vector to calculate the ground plane of the virtual scene, thereby ensuring that the ground plane of the virtual scene is parallel to the ground plane of the real scene.

[0049] Specifically, this application proposes a virtual ground plane determination method according to the first embodiment, applied to a VR device. The VR device includes an inertial sensor. Please refer to... Figure 1 The virtual ground plane determination method includes:

[0050] Step S10: Collect motion parameters of the VR device based on the inertial sensor;

[0051] It should be noted that the inertial sensor is used to collect the motion parameters of the VR device. The inertial sensor can be an IMU. The motion parameters are used to characterize the motion of the VR device, and these motion parameters can include movement acceleration, rotational angular velocity, etc.

[0052] Step S20: Determine the direction of the gravity axis of the VR device based on the motion parameters;

[0053] In one possible implementation, the direction of motion related to gravitational acceleration in the motion parameters can be directly used as the direction of gravity axis. In another possible implementation, the motion state of the VR device can be determined first based on the motion parameters, and then the direction of gravity axis of the VR device can be determined in a corresponding manner according to the motion state of the VR device. For example, when the VR device is in a stationary state, the direction of the motion acceleration can be optimized to obtain the direction of gravity axis. When the VR device is in a moving state, the direction of the motion acceleration can be directly used as the direction of gravity axis.

[0054] Step S30: Determine the virtual ground plane of the VR device based on the gravity axis direction, wherein the gravity axis direction is the normal vector direction of the virtual ground plane.

[0055] Determining the virtual ground plane of a VR device based on the direction of the gravity axis can be achieved by using the gravity axis direction as the normal vector direction. Alternatively, it can be done by using the gravity axis direction as the normal vector direction of the virtual scene's ground plane, while simultaneously adjusting the height of the virtual scene's ground plane, and finally using the resulting virtual plane as the virtual ground plane.

[0056] Specifically, in one possible implementation, if the VR device is in the process of initializing the virtual scene, the virtual ground plane obtained by using the gravity axis direction as the normal vector direction can be: generating the virtual ground plane by using the gravity axis direction as the normal vector direction; in another possible implementation, if the VR device has already completed the initialization of the virtual scene, and an initialized virtual ground plane has already been generated, the virtual ground plane obtained by using the gravity axis direction as the normal vector direction can be: rotating, offsetting, or performing other operations on the initialized virtual ground plane by using the gravity axis direction as the normal vector direction to obtain the actual virtual ground plane of the VR device.

[0057] It should be noted that the virtual ground plane determination process proposed in this embodiment can occur during virtual scene initialization or during the use of the VR device after virtual scene initialization; no limitation is made here. Specifically, virtual ground plane adjustment can be triggered by pre-set conditions. For example, it can be triggered by user commands or by detecting that the user is in a stationary state; no specific limitation is made here.

[0058] In this embodiment, motion parameters of the VR device are first collected based on inertial sensors; then the direction of the gravity axis of the VR device is determined based on the motion parameters; finally, the virtual ground plane of the VR device is determined based on the direction of the gravity axis, wherein the direction of the gravity axis is the normal vector direction of the virtual ground plane.

[0059] The gravity axis typically points perpendicularly to the ground plane. In this embodiment, the normal vector of the virtual ground plane is aligned with the gravity axis to ensure that the virtual ground plane is perpendicular to the gravity axis, thus aligning the virtual ground plane with the real ground plane. This solves the technical problem of poor ground plane simulation in existing virtual scenes and improves the simulation effect of the ground plane in virtual scenes.

[0060] It should be noted that using the gravity axis direction as the normal vector direction for the virtual ground plane only guarantees that the virtual ground plane is parallel to the real ground plane. However, in the actual use of VR devices, there may be situations where the virtual ground plane and the real scene's ground plane are not aligned at the same height. In this case, when users walk in the simulated scene, they may experience the problem of walking below the ground or into the air. Therefore, using the gravity axis direction as the normal vector direction for the virtual scene's ground plane, while adjusting the height of the virtual scene's ground plane, can ensure that the determined virtual ground plane coincides with the real ground plane, allowing the virtual ground plane simulated in the VR device to be more accurately aligned with the real ground plane.

[0061] Based on this, in one possible implementation, after determining the virtual ground plane of the VR device according to the gravity axis direction, the method further includes: aligning the heights of the virtual ground plane and the real ground plane. The specific alignment process is not limited here; for example, it can be to obtain the height of the real ground plane using ground images, ground depth images, etc., and then align the heights of the virtual ground plane and the real ground plane; or it can be to obtain the user's visual height relative to the real ground plane of the external environment in which the VR device is located; obtain the user's visual height relative to the real ground plane of the external environment in which the VR device is located; and align the heights of the virtual ground plane and the real ground plane according to the visual height. This implementation ensures that the virtual ground plane and the real ground plane are aligned, and also aligns their heights, thereby improving the simulation effect of the ground plane in the virtual scene.

[0062] Example 2

[0063] Based on the first embodiment of this application, in another embodiment of this application, the same or similar content as in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 The motion parameters include acceleration, and the step of determining the gravity axis direction of the VR device based on the motion parameters includes:

[0064] Step S21: Detect whether the VR device is in a stationary state based on the motion parameters;

[0065] When a VR device moves, the inertial sensor generates motion data in any direction. Therefore, based on motion parameters, it is possible to detect whether the VR device is stationary. There are no restrictions on the specific parameters used for detection; they can be set according to actual needs.

[0066] Step S22: If the VR device is stationary, the direction of the acceleration is taken as the initial direction of the VR device's gravity axis, and the direction of the VR device's gravity axis is determined based on the initial direction.

[0067] If the VR device is stationary, the inertial sensor is only affected by gravity, and the direction of the gravity axis can be determined at this time. It should be noted that when determining the gravity axis direction of the VR device based on the initial gravity axis direction, the initial gravity axis direction can be directly used as the gravity axis direction, or the optimized initial gravity axis direction can be used as the gravity axis direction after optimization. This embodiment does not limit this.

[0068] In this embodiment, when the VR device is in the virtual scene initialization state, it is first detected whether the VR device is in a stationary state. If the VR device is in a stationary state, it means that the VR device is only affected by gravity and not by other external forces. Therefore, the direction of the acceleration of movement can be taken as the initial gravity axis direction of the VR device. Finally, based on the initial gravity axis direction, the gravity axis direction of the VR device is determined. Thus, this embodiment determines the gravity axis direction when the VR device is in a stationary state to ensure that the determined gravity axis direction is the direction of the VR device's gravity, thereby ensuring the accuracy of the gravity axis direction determination, that is, ensuring that the determined gravity axis direction is close to the actual gravity direction of the VR device.

[0069] Furthermore, in this embodiment, if the VR device is not stationary, the direction of the gravity axis cannot be accurately obtained. In one possible embodiment, the direction of the VR device's acceleration can be directly used as the direction of the gravity axis. Although the direction of the VR device's acceleration is not the direction of gravity, it can provide a definite basis for determining the virtual ground plane later. In another possible embodiment, the direction of the gravity axis can be separated based on motion parameters, and no specific limitation is made here.

[0070] In one possible implementation, the motion parameters include rotational angular velocity, and the step of determining the gravity axis orientation of the VR device based on the initial gravity axis orientation includes:

[0071] Step S221: Integrate the rotational angular velocity to obtain the rotation angle;

[0072] The rotational angular velocity of the inertial sensor is integrated to obtain the cumulative rotational angular velocity, which in turn yields the rotation angle of the VR device. The specific integration process is not limited here.

[0073] It should be noted that the integral error of the rotational angular velocity will gradually accumulate over time. This is because the inertial sensor itself may have a small error, which means that the obtained rotation angle may drift over time. In order to reduce the integral error, other sensor data, such as accelerometer data, can be used for attitude calibration to help eliminate or reduce the error introduced by the inertial sensor integration.

[0074] Step S222: Optimize the initial gravity axis direction based on the rotation angle to obtain the gravity axis direction of the VR device.

[0075] It should be noted that when optimizing the initial gravity axis direction based on the rotation angle, the Visual Inertial Alignment (VIA) algorithm and the Refine Gravity (RG) algorithm can be used to optimize the initial gravity axis direction based on the rotation angle. Alternatively, various toolkits (Manifold ToolKit, MTK) can be used to optimize the initial gravity axis direction based on the rotation angle. This embodiment does not specifically limit the specific optimization method.

[0076] In this embodiment, the rotational angular velocity is first integrated to obtain the rotation angle. Then, the initial gravity axis direction is optimized based on the rotation angle so that the final determined gravity axis direction of the VR device can approximately approximate the direction of gravity experienced by the VR device in the current pose. This improves the accuracy of the gravity axis direction determination and, consequently, the accuracy of the subsequently determined virtual ground plane.

[0077] In one possible implementation, the motion parameters include rotational angular velocity, and the step of detecting whether the VR device is stationary based on the motion parameters includes:

[0078] Step S211: Determine the acceleration difference between the moving acceleration and the gravitational acceleration;

[0079] It should be noted that motion acceleration refers to the acceleration of the VR device along the direction of movement. This motion acceleration is actually a combined value of gravitational acceleration and the acceleration generated by the movement of the VR device.

[0080] Step S212: Verify whether the acceleration difference is less than a preset acceleration difference and whether the rotational angular velocity is less than a preset angular velocity;

[0081] The difference between the acceleration due to movement and the acceleration due to gravity can reflect whether the VR device is moving in a direction other than the direction of gravity, and the rotational angular velocity can reflect whether the VR device is rotating. Therefore, in this embodiment, the VR device is determined to be stationary by checking whether the acceleration difference is less than a preset acceleration difference and whether the rotational angular velocity is less than a preset angular velocity.

[0082] Step S213: If the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, then the VR device is determined to be in a stationary state.

[0083] If the acceleration difference is less than the preset acceleration difference, it means that the VR device has not moved in any direction other than the direction of gravity. If the rotational angular velocity is less than the preset angular velocity, it means that the VR device has not rotated. Therefore, if the acceleration difference is less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, it is determined that the VR device is in a stationary state.

[0084] In this embodiment, when the VR device is in the virtual scene initialization state, the acceleration difference between the movement acceleration and the gravitational acceleration is first determined. Then, it is checked whether this acceleration difference is less than a preset acceleration difference and whether the rotational angular velocity is less than a preset angular velocity. If the acceleration difference is less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, it indicates that the VR device is only affected by gravity and not by other external forces. Therefore, it can be determined that the VR device is in a stationary state. When the VR device is in a stationary state, the direction of the movement acceleration can be used as the initial gravity axis direction of the VR device. Finally, based on this initial gravity axis direction, the gravity axis direction of the VR device is determined. Thus, this embodiment ensures that the determined gravity axis direction is the direction of the VR device's gravity by determining the gravity axis direction when the VR device is in a stationary state, thereby ensuring the accuracy of the gravity axis direction determination, that is, ensuring that the determined gravity axis direction is close to the actual gravity direction of the VR device.

[0085] In one possible implementation, the step of determining that the VR device is in a stationary state if the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity includes:

[0086] Step S2131: If the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, then the VR device is determined to be in a stationary state.

[0087] If the acceleration difference is less than a preset acceleration difference and the rotational angular velocity is less than a preset angular velocity, the VR device is determined to be in a stationary state at a given time step. This determined state is instantaneous; the VR device may move in the next time step, and it cannot be completely guaranteed that the VR device is stationary. To accurately determine the stationary state of the VR device, a duration detection is introduced when detecting whether the VR device is stationary. That is, if a prolonged period of stationary state is detected, the VR device is determined to be stationary.

[0088] Step S2132: Verify whether the duration of the VR device remaining in a stationary state has reached the preset duration;

[0089] Verify whether the duration of the VR device remaining stationary has reached a preset duration. In one possible implementation, the duration can be directly detected; in another possible implementation, the duration can be detected by accumulating the number of consecutive times the device remains stationary.

[0090] Step S2133: If the duration reaches the preset duration, then the VR device is determined to be in a static state.

[0091] If the duration reaches the preset duration, it is determined that the VR device has been in a stationary state for an extended period of time.

[0092] In this embodiment, the VR device can be determined to be stationary when it has been stationary for an extended period of time. This more accurate determination of the stationary state ensures that the gravity axis direction is the direction of the VR device's gravity when the VR device is stationary, thereby improving the accuracy of the gravity axis direction.

[0093] In one possible implementation, the step of verifying whether the duration of the VR device remaining in a stationary state has reached a preset duration includes:

[0094] Step S21321: Increment the number of determinations that the VR device is in a stationary state by one, and verify whether the number of determinations is greater than the preset number of determinations;

[0095] In this embodiment, the number of consecutive times the VR device is in a stationary state is accumulated, and the duration is detected by detecting the number of consecutive counts. Specifically, the number of times the VR device is determined to be in a stationary state is incremented by one, and it is verified whether the number of determinations is greater than a preset number of determinations.

[0096] Step S21322: If the number of determinations is found to be greater than the preset number of determinations, then it is determined that the duration of the VR device in the stationary state has reached the preset duration.

[0097] If the number of checks is greater than the preset number of checks, then it is determined that the duration of the VR device remaining in a stationary state has reached the preset duration.

[0098] Step S2133: If the number of determinations is less than or equal to the preset number of determinations, then it is determined that the duration has not reached the preset duration.

[0099] If the number of judgments is less than or equal to the preset number of judgments, it is determined that the duration has not reached the preset duration. At this time, the VR device's acceleration and rotational angular velocity in the next time step can be collected based on the inertial sensor, and the steps of checking whether the acceleration difference is less than the preset acceleration difference and whether the rotational angular velocity is less than the preset angular velocity can be returned.

[0100] Furthermore, in this embodiment, if the acceleration difference is found to be greater than or equal to a preset acceleration difference, or the rotational angular velocity is greater than or equal to a preset angular velocity, the number of determinations that the VR device is in a stationary state is set to zero, so as to avoid the current count interfering with the subsequent determination of the VR device state.

[0101] In this embodiment, if the acceleration difference is found to be less than a preset acceleration difference and the rotational angular velocity is found to be less than a preset angular velocity, the number of determinations that the VR device is in a stationary state is incremented by one, and it is checked whether this number of determinations is greater than a preset number of determinations. If the number of determinations is greater than the preset number of determinations, it indicates that the VR device has been stationary for a long time, and the direction of the acceleration is taken as the initial gravity axis direction of the VR device. If the number of determinations is less than or equal to the preset number of determinations, the acceleration and rotational angular velocity of the VR device in the next time step are collected based on the inertial sensor, and the above process of determining that the VR device is stationary is repeated. During this process, if the acceleration difference is found to be greater than or equal to the preset acceleration difference, or the rotational angular velocity is found to be greater than or equal to the preset angular velocity, it indicates that the VR device is not only affected by gravity but also by other external forces, i.e., by the force generated by the movement of the VR device. Therefore, it can be determined that the VR device is in motion. This embodiment ensures that the gravity axis direction is closer to the actual gravity direction of the VR device by ensuring that the VR device is stationary for a long time before determining the gravity axis direction, thus further improving the accuracy of the gravity axis direction determination.

[0102] In one possible implementation, after the step of detecting whether the VR device is stationary based on the motion parameters, the method further includes:

[0103] Step S23: If the VR device is not stationary, estimate the linear acceleration on each axis based on the angular velocity of each axis in the inertial sensor.

[0104] In this embodiment, if the VR device is not stationary, the inertial sensor may be affected by forces in multiple directions. In this case, it is necessary to separate the gravitational acceleration from the data collected by the inertial sensor and determine the direction of the gravitational acceleration so as to use the direction of the gravitational acceleration as the direction of the gravity axis.

[0105] Specifically, the linear acceleration on each axis is estimated based on the angular velocity of each axis in the inertial sensor, that is, the non-gravitational acceleration. Specifically, the angular velocity can be integrated, the angle can be obtained by integrating the angular velocity, and the linear acceleration can be obtained by integrating the angle. The specific integration process is not limited here.

[0106] Step S24: Subtract the linear acceleration from the accelerometer data in the motion parameters to obtain the gravitational acceleration of the VR device, normalize the gravitational acceleration data to obtain a unit vector representing the gravity vector, and take the direction of the unit vector as the direction of the gravity axis.

[0107] Accelerometer data is a superposition of gravitational acceleration and linear acceleration. Therefore, the gravitational acceleration of the VR device can be obtained by subtracting the linear acceleration from the accelerometer data in the motion parameters.

[0108] Normalizing gravitational acceleration data involves obtaining a unit vector representing the direction of gravity from the data, and using the direction of this unit vector as the direction of the gravity axis. The specific implementation of normalizing gravitational acceleration data can be as follows: Construct a unit vector using the three-axis components of gravitational acceleration; that is, calculate the proportion of each component of gravitational acceleration and then combine them into a single unit vector, which represents the direction of gravity.

[0109] In this embodiment, the separation of gravitational acceleration is determined when the VR device is not stationary, and the direction of gravitational acceleration is taken as the direction of gravity axis. Then, the virtual ground plane can be determined according to the direction of gravity axis. This embodiment enables the virtual ground plane to be aligned with the real ground plane even when the VR device is not stationary, thereby improving the simulation effect of the ground plane in the virtual scene.

[0110] For example, to aid in understanding the technical concept or principles of this application, please refer to Figure 3 , Figure 3 A simplified flowchart for determining a virtual ground plane is provided. Assuming a preset angular velocity of 1° / s, a preset acceleration difference of 0.1, a gravitational acceleration of 9.8, and a preset number of judgments of 300, the specific process for determining the virtual ground plane is as follows:

[0111] First, after acquiring the movement acceleration and rotational angular velocity of the VR device through the IMU in the VR device, the difference between the movement acceleration and the gravitational acceleration of 9.8 is determined. If the acceleration difference is less than 0.1 and the rotational angular velocity is less than 1° / s, the VR device is determined to be in a stationary state (that is, if the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, the VR device is determined to be in a stationary state).

[0112] The number of times C is used to determine if the VR device is in a static state. 静止 Increment the count by one and continue the above judgment process, checking if the cumulative number of judgments exceeds 300. The count C is the number of judgments that determined the VR device to be in a static state. 静止 When the number of judgments exceeds 300, the gravity axis is calibrated, and the cumulative number of judgments C is recorded. 静止 The process involves resetting the count (which increments the count of the VR device being in a stationary state by one and checks if the count is greater than a preset count; if the count is greater than the preset count, the duration of the VR device being in a stationary state is determined to have reached a preset duration; if the duration reaches the preset duration, the VR device is determined to be in a stationary state).

[0113] The gravity axis is calibrated, that is, the direction of the movement acceleration is used as the initial gravity axis direction to complete the initialization of the gravity axis. Then, the initial gravity axis direction is optimized using the acceleration difference to obtain the final gravity axis direction of the VR device (that is, if the VR device is in a stationary state, the direction of the movement acceleration of the VR device is used as the initial direction of the gravity axis of the VR device, and the gravity axis direction of the VR device is determined based on the initial direction).

[0114] If the acceleration difference is not less than 0.1 and the rotational angular velocity is not less than 1° / s, then the VR device is determined not to be stationary, and the cumulative number of determinations is C. 静止 The system is then reset to zero. At this point, the acceleration and direction are recalculated to obtain the direction of the gravity axis (i.e., if the VR device is not stationary, the linear acceleration on each axis is estimated based on the angular velocity of each axis in the inertial sensor; the linear acceleration is subtracted from the accelerometer data in the motion parameters to obtain the gravitational acceleration of the VR device; the gravitational acceleration data is normalized to obtain a unit vector representing the gravity vector; and the direction of the unit vector is taken as the direction of the gravity axis).

[0115] After obtaining the direction of the gravity axis, the ground plane is calculated using the gravity axis as the normal vector (that is, the virtual ground plane of the VR device is determined based on the direction of the gravity axis, wherein the direction of the gravity axis is the normal vector direction of the virtual ground plane).

[0116] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the virtual ground plane determination method of this application. Any simple transformations based on this technical concept are all within the protection scope of this application.

[0117] Example 3

[0118] This invention also provides a virtual ground plane determination device for use in VR devices. The VR device includes an inertial sensor. (Please refer to...) Figure 4 The virtual ground plane determination device includes:

[0119] Acquisition module 10 is used to acquire motion parameters of the VR device based on the inertial sensor;

[0120] The first determining module 20 is used to determine the direction of the gravity axis of the VR device based on the motion parameters;

[0121] The second determining module 30 is used to determine the virtual ground plane of the VR device based on the gravity axis direction, wherein the gravity axis direction is the normal vector direction of the virtual ground plane.

[0122] Optionally, the motion parameters include movement acceleration, and the first determining module 20 is further configured to:

[0123] The VR device is detected as being stationary based on the motion parameters.

[0124] If the VR device is stationary, the direction of the acceleration is taken as the initial direction of the VR device's gravity axis, and the direction of the VR device's gravity axis is determined based on the initial direction.

[0125] Optionally, the motion parameters include rotational angular velocity, and the first determining module 20 is further configured to:

[0126] Integrating the rotational angular velocity yields the rotation angle;

[0127] The initial direction is adjusted according to the rotation angle, and the adjusted initial direction is taken as the direction of the gravity axis.

[0128] Optionally, the motion parameters include rotational angular velocity, and the first determining module 20 is further configured to:

[0129] Determine the acceleration difference between the moving acceleration and the gravitational acceleration;

[0130] Verify whether the acceleration difference is less than a preset acceleration difference and whether the rotational angular velocity is less than a preset angular velocity;

[0131] If the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, then the VR device is determined to be in a stationary state.

[0132] Optionally, the first determining module 20 is further configured to:

[0133] If the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, then the VR device is determined to be in a stationary state, and it is verified whether the duration of the VR device being in a stationary state has reached the preset duration.

[0134] If the duration reaches the preset duration, the VR device is determined to be in a static state.

[0135] Optionally, the first determining module 20 is further configured to:

[0136] Increment the number of determinations that the VR device is in a stationary state by one, and verify whether the number of determinations is greater than the preset number of determinations;

[0137] If the number of determinations is greater than the preset number of determinations, then it is determined that the duration of the VR device remaining in a stationary state has reached the preset duration.

[0138] If the number of determinations is less than or equal to the preset number of determinations, then it is determined that the duration has not reached the preset duration.

[0139] Optionally, the first determining module 20 is further configured to:

[0140] If the VR device is not stationary, the linear acceleration on each axis is estimated based on the angular velocity of each axis in the inertial sensor.

[0141] The gravitational acceleration of the VR device is obtained by subtracting the linear acceleration from the accelerometer data in the motion parameters. The gravitational acceleration data is normalized to obtain a unit vector representing the gravity vector, and the direction of the unit vector is taken as the direction of the gravity axis.

[0142] The virtual ground plane determination device provided by this invention, employing the virtual ground plane determination method in Embodiment 1 or Embodiment 2 described above, can solve the technical problem of poor simulation effect of the ground plane in existing virtual scenes. Compared with the prior art, the beneficial effects of the virtual ground plane determination device provided by this invention are the same as the beneficial effects of the virtual ground plane determination method provided in the above embodiments, and other technical features in the virtual ground plane determination device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0143] Example 4

[0144] This invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the virtual ground plane determination method in Embodiment 1 above.

[0145] The following is for reference. Figure 5 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0146] like Figure 5 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. While electronic devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0147] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of embodiments of this disclosure.

[0148] The electronic device provided by this invention, employing the virtual ground plane determination method in the above embodiments, can solve the technical problem of poor simulation effect of the ground plane in existing virtual scenes. Compared with the prior art, the beneficial effects of the electronic device provided by this invention are the same as those of the virtual ground plane determination method provided in the above embodiments, and other technical features of this electronic device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0149] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0150] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0151] Example 5

[0152] This invention provides a computer-readable storage medium having computer-readable program instructions stored thereon, which are used to execute the virtual ground plane determination method in Embodiment 1 above.

[0153] The computer-readable storage medium provided in this embodiment of the invention may be, for example, a USB flash drive, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0154] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0155] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to: acquire a ground plane depth map of the gravity axis direction of the VR device and the real ground plane; determine each connected plane in the ground plane depth map; select, from each of the connected planes, a connected plane whose angle with the perpendicular direction to the gravity axis direction is less than a preset angle as a target plane; and select the target plane with the lowest height among the target planes as a virtual ground plane.

[0156] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0157] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0158] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0159] The readable storage medium provided by this invention is a computer-readable storage medium that stores computer-readable program instructions for executing the above-described virtual ground plane determination method, thereby solving the technical problem of poor ground plane simulation in existing virtual scenes. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this embodiment are the same as those of the virtual ground plane determination method provided in Embodiment 1 or Embodiment 2, and will not be repeated here.

[0160] Example 6

[0161] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the virtual ground plane determination method described above.

[0162] The computer program product provided in this application can solve the technical problem of poor simulation effect of the ground plane in existing virtual scenes. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of the present invention are the same as the beneficial effects of the virtual ground plane determination method provided in Embodiment 1 or Embodiment 2 above, and will not be repeated here.

[0163] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A method for determining a virtual ground plane, characterized in that, Applied to VR devices, the VR devices including inertial sensors, the virtual ground plane determination method includes: The motion parameters of the VR device are collected based on the inertial sensor. The direction of the gravity axis of the VR device is determined based on the motion parameters. The virtual ground plane of the VR device is determined based on the direction of the gravity axis, wherein the direction of the gravity axis is the normal vector direction of the virtual ground plane; The motion parameters include movement acceleration, and the step of determining the gravity axis direction of the VR device based on the motion parameters includes: If the VR device is stationary, the direction of the acceleration is taken as the initial direction of the VR device's gravity axis, and the direction of the VR device's gravity axis is determined based on the initial direction. If the VR device is not in the stationary state, the linear acceleration on each axis is estimated based on the angular velocity of each axis in the inertial sensor. The linear acceleration is obtained by subtracting the accelerometer data in the motion parameters from the linear acceleration. The gravitational acceleration data is normalized to obtain a unit vector representing the gravity vector. The direction of the unit vector is taken as the direction of the gravity axis. The step of determining the virtual ground plane of the VR device based on the direction of the gravity axis includes: Obtain a ground plane depth map of the actual ground plane, and determine each connected plane in the ground plane depth map; Among the connected planes, those whose angle with the perpendicular direction to the direction of the gravity axis is less than a preset angle are selected as target planes; The target plane with the lowest height among all the target planes is used as the virtual ground plane of the VR device.

2. The virtual ground plane determination method as described in claim 1, characterized in that, The motion parameters include rotational angular velocity, and the step of determining the gravity axis direction of the VR device based on the initial direction includes: Integrating the rotational angular velocity yields the rotation angle; The initial direction is adjusted according to the rotation angle, and the adjusted initial direction is taken as the direction of the gravity axis.

3. The virtual ground plane determination method as described in claim 1, characterized in that, The motion parameters include rotational angular velocity, and the step of detecting whether the VR device is stationary based on the motion parameters includes: Determine the acceleration difference between the moving acceleration and the gravitational acceleration; Verify whether the acceleration difference is less than a preset acceleration difference and whether the rotational angular velocity is less than a preset angular velocity; If the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, then the VR device is determined to be in a stationary state.

4. The virtual ground plane determination method as described in claim 3, characterized in that, The step of determining that the VR device is in a stationary state if the acceleration difference is less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity includes: If the acceleration difference is found to be less than the preset acceleration difference and the rotational angular velocity is less than the preset angular velocity, then the VR device is determined to be in a stationary state, and it is verified whether the duration of the VR device being in a stationary state has reached the preset duration. If the duration reaches the preset duration, the VR device is determined to be in a static state.

5. The virtual ground plane determination method as described in claim 4, characterized in that, The step of verifying whether the duration of the VR device remaining in a stationary state has reached the preset duration includes: Increment the number of determinations that the VR device is in a stationary state by one, and verify whether the number of determinations is greater than the preset number of determinations; If the number of determinations is greater than the preset number of determinations, then it is determined that the duration of the VR device remaining in a stationary state has reached the preset duration. If the number of determinations is less than or equal to the preset number of determinations, then it is determined that the duration has not reached the preset duration.

6. A virtual ground plane determination device, characterized in that, Applied to VR devices, the VR devices including inertial sensors, the virtual ground plane determination device includes: The acquisition module is used to acquire motion parameters of the VR device based on the inertial sensor; The first determining module is used to determine the direction of the gravity axis of the VR device based on the motion parameters; The second determining module is used to determine the virtual ground plane of the VR device based on the direction of the gravity axis, wherein the direction of the gravity axis is the normal vector direction of the virtual ground plane; The motion parameters include motion acceleration. The first determining module is further configured to: if the VR device is stationary, use the direction of the motion acceleration as the initial direction of the VR device's gravity axis, and determine the direction of the VR device's gravity axis based on the initial direction; if the VR device is not stationary, estimate the linear acceleration on each axis based on the angular velocity of each axis in the inertial sensor, subtract the linear acceleration from the accelerometer data in the motion parameters to obtain the gravitational acceleration of the VR device, normalize the gravitational acceleration data to obtain a unit vector representing the gravity vector, and use the direction of the unit vector as the direction of the gravity axis. The second determining module is further configured to: acquire a ground plane depth map of the real ground plane, determine each connected plane in the ground plane depth map; select the connected planes among the connected planes whose angle with the perpendicular direction to the gravity axis is less than a preset angle as target planes; and select the target plane with the lowest height among the target planes as the virtual ground plane of the VR device.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the steps of the virtual ground plane determination method as described in any one of claims 1 to 5.

8. A readable storage medium, characterized in that, The readable storage medium is a computer-readable storage medium, and the computer-readable storage medium stores a program that implements the virtual ground plane determination method. The program that implements the virtual ground plane determination method is executed by a processor to implement the steps of the virtual ground plane determination method as described in any one of claims 1 to 5.

Images