Spatial positioning method and device of handle and virtual reality system

Abstract

The embodiment of the invention discloses a spatial positioning method and device of a handle and a virtual reality system, and relates to the technical field of virtual reality. The method comprises the steps of determining whether a handle is shielded or not in the process of performing spatial positioning on the handle based on an inside-out positioning system; when the handle is shielded, acquiring historical positioning data of the handle; according to the historical positioning data, motion parameters of the handle are obtained through calculation, and according to the motion parameters, predicted poses of the handle at the subsequent sampling moment are obtained through calculation and serve as space poses of the handle at the subsequent sampling moment. Spatial positioning of the handle can still be kept when the handle is shielded, the corresponding spatial pose of the handle is obtained, and continuous tracking of the pose of the handle is guaranteed.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, and in particular to a spatial positioning method, device and virtual reality system for a controller. Background Technology

[0002] With the development of virtual reality technology, immersive experiences in large spaces with multiple users have become an industry trend. In this scenario, accurate, stable, and continuous spatial positioning of the head-mounted display and controllers (i.e., hands) is crucial for the user interaction experience.

[0003] In large, multi-person environments, head-mounted displays typically employ Lighthouse positioning based on laser scanning principles to provide high-precision and stable positioning. The controllers, on the other hand, use inside-out positioning, which relies on cameras mounted on the head-mounted display to identify markers on the controllers for positioning.

[0004] However, in this method, when the controller is blocked by the user's body or other objects, the head-mounted display's camera cannot properly recognize the markers on the controller, which will cause the controller's spatial positioning to be interrupted, seriously affecting the user experience. Summary of the Invention

[0005] In view of this, this application provides a spatial positioning method, device, and virtual reality system for a controller, so as to improve the continuity of the controller's spatial positioning and ensure user experience.

[0006] This application provides the following solution: Firstly, a spatial positioning method for a controller is provided, the controller being used to enable users to perform virtual reality interactions in a virtual reality scene, the method including: During the spatial positioning of the handle based on the inside-out positioning system, it is determined whether the handle is obstructed. When the controller is obstructed, acquire the controller's historical positioning data; The motion parameters of the handle are calculated based on historical positioning data; Based on the motion parameters, the predicted pose of the handle at subsequent sampling times is calculated and used as the spatial pose of the handle at subsequent sampling times.

[0007] Optionally, during the spatial positioning of the handle based on the inside-out positioning system, determining whether the handle is obstructed includes: During the spatial positioning of the handle based on the inside-out positioning system, if an abnormality in the handling recognition is detected and / or a sudden change in the spatial pose of the handle is obtained, it is determined that the handle is occluded.

[0008] Optionally, motion parameters include the speed and acceleration of the handle during movement.

[0009] Optionally, based on the motion parameters, the predicted pose of the handle at subsequent sampling times is calculated, including: The predicted pose is calculated based on the motion parameters using a uniform motion model or a uniformly accelerated motion model.

[0010] Optionally, the predicted pose is calculated based on the motion parameters using a uniform motion model or a uniformly accelerated motion model, including: The predicted pose is calculated based on the motion parameters using a uniform motion model or a uniformly accelerated motion model, and then optimized using a Kalman filter algorithm.

[0011] Optionally, the motion parameters include the angular velocity and angular acceleration of the handle during movement.

[0012] Optionally, based on the motion parameters, the predicted pose of the handle at subsequent sampling times is calculated, including: The predicted pose is calculated based on the motion parameters using a uniform angular motion model or a uniformly accelerated angular motion model.

[0013] Optionally, the predicted pose is calculated based on the motion parameters using a uniform angular motion model or a uniformly accelerated angular motion model, including: The predicted pose is calculated based on the motion parameters using a uniform angular motion model or a uniformly accelerated angular motion model, and then optimized using a Kalman filter algorithm.

[0014] Optionally, based on the motion parameters, the predicted pose of the handle at subsequent sampling times is calculated as the spatial pose of the handle at subsequent sampling times, including: Based on the motion parameters, the predicted pose of the handle at subsequent sampling times is calculated; When the distance between the predicted pose and the current spatial pose of the head-mounted display is less than or equal to a preset threshold, the predicted pose is used as the spatial pose of the handle at subsequent sampling times.

[0015] Optionally, the method further includes: When the handle is no longer obstructed, acquire the current measured spatial pose of the handle based on the inside-out positioning system for spatial positioning. The spatial pose of the handle is obtained by fusing the current measured spatial pose and the predicted pose at the current sampling time.

[0016] Optionally, the spatial pose of the handle is obtained by fusing the current measured spatial pose and the predicted pose at the current sampling time, including: The pose located between the current measurement spatial pose and the predicted pose at the current sampling time is taken as the spatial pose of the handle.

[0017] Secondly, a spatial positioning device for a controller is provided. The controller is used to enable users to perform virtual reality interaction in a virtual reality scene. The device includes: The determination module is used to determine whether the handle is obstructed during the spatial positioning of the handle based on the inside-out positioning system. The prediction module is used to acquire the historical positioning data of the handle when it is occluded; calculate the motion parameters of the handle based on the historical positioning data; and calculate the predicted pose of the handle at subsequent sampling times based on the motion parameters, so as to serve as the spatial pose of the handle at subsequent sampling times.

[0018] Thirdly, a virtual reality system is provided, including a controller; a head-mounted display device; and the apparatus described in the second aspect above.

[0019] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0020] Fifthly, an electronic device is provided, comprising: One or more processors; and A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first aspects above.

[0021] A sixth aspect provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0022] According to the specific embodiments provided in this application, the following technical effects are disclosed: The solution provided in this application allows for the determination of whether the controller is occluded during spatial positioning based on an inside-out positioning system. When the controller is occluded, its motion parameters can be calculated using historical positioning data. Then, based on these motion parameters, the predicted pose of the controller at subsequent sampling times can be calculated as its corresponding spatial pose. This method enables the prediction of the controller's subsequent motion after occlusion based on its historical positioning data before occlusion, thus obtaining the predicted pose as its corresponding spatial pose. This ensures continuous tracking of the controller's pose while it is occluded, maintaining spatial positioning and obtaining the corresponding spatial pose even during occlusion. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the virtual reality system corresponding to this application.

[0025] Figure 2 This is a flowchart illustrating the spatial positioning method for a handle provided in an embodiment of this application.

[0026] Figure 3 This application provides a schematic diagram of the spatial positioning device for a handle.

[0027] Figure 4 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0029] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0030] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0031] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0032] With the development of virtual reality technology, immersive experiences in large spaces with multiple users have become an industry trend. In this scenario, accurate, stable, and continuous spatial positioning of the head-mounted display and controllers (i.e., hands) is crucial for the user interaction experience.

[0033] In large, multi-person environments, head-mounted displays typically employ Lighthouse positioning based on laser scanning principles to provide high-precision and stable positioning. The controllers, on the other hand, use inside-out positioning, which relies on cameras mounted on the head-mounted display to identify markers on the controllers for positioning.

[0034] However, in this method, when the controller is blocked by the user's body or other objects, the head-mounted display's camera cannot properly recognize the markers on the controller, which will cause the controller's spatial positioning to be interrupted, seriously affecting the user experience.

[0035] In view of this, this application provides a spatial positioning method for a controller, which can determine whether the controller is occluded during the spatial positioning process based on an inside-out positioning system. When the controller is occluded, its motion parameters can be calculated using historical positioning data, and then the predicted pose of the controller at subsequent sampling times can be calculated based on these motion parameters, serving as the corresponding spatial pose of the controller. In this way, when the controller is occluded, the subsequent motion of the controller after occlusion can be predicted based on its historical positioning data before occlusion, thereby obtaining the predicted pose as its corresponding spatial pose. This ensures that the spatial positioning of the controller is maintained even during occlusion, guaranteeing continuous tracking of the controller's pose.

[0036] The controller is a device used to enable users to interact with virtual reality in a virtual reality scene. For example, the controller can be a handheld prop corresponding to a specific virtual reality scene, or a controller used to interact with and control the virtual reality scene. There are no restrictions on the shape and configuration of the controller; it can be a controller in any virtual reality system.

[0037] To facilitate understanding of this application, the implementation environment corresponding to this application, namely the corresponding virtual reality system, will be described first. Figure 1 A schematic diagram of the virtual reality system corresponding to this application is shown, such as... Figure 1 As shown, the virtual reality system may include: a head-mounted display (HUD) and a controller. The HUD and controller are paired and interconnected to provide the user with a virtual reality experience. The diagram illustrates four virtual reality systems: HUD A and its corresponding controller A, HUD B and its corresponding controller B, HUD C and its corresponding controller C, and HUD D and its corresponding controller D.

[0038] A head-mounted display (HMD) can be worn on a user's head and provides visual content to the user through an internal display (not shown). The HMD can be an all-in-one device, containing a computing system such as a CPU, GPU, and power management module, which will not be elaborated further. Alternatively, the HMD can communicate with an external computing device (not shown). Virtual reality technology-related processing can be performed through the aforementioned computing system or device; for example, when performing spatial positioning of the HMD and controllers, corresponding computational processing can be performed by the computing system or device. The method provided in this application can also be executed by the aforementioned computing system or device.

[0039] In this virtual reality system, the head-mounted display device can be spatially positioned using Lighthouse positioning technology via base stations (i.e., Lighthouse base stations). One or more base stations can be used; the diagram illustrates two base stations, 1 and 2. Specific methods for positioning the head-mounted display device can be found in implementation details of Lighthouse positioning technology, which will not be elaborated upon here.

[0040] The head-mounted display (HMD) can also have an outward-facing camera (not shown), and the handle can be equipped with an inertial measurement unit (IMU) and corresponding markers (not shown), such as markers composed of infrared LEDs, thus forming an inside-out positioning system. This allows the HMD's camera to identify the markers on the handle and, combined with the measurement data from the IMU, perform spatial positioning of the handle based on inside-out positioning technology. Specific implementation methods for spatial positioning of the handle based on the inside-out system can be found in related implementation methods of inside-out positioning technology, and will not be elaborated upon here.

[0041] It should be understood that Figure 1 This is only a brief illustrative example of a virtual reality system. In practical applications, the system based on virtual reality technology may include other related devices or apparatuses, and there are no limitations here.

[0042] Figure 2 This is a flowchart illustrating the spatial positioning method for a handle provided in an embodiment of this application. Figure 2 As shown, the method may include the following S201-S204.

[0043] S201. During the process of spatial positioning of the handle based on the inside-out positioning system, determine whether the handle is obstructed.

[0044] The inside-out positioning system, also known as the insid-out positioning system, uses a camera mounted on the head-mounted display to identify markers on the controllers to achieve spatial positioning of the controllers. Therefore, if the controllers are obstructed, preventing the camera on the head-mounted display from properly identifying the markers on the controllers, the spatial positioning of the controllers by the inside-out positioning system will be abnormal, resulting in problems such as instantaneous shifts and jumps in the spatial pose of the controllers.

[0045] Therefore, in some possible implementations, during the spatial positioning of the handle based on the inside-out positioning system, it can be determined whether the handle is occluded based on whether the inside-out positioning system identifies the handle abnormally and / or whether the handle's spatial pose changes abruptly. That is, if an abnormality in handle identification and / or abrupt changes in the obtained handle's spatial pose are detected during the spatial positioning of the handle based on the inside-out positioning system, it can be determined that the handle is occluded. In this way, when the inside-out positioning system cannot accurately position the handle, the spatial pose of the handle can be determined promptly using the method provided in this application, avoiding problems such as instantaneous shifts or jumps in the handle's spatial pose caused by inaccurate spatial positioning of the handle by the inside-out positioning system.

[0046] For example, during the spatial localization of the handle based on the inside-out positioning system, if the number of identified handle markers is less than a threshold, and / or the confidence level of the obtained spatial pose is lower than a confidence threshold, and / or the confidence level of identifying the handle based on the image captured by the head-mounted display device's camera is lower than a confidence threshold, and / or the head-mounted display device's camera cannot acquire an image containing the handle, and / or the distance between the obtained spatial pose of the handle and the previously obtained spatial pose is greater than a threshold (i.e., the obtained spatial pose of the handle undergoes a sudden change), then it can be determined that the handle is occluded.

[0047] Of course, the above is just an example of how to determine whether the controller is obstructed. In practical applications, any other method can be used to determine whether the controller is obstructed. For example, image recognition algorithms can be used to determine whether the controller is obstructed by recognizing the image captured by the camera of the head-mounted display device, thereby improving the efficiency of determining whether the controller is obstructed. There are no restrictions here.

[0048] S202. When the controller is obstructed, acquire the controller's historical positioning data.

[0049] Among them, the historical positioning data is the set of spatial poses obtained by the inside-out positioning system to spatially locate the handle before it is obscured (i.e., the historical trajectory of the handle).

[0050] S203. Based on the historical positioning data, calculate the motion parameters of the handle.

[0051] Typically, each spatial pose in the historical positioning data of a controller is marked with the sampling time corresponding to that spatial pose. Therefore, relevant motion parameters of the controller can be calculated based on the historical positioning data. For example, the displacement-time curve of the controller before it is occluded can be obtained based on the historical positioning data, and the velocity and acceleration of the controller during movement can be calculated based on the first and second derivatives of this curve. As another example, the rotation angle-time curve corresponding to the controller's own rotation before it is occluded can be obtained based on the historical positioning data, and the angular velocity and angular acceleration of the controller during its rotation can be calculated based on the first and second derivatives of this curve. Therefore, the motion of the controller at subsequent sampling moments during the occlusion period can be predicted based on these motion parameters to obtain the corresponding predicted pose. Of course, the above is only an exemplary description of obtaining motion parameters from historical positioning data. In this embodiment, any other related technology can be used to obtain the motion parameters of the controller from historical positioning data, and no limitation is made here.

[0052] S204. Based on the motion parameters, calculate the predicted pose of the handle at subsequent sampling times, and use it as the spatial pose of the handle at subsequent sampling times.

[0053] As an example, the predicted pose of the controller at subsequent sampling moments (i.e., sampling moments during the period when the controller is occluded) can be calculated using the corresponding motion model based on motion parameters. This allows for quick and efficient prediction of the controller's motion during occlusion using the appropriate motion model, thereby reducing computational complexity, increasing the efficiency of calculating the predicted pose, reducing the computational delay of obtaining the controller's spatial pose, and improving the user experience.

[0054] For example, taking the velocity and acceleration of the handle as motion parameters, the corresponding predicted pose can be calculated using a uniform motion model or a uniformly accelerated motion model based on the motion parameters. Specifically, when the acceleration of the handle is zero, the coordinate values ​​of subsequent sampling moments are calculated using a uniform motion model in each coordinate axis direction based on the components of the handle's velocity on different coordinate axes, thus obtaining the coordinate values ​​of the handle at subsequent sampling moments, i.e., the predicted pose. When the acceleration of the handle is not zero, the coordinate values ​​of subsequent sampling moments are calculated using a uniformly accelerated motion model in each coordinate axis direction based on the components of the handle's velocity and acceleration on different coordinate axes, thus obtaining the coordinate values ​​of the handle at subsequent sampling moments, i.e., the predicted pose. When the motion trajectory of the handle is determined to be circular motion based on historical positioning data, the uniformly accelerated motion model used can specifically be a uniformly accelerated circular motion model. That is, in some possible implementations of this application, a corresponding motion model can be pre-matched based on the specific motion parameters and motion trajectory to obtain a corresponding matching relationship, so that the method of this application can select the corresponding motion model to calculate the predicted pose through this matching relationship.

[0055] For example, taking the angular velocity and angular acceleration of the handle as motion parameters, the corresponding predicted pose can be calculated using a uniform angular motion model or a uniformly accelerated angular motion model based on these motion parameters. Specifically, when the angular acceleration of the handle is zero, the rotation angle at subsequent sampling moments is calculated based on the angular velocity of the handle using a uniform angular motion model, thus obtaining the predicted pose corresponding to the handle's own rotation at subsequent sampling moments. Conversely, when the angular acceleration of the handle is not zero, the rotation angle at subsequent sampling moments is calculated based on the angular velocity and angular acceleration of the handle using a uniformly accelerated angular motion model, thus obtaining the predicted pose corresponding to the handle's own rotation at subsequent sampling moments.

[0056] In some possible embodiments of this application, the speed, acceleration, angular velocity and angular acceleration of the handle described above can be combined to predict and calculate the spatial movement and self-rotation of the handle, thereby obtaining the predicted pose of the handle that is simultaneously moving in space and rotating on its own.

[0057] Of course, in other possible embodiments of this application, a pre-trained motion prediction model can be used to obtain the predicted pose of the handle at subsequent sampling times based on the handle's historical positioning data and / or motion parameters, thereby improving the accuracy of the obtained predicted pose. Therefore, this application does not limit the specific implementation method of how to calculate the corresponding predicted pose based on motion parameters.

[0058] Optionally, after calculating the predicted pose using the corresponding motion model, the predicted pose can be optimized based on the Kalman filter algorithm to obtain a more accurate predicted pose, so that the spatial pose trajectory of the handle after it is occluded is closer to the actual motion trajectory of the handle.

[0059] Among them, optimizing the predicted pose based on the Kalman filter algorithm can be achieved by taking the spatial pose obtained by the inside-out positioning system for spatial positioning of the handle during the period when the handle is occluded as a reference, and optimizing and updating the predicted pose calculated by the corresponding motion model based on the Kalman filter algorithm to obtain the optimized predicted pose.

[0060] In some possible implementations of the embodiments of this application, after the predicted pose is calculated, the predicted pose can be verified, and if the verification is successful, the predicted pose can be used as the spatial pose corresponding to the handle.

[0061] For example, the predicted pose can be validated based on the distance between the predicted pose and the current spatial pose of the head-mounted display (HMD). If this distance is less than or equal to a preset threshold, the predicted pose is then used as the spatial pose corresponding to the controller. The current spatial pose of the HMD, i.e., the spatial pose corresponding to the HMD at the current sampling moment, can be determined using HMD positioning technologies such as Lighthouse. This method allows for validation of the obtained predicted pose based on the constraints of the HMD's spatial pose, avoiding drift or divergence in the controller's spatial pose determined by the predicted pose, and ensuring a smoother and more stable motion trajectory formed by the controller's spatial pose during occlusion.

[0062] For example, in this embodiment of the application, when the handle is no longer obstructed (i.e., the inside-out positioning system can accurately re-identify the handle), the current measured spatial pose (i.e., the spatial pose measured at the current sampling time) obtained by spatial positioning of the handle based on the inside-out positioning system can also be obtained, and then fused with the predicted pose corresponding to the current sampling time to obtain the spatial pose of the handle. In this way, it is possible to avoid the situation where the spatial pose obtained by spatial positioning of the handle based on the inside-out positioning system is directly used as the current spatial pose of the handle when the handle is no longer obstructed, which would cause a jump in the spatial pose compared with the spatial pose obtained when the handle is obstructed. This allows the trajectory corresponding to the spatial pose when the handle is obstructed and the trajectory corresponding to the spatial pose after the handle is no longer obstructed to transition smoothly, maintaining the continuity of the handle's motion trajectory obtained from the spatial pose when the handle is obstructed and when it is no longer obstructed, thus achieving a seamless user experience.

[0063] As an example, one way to fuse the current measured spatial pose and the predicted pose at the current sampling time is to use the pose located between the current measured spatial pose and the predicted pose at the current sampling time as the handle's spatial pose. For example, for spatial movement, a point (e.g., the midpoint, or a segmentation point based on a preset ratio) on the spatial line connecting the current measured spatial pose and the predicted pose at the current sampling time is used as the fused spatial pose to serve as the handle's spatial pose. For self-rotation, an angle (e.g., the midpoint, or a segmentation angle based on a preset ratio) between the rotation angle of the current measured spatial pose and the predicted pose at the current sampling time is used as the fused spatial pose to serve as the handle's spatial pose. Of course, when the handle simultaneously performs spatial movement and self-rotation, the above two methods can be combined to fuse the handle's spatial position in the spatial movement dimension and its spatial pose in the self-rotation dimension, thereby obtaining the handle's spatial pose. Alternatively, in some possible implementations, the current measured spatial pose and the predicted pose at the current sampling time can be fused based on any pose point fusion algorithm to obtain the handle's spatial pose; this is not limited here.

[0064] Of course, in other possible implementations of this application, the current measured spatial pose and the predicted pose at the current sampling time may not be fused. When the handle is no longer obscured, the current measured spatial pose obtained by spatially locating the handle based on the inside-out positioning system may be directly used as the current spatial pose of the handle. This is not a limitation.

[0065] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0066] According to another embodiment, a spatial positioning device for a handle is provided. Figure 3 A schematic diagram illustrating the components of the spatial positioning device for the handle is shown. Figure 3 As shown, the device includes: a determination module 301, used to determine whether the handle is occluded during the spatial positioning of the handle based on the inside-out positioning system; a prediction module 302, used to acquire historical positioning data of the handle when the handle is occluded; calculate the motion parameters of the handle based on the historical positioning data; and calculate the predicted pose of the handle at subsequent sampling times based on the motion parameters, so as to serve as the spatial pose of the handle at subsequent sampling times.

[0067] Optionally, the determining module 301 is specifically used to determine that the handle is occluded if an abnormality in the recognition of the handle and / or a sudden change in the spatial pose of the handle is detected during the process of spatial positioning of the handle based on the inside-out positioning system.

[0068] Optionally, motion parameters include the speed and acceleration of the handle during movement.

[0069] Optionally, the prediction module 302 is specifically used to calculate the predicted pose based on the motion parameters using a uniform motion model or a uniformly accelerated motion model.

[0070] Optionally, the prediction module 302 is specifically used to calculate the predicted pose based on the motion parameters through a uniform motion model or a uniformly accelerated motion model, and to optimize the predicted pose based on the Kalman filter algorithm.

[0071] Optionally, the motion parameters include the angular velocity and angular acceleration of the handle during movement.

[0072] Optionally, the prediction module 302 is specifically used to calculate the predicted pose based on the motion parameters using a uniform angular motion model or a uniformly accelerated angular motion model.

[0073] Optionally, the prediction module 302 is specifically used to calculate the predicted pose based on the motion parameters through a uniform angular motion model or a uniformly accelerated angular motion model, and to optimize the predicted pose based on the Kalman filter algorithm.

[0074] Optionally, the prediction module 302 is specifically used to calculate the predicted pose of the handle at the subsequent sampling time based on the motion parameters; when the distance between the predicted pose and the current spatial pose of the head-mounted display device is less than or equal to a preset threshold, the predicted pose is used as the spatial pose of the handle at the subsequent sampling time.

[0075] Optionally, the prediction module 302 is further configured to obtain the current measured spatial pose of the handle when the handle is no longer obstructed; and to fuse the current measured spatial pose and the predicted pose at the current sampling time to obtain the spatial pose of the handle.

[0076] Optionally, the prediction module 302 is specifically used to take the pose located between the current measurement spatial pose and the predicted pose at the current sampling time as the spatial pose of the handle.

[0077] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments. The system and device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0078] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0079] In addition, this application also provides a virtual reality system, including a controller; a head-mounted display device; and the aforementioned spatial positioning device for the controller.

[0080] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0081] And an electronic device, comprising: One or more processors; and A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0082] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0083] in, Figure 4An exemplary architecture of an electronic device is illustrated, which may specifically include a bus 401, a processor 402, a memory 403, and a communication interface 404. The processor 402, memory 403, and communication interface 404 communicate via the bus 401. The memory 403 stores computer program code, including computer instructions. When these instructions are executed by the processor 402, the electronic device performs the methods provided in the foregoing embodiments. It should be understood that this application does not limit the number of processors 402 and memories 403 in the electronic device.

[0084] Bus 401 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 401 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus 401 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 401 may include a path for transmitting information between various components of an electronic device (e.g., memory 403, processor 402, communication interface 404).

[0085] Processor 402 may include any one or more processors such as a central processing unit, a graphics processing unit, a microprocessor (MP), or a digital signal processor (DSP).

[0086] Memory 403 may include volatile memory, such as random access memory (RAM). Memory 403 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0087] Communication interface 404 uses command distribution modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the device and other devices or communication networks.

[0088] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0089] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for spatial positioning of a handle, characterized in that, The controller is used to enable users to perform virtual reality interactions in a virtual reality scene, and the method includes: During the process of spatially positioning the handle based on the inside-out positioning system, it is determined whether the handle is obstructed; When the handle is blocked, acquire the handle's historical positioning data; Based on the historical positioning data, the motion parameters of the handle are calculated; Based on the motion parameters, the predicted pose of the handle at subsequent sampling times is calculated and used as the spatial pose of the handle at subsequent sampling times.

2. The method according to claim 1, characterized in that, During the process of spatially locating the handle based on the inside-out positioning system, determining whether the handle is obstructed includes: During the spatial positioning of the handle based on the inside-out positioning system, if an abnormality in the identification of the handle is detected and / or a sudden change in the spatial pose of the handle is obtained, it is determined that the handle is occluded.

3. The method according to claim 1, characterized in that, The motion parameters include the speed and acceleration of the handle during movement.

4. The method according to claim 3, characterized in that, The step of calculating the predicted pose of the handle at subsequent sampling times based on the motion parameters includes: The predicted pose is calculated based on the motion parameters using a uniform motion model or a uniformly accelerated motion model.

5. The method according to claim 4, characterized in that, The step of calculating the predicted pose based on the motion parameters using a uniform motion model or a uniformly accelerated motion model includes: The predicted pose is calculated based on the motion parameters using a uniform motion model or a uniformly accelerated motion model, and then optimized using a Kalman filter algorithm.

6. The method according to claim 1, characterized in that, The motion parameters include the angular velocity and angular acceleration of the handle during movement.

7. The method according to claim 6, characterized in that, The step of calculating the predicted pose of the handle at subsequent sampling times based on the motion parameters includes: The predicted pose is calculated based on the motion parameters using a uniform angular motion model or a uniformly accelerated angular motion model.

8. The method according to claim 7, characterized in that, The step of calculating the predicted pose based on the motion parameters using a uniform angular motion model or a uniformly accelerated angular motion model includes: The predicted pose is calculated based on the motion parameters using a uniform angular motion model or a uniformly accelerated angular motion model, and then optimized using a Kalman filter algorithm.

9. The method according to any one of claims 1-8, characterized in that, The step of calculating the predicted pose of the handle at subsequent sampling times based on the motion parameters, and using it as the spatial pose of the handle at subsequent sampling times, includes: Based on the motion parameters, the predicted pose of the handle at subsequent sampling times is calculated; When the distance between the predicted pose and the current spatial pose of the head-mounted display is less than or equal to a preset threshold, the predicted pose is used as the spatial pose of the handle at subsequent sampling times.

10. The method according to any one of claims 1-8, characterized in that, The method further includes: When the handle is no longer obstructed, the current measured spatial pose of the handle is obtained based on the spatial positioning of the handle using the inside-out positioning system; The spatial pose of the handle is obtained by fusing the current measured spatial pose and the predicted pose at the current sampling time.

11. The method according to claim 10, characterized in that, The process of fusing the current measured spatial pose and the predicted pose at the current sampling time to obtain the spatial pose of the handle includes: The pose located between the current measured spatial pose and the predicted pose at the current sampling time is taken as the spatial pose of the handle.

12. A spatial positioning device for a handle, characterized in that, The handle is used to enable users to perform virtual reality interaction in a virtual reality scene, and the device includes: The determination module is used to determine whether the handle is obstructed during the process of spatial positioning of the handle based on the inside-out positioning system; The prediction module is used to acquire historical positioning data of the handle when the handle is occluded; calculate motion parameters of the handle based on the historical positioning data; and calculate the predicted pose of the handle at subsequent sampling times based on the motion parameters, so as to serve as the spatial pose of the handle at subsequent sampling times.

13. A virtual reality system, characterized in that, include: handle; Head-mounted display devices; as well as The spatial positioning device for the handle as described in claim 12.

Images