VR positioning compensation method and system for complex environment

Abstract

The present application relates to the technical field of virtual reality (VR), and particularly relates to a VR positioning compensation method and system for complex environments, comprising a multi-source perception synchronous acquisition module, a scene data preprocessing module, an initial pose solution and abnormal state identification module, a multi-source dynamic weighted fusion first-level compensation module, a scene semantic constraint second-level accurate compensation module, a closed-loop verification and pose smoothing output module; comprising the following steps: S1, a multi-source data acquisition and preprocessing stage; S2, an initial pose solution and abnormal identification stage; S3, a multi-source dynamic weighted fusion first-level compensation stage; S4, a scene semantic constraint second-level accurate compensation stage; S5, a closed-loop verification and pose output stage; through the architecture design of multi-source dynamic weighted fusion and two-level compensation, the present application effectively solves the core problems of VR positioning in complex environments, such as being easily disturbed, easily interrupted and insufficient accuracy, and can adapt to dynamic and static interference factors in various scenes.

Description

Technical Field

[0001] This invention relates to the field of virtual reality (VR) technology, and in particular to a VR positioning compensation method and system for complex environments. Background Technology

[0002] With the rapid development of virtual reality technology, VR technology has been widely used in many fields such as real estate display, medical device training, and red culture education, bringing users an immersive interactive experience; Spatial positioning technology is the core foundation of VR systems. The accuracy, continuity, and stability of positioning directly determine the smoothness of VR interaction and the immersive experience for users. In practical applications, VR systems often need to face various complex usage environments. Factors such as occlusion, reflection, electromagnetic interference, and weak texture areas in the environment can significantly interfere with VR positioning, easily leading to problems such as inaccurate positioning, interruption, and screen jumps, which seriously affect the user experience and may even cause users to feel dizzy, thus limiting the large-scale application of VR technology in more complex scenarios. Therefore, developing VR positioning compensation technology that can adapt to complex environments has become a core technical problem that urgently needs to be solved in the current VR field. Summary of the Invention

[0003] The purpose of this invention is to provide a VR positioning compensation method and system for complex environments, so as to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: The VR positioning compensation system for complex environments includes a multi-source perception synchronous acquisition module, a scene-based data preprocessing module, an initial pose calculation and abnormal state identification module, a multi-source dynamic weighted fusion primary compensation module, a scene semantic constraint secondary accurate compensation module, and a closed-loop verification and pose smoothing output module.

[0005] As a further improvement to this technical solution: the initial pose calculation and abnormal state recognition module includes an initial pose calculation submodule and an abnormal state real-time recognition submodule. The initial pose calculation submodule completes visual feature point extraction and matching, VR head-mounted display six-degree-of-freedom initial pose calculation, IMU data pre-integration and pose prediction value output. The abnormal state real-time recognition submodule completes the real-time judgment and level classification of three types of positioning abnormal states: visual positioning abnormality, inertial positioning abnormality, and spatial positioning abnormality.

[0006] As a further improvement to this technical solution: the multi-source dynamic weighted fusion first-level compensation module includes an adaptive dynamic weight allocation submodule and a multi-source data tightly coupled fusion compensation solution submodule. The adaptive dynamic weight allocation submodule completes the dynamic weight allocation of visual positioning data, IMU inertial positioning data, and UWB spatial positioning data based on the abnormal state recognition results. The weight allocation is negatively correlated with the abnormality level of the corresponding sensor data. The core calculation formula is as follows: ; In the formula, The weight values ​​for visual positioning data. Weight values ​​for IMU inertial positioning data. Weight values ​​for UWB spatial positioning data, Let i be the weight value of the i-th type of sensor data. The anomaly level of the i-th type of sensor data, The maximum anomaly level is preset. When a certain type of sensor data is determined to be completely invalid, the corresponding weight value is set to 0, and the weight values ​​of the remaining valid sensor data are redistributed according to the above formula.

[0007] As a further improvement to this technical solution: the multi-source data tightly coupled fusion compensation solution submodule uses extended Kalman filtering to complete the tightly coupled fusion of weighted multi-source data, and outputs the VR head-mounted display pose data after primary compensation. The core calculation formula is as follows: In the formula, The Kalman gain at time k, Let be the prior covariance matrix at time k. Let be the observation matrix at time k. Let be the transpose of the observation matrix. To observe the noise covariance matrix, the diagonal elements of the noise covariance matrix are negatively correlated with the weight values ​​of the corresponding sensor data.

[0008] As a further improvement to this technical solution: the scene semantic constraint secondary precision compensation module includes a scene semantic map pre-construction sub-module and a semantic constraint pose correction sub-module. The scene semantic map pre-construction sub-module completes the construction of a three-dimensional semantic map of the target application scene, marking the three-dimensional coordinates and physical boundaries, walkable areas and impenetrable areas of fixed semantic objects in the scene. The semantic constraint pose correction sub-module completes the secondary precision correction of the primary compensation pose. The core calculation formula is as follows: In the formula, This is the corrected spatial position vector for the VR headset. This is the VR headset spatial position vector after initial compensation. for The vertical distance to the nearest boundary of the impenetrable region. It is the unit vector of the outward normal of the nearest boundary of the impenetrable region.

[0009] As a further improvement to this technical solution: the closed-loop verification and pose smoothing output module includes a closed-loop verification submodule and a pose smoothing output submodule. The closed-loop verification submodule completes the motion continuity verification of the compensated pose and completes the closed-loop feedback optimization of the compensation parameters. The pose smoothing output submodule completes the inter-frame smoothing processing of the pose data and completes the real-time output of the pose data to the VR rendering engine.

[0010] A VR positioning compensation method for complex environments includes the following steps: S1. Multi-source data acquisition and preprocessing stage: The VR headset full-link operation data is acquired through the multi-source perception synchronous acquisition module, the time base of multi-source data is unified and standardized preprocessing is completed, and a standardized dataset that meets the solution requirements is output. S2, Initial pose calculation and anomaly identification stage: Based on the preprocessed standardized dataset, the initial pose calculation of the VR headset with six degrees of freedom is completed, and the real-time judgment and level classification of three types of abnormal states, namely visual positioning anomaly, inertial positioning anomaly and spatial positioning anomaly, are completed simultaneously. S3, Multi-source dynamic weighted fusion first-level compensation stage: Based on the anomaly identification results, adaptive weight allocation of each sensor data is completed, and extended Kalman filtering is used to complete the tight coupling fusion solution of weighted multi-source data, outputting VR head-mounted display pose data after primary compensation; S4, the second-level precise compensation stage of scene semantic constraints, matches the pose data after the initial compensation with the pre-built scene semantic map in real time, completes the pose compliance judgment, performs a second correction on poses that exceed semantic constraints, and outputs compliant precise pose data. S5, Closed-loop verification and pose output stage: After completing the motion continuity verification of the compensated pose, the verified pose data is smoothed between frames and output to the VR rendering engine in real time. At the same time, the parameter data of this compensation is fed back to the anomaly recognition module in a closed loop to complete the threshold optimization.

[0011] As a further improvement to this technical solution: the weight allocation in the S3 stage is negatively correlated with the anomaly level of the corresponding sensor data; the pose correction in the S4 stage only adjusts the spatial position vector and does not change the attitude vector; after the correction is completed, the pose motion continuity verification is completed simultaneously; if the verification fails, the pose smooth transition is completed based on the inter-frame optical flow data.

[0012] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention effectively solves the core problems of VR positioning being susceptible to interference, interruption, and insufficient accuracy in complex environments through a multi-source dynamic weighted fusion and two-level compensation architecture design. It can adapt to dynamic and static interference factors in various scenarios, avoid problems such as VR screen jumps, wall penetration, and spatial misalignment caused by single sensor positioning failure, significantly improve the continuity, robustness, and accuracy of VR positioning, effectively reduce user dizziness caused by inaccurate positioning, and comprehensively optimize the immersive interactive experience of VR in all scenarios.

[0013] 2. This invention has strong scene adaptability and can cover complex usage environments in multiple mainstream VR application fields such as real estate display, medical device training, and red culture publicity. It can achieve stable positioning compensation effect without making significant system adjustments for different scenarios. At the same time, it realizes the self-adaptive adjustment of system parameters through a closed-loop feedback optimization mechanism, reducing the deployment and maintenance costs of VR systems and providing stable and reliable positioning technology support for the large-scale application of VR technology in various complex scenarios.

[0014] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a schematic diagram of a VR positioning compensation method and system for complex environments. Detailed Implementation

[0016] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are for illustrative purposes only and are not intended to limit the scope of the invention. The invention is described more specifically in the following paragraphs by way of example with reference to the accompanying drawings. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0017] Please see Figure 1 In this embodiment of the invention, the VR positioning compensation system for complex environments includes a multi-source perception synchronous acquisition module, a scene-based data preprocessing module, an initial pose calculation and abnormal state identification module, a multi-source dynamic weighted fusion first-level compensation module, a scene semantic constraint second-level accurate compensation module, and a closed-loop verification and pose smoothing output module. Specifically, the multi-source sensing synchronous acquisition module is the system's data source entry point, responsible for collecting all-dimensional raw sensor data required for VR positioning, providing basic data support for subsequent pose calculation and compensation; The scenario-based data preprocessing module is responsible for standardizing the raw collected data, eliminating data noise, transmission delay differences and data distortion caused by environmental interference, and providing a compliant standardized dataset for subsequent pose calculation. The initial pose calculation and abnormal state recognition module is responsible for calculating the initial spatial pose of the VR headset, and at the same time, it identifies various abnormal positioning states in complex environments, providing a basis for judgment and quantitative indicators for subsequent compensation. The multi-source dynamic weighted fusion primary compensation module is responsible for completing the first level of pose compensation for the identified positioning anomalies, eliminating pose errors caused by the failure of a single sensor data, and ensuring the continuity of the positioning link. The scene semantic constraint level 2 accurate compensation module is responsible for completing the second level of pose accuracy correction based on scene spatial physical rules, eliminating positioning problems that do not conform to the actual scene, such as spatial misalignment and wall penetration, and improving positioning accuracy; The closed-loop verification and pose smoothing output module is responsible for the final verification and smoothing of the compensated pose, completing the final output of pose data, and forming a full-process parameter feedback optimization link to ensure the stability of system operation.

[0018] The initial pose calculation and abnormal state recognition module includes an initial pose calculation submodule and an abnormal state real-time recognition submodule. The initial pose calculation submodule completes visual feature point extraction and matching, VR head-mounted display six-DOF initial pose calculation, IMU data pre-integration and pose prediction value output. The abnormal state real-time recognition submodule completes the real-time judgment and level classification of three types of positioning abnormal states: visual positioning abnormality, inertial positioning abnormality, and spatial positioning abnormality. Specifically, the execution logic and function of the initial pose calculation submodule are as follows: by extracting feature points from the binocular vision image and matching them with the spatial marker points preset in the scene, and combining the spatial coordinates of the UWB positioning anchor point, the initial pose of the VR headset with six degrees of freedom is calculated. At the same time, the inertial data collected by the IMU is pre-integrated and the inter-frame pose prediction value is output, providing a dual reference for pose calculation and ensuring the accuracy of the initial pose calculation. The execution logic and function of the real-time abnormal state identification submodule are as follows: For three core positioning interference scenarios in complex environments, it performs real-time judgment and level classification of corresponding abnormal states. Specifically, visual positioning anomalies, targeting visual positioning failure scenarios caused by occlusion, reflection, and weak texture within the scene, are judged using two core indicators: the number of effectively matched feature points and the feature point reprojection error. Inertial positioning anomalies, targeting IMU data drift scenarios caused by electromagnetic interference and rapid movement, are judged using the spatial position residual between the IMU pre-integrated pose and the visually calculated pose. Spatial positioning anomalies, targeting UWB anchor point positioning failure scenarios caused by obstacle occlusion and multipath effects, are judged using the number of effective signal anchor points. Simultaneously, the anomaly level is classified based on the exceedance of various indicators, providing a quantitative basis for subsequent weight allocation.

[0019] The multi-source dynamic weighted fusion primary compensation module includes an adaptive dynamic weight allocation submodule and a multi-source data tightly coupled fusion compensation solution submodule. The adaptive dynamic weight allocation submodule dynamically allocates weights to visual positioning data, IMU inertial positioning data, and UWB spatial positioning data based on the abnormal state recognition results. The weight allocation is negatively correlated with the anomaly level of the corresponding sensor data. The core calculation formula is as follows: ; In the formula, The weight values ​​for visual positioning data. Weight values ​​for IMU inertial positioning data. Weight values ​​for UWB spatial positioning data, Let i be the weight value of the i-th type of sensor data. The anomaly level of the i-th type of sensor data, The maximum anomaly level is preset. When a certain type of sensor data is determined to be completely invalid, the corresponding weight value is set to 0, and the weight values ​​of the remaining valid sensor data are reallocated according to the above formula. Specifically, the execution logic and function of the adaptive dynamic weight allocation submodule are as follows: based on the anomaly type and anomaly level output by the real-time anomaly identification submodule, dynamic weight allocation is performed for the three core sensor data: visual positioning data, IMU inertial positioning data, and UWB spatial positioning data. The weight allocation value is negatively correlated with the anomaly level of the corresponding sensor data. That is, the higher the anomaly level of the sensor data, the lower the weight value is allocated, thereby reducing the interference of abnormal data on pose calculation and ensuring the reliability of fusion calculation. This claim includes two core calculation formulas. The first formula is a weight normalization formula, which ensures that the sum of the weights of the three types of sensor data is always 1, guaranteeing the normalization of weight allocation and avoiding weight imbalance during multi-source data fusion, thus ensuring the rationality of the fusion solution. The second formula is a dynamic weight allocation formula, which quantifies the corresponding weights based on the anomaly level of the sensor data, enabling adaptive dynamic adjustment of the weights without the need for manual preset of fixed weights, and adapting to dynamic interference scenarios in different complex environments. When a certain type of sensor data is determined to be completely invalid, the corresponding weight value is set to 0, and the weight values ​​of the remaining valid sensor data are reallocated according to the above two formulas to ensure that the positioning link is not interrupted. The function of the multi-source data tightly coupled fusion compensation solution submodule is to perform tightly coupled fusion solution on the multi-source sensor data after dynamic weight allocation, eliminate the errors between the multi-source data, and output the VR head-mounted display six-degree-of-freedom pose data after primary compensation.

[0020] The multi-source data tightly coupled fusion compensation solution submodule uses extended Kalman filtering to complete the tightly coupled fusion of weighted multi-source data, and outputs the VR head-mounted display pose data after primary compensation. The core calculation formula is as follows: In the formula, The Kalman gain at time k, Let be the prior covariance matrix at time k. Let be the observation matrix at time k. Let be the transpose of the observation matrix. To observe the noise covariance matrix, the diagonal elements of the observation noise covariance matrix are negatively correlated with the weight values ​​of the corresponding sensor data; Specifically, this submodule uses the extended Kalman filter algorithm to complete the tight coupling fusion of multi-source data. The extended Kalman filter can adapt to nonlinear motion scenarios in the VR positioning process and achieve the optimal estimation of VR head-mounted display pose. It is the core solution algorithm in the VR positioning field. The formula included in this claim is the core Kalman gain calculation formula of the extended Kalman filter. The function of this formula is to calculate the Kalman gain in the extended Kalman filter process. The Kalman gain is used to balance the weight between the prior state prediction value and the actual observation value, and to determine the correction magnitude of the current observation value to the state update. It is the core calculation link of the extended Kalman filter algorithm. In the formula, the prior covariance matrix The observation matrix is ​​used to characterize the error distribution of the predicted values ​​from prior states. Used to establish the mapping relationship between the state vector and the observation vector, the observation noise covariance matrix. Used to characterize the noise distribution of observation data, the diagonal elements of the observation noise covariance matrix are negatively correlated with the weight values ​​of the corresponding sensor data. That is, the higher the weight value of the sensor data, the lower the corresponding observation noise value, and the greater the correction amplitude in the state update process, so as to realize the dominant role of high reliability data in the solution results. After calculating the Kalman gain using this formula, the state update and covariance update of the extended Kalman filter can be completed. Finally, the 3D position and 3D pose of the VR headset can be extracted from the updated posterior state vector, i.e., the six-DOF pose data after primary compensation. The scene semantic constraint secondary precision compensation module includes a scene semantic map pre-construction sub-module and a semantic constraint pose correction sub-module. The scene semantic map pre-construction sub-module completes the construction of a 3D semantic map of the target application scene, marking the 3D coordinates and physical boundaries of fixed semantic objects in the scene, as well as walkable and impenetrable areas. The semantic constraint pose correction sub-module completes the secondary precision correction of the primary compensation pose. The core calculation formula is as follows: In the formula, This is the corrected spatial position vector for the VR headset. This is the VR headset spatial position vector after initial compensation. for The vertical distance to the nearest boundary of the impenetrable region. The unit vector of the outward normal to the nearest boundary of the impenetrable region; Specifically, the execution logic and function of the scene semantic map pre-construction sub-module are as follows: to complete the construction of a three-dimensional semantic map for the target scene of VR application, mark the three-dimensional coordinates and physical boundaries of fixed semantic objects in the scene, and mark the walkable and impenetrable areas in the scene, providing a spatial hard constraint basis for pose correction, adapting to the spatial rules of various VR application scenarios such as real estate, medical devices, and red culture exhibition halls, and ensuring that pose correction conforms to the actual physical boundaries of the scene. The execution logic and function of the semantic constraint pose correction submodule are as follows: for the pose data after the initial compensation, a second precise correction is performed based on the spatial constraints of the scene semantic map to eliminate positioning errors that do not conform to the physical rules of the scene, such as pose passing through walls and spatial misalignment, thereby further improving the accuracy and rationality of positioning. The formula is the core calculation formula for pose correction. Its function is to quantitatively correct poses that extend beyond impenetrable areas, adjusting them to a reasonable position that conforms to the physical rules of the scene. The correction process only adjusts the spatial position vector of the VR headset, without changing the posture vector, thus ensuring the continuity of the VR viewpoint and avoiding user dizziness caused by abrupt screen transitions. In the formula, vertical distance... With the outer normal unit vector All poses are extracted directly from a pre-built 3D semantic map, ensuring that the corrected poses perfectly match the actual physical boundaries of the scene.

[0021] The closed-loop verification and pose smoothing output module includes a closed-loop verification submodule and a pose smoothing output submodule. The closed-loop verification submodule completes the motion continuity verification of the compensated pose and completes the closed-loop feedback optimization of the compensation parameters. The pose smoothing output submodule completes the inter-frame smoothing processing of the pose data and completes the real-time output of the pose data to the VR rendering engine. Specifically, the execution logic and function of the closed-loop verification submodule are as follows: to perform motion continuity verification on the pose data after the completion of the second-level compensation, to determine whether the change amplitude between pose frames conforms to the reasonable range of human head movement, to remove abnormal pose data that does not conform to the motion logic, and at the same time to feed back the weight parameters and error data in this compensation process to the abnormal state real-time identification submodule, to complete the dynamic optimization of the abnormal judgment threshold, to form a closed-loop optimization link for the entire process, and to continuously improve the system's adaptability to complex environments; The execution logic and function of the pose smoothing output submodule are as follows: perform inter-frame smoothing on the verified pose data to eliminate screen flickering caused by slight pose jitter, and output the processed pose data to the VR rendering engine in real time to drive the synchronous update of the VR screen, ensuring the synchronization between the VR screen and head movement, and improving the smoothness and comfort of the VR experience.

[0022] A VR positioning compensation method for complex environments includes the following steps: S1. Multi-source data acquisition and preprocessing stage: The VR headset full-link operation data is acquired through the multi-source perception synchronous acquisition module, the time base of multi-source data is unified and standardized preprocessing is completed, and a standardized dataset that meets the solution requirements is output. S2, Initial pose calculation and anomaly identification stage: Based on the preprocessed standardized dataset, the initial pose calculation of the VR headset with six degrees of freedom is completed, and the real-time judgment and level classification of three types of abnormal states, namely visual positioning anomaly, inertial positioning anomaly and spatial positioning anomaly, are completed simultaneously. S3, Multi-source dynamic weighted fusion first-level compensation stage: Based on the anomaly identification results, adaptive weight allocation of each sensor data is completed, and extended Kalman filtering is used to complete the tight coupling fusion solution of weighted multi-source data, outputting VR head-mounted display pose data after primary compensation; S4, the second-level precise compensation stage of scene semantic constraints, matches the pose data after the initial compensation with the pre-built scene semantic map in real time, completes the pose compliance judgment, performs a second correction on poses that exceed semantic constraints, and outputs compliant precise pose data. S5, Closed-loop verification and pose output stage: After completing the motion continuity verification of the compensated pose, the verified pose data is smoothed between frames and output to the VR rendering engine in real time. At the same time, the parameter data of this compensation is fed back to the anomaly recognition module in a closed loop to complete the threshold optimization. Specifically, in the S1 multi-source data acquisition and preprocessing stage, corresponding to the system's multi-source perception synchronous acquisition module and scene-based data preprocessing module, the multi-source perception synchronous acquisition module acquires the full-link raw data during the VR headset operation process, including IMU inertial data, binocular vision image data, inter-frame optical flow data, and UWB anchor point ranging data. At the same time, it unifies the time base of multi-source data, eliminates the transmission delay difference between different sensors, performs distortion removal, motion blur removal, and weak texture enhancement processing on image data, and performs zero bias correction, gravity component separation, and noise filtering processing on IMU data. Finally, it outputs a standardized dataset that meets the requirements of pose calculation. In the S2 initial pose calculation and anomaly identification stage, the corresponding system's initial pose calculation and anomaly identification module, based on the preprocessed standardized dataset, completes the calculation of the VR headset's six-degree-of-freedom initial pose, and simultaneously completes the real-time judgment of three types of abnormal states: visual positioning anomaly, inertial positioning anomaly, and spatial positioning anomaly, and classifies the anomaly level according to the index exceeding the limit. In the S3 multi-source dynamic weighted fusion first-level compensation stage, which corresponds to the multi-source dynamic weighted fusion first-level compensation module of the system, based on the anomaly identification results output from the S2 stage, it completes the adaptive weight allocation of the three types of sensor data, uses the extended Kalman filter algorithm to complete the tight coupling fusion solution of the weighted multi-source data, and outputs the VR head-mounted display pose data after primary compensation. In the S4 scene semantic constraint secondary precise compensation stage, the corresponding system scene semantic constraint secondary precise compensation module matches the pose data after primary compensation with the pre-built scene semantic map in real time, completes the pose compliance judgment, determines whether the pose falls into the impenetrable area, performs secondary correction on the pose that exceeds the semantic constraints, and outputs precise pose data that conforms to the scene space rules. In the S5 closed-loop verification and pose output stage, the corresponding closed-loop verification and pose smoothing output module of the system completes the motion continuity verification of the compensated pose, performs inter-frame smoothing processing on the verified pose data, and outputs it to the VR rendering engine in real time. At the same time, the parameter data of this compensation is fed back to the anomaly identification stage in a closed loop to complete the optimization and update of the anomaly judgment threshold.

[0023] The weight allocation in stage S3 is negatively correlated with the anomaly level of the corresponding sensor data. In stage S4, the pose correction only adjusts the spatial position vector without changing the attitude vector. After the correction is completed, the pose motion continuity verification is completed simultaneously. If the verification fails, the pose smooth transition is completed based on the inter-frame optical flow data. Specifically, supplementary restrictions are made to the weight allocation rules in the S3 stage, clarifying that the weight allocation is negatively correlated with the anomaly level of the corresponding sensor data. That is, the higher the anomaly level of the sensor data, the lower the weight value allocated, ensuring that the interference of abnormal data on the fusion solution is minimized and further enhancing the reliability of the first-level compensation. The pose correction rules in the S4 stage are supplemented and restricted to clarify that pose correction only adjusts the spatial position vector and does not change the posture vector, thus ensuring the continuity of the VR screen view and avoiding user dizziness caused by screen jumps. At the same time, it is clarified that after the pose correction is completed, the pose motion continuity is checked simultaneously. When the check fails, the pose is smoothly transitioned based on the inter-frame optical flow data, further eliminating the screen jitter caused by pose jumps and improving the comfort and smoothness of the VR experience.

[0024] The method of use and working principle of this invention are as follows: Usage: Before formal use, first complete the construction of a 3D semantic map of the target VR application scene and the deployment of positioning anchor points. After the VR system is powered on, firstly, the multi-source perception synchronous acquisition unit synchronously acquires full-dimensional sensor data during the operation of the VR headset. The acquired multi-source data is preprocessed with unified and standardized time reference. Then, based on the preprocessed dataset, the initial spatial pose of the VR headset is calculated. Simultaneously, various positioning anomalies that occur during system operation are identified and their corresponding levels are classified. Subsequently, based on the anomaly identification results, adaptive weight allocation of multi-source sensor data is completed. The corresponding filtering algorithm is used to complete the tight coupling fusion calculation of the weighted multi-source data, and the pose data after primary compensation is output. Then, the pose data after primary compensation is matched and verified with the pre-built scene semantic map in real time. The poses that do not conform to the scene spatial rules are corrected in a second precise manner. Finally, the corrected pose data is subjected to motion continuity verification and inter-frame smoothing processing, and then output to the VR rendering engine in real time. At the same time, the compensation parameters during this operation are fed back to the anomaly identification stage in a closed loop to complete the dynamic optimization and adjustment of the system operation parameters.

[0025] Working Principle: Based on a dual compensation architecture of multi-source sensor complementary fusion and scene semantic constraints, and relying on the environmental anti-interference complementary characteristics of different types of sensor data, this system solves the core problem of single-sensor positioning schemes being prone to failure in complex environments. Through real-time identification and quantification of positioning anomalies, it achieves adaptive dynamic allocation of multi-source sensor data weights, reducing the impact of disturbed anomalies on the positioning solution results and completing the first level of robust positioning compensation. Then, combined with a pre-constructed 3D semantic map of the target application scenario, and using fixed physical boundaries and spatial operation rules within the scene as hard constraints, it performs a second level of precise correction on the fused pose data, eliminating positioning errors that do not conform to the actual physical rules of the scene. At the same time, through a closed-loop feedback mechanism throughout the entire process, it continuously optimizes the system's anomaly identification and compensation parameters, adapting to dynamic and static interference in different complex scenarios, ultimately achieving continuous, accurate, and stable operation of VR positioning in complex environments.

[0026] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Those skilled in the art can readily implement the present invention based on the description and drawings above. However, any modifications, alterations, and variations made by those skilled in the art without departing from the scope of the present invention using the disclosed technical content are equivalent embodiments of the present invention. Furthermore, any modifications, alterations, and variations made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the present invention.

Claims

1. A VR positioning compensation system for complex environments, characterized in that, It includes a multi-source sensing synchronous acquisition module, a scenario-based data preprocessing module, an initial pose calculation and abnormal state identification module, a multi-source dynamic weighted fusion first-level compensation module, a scenario semantic constraint second-level precise compensation module, and a closed-loop verification and pose smoothing output module.

2. The VR positioning compensation system for complex environments according to claim 1, characterized in that, The initial pose calculation and abnormal state recognition module includes an initial pose calculation submodule and an abnormal state real-time recognition submodule. The initial pose calculation submodule completes visual feature point extraction and matching, VR head-mounted display six-DOF initial pose calculation, IMU data pre-integration and pose prediction value output. The abnormal state real-time recognition submodule completes real-time judgment and level classification of three types of positioning abnormal states: visual positioning abnormality, inertial positioning abnormality, and spatial positioning abnormality.

3. The VR positioning compensation system for complex environments according to claim 1, characterized in that, The multi-source dynamic weighted fusion primary compensation module includes an adaptive dynamic weight allocation submodule and a multi-source data tightly coupled fusion compensation solution submodule. The adaptive dynamic weight allocation submodule dynamically allocates weights to visual positioning data, IMU inertial positioning data, and UWB spatial positioning data based on the abnormal state recognition results. The weight allocation is negatively correlated with the anomaly level of the corresponding sensor data. The core calculation formula is as follows: ; In the formula, The weight values ​​for visual positioning data. Weight values ​​for IMU inertial positioning data. Weight values ​​for UWB spatial positioning data, Let i be the weight value of the i-th type of sensor data. The anomaly level of the i-th type of sensor data, The maximum anomaly level is preset. When a certain type of sensor data is determined to be completely invalid, the corresponding weight value is set to 0, and the weight values ​​of the remaining valid sensor data are redistributed according to the above formula.

4. The VR positioning compensation system for complex environments according to claim 3, characterized in that, The multi-source data tightly coupled fusion compensation solution submodule uses extended Kalman filtering to complete the tightly coupled fusion of weighted multi-source data, and outputs the VR head-mounted display pose data after primary compensation. The core calculation formula is as follows: In the formula, The Kalman gain at time k, Let be the prior covariance matrix at time k. Let be the observation matrix at time k. Let be the transpose of the observation matrix. To observe the noise covariance matrix, the diagonal elements of the noise covariance matrix are negatively correlated with the weight values ​​of the corresponding sensor data.

5. The VR positioning compensation system for complex environments according to claim 1, characterized in that, The scene semantic constraint secondary precision compensation module includes a scene semantic map pre-construction sub-module and a semantic constraint pose correction sub-module. The scene semantic map pre-construction sub-module completes the construction of a 3D semantic map of the target application scene, marking the 3D coordinates and physical boundaries of fixed semantic objects in the scene, as well as walkable and impenetrable areas. The semantic constraint pose correction sub-module completes the secondary precision correction of the primary compensation pose. The core calculation formula is as follows: In the formula, This is the corrected spatial position vector for the VR headset. This is the VR headset spatial position vector after initial compensation. for The vertical distance to the nearest boundary of the impenetrable region. It is the unit vector of the outward normal of the nearest boundary of the impenetrable region.

6. The VR positioning compensation system for complex environments according to claim 1, characterized in that, The closed-loop verification and pose smoothing output module includes a closed-loop verification submodule and a pose smoothing output submodule. The closed-loop verification submodule performs motion continuity verification of the compensated pose and performs closed-loop feedback optimization of the compensation parameters. The pose smoothing output submodule performs inter-frame smoothing processing of the pose data and performs real-time output of the pose data to the VR rendering engine.

7. A VR positioning compensation method for complex environments, applied to the VR positioning compensation system for complex environments as described in any one of claims 1-6, characterized in that, Includes the following steps: S1. Multi-source data acquisition and preprocessing stage: The VR headset full-link operation data is acquired through the multi-source perception synchronous acquisition module, the time base of multi-source data is unified and standardized preprocessing is completed, and a standardized dataset that meets the solution requirements is output. S2, Initial pose calculation and anomaly identification stage: Based on the preprocessed standardized dataset, the initial pose calculation of the VR headset with six degrees of freedom is completed, and the real-time judgment and level classification of three types of abnormal states, namely visual positioning anomaly, inertial positioning anomaly and spatial positioning anomaly, are completed simultaneously. S3, Multi-source dynamic weighted fusion first-level compensation stage: Based on the anomaly identification results, adaptive weight allocation of each sensor data is completed, and extended Kalman filtering is used to complete the tight coupling fusion solution of weighted multi-source data, outputting VR head-mounted display pose data after primary compensation; S4, the second-level precise compensation stage of scene semantic constraints, matches the pose data after the initial compensation with the pre-built scene semantic map in real time, completes the pose compliance judgment, performs a second correction on poses that exceed semantic constraints, and outputs compliant precise pose data. S5, Closed-loop verification and pose output stage: After completing the motion continuity verification of the compensated pose, the verified pose data is smoothed between frames and output to the VR rendering engine in real time. At the same time, the parameter data of this compensation is fed back to the anomaly recognition module in a closed loop to complete the threshold optimization.

8. The VR positioning compensation method for complex environments according to claim 7, characterized in that, The weight allocation in the S3 stage is negatively correlated with the anomaly level of the corresponding sensor data. The pose correction in the S4 stage only adjusts the spatial position vector and does not change the attitude vector. After the correction is completed, the pose motion continuity verification is completed simultaneously. If the verification fails, the pose smooth transition is completed based on the inter-frame optical flow data.

Images