Virtual object pose control method in mixed reality environment
By dynamically adjusting the sensitivity and freedom of virtual objects in a mixed reality environment, combined with coordinate system mapping, the fine demand for position control of virtual objects is solved, the human-computer interaction experience is improved, and it is suitable for a variety of application scenarios.
Patent Information
- Application Number
- CN202410839877.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-26
- Publication Date
- 2024-11-15
AI Technical Summary
The spatial position of virtual objects in mixed reality environments is difficult to accurately control, resulting in poor human-computer interaction experience, and the existing technology is difficult to meet the fine demands of virtual objects in different application scenarios, especially in scenarios such as assembly and education and training. .
Provide a virtual object position control method, through the acquisition and processing of mixed reality gesture data, dynamic adjustment of sensitivity and degree of freedom, and on-demand switching of coordinate systems, including virtual object spatial position sensitivity adjustment, degree of freedom control and coordinate system mapping , combining the Unity engine and MRTK development framework to achieve flexible settings of sensitivity and freedom.
It realizes fine control and accurate operation of virtual object position, reduces erroneous operation, reduces user learning costs, improves human-computer interactive experience, and is suitable for a variety of application scenarios.
Smart Images

Figure CN118963538A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of mixed reality, virtual reality and human-computer interaction, and in particular to a method for controlling the position and posture of a virtual object in a mixed reality environment. Background Art
[0002] As an emerging technology in the field of physical and digital interaction, Mixed Reality (MR) technology can combine the real and virtual worlds to create new environments and visualizations, forming a mixed reality space that combines a person, a computer, and an environment, so that people can combine computer digital information to enhance their understanding of the real environment; at the same time, users can interact with the virtual world and the real world in this environment to achieve real-time interaction between virtual and reality and enhance the sense of reality of user experience. With the explosive development of concepts such as the metaverse and digital twins, mixed reality technology has been widely and deeply applied and developed in industries and fields such as education and training, medical care, manufacturing and maintenance, construction and civil engineering, military and security, tourism and culture, entertainment and games.
[0003] The spatial position of virtual objects in a mixed reality environment is the key to the presentation effect of virtual objects in the real world, and it is also an important factor affecting the human-computer interaction experience. However, the current virtual-reality interaction and human-computer interaction of mixed reality technology are mainly based on gestures, which poses certain challenges to the spatial position control of virtual objects: 1) In the current mixed reality environment, the virtual object interaction logic based on gestures, the slight movement of gestures will cause a large change in the three-dimensional position of the virtual object, making it difficult to accurately control the spatial position of the virtual object in the real physical space, resulting in a poor human-computer interaction experience and increasing the user's learning cost; 2) Different application scenarios have different requirements for virtual object position control. The current six-degree-of-freedom virtual object position control method in the mixed reality environment is difficult to meet the needs of more precise or special virtual objects. 3) From the perspective of the principle of spatial rotation posture, there are usually two methods: quaternion and Euler angle. However, for the independent control of specific axial degrees of freedom of the spatial rotation posture of virtual objects, quaternion processing is usually very complicated, and there is no simple and direct way to disable a certain axial rotation of the quaternion. Although the Euler angle can intuitively map a specific rotation axis, as an ordered non-continuous transformation, there is a "universal lock" problem. At present, the independent control of the axial degrees of freedom of the spatial rotation posture of virtual objects faces technical challenges.
[0004] Therefore, it is of great practical significance to realize the virtual object posture control in mixed reality environment. Applying it in the fields of mixed reality, virtual reality and human-computer interaction to realize virtual object posture control in mixed reality environment with dynamic sensitivity adjustment, flexible degree of freedom setting and on-demand switching of coordinate system can supplement and improve the human-computer interaction logic of virtual object posture control based on gesture in mixed reality, meet the application requirements of virtual object operation in different scenarios, reduce misoperation, lighten the learning burden of users, and improve the human-computer interaction experience. Summary of the invention
[0005] In order to dynamically adjust the sensitivity, degree of freedom and coordinate system reference of virtual object posture control in a mixed reality environment based on gesture interaction according to scene requirements, the present invention provides a virtual object posture control method in a mixed reality environment, the method comprising the following steps:
[0006] Step 1: Acquisition and processing of mixed reality gesture data;
[0007] Step 2: virtual object spatial posture sensitivity adjustment;
[0008] Step 3, control of the degree of freedom of the virtual object's spatial position;
[0009] Step 4: mapping the virtual object space pose coordinate system;
[0010] Step 5: Packaging, testing and release.
[0011] Furthermore, in step 1, the following steps may be performed in sequence in the Unity engine:
[0012] Step 1.1: Configuration of mixed reality development framework based on MRTK;
[0013] Step 1.2: Implement gesture interaction data interface;
[0014] Step 1.3: Acquisition of gesture interaction data based on MRTK;
[0015] Step 1.4: Gesture data processing.
[0016] Further, in step 2, the following steps may be performed in sequence:
[0017] Step 2.1: Adjust the sensitivity of the virtual object spatial position translation;
[0018] Step 2.2: Adjust the rotation sensitivity of the virtual object’s spatial position.
[0019] Further, in step 3, the following steps may be performed in sequence:
[0020] Step 3.1: Control of the degree of freedom of translation of the virtual object’s spatial position;
[0021] Step 3.2: Control the rotational freedom of the virtual object’s spatial position.
[0022] Further, in step 4, the following steps may be performed in sequence:
[0023] Step 4.1: In Visual Studio, and R2 are mapped to global or local coordinates simultaneously;
[0024] Step 4.2: Implement virtual object space pose update in Unity.
[0025] Step 4.3: Define the matching relationship between prefab, material ball, and virtual carrier mesh.
[0026] Further, in step 5, the following steps may be performed in sequence:
[0027] Step 5.1: Build a mixed reality human-computer interaction interface for spatial posture control;
[0028] Step 5.2: Open the saved project with Visual Studio, compile the project into ARM64 format, and publish it as an installation package; connect HoloLens 2, deploy and debug the usage environment of HoloLens 2, set HoloLens 2 to use Windows Device Portal, connect HoloLens 2 device via Wi-Fi, and publish the project;
[0029] Step 5.3: Wear the HoloLens 2 device and select different sensitivities, degrees of freedom, and coordinate axis references according to the human-computer interaction interface to perform virtual object space posture sensitivity adjustment test, degree of freedom control test, and coordinate system mapping test.
[0030] The advantages and positive effects of the present invention are:
[0031] (1) Flexibly adjust the spatial position of virtual objects in the global coordinate system or local coordinate system to achieve on-demand control of the spatial position;
[0032] (2) Flexibly and dynamically control the degrees of freedom of translation and rotation of virtual objects to achieve accurate control of spatial position;
[0033] (3) Flexibly and dynamically adjust the sensitivity of virtual object translation and rotation to achieve fine control of spatial posture;
[0034] (4) Encapsulated in script form, fast, flexible and low-cost deployment.
[0035] Based on this, the present invention proposes a virtual object posture control method in a mixed reality environment, in which the sensitivity can be dynamically adjusted, the degrees of freedom can be flexibly set, and the coordinate system can be switched on demand, which supplements and improves the human-computer interaction logic of virtual object posture control based on gestures in mixed reality; meets the application needs of virtual object operations in different scenarios, and greatly improves the application applicability of mixed reality technology in virtual assembly, education and training, remote operation, and other scenarios where virtual objects need to be operated more finely or with specific constraints; reduces misoperation, alleviates the user's learning burden, and improves the mixed reality virtual-reality interaction and human-computer interaction experience; provides a more efficient and practical method for virtual object posture control in the fields of mixed reality, virtual reality, and human-computer interaction, and promotes innovation and sustainable development in the fields of mixed reality, virtual reality, and human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The above and / or other aspects and advantages of the present invention will become clearer and easier to understand through the detailed description made in conjunction with the following drawings, which are only exemplary and do not limit the present invention, wherein:
[0037] Figure 1 It is a flow chart of the virtual object posture control method in a mixed reality environment.
[0038] Figure 2 This is an example diagram of configuring a mixed reality development environment in the Unity engine.
[0039] Figure 3 This is an example diagram of a mixed reality human-computer interaction interface for spatial posture control.
[0040] Figure 4 This is an example of operating and testing a HoloLens 2 device.
[0041] FIG. 5 is an example diagram of virtual object spatial posture sensitivity adjustment.
[0042] Figure 6 This is an example diagram of the control of the spatial position and freedom of a virtual object.
[0043] Figure 7-1 It is a schematic diagram of the global coordinate system of the virtual object;
[0044] Figure 7-2 This is a diagram of the virtual object being translated in the global coordinate system. Figure 1 ;
[0045] Figure 7-3 This is a diagram of the virtual object being translated in the global coordinate system. Figure 2 ;
[0046] Figure 7-4 A schematic diagram of the local coordinate system of a virtual object is shown;
[0047] Figure 7-5 Shows the translation of a virtual object in a local coordinate system Figure 1 ;
[0048] Figure 7-6 Shows the translation of a virtual object in a local coordinate system Figure 2 ; DETAILED DESCRIPTION
[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0050] like Figure 1 As shown, the present invention provides a method for controlling the posture of a virtual object in a mixed reality environment, the method comprising the following steps:
[0051] Step 1: Acquisition and processing of mixed reality gesture data;
[0052] Step 2: virtual object spatial posture sensitivity adjustment;
[0053] Step 3, control of the degree of freedom of the virtual object's spatial position;
[0054] Step 4: mapping the virtual object space pose coordinate system;
[0055] Step 5: Packaging, testing and release.
[0056] Furthermore, in step 1, the following steps can be performed in the Unity engine in sequence:
[0057] Step 1.1: Use the MixedRealityFeatureTool tool in the mixed reality development platform Unity engine to import the Mixed Reality Toolkit (MRTK) mixed reality development toolkit, which mainly includes the "Mixed RealityToolkit Examples", "Mixed Reality Toolkit Extensions", "Mixed Reality ToolkitUtilities" and "Mixed Reality Toolkit Tools" resource packages to complete the configuration of the mixed reality development environment, such as Figure 2 As shown;
[0058] Step 1.2: Use the C# programming language in Visual Studio to register the mixed reality gesture processor, define the gesture recognition logic, and listen to the gesture update event to implement the gesture interaction data interface, which mainly includes the following two MRTK methods "IMixedRealityGestureHandler <vector3>”"IMixedRealityGestureHandler <quaternion>”;
[0059] Step 1.3: Write a gesture data acquisition path based on C# language in Visual Studio, and obtain the Vector3-based translation gesture and Quaternion-based rotation gesture interaction data based on MRTK;
[0060] Step 1.3: Assign the mixed reality gesture data obtained from MRTK to the virtual object's spatial position "targetP osition" and rotation "targetRotation" data.
[0061] Further, in step 2, the following steps may be performed in sequence:
[0062] Step 2.1: Innovatively propose and introduce sensitivity parameters, including translation sensitivity S t and rotation sensitivity S r First, adjust the translation sensitivity according to the translation sensitivity parameter S t Incremental translation of spatial pose data based on Vector3 Scaling, where i x and i y The input values along the x-axis and y-axis are respectively, and then the processed spatial pose translation increment data is combined with the current spatial pose translation position data Add and output the intermediate data of spatial pose translation target position The specific expression is as follows:
[0063]
[0064] Step 2.2: Adjust the rotation sensitivity and increment R of the spatial posture rotation data based on Quaternion according to the rotation sensitivity parameter Sr Δ To scale, you need to convert the spatial position rotation quaternion into Euler angle data Multiply the rotation sensitivity parameter in the Euler angle to achieve the scaling of the rotation increment Then the processed Euler angle Reconvert it to the rotation quaternion Rscaled, and then multiply the processed spatial posture rotation increment quaternion data with the current spatial posture rotation position quaternion data R based on quaternion multiplication initial Multiply them together to output the intermediate data R1 of the spatial pose rotation target position. The specific expression is as follows:
[0065]
[0066] Further, in step 3, the following steps may be performed in sequence:
[0067] Step 3.1: Control the spatial posture freedom of the virtual object. An innovative method for independent control of the axial freedom is proposed. First, the translation freedom control is performed to translate the spatial posture to the target position intermediate data. Map to the xyz three axes to form V 1x 、V 1y and V 1z Three degrees of freedom separate data, At the same time, the current spatial pose position data is translated Map to the xyz three axes to form V C1x 、V C1y and V C1z Three degrees of freedom separate data, According to the axial translation freedom control requirements, in V 1x 、V 1y 、V 1z 、V C1x 、V C1y and V C1z The selection process can be described by introducing the function F, which acts on each axial translational degree of freedom. or The corresponding component of the control vector Each F i When (i=x,y,z) is 1, select Select when it is 0 Form new degrees of freedom data and output the final data of the spatial pose translation target position Where V 2i =F i ·V 1i +(1-F i )·V C1i , i represents x, y or z, the specific expression is as follows:
[0068]
[0069] Step 3.2: Perform rotational freedom control. The present invention innovatively proposes a spatial posture rotational freedom independent control method combining quaternion and Euler angle, converting the quaternion data of the spatial posture rotation target position intermediate data R1 into the Euler angle data O1 of the spatial posture rotation target position intermediate data, O1 = Euler (R1), and then mapping the Euler angle data O1 to the xyz three-axis, O1 = (O 1x , O 1y , O 1z ), forming O 1x , O 1y and O 1z The three degrees of freedom are separated into data, and then the current spatial position rotation position quaternion data R C1 Convert to Euler angle data O C1 , O C1 =Euler(R C1 ), and mapped to the xyz three axes, O C1 =(O C1x , O C1y , O C1z ), forming O C1x , O C1y and O C1z The three degrees of freedom are independent data, according to the control requirements of the axial rotational degree of freedom, in O 1x , O 1y , O 1z , O C1x , O C1y and O C1z Select the corresponding degree of freedom data from the selection process, the selection process is similar to the translational degree of freedom control, and introduce a control vector Each F Ri (i = x, y, z) determines whether to select the component of O1 or O C1 The components form the new spatial posture rotation target position final data Euler angle data O2 = (O 2x , O 2y , O 2z ), where O 2i =F Ri ·O 1i +(1-F Ri )·O C1i , i represents x, y or z, and finally outputs the quaternion data R2 of the final data of the spatial posture rotation target position. The specific expression is as follows:
[0070] R2=Quaternion(O2).
[0071] Further, in step 4, the following steps may be performed in sequence:
[0072] Step 4.1: Virtual object space pose coordinate system mapping. The present invention proposes a virtual object coordinate system reference conversion method. According to the coordinate system reference requirements, and R2 are simultaneously mapped to the global coordinates or local coordinates of the virtual object, and the global coordinate system is translated as The rotation is transform.rotation = R2, and the local coordinate system translation is Rotation is transform.localRotation = R2;
[0073] Step 4.2: Update the spatial pose of the virtual object in the global coordinate system or in the local coordinate system relative to the parent object. The spatial pose translation in the global coordinate system is expressed as targetPosition = transform.position, and the rotation is expressed as targetRotation = transform.rotation; the spatial pose translation in the local coordinate system is expressed as targetPosition = transform.localposition, and the rotation is expressed as targetRotation = transform.localrotation.
[0074] Furthermore, in step 5, the following steps may be performed in sequence in the Unity engine:
[0075] Step 5.1: Build a mixed reality human-computer interaction interface for spatial posture control in the Unity engine, and call the Button component to build interactive buttons, including translation and rotation sensitivity sliding adjustment buttons, degree of freedom independent control buttons, and coordinate system reference switching buttons, such as Figure 3 As shown;
[0076] Step 5.2: Open the saved project with Visual Studio, compile the project into a format suitable for HoloLens 2 (ARM64 format), publish it into an installation package, connect HoloLens 2, deploy and debug the usage environment of HoloLens 2, set HoloLens to use Windows Device Portal, connect HoloLens 2 device via Wi-Fi, and publish the project;
[0077] Step 5.3: Wear the HoloLens 2 device in real space, and select different sensitivities, degrees of freedom, and coordinate axis references according to the human-computer interaction interface to perform virtual object space posture sensitivity adjustment test, degree of freedom control test, and coordinate system mapping test, such as Figure 4 - As shown in Figure 7, as shown in Figure 5, taking the translation sensitivity control as an example, only the x-axis translation freedom is released, the solid line represents the moving distance of the gesture, 5-1 indicates that the sensitivity is set to the highest setting, it can be seen that the position change of the virtual object responds almost in real time to the gesture movement, the white solid line is a straight line, 5-2 indicates that when the sensitivity is set to low sensitivity, it can be seen that the position change of the virtual object responds slower to the gesture movement, and the white solid line begins to bend, indicating that a larger gesture moving distance is required to make a smaller translation position change of the virtual object, proving that the sensitivity adjustment has taken effect. Figure 6 The x-axis translation is locked. You can see that when the virtual object is dragged to the right along the x-axis, the virtual object's spatial position will not move along the x-axis, indicating that the x-axis translation freedom has been independently controlled. The same is true for the rotation freedom. Figure 7-1 The global coordinate system of the virtual object is displayed. It can be observed that the positive direction of the y-axis is toward the north. At this time, only the y-axis translation freedom is released. In the global coordinate system, the gesture moves toward the north, and the virtual object will only translate in the positive direction of the y-axis, that is, the north direction. For example, Figure 7-2 and Figure 7-3 As shown, Figure 7-4 The local coordinate system of the virtual object is displayed. It can be observed that the positive direction of the y-axis is 45 degrees northwest. At this time, only the y-axis translation freedom is released. In the local coordinate system, the gesture moves toward the north, but the virtual object will only translate in the positive direction of the y-axis, that is, 45 degrees northwest. Figure 7-5 and Figure 7-6 As shown, this proves that the coordinate axis reference has been switched successfully.
[0078] The method for controlling the posture of a virtual object in a mixed reality environment provided by the present invention innovatively proposes and introduces a sensitivity parameter, flexibly adjusts the response degree of the virtual object to gesture movement, and realizes fine control of the spatial posture; at the same time, a method for independently controlling the axial degree of freedom of the virtual object is proposed to realize accurate control of the spatial position; in addition, a method for reference conversion of the virtual object coordinate system is proposed to realize on-demand control of the spatial posture; the present invention solves the common problem that it is difficult to dynamically adjust the degrees of freedom and sensitivity of the posture control of virtual objects in a mixed reality environment based on gesture interaction according to scene requirements in current mixed reality applications; the present invention supplements and improves the human-computer interaction logic of virtual objects based on gestures in mixed reality, meets the application requirements of virtual object operations in different scenes, reduces the user's learning burden, improves the human-computer interaction experience, and provides a more efficient and practical method for virtual object posture control in the fields of mixed reality, virtual reality and human-computer interaction.
[0079] The present invention is not limited to the above-mentioned embodiments. Anyone can derive other various forms of products under the inspiration of the present invention. However, no matter what changes are made in the shape or structure, any technical solution that is the same or similar to that of the present application falls within the protection scope of the present invention.< / quaternion>
Claims
1. A method for controlling the posture of a virtual object in a mixed reality environment, characterized in that: The method comprises the following steps: Step 1: Acquisition and processing of mixed reality gesture data. Use MixedReality Toolkit (MRTK) in the mixed reality development platform Unity engine to configure the mixed reality development framework. Use C# programming language in Visual Studio to register the mixed reality gesture processor, define gesture recognition logic, and monitor gesture update events to implement the gesture interaction data interface. Based on MRTK, obtain the translation gesture based on Vector3 and the rotation gesture interaction data based on Quaternion, and convert the gesture data into virtual object space pose data. Step 2: Innovatively propose and introduce sensitivity parameters, including translation sensitivity and rotation sensitivity , first adjust the translation sensitivity, according to the translation sensitivity parameter Incremental translation of spatial pose data based on Vector3 To scale, and The input values along the X-axis and Y-axis are respectively, and then the processed spatial pose translation increment data is combined with the current spatial pose translation position data Add and output the intermediate data of spatial pose translation target position , the specific expression is as follows: Adjust the rotation sensitivity according to the rotation sensitivity parameters Incremental Quaternion-based spatial pose rotation data To scale, you need to convert the spatial position rotation quaternion into Euler angle data , multiply the rotation sensitivity parameter in the Euler angle to achieve the scaling of the rotation increment , and then the processed Euler angle Convert back to rotation quaternion , and then based on quaternion multiplication, the processed spatial posture rotation increment quaternion data and the current spatial posture rotation position quaternion data Multiply them together to output the intermediate data R1 of the spatial pose rotation target position. The specific expression is as follows: ; Step 3: Control the spatial posture freedom of the virtual object. An innovative method for independent control of the axial freedom is proposed. First, the translation freedom control is performed to translate the spatial posture to the target position intermediate data. Map to the xyz three axes to form , and Three degrees of freedom separate data, , while translating the current spatial pose position data Map to the xyz three axes to form , and Three degrees of freedom separate data, According to the control requirements of axial translation freedom, , , , , and Select the corresponding degree of freedom data from the selection process, and the selection process can be introduced by introducing the function To describe, Acting on each axial translational degree of freedom, it is decided to use or The corresponding component of the control vector , where each When 1, select , when it is 0, select , forming new degrees of freedom data, and outputting the final data of the spatial pose translation target position ,in , Represents x, y or z. The specific expression is as follows: Secondly, the rotational freedom control is carried out. The present invention innovatively proposes a spatial posture rotational freedom independent control method combining quaternion and Euler angle to convert the spatial posture rotation target position intermediate data into The quaternion data is converted into the Euler angle data of the intermediate data of the spatial position rotation target position , , and then use the Euler angle data Mapping to the xyz three axes, ,form , and Three degrees of freedom separate data, and then the current spatial position rotation position quaternion data Convert to Euler angle data , , and mapped to the xyz three axes, ,form , and The three degrees of freedom are separated into separate data, according to the control requirements of the axial rotational degree of freedom. , , , , and Select the corresponding degree of freedom data from the selection process, the selection process is similar to the translational degree of freedom control, and introduce a control vector , where each Decided is the choice The amount is still The components form the Euler angle data of the final data of the new spatial posture rotation target position ,in , Represents x, y or z, and finally outputs the quaternion data R2 of the final data of the spatial posture rotation target position. The specific expression is as follows: Quaternion ; Step 4, virtual object space pose coordinate system mapping, the present invention proposes a virtual object coordinate system reference conversion method, according to the coordinate system reference requirements, and At the same time, the virtual object is mapped to the global coordinates or local coordinates to achieve the spatial posture update of the virtual object in the global coordinate system or in the local coordinate system relative to the parent object, also known as the component coordinate system; Step 5: Packaging, publishing, and testing. Build a mixed reality human-computer interaction interface for spatial posture control in the Unity engine, including translation and rotation sensitivity sliding adjustment buttons, independent degree of freedom control buttons, and coordinate system reference switching buttons. Use Visual Studio to compile the project into ARM64 format and connect HoloLens 2 for publishing. Wear the HoloLens 2 device in real space, and select different sensitivities, degrees of freedom, and coordinate axis references according to the human-computer interaction interface to perform virtual object spatial posture sensitivity adjustment tests, degree of freedom control tests, and coordinate system mapping tests.
2. The method for controlling the position and posture of a virtual object in a mixed reality environment according to claim 1, characterized in that: In step 1, you can perform the following steps in the Unity engine: Step 1.1: Configuration of mixed reality development framework based on MRTK; Step 1.2: Implement gesture interaction data interface; Step 1.3: Acquisition of gesture interaction data based on MRTK; Step 1.4: Gesture data processing.
3. The method for controlling the position and posture of a virtual object in a mixed reality environment according to claim 1, wherein: In step 2, the following steps can be performed in sequence: Step 2.1: Adjust the sensitivity of the virtual object spatial position translation; Step 2.2: Adjust the rotation sensitivity of the virtual object’s spatial position.
4. The method for controlling the position and posture of a virtual object in a mixed reality environment according to claim 1, wherein: In step 3, the following steps may be performed in sequence: Step 3.1: Control of the degree of freedom of translation of the virtual object’s spatial position; Step 3.2: Control the rotational freedom of the virtual object’s spatial position.
5. The method for controlling the position and posture of a virtual object in a mixed reality environment according to claim 1, wherein: In step 4, the following steps may be performed in sequence: Step 4.1: In Visual Studio, and Map to global coordinates or local coordinates at the same time; Step 4.2: Implement virtual object space pose update in Unity.
6. The method for controlling the position and posture of a virtual object in a mixed reality environment according to claim 1, wherein: In step 5, the following steps may be performed in sequence: Step 5.1: Build a mixed reality human-computer interaction interface for spatial posture control; Step 5.2: Open the saved project with Visual Studio, compile the project into ARM64 format, and publish it as an installation package; connect HoloLens 2, deploy and debug the usage environment of HoloLens 2, set HoloLens 2 to use Windows Device Portal, connect HoloLens 2 device via Wi-Fi, and publish the project; Step 5.3: Wear the HoloLens 2 device and select different sensitivities, degrees of freedom, and coordinate axis references according to the human-computer interaction interface to perform virtual object space posture sensitivity adjustment test, degree of freedom control test, and coordinate system mapping test.
