Interaction control method and apparatus, readable storage medium, and electronic device
By running a simulator on a PC and adding simulated sensor components, a virtual reality space and target markers are generated, solving the problem that PC devices cannot simulate the 6 degrees of freedom interaction of XR devices, and realizing low-cost XR system testing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING ZITIAO NETWORK TECH CO LTD
- Filing Date
- 2025-01-06
- Publication Date
- 2026-07-14
AI Technical Summary
In extended reality scenarios, using a PC's mouse and keyboard cannot effectively simulate the 6 degrees of freedom interactive control of an XR device, resulting in high testing costs.
By running a simulator on a PC device to simulate an extended reality operating system and adding simulated sensor components to the simulator's driver layer, a virtual reality space is generated, virtual objects are displayed, and pose information is determined by the position of the target marker, thus realizing the simulation of 6-DOF data.
It reduces the testing cost of XR systems, enables the simulation of XR device operation through PC devices, and provides a similar interactive control experience.
Smart Images
Figure CN122387291A_ABST
Abstract
Description
Technical Field
[0001] This disclosure belongs to the field of virtual machine technology, specifically relating to an interactive control method and apparatus, a readable storage medium, and an electronic device. Background Technology
[0002] In related technologies, in extended reality (XR) scenarios, users can change their position and perspective in the virtual reality scene by moving their heads, and interact with virtual objects in the virtual reality scene through controllers such as extended reality input controllers.
[0003] Unlike the mouse and keyboard of a personal computer (PC), extended reality input controllers have 6 degrees of freedom (DoF), while the mouse and keyboard only have 2 degrees of freedom. Therefore, it is impossible to interact with the system through conventional input methods such as mouse or keyboard. As a result, when testing an XR system, it is necessary to set up an XR device as a test environment, which results in high testing costs. Summary of the Invention
[0004] The purpose of this disclosure is to provide an interactive control method and apparatus, a readable storage medium and an electronic device, which can simulate the operation of XR devices through a PC device, thereby reducing the testing cost of XR systems.
[0005] In a first aspect, embodiments of this disclosure provide an interactive control method applied to an electronic device, the interactive control method comprising:
[0006] An extended reality operating system is run through a simulator to generate a virtual reality space; the simulator includes a simulated sensor module.
[0007] Displaying a virtual reality interface; wherein, the virtual reality interface is used to display virtual objects in the virtual reality space, and the virtual reality interface includes target identifiers, the display position of which is associated with the user's interactive input to the virtual reality interface;
[0008] Determine the first pose information based on the display position of the target identifier;
[0009] The first pose information is sent to the extended reality operating system via the analog sensor module, so that the extended reality operating system can determine the interactive input information of the extended reality input controller based on the first pose information.
[0010] Secondly, embodiments of this disclosure provide an interactive control device applied to an electronic device, the interactive control device comprising:
[0011] The runtime module is used to run the extended reality operating system through the simulator to generate a virtual reality space; wherein, the simulator includes a simulated sensor module;
[0012] The display module is used to display the virtual reality interface; wherein, the virtual reality interface is used to display virtual objects in the virtual reality space, and the virtual reality interface includes target identifiers, the display position of which is associated with the user's interactive input to the virtual reality interface;
[0013] The determination module is used to determine the first pose information based on the display position of the target identifier;
[0014] The transmitting module is used to send the first pose information to the extended reality operating system through the analog sensor module, so that the extended reality operating system can determine the interactive input information of the extended reality input controller based on the first pose information.
[0015] Thirdly, embodiments of this disclosure provide an interactive control device, which includes:
[0016] Memory, used to store programs or instructions;
[0017] A processor is used to implement the interactive control method as described in the first aspect when executing the aforementioned program or instructions.
[0018] Fourthly, embodiments of this disclosure provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method as described in the first aspect.
[0019] Fifthly, embodiments of this disclosure provide an electronic device, including an interactive control device as described in the second or third aspect; and / or a readable storage medium as described in the fourth aspect.
[0020] In a sixth aspect, embodiments of this disclosure provide a chip including a processor and a communication interface coupled to the processor, the processor being used to run programs or instructions to implement the steps of the interactive control method of the first aspect.
[0021] In a seventh aspect, embodiments of this disclosure provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the steps of the interactive control method of the first aspect.
[0022] The summary section is provided to introduce a series of concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or essential features of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0023] Figure 1 Flowcharts of interactive control methods according to some embodiments of this disclosure are shown;
[0024] Figure 2 A schematic diagram of a virtual reality interface according to some embodiments of the present disclosure is shown;
[0025] Figure 3 A schematic diagram of the architecture of an extended reality operating system according to some embodiments of this disclosure is shown;
[0026] Figure 4 A schematic diagram of the coordinate system of a virtual reality space according to some embodiments of the present disclosure is shown;
[0027] Figure 5 A schematic diagram of the projection of virtual reality space onto a virtual reality interface is shown, representing some embodiments of the present disclosure.
[0028] Figure 6 One of the perspective projection schematic diagrams of an analog extended reality input controller according to some embodiments of the present disclosure is shown;
[0029] Figure 7 One of the schematic diagrams showing the effect of a virtual reality interface according to some embodiments of the present disclosure is illustrated.
[0030] Figure 8 A second perspective projection schematic diagram of an analog extended reality input controller according to some embodiments of the present disclosure is shown;
[0031] Figure 9 This is the second schematic diagram showing the effect of a virtual reality interface according to some embodiments of the present disclosure;
[0032] Figure 10 Structural block diagrams of interactive control devices according to some embodiments of the present disclosure are shown;
[0033] Figure 11 Structural block diagrams of interactive control devices according to some embodiments of the present disclosure are shown;
[0034] Figure 12 A structural block diagram of an electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation
[0035] The technical solutions of the embodiments of this disclosure will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure are within the scope of protection of this disclosure.
[0036] The terms "first," "second," etc., used in this disclosure and in the claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this disclosure can be implemented in orders other than those illustrated or described herein, and that the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0037] The interactive control method and apparatus, readable storage medium and electronic device provided in this disclosure will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.
[0038] As mentioned above, extended reality input controllers have 6 degrees of freedom, while PC mice and keyboards only have 2 degrees of freedom, making interaction impossible through conventional input methods like mice or keyboards. Testing XR systems requires setting up XR equipment as a test environment, resulting in high testing costs.
[0039] In view of this, in some embodiments of this disclosure, an interactive control method is provided. Figure 1 Flowcharts of interactive control methods according to some embodiments of this disclosure are shown, such as Figure 1 As shown, the interactive control methods include:
[0040] Step 102: Run the extended reality operating system through a simulator to generate a virtual reality space; wherein the simulator includes a simulated sensor module.
[0041] This disclosure aims to provide a method for simulating the operation of an extended reality operating system using an emulator on a PC. An emulator is software that can simulate the functionality of one device using another. The core working principle of an emulator lies in creating a virtual machine environment, which then simulates the operation system (OS) of the target device. For example, this disclosure uses an emulator running on a PC to simulate the operation of an XR device's operating system, i.e., the aforementioned extended reality operating system.
[0042] An extended reality operating system refers to the operating system for extended reality devices, also known as XR devices. Extended reality is an interactive environment combining real and virtual elements, created through computer technology and wearable devices. It encompasses the concepts of Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR). Through an extended reality operating system and corresponding applications, it is possible to simulate and create 3D virtual reality spaces. 3D virtual objects can be generated within these virtual reality spaces, or real-world objects can be projected into them.
[0043] To simulate the operation of XR devices, this disclosure adds a virtual sensor component to the simulator's driver layer. The virtual sensor component can simulate the 6 degrees of freedom information collected by the head-mounted display device, or the 6 degrees of freedom information collected by the sensors in the extended reality input controller, thereby simulating a realistic XR environment.
[0044] Step 104: Display the virtual reality interface; wherein, the virtual reality interface is used to display virtual objects in the virtual reality space, and the virtual reality interface includes target identifiers, the display position of which is associated with the user's interactive input to the virtual reality interface.
[0045] In this embodiment of the disclosure, the virtual reality interface is the interface displayed when a PC or other device simulates the running of an XR system or software. That is, it projects virtual objects in a 3D virtual reality space onto a display plane from a single viewpoint. For example, the target identifier is the mouse pointer on the PC device, and the target identifier in the virtual reality interface corresponds to the position of the controller ray in the virtual reality scene.
[0046] The display position of the target icon is associated with the user's interactive input to the virtual reality interface. For example, the user can change the display position of the target icon by moving the mouse pointer within the virtual reality interface.
[0047] For example, Figure 2 The following are schematic diagrams illustrating virtual reality interfaces of some embodiments of this disclosure, such as... Figure 2 As shown, the virtual reality interface 200 includes a virtual object 202 and a target identifier 204. The display position of the target identifier 204 in the virtual reality interface 200 is related to the user's interactive input. For example, the target identifier 204 is a mouse pointer; when the user moves the mouse, the display position of the target identifier 204 in the virtual reality interface 200 can be changed.
[0048] Step 106: Determine the first pose information based on the display position of the target identifier.
[0049] In this technical solution, taking the target identifier corresponding to the mouse pointer as an example, the display position of the target identifier is also the position of the mouse pointer in the virtual reality interface. The simulator uses the mouse to simulate and generate 6-DOF data of the extended reality input controller. Since the changes that the mouse can simulate are two-dimensional, when simulating the 6-DOF data of the extended reality input controller, one can be selected as a fixed position or direction, and then the other can be simulated.
[0050] For example, the orientation of the extended reality input controller can be fixed, and the position movement of the extended reality input controller can be simulated by mouse operation.
[0051] For example, the position of the extended reality input controller can be fixed, and the direction of the extended reality input controller can be simulated by mouse operation.
[0052] Step 108: The first pose information is sent to the extended reality operating system through the analog sensor module, so that the extended reality operating system can determine the interactive input information of the extended reality input controller based on the first pose information.
[0053] In this embodiment, the actual XR device includes a head-mounted display (HUD) and an extended reality input controller (ARIC). Sensors embedded in the HUD and ADI acquire 6-DOF (6 degrees of freedom) information at specified frequencies and report it to the system. The system calculates the device's position and orientation in the virtual 3D scene based on the pose information reported by the sensors in the real-world scene. XR applications use a software development kit (SDK) to acquire pose information in the virtual reality space, thereby determining the final display effect of the application.
[0054] For example, Figure 3 The following are schematic diagrams illustrating the architecture of an extended reality operating system according to some embodiments of this disclosure, such as... Figure 3As shown, taking the simulation of an extended reality operating system using the QEMU (a type of processor simulation software) emulator as an example, the system architecture includes an emulator part and a simulated extended reality operating system part. The emulator user interface (EmulatorUI) sends data to the driver layer via the emulator pipeline (QEMU pipe). In addition to the drivers for the real sensors (sensor 1 and sensor 2), the driver layer also includes virtual sensor components. User input via input devices such as a mouse or keyboard is sent to the driver layer through the emulator pipeline. When the current operating environment is recognized as an emulator environment, the loaded extended reality operating system reads the 6 degrees of freedom (DOF) information from the virtual sensor components and provides it to the upper application layer. One or more applications in the application layer can determine the final display effect based on the 6 DFB information reported by the virtual sensor components.
[0055] For example, VSOCK (Virtual Socket) is a mechanism used in a virtual machine environment for efficient communication between the virtual machine and the host. This embodiment of the disclosure creates a new QEMU-VSOCK channel in the QEMU emulator to provide 6-DOF data generated by the emulator to upper-layer applications. This allows upper-layer applications to be unaware of the emulator environment, reducing the scope of modifications to the extended reality operating system and extended reality applications, and enabling the transfer of arbitrary custom data types.
[0056] This disclosure proposes a method for running an extended reality operating system via a simulator, enabling control interaction with a 6-DOF (DoF) extended reality input controller in a simulated XR device using a conventional 2-DOF input device. The position of the controller ray in a virtual reality scene is simulated based on the position of target markers such as the mouse pointer on the display interface. An algorithm converts the coordinates of the mouse pointer into pose data for the extended reality input controller, thus simulating XR device operation via a PC. Therefore, a test environment for XR systems or software can be built using a PC, reducing the testing cost of XR systems.
[0057] In some embodiments of this disclosure, the step of displaying a virtual reality interface includes: displaying the virtual reality interface with a virtual camera in the virtual reality space as the viewpoint; the step of determining first pose information based on the display position of a target identifier includes: determining the planar coordinates of the target identifier in the virtual reality interface based on the display position of the target identifier; determining the spatial coordinates corresponding to the display position of the target identifier in the virtual reality space based on the planar coordinates and the interface parameters of the virtual reality interface; wherein the interface parameters include interface size and field of view information; and determining the first pose information based on the spatial coordinates and the second pose information of the virtual camera.
[0058] In this embodiment, to project the virtual reality space onto a display plane, i.e., into the virtual reality interface, a viewpoint needs to be determined. The aforementioned virtual camera is the viewpoint corresponding to the virtual reality interface. Exemplarily, common extended reality devices include head-mounted displays (HUDs) and extended reality input controllers. Both HUDs and input controllers are 6 degrees of freedom (6DoF) devices. They can not only represent the rotation of the user's visual direction but also the movement of the user's spatial position. Typically, 6DoF data includes a three-dimensional coordinate representing the spatial position and data representing the direction. Exemplarily, a direction can be represented by a rotation angle relative to a default direction. For example, a direction can be represented by Euler angles or quaternions. This embodiment illustrates the use of quaternions to identify direction, meaning that a 6DoF data set includes a spatial coordinate and a rotation quaternion.
[0059] For example, when the simulator program starts, the coordinate system is set to a right-handed coordinate system. Figure 4 A schematic diagram of the coordinate system of virtual reality space according to some embodiments of this disclosure is shown, such as... Figure 4 As shown, the coordinate system of the virtual reality space includes the x-axis, y-axis, and z-axis, where the z-axis points vertically outward from the virtual reality interface. The position of the head-mounted display is initialized to the spatial origin (0, 0, 0). The initial orientation of the head-mounted display is initialized to the z-axis direction (0, 0, -1). The corresponding rotation quaternion is (1, 0, 0, 0). Therefore, the positions of the two virtual cameras corresponding to the left and right eyes are (N, 0, 0) and (-N, 0, 0) respectively. Here, N is half the interpupillary distance.
[0060] After displaying the virtual reality (VR) interface, the interface includes a target marker indicating the position of the extended reality input controller (ERC) ray. This target marker can be a mouse pointer. The target marker moves in two dimensions within the VR interface, with a planar coordinate structure of (x, y). The VR space is a three-dimensional space, and the coordinates of the ERC ray position within this three-dimensional space are also three-dimensional coordinates. Since the second pose information of the virtual camera in the VR space follows the simulated head-mounted display pose, the virtual camera's pose information is known in the simulator. Given the second pose information and the interface parameters of the VR interface, the three-dimensional spatial data corresponding to the target marker in the VR interface can be calculated using a perspective projection algorithm from three dimensions to two dimensions. This includes, for example, the three-dimensional coordinates of the target marker in the VR space and the first pose information of the simulated ERC input controller.
[0061] For example, Figure 5The following are schematic diagrams illustrating the projection of virtual reality space onto a virtual reality interface according to some embodiments of the present disclosure, such as... Figure 5 As shown, point A is a feature point of the simulated extended reality input controller. Plane a is the display plane corresponding to the virtual reality interface. Point B is the target identifier, i.e., the intersection of the ray from the extended reality input controller and the display plane corresponding to the virtual reality interface.
[0062] By using the planar coordinates of the target icon in the virtual reality interface and employing a perspective projection algorithm, the spatial coordinates of the target icon in the virtual reality space are obtained. Combined with the second pose information of the virtual camera, the first pose information of the virtual extended reality input controller can be calculated. This enables the mapping of 2-DOF mouse operations to 6-DOF extended reality input controller operations, allowing the simulation of XR device operation via PCs and other devices.
[0063] In some embodiments of this disclosure, the second pose information further includes the position coordinates of the virtual camera in the virtual reality space; wherein the position coordinates of the extended reality input controller in the virtual reality space are the same as the position coordinates of the virtual camera in the virtual reality space.
[0064] In this embodiment of the disclosure, the simulator uses the 2-DOF data of the mouse to simulate the 6-DOF data of the extended reality input controller. Therefore, when simulating the extended reality input controller, either the position or the orientation needs to be fixed.
[0065] For example, a scheme that simulates the positional movement of the extended reality input controller while keeping its orientation fixed. Figure 6 One of the perspective projection schematic diagrams of an analog extended reality input controller according to some embodiments of this disclosure is shown. Figure 7 One of the schematic diagrams illustrating the effect of a virtual reality interface according to some embodiments of this disclosure is shown. For example... Figure 6 and Figure 7 As shown, point A is a feature point of the simulated Extended Reality Input Controller. Points B and B' are the center viewpoint. Point C is the mouse pointer. Point C' is the collision response point of the ray from the Extended Reality Input Controller.
[0066] In this situation, such as Figure 7 As shown, the simulated collision response point of the extended reality input controller's ray at the mouse pointer's location is not the actual mouse pointer position. When using a real XR device, without a specific anchor point, the user actively adjusts the extended reality input controller's pose based on the current ray position. However, when using an emulator on a PC, users are accustomed to using the mouse pointer as the operation anchor point when using PC programs. If the operation response point is inconsistent with the mouse pointer's indicated point, it will severely impact the user experience.
[0067] Therefore, this disclosure selects a scheme to fix the position of the extended reality input controller and simulate the rotation of the extended reality input controller.
[0068] For example, Figure 8 A second perspective projection schematic diagram of an analog extended reality input controller according to some embodiments of this disclosure is shown. Figure 9 This is a second schematic diagram illustrating the effect of a virtual reality interface according to some embodiments of the present disclosure. For example... Figure 8 and Figure 9 As shown, point A is a feature point of the simulated extended reality input controller, point B is the center viewpoint, and point C coincides with point C'.
[0069] This disclosure sets the spatial coordinates of the extended reality input controller to be the same as those of the virtual camera. By fixing the position of the extended reality input controller and simulating its rotation, the logic for simulating extended reality input controller operation via mouse input is defined. This ensures that the ray position of the extended reality input controller coincides with the mouse pointer position. This guarantees that the ray of the extended reality input controller and the mouse pointer coincide, and also allows the ray of the extended reality input controller to be mapped as a point in the virtual reality interface, thus ensuring a good user experience when simulating XR device operation through the simulator.
[0070] In some embodiments of this disclosure, the interface size includes the interface width, and the field of view information includes the horizontal field of view.
[0071] The steps for determining the spatial coordinates of the target identifier's display position in virtual reality space based on planar coordinates and interface parameters of the virtual reality interface specifically include: determining the depth coordinates of the target identifier's display position in virtual reality space based on the interface width and horizontal field of view; and determining the spatial coordinates based on the planar coordinates and depth coordinates.
[0072] In this embodiment of the disclosure, the interface size of the virtual reality interface includes the interface width, and the field of view information includes the horizontal field of view (Fov). It is assumed that the planar coordinates of the target identifier in the virtual reality interface are (x, y), and the spatial coordinates of the target identifier in the virtual reality space are (x, y, z). near The process of determining the spatial coordinates of the target identifier can actually be viewed as determining the depth coordinate value z. near The process.
[0073] For example, the depth coordinate value z is calculated using the following formula (1). near :
[0074]
[0075] Among them, z near Here are the depth coordinates, width is the interface width, and horizontalFov is the horizontal field of view.
[0076] Based on the interface width and horizontal field of view of the virtual reality interface, this disclosure can convert two-dimensional planar coordinates into three-dimensional spatial coordinates, enabling the operation of an extended reality input controller to be simulated through mouse operation.
[0077] In some embodiments of this disclosure, the interface size includes the interface height, and the field of view information includes the vertical field of view. The step of determining the spatial coordinates of the display position of the target identifier in the virtual reality space based on the planar coordinates and the interface parameters of the virtual reality interface specifically includes: determining the depth coordinate value of the display position of the target identifier in the virtual reality space based on the interface height and the vertical field of view; and determining the spatial coordinates based on the coordinate values of the planar coordinates and the depth coordinate values.
[0078] In this embodiment of the disclosure, the interface dimensions of the virtual reality interface include the interface height, and the field of view information includes the vertical field of view (Fov). Assuming the planar coordinates of the target identifier in the virtual reality interface are (x, y), and the spatial coordinates of the target identifier in the virtual reality space are (x, y, z),... near Therefore, the process of determining the spatial coordinates of the target identifier can actually be regarded as determining the depth coordinate value z. near The process.
[0079] For example, the depth coordinate value z is calculated using the following formula (2). near :
[0080]
[0081] Among them, z near Here, is the depth coordinate value, height is the interface height, and verticalFov is the vertical field of view angle.
[0082] Based on the interface height and vertical field of view of the virtual reality interface, this disclosure can convert two-dimensional planar coordinates into three-dimensional spatial coordinates, enabling the operation of an extended reality input controller to be simulated through mouse operation.
[0083] In some embodiments of this disclosure, the step of determining the first pose information based on spatial coordinates and the second pose information of the virtual camera includes: determining the target orientation vector of the extended reality input controller based on the spatial coordinates and the second pose information; wherein the second pose information includes the rotation state of the virtual camera; determining the rotation state of the extended reality input controller based on the target orientation vector, a preset initial orientation vector, and the rotation state of the virtual camera; and determining the first pose information based on the target orientation vector, the rotation state, and the spatial coordinates.
[0084] In this embodiment of the disclosure, the first pose information of the extended reality input controller specifically includes the spatial coordinates of the extended reality input controller in the virtual reality space, and the orientation of the extended reality input controller. The orientation of the extended reality input controller can be represented by a target orientation vector, the rotation state of the extended reality input controller, and its spatial coordinates.
[0085] For example, the spatial position of the extended reality input controller is set to be the same as the spatial position of the virtual camera, that is, their spatial coordinates are the same. The content displayed on the virtual reality interface is actually related to the pose of the virtual camera. Therefore, based on the second pose information of the camera, including the rotation state of the virtual camera, the rotation state of the extended reality input controller and the target orientation vector can be determined, thereby obtaining the first pose information.
[0086] This disclosure enables the conversion of 2-DOF data of a mouse into 6-DOF data of an extended reality input controller, thereby simulating the operation of an XR device through a PC emulator. Therefore, it eliminates the need to build an actual XR device to test the extended reality operating system, reducing testing costs.
[0087] In some embodiments of this disclosure, the display plane of the virtual reality interface coincides with a first plane in the virtual reality space, and the midline point of the target identifier is the intersection of the first ray and the first plane; or, the display content of the virtual reality interface includes a first virtual object, and the midline point of the target identifier is the first intersection of the first ray and the first virtual object; wherein, the extended reality input controller is located in the virtual reality space, the first ray is a ray with any point on the extended reality input controller as its endpoint, and the direction of the first ray is the same as the vector direction of the target orientation vector.
[0088] In this embodiment of the disclosure, when the extended reality operating system is running in a simulator, the virtual reality space generated by the extended reality operating system is projected onto the real plane of the display screen using a set virtual camera as the viewpoint. This displays the virtual reality interface. The ray of the extended reality input controller specifically refers to a ray whose endpoint is the recognition point of the extended reality input controller in the simulator. This ray actually reflects the orientation of the extended reality input controller. The function of the ray of the extended reality input controller is specifically to indicate the virtual object that the user wishes to interact with.
[0089] In some cases, the display plane of the virtual reality interface is defined as the first plane in the virtual reality space. The ray collision response point of the extended reality input controller is the intersection of the ray and the first plane, and this point is also the center point of the target marker in the virtual reality interface.
[0090] In other cases, although the virtual reality interface is a display plane, the virtual objects displayed within it are 3D objects. These can be considered as objects seen through the "window" of the virtual reality interface, possessing a certain depth. Since the extended reality input controller's ray interacts with the virtual objects, the collision response point of the extended reality input controller's ray, i.e., the center point of the target identifier, can be defined as the first collision point between the ray and the virtual object displayed in the virtual reality interface.
[0091] This disclosure enables the mapping of a ray from an extended reality input controller to a point in a virtual reality interface, allowing users to more intuitively simulate the operation of an XR device through a simulator.
[0092] In some embodiments of this disclosure, the first pose information includes the spatial coordinates of the extended reality input controller and the rotation state of the extended reality input controller; the second pose information includes the viewpoint coordinates of the virtual camera and the rotation state of the virtual camera.
[0093] In this embodiment of the disclosure, the first pose information indicates the position and orientation of the extended reality input controller in the virtual reality space. Exemplarily, the first pose information includes the spatial coordinates of the extended reality input controller in the virtual reality space, and the rotation state of the extended reality input controller. The rotation state of the extended reality input controller can be represented by a quaternion, a position vector, a rotation matrix, or Euler angles.
[0094] The second pose information indicates the position and orientation of the virtual camera in the virtual reality space. For example, the second pose information includes viewpoint coordinates, the initial value of which is the origin of the spatial coordinate system of the virtual reality space. The second pose information also includes the rotation state of the virtual camera. The rotation state of the virtual camera can be represented by quaternions, position vectors, rotation matrices, or Euler angles.
[0095] In some embodiments of this disclosure, the step of determining the target orientation vector of the extended reality input controller based on spatial coordinates and second pose information specifically includes: determining the target orientation vector using the following formula (3):
[0096] targetVec = (x, y, z) near )-cameraPos;(3)
[0097] Where targetVec is the target orientation vector, (x, y, z) near ) represents the spatial coordinates, and cameraPos represents the second pose information.
[0098] In this embodiment of the disclosure, the target orientation vector is determined by the above formula (3).
[0099] For example, the target orientation vector is a vector from the recognition point of the extended reality input controller to the center point of the target identifier, and the second pose information specifically includes the spatial coordinates of the virtual camera and the orientation vector of the virtual camera.
[0100] In some embodiments of this disclosure, the step of determining the rotation state of the extended reality input controller based on the target orientation vector, a preset initial orientation vector, and the rotation state of the virtual camera specifically includes: determining the rotation state using the following formula (4):
[0101] controllerRot=rotationTo(baseVec, targetVec)×hmdRot; (4)
[0102] Where controllerRot is the rotation state, baseVec is the initial orientation vector, targetVec is the target orientation vector, hmdRot is the rotation state of the virtual camera, and rotationTo is the rotation function.
[0103] In this embodiment of the disclosure, the target orientation vector is determined by the above formula (4).
[0104] For example, the rotation state is used to represent the angle by which the virtual camera rotates from the initial orientation vector to the target orientation vector, and the rotation state of the virtual camera can be determined according to the built-in parameters of the simulator.
[0105] In some embodiments of this disclosure, after the step of displaying the virtual reality interface, the method further includes: receiving a first input from a user to an electronic device; wherein the electronic device is an electronic device running a simulator; and adjusting second pose information in response to the first input to update the display content of the virtual reality interface; wherein the second pose information is the pose information of a virtual camera, and the virtual reality interface is an interface displayed with the virtual camera as the viewpoint.
[0106] In this embodiment of the disclosure, the virtual reality interface is an interface displayed using a virtual camera in the virtual reality space as the viewpoint. The pose of the virtual camera, i.e., the second pose information, determines the content displayed on the virtual reality interface. Users can change the second pose information of the virtual camera by making a first input to the electronic device, thereby changing the spatial coordinates, orientation, and rotation angle of the virtual camera in the virtual reality space, and achieving dynamic adjustment of the content displayed on the virtual reality interface.
[0107] For example, the aforementioned electronic devices include, but are not limited to, PC devices, mobile phones, and tablet computers. When the electronic device is a PC device, the first input can be input made by the user through a keyboard. For example, the user presses the arrow keys on the keyboard, or presses the W, A, S, D, etc. keys on the keyboard. When the electronic device is a mobile phone or tablet computer, the first input can be the user's swipe touch input on the touchscreen.
[0108] In some embodiments of this disclosure, after the step of displaying the virtual reality interface, the method further includes: receiving a second input from a user to an electronic device; wherein the electronic device is an electronic device running a simulator; and in response to the second input, performing a corresponding interactive operation with the object indicated by the target identifier as the target; wherein the interactive operation includes any or a combination of the following: a confirmation operation, a selection operation, and a shortcut function operation.
[0109] In this embodiment of the disclosure, the extended reality device's extended reality input controller typically has one or more buttons. These buttons are each associated with different functions, such as clicking "OK," selecting a target, or using a shortcut function. When simulating the extended reality device using a simulator, the buttons of the extended reality input controller can be associated with mouse buttons or keyboard buttons. This allows users to simulate the extended reality input controller using scene interaction devices such as a mouse or keyboard.
[0110] For example, a user moves the mouse to bring the target marker into contact with the first virtual object in the virtual reality interface. The user can then press the left mouse button to "pick up" the first virtual object, causing it to move synchronously with the target marker.
[0111] The interactive control method provided in this disclosure can be executed by an interactive control device. This disclosure uses an interactive control device executing the interactive control method as an example to illustrate the interactive control device provided in this disclosure.
[0112] In some embodiments of this disclosure, an interactive control device is provided, applied to an electronic device. Figure 10 Structural block diagrams of interactive control devices according to some embodiments of this disclosure are shown, such as... Figure 10 As shown, the interactive control device 1000 includes:
[0113] The running module 1002 is used to run an extended reality operating system through a simulator to generate a virtual reality space; wherein the simulator includes a simulated sensor module;
[0114] Display module 1004 is used to display a virtual reality interface; wherein, the virtual reality interface is used to display virtual objects in the virtual reality space, and the virtual reality interface includes target identifiers, the display position of which is associated with the user's interactive input to the virtual reality interface;
[0115] The determination module 1006 is used to determine the first pose information based on the display position of the target identifier;
[0116] The transmitting module 1008 is used to transmit the first pose information to the extended reality operating system through the analog sensor module, so that the extended reality operating system can determine the interactive input information of the extended reality input controller based on the first pose information.
[0117] This disclosure proposes a method to run an extended reality operating system via a simulator and, using a conventional 2-DOF input device, simulate the control interaction of a 6-DOF extended reality input controller in an XR device. The position of the controller ray in a virtual reality scene is simulated based on the position of target markers such as the mouse pointer on the display interface. An algorithm converts the coordinates of the mouse pointer into pose data for the extended reality input controller, enabling the simulation of XR device operation via a PC. Therefore, a test environment for XR systems or software can be built using a PC, reducing the testing cost of XR systems.
[0118] In some embodiments of this disclosure, the display module 1004 is further configured to display a virtual reality interface with a virtual camera in the virtual reality space as the viewpoint; the determination module 1006 is further configured to determine the planar coordinates of the target identifier in the virtual reality interface based on the display position of the target identifier; determine the spatial coordinates of the display position of the target identifier in the virtual reality space based on the planar coordinates and the interface parameters of the virtual reality interface; wherein, the interface parameters include interface size and field of view information; and determine the first pose information based on the spatial coordinates and the second pose information of the virtual camera.
[0119] By using the planar coordinates of the target icon in the virtual reality interface and employing a perspective projection algorithm, the spatial coordinates of the target icon in the virtual reality space are obtained. Combined with the second pose information of the virtual camera, the first pose information of the virtual extended reality input controller can be calculated. This enables the mapping of 2-DOF mouse operations to 6-DOF extended reality input controller operations, allowing the simulation of XR device operation via PCs and other devices.
[0120] In some embodiments of this disclosure, the second pose information further includes the position coordinates of the virtual camera in the virtual reality space; wherein the position coordinates of the extended reality input controller in the virtual reality space are the same as the position coordinates of the virtual camera in the virtual reality space.
[0121] This disclosure sets the spatial coordinates of the extended reality input controller to be the same as those of the virtual camera. By fixing the position of the extended reality input controller and simulating its rotation, the logic for simulating extended reality input controller operation via mouse input is defined. This ensures that the ray collision response point of the extended reality input controller coincides with the mouse pointer position. This not only guarantees that the ray collision response point of the extended reality input controller coincides with the mouse pointer, but also allows the ray of the extended reality input controller to be mapped to a point in the virtual reality interface, thus ensuring a good user experience when simulating XR device operation through the simulator.
[0122] In some embodiments of this disclosure, the interface size includes the interface width, and the field of view information includes the horizontal field of view; the determining module 1006 is further configured to determine the depth coordinate value corresponding to the display position of the target identifier in the virtual reality space based on the interface width and the horizontal field of view; and to determine the spatial coordinates based on the coordinate values of the planar coordinates and the depth coordinate values.
[0123] Based on the interface width and horizontal field of view of the virtual reality interface, this disclosure can convert two-dimensional planar coordinates into three-dimensional spatial coordinates, enabling the operation of an extended reality input controller to be simulated through mouse operation.
[0124] In some embodiments of this disclosure, the interface size includes the interface height, and the field of view information includes the vertical field of view; the determining module 1006 is further configured to determine the depth coordinate value corresponding to the display position of the target identifier in the virtual reality space based on the interface height and the vertical field of view; and to determine the spatial coordinates based on the coordinate values of the planar coordinates and the depth coordinate values.
[0125] Based on the interface height and vertical field of view of the virtual reality interface, this disclosure can convert two-dimensional planar coordinates into three-dimensional spatial coordinates, enabling the operation of an extended reality input controller to be simulated through mouse operation.
[0126] In some embodiments of this disclosure, the determining module 1006 is further configured to determine the target orientation vector of the extended reality input controller based on spatial coordinates and second pose information; wherein the second pose information includes the rotation state of the virtual camera; the rotation state of the extended reality input controller is determined based on the target orientation vector, a preset initial orientation vector and the rotation state of the virtual camera; and the first pose information is determined based on the target orientation vector, the rotation state and the spatial coordinates.
[0127] This disclosure enables the conversion of 2-DOF data of a mouse into 6-DOF data of an extended reality input controller, thereby simulating the operation of an XR device through a PC emulator. Therefore, it eliminates the need to build an actual XR device to test the extended reality operating system, reducing testing costs.
[0128] In some embodiments of this disclosure, the display plane of the virtual reality interface coincides with a first plane in the virtual reality space, and the midline point of the target identifier is the intersection of the first ray and the first plane; or, the display content of the virtual reality interface includes a first virtual object, and the midline point of the target identifier is the first intersection of the first ray and the first virtual object; wherein, the extended reality input controller is located in the virtual reality space, the first ray is a ray with any point on the extended reality input controller as its endpoint, and the direction of the first ray is the same as the vector direction of the target orientation vector.
[0129] This disclosure enables the mapping of a ray from an extended reality input controller to a point in a virtual reality interface, allowing users to more intuitively simulate the operation of an XR device through a simulator.
[0130] In some embodiments of this disclosure, the first pose information includes the spatial coordinates of the extended reality input controller and the rotation state of the extended reality input controller; the second pose information includes the viewpoint coordinates of the virtual camera and the rotation state of the virtual camera.
[0131] In this embodiment, the first pose information indicates the position and orientation of the extended reality input controller in the virtual reality space, and the second pose information indicates the position and rotation state of the virtual camera in the virtual reality space. The rotation state can be represented by a quaternion, a position vector, a rotation matrix, or Euler angles.
[0132] In some embodiments of this disclosure, the interactive control device further includes: a receiving module for receiving a first input from a user to an electronic device; wherein the electronic device is an electronic device running a simulator; and an adjustment module for adjusting second pose information in response to the first input to update the display content of the virtual reality interface; wherein the second pose information is the pose information of a virtual camera, and the virtual reality interface is an interface displayed with the virtual camera as the viewpoint.
[0133] In this embodiment of the disclosure, the virtual reality interface is an interface displayed using a virtual camera in the virtual reality space as the viewpoint. The pose of the virtual camera, i.e., the second pose information, determines the content displayed on the virtual reality interface. Users can change the second pose information of the virtual camera by making a first input to the electronic device, thereby changing the spatial coordinates, orientation, and rotation angle of the virtual camera in the virtual reality space, and achieving dynamic adjustment of the content displayed on the virtual reality interface.
[0134] In some embodiments of this disclosure, the interactive control device further includes: a receiving module for receiving a second input from a user to an electronic device; wherein the electronic device is an electronic device running a simulator; and an execution module for responding to the second input and performing a corresponding interactive operation with the object indicated by the target identifier as the target; wherein the interactive operation includes any or a combination of the following: a confirmation operation, a selection operation, and a shortcut function operation.
[0135] In this embodiment of the disclosure, the extended reality device's extended reality input controller typically has one or more buttons. These buttons are each associated with different functions, such as clicking "OK," selecting a target, or using a shortcut function. When simulating the extended reality device using a simulator, the buttons of the extended reality input controller can be associated with mouse buttons or keyboard buttons. This allows users to simulate the extended reality input controller using scene interaction devices such as a mouse or keyboard.
[0136] The interactive control device in this disclosure can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This disclosure does not impose specific limitations.
[0137] The interactive control device in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not impose specific limitations.
[0138] The interactive control device provided in this disclosure can implement the various processes implemented in the above method embodiments, and will not be described again here to avoid repetition.
[0139] In some embodiments of this disclosure, an interactive control device is provided. Figure 11 Structural block diagrams of interactive control devices according to some embodiments of this disclosure are shown, such as... Figure 11 As shown, the interactive control device 1100 includes a processor 1102, a memory 1104, and a program or instructions stored in the memory 1104 and executable on the processor 1102. When the program or instructions are executed by the processor 1102, they implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.
[0140] In some embodiments of this disclosure, a readable storage medium is provided on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements the various processes of the above method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0141] In some embodiments of this disclosure, an electronic device is provided, including the interactive control device as provided in any of the above embodiments; and / or the readable storage medium as provided in any of the above embodiments, thus achieving the same technical effect. To avoid repetition, it will not be described again here.
[0142] Figure 12 A structural block diagram of an electronic device that can be used to implement embodiments of the present disclosure is shown, such as... Figure 12 As shown, electronic device 1200 is in the form of a general-purpose electronic device. Components of electronic device 1200 may include, but are not limited to, one or more processors or processing units 1210, memory 1220, storage device 1230, one or more communication units 1240, one or more input devices 1250, and one or more output devices 1260. Processing unit 1210 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 1220. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 1200.
[0143] Electronic device 1200 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 1200, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 1220 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof). Storage device 1230 can be removable or non-removable media and may include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data (e.g., training data for training) and accessible within electronic device 1200.
[0144] Electronic device 1200 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 12 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 1220 may include computer program product 1225 having one or more program modules configured to perform various methods or actions of various implementations of this disclosure.
[0145] The communication unit 1240 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 1200 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 1200 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.
[0146] Input device 1250 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 1260 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 1200 can also communicate with one or more external devices (not shown) via communication unit 1240 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 1200, or with any device that enables electronic device 1200 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).
[0147] It should be noted that the electronic devices in this embodiment include the mobile electronic devices and non-mobile electronic devices described above.
[0148] This disclosure also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.
[0149] It should be understood that the chip mentioned in the embodiments of this disclosure may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.
[0150] This disclosure provides a computer program product stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, further details are omitted here.
[0151] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this disclosure is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0152] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solutions of this disclosure, in essence or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this disclosure.
[0153] The embodiments of this disclosure have been described above with reference to the accompanying drawings. However, this disclosure is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this disclosure without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this disclosure.
Claims
1. An interactive control method applied to electronic devices, characterized in that, The method includes: An extended reality operating system is run through a simulator to generate a virtual reality space; wherein the simulator includes a simulated sensor module; Displaying a virtual reality interface; wherein the virtual reality interface is used to display virtual objects in the virtual reality space, the virtual reality interface includes a target identifier, and the display position of the target identifier is associated with the user's interactive input to the virtual reality interface; Determine the first pose information based on the display position of the target identifier; The first pose information is sent to the extended reality operating system via the analog sensor module, so that the extended reality operating system can determine the interactive input information of the extended reality input controller based on the first pose information.
2. The interactive control method according to claim 1, characterized in that, The step of displaying the virtual reality interface includes: The virtual reality interface is displayed using a virtual camera in the virtual reality space as the viewpoint; The step of determining the first pose information based on the display position of the target identifier includes: Based on the display position of the target identifier, determine the planar coordinates of the target identifier in the virtual reality interface; Based on the planar coordinates and the interface parameters of the virtual reality interface, the spatial coordinates corresponding to the display position of the target identifier in the virtual reality space are determined; wherein, the interface parameters include interface size and field of view information; The first pose information is determined based on the spatial coordinates and the second pose information of the virtual camera.
3. The interactive control method according to claim 2, characterized in that, The second pose information also includes the position coordinates of the virtual camera in the virtual reality space; wherein the position coordinates of the extended reality input controller in the virtual reality space are the same as the position coordinates of the virtual camera in the virtual reality space.
4. The interactive control method according to claim 2, characterized in that, The interface dimensions include the interface width, and the field of view information includes the horizontal field of view. The step of determining the spatial coordinates of the display position of the target identifier in the virtual reality space based on the planar coordinates and the interface parameters of the virtual reality interface specifically includes: Based on the interface width and the horizontal field of view, determine the depth coordinate value of the display position of the target identifier in the virtual reality space. The spatial coordinates are determined based on the coordinate values of the planar coordinates and the depth coordinates.
5. The interactive control method according to claim 2, characterized in that, The interface dimensions include the interface height, and the field of view information includes the vertical field of view. The step of determining the spatial coordinates of the display position of the target identifier in the virtual reality space based on the planar coordinates and the interface parameters of the virtual reality interface specifically includes: Based on the interface height and the vertical field of view, determine the depth coordinate value of the display position of the target identifier in the virtual reality space; The spatial coordinates are determined based on the coordinate values of the planar coordinates and the depth coordinates.
6. The interactive control method according to claim 2, characterized in that, The step of determining the first pose information based on the spatial coordinates and the second pose information of the virtual camera includes: Based on the spatial coordinates and the second pose information, the target orientation vector of the extended reality input controller is determined; wherein, the second pose information includes the rotation state of the virtual camera; The rotation state of the extended reality input controller is determined based on the target orientation vector, the preset initial orientation vector, and the rotation state of the virtual camera. The first pose information is determined based on the target orientation vector, the rotation state of the extended reality input controller, and the spatial coordinates.
7. The interactive control method according to claim 6, characterized in that, The display plane of the virtual reality interface coincides with the first plane in the virtual reality space, and the midline point of the target identifier is the intersection of the first ray and the first plane; Alternatively, the content displayed on the virtual reality interface includes a first virtual object, and the midline point of the target identifier is the first intersection point of the first ray and the first virtual object; The extended reality input controller is located within the virtual reality space, and the first ray is a ray with any point on the extended reality input controller as its endpoint, and the direction of the first ray is the same as the vector direction of the target orientation vector.
8. The interactive control method according to any one of claims 1 to 7, characterized in that, Following the step of displaying the virtual reality interface, the method further includes: Receive first input from the user to the electronic device; wherein, the electronic device is an electronic device running the simulator; In response to the first input, the second pose information is adjusted to update the display content of the virtual reality interface; wherein, the second pose information is the pose information of the virtual camera, and the virtual reality interface is an interface displayed with the virtual camera as the viewpoint.
9. The interactive control method according to claim 8, characterized in that, The first pose information includes the spatial coordinates of the extended reality input controller and the rotation state of the extended reality input controller; The second pose information includes the viewpoint coordinates of the virtual camera and the rotation state of the virtual camera.
10. The interactive control method according to any one of claims 1 to 7, characterized in that, Following the step of displaying the virtual reality interface, the method further includes: Receive a second input from the user to the electronic device; wherein the electronic device is an electronic device running the simulator; In response to the second input, with the object indicated by the target identifier as the target, a corresponding interactive operation is performed; wherein the interactive operation includes any or a combination of the following: confirmation operation, selection operation, and shortcut function operation.
11. An interactive control device, applied to electronic devices, characterized in that, The interactive control device includes: A runtime module is used to run an extended reality operating system through a simulator to generate a virtual reality space; wherein the simulator includes a simulated sensor module; A display module is used to display a virtual reality interface; wherein the virtual reality interface is used to display virtual objects in the virtual reality space, the virtual reality interface includes a target identifier, and the display position of the target identifier is associated with the user's interactive input to the virtual reality interface; The determination module is used to determine the first pose information based on the display position of the target identifier; The sending module is used to send the first pose information to the extended reality operating system through the analog sensor module, so that the extended reality operating system can determine the interactive input information of the extended reality input controller based on the first pose information.
12. An interactive control device, characterized in that, The interactive control device includes: Memory, used to store programs or instructions; A processor for implementing the steps of the interactive control method as described in any one of claims 1 to 10 when executing the program or instructions.
13. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the interactive control method as described in any one of claims 1 to 10.
14. An electronic device, characterized in that, include: The interactive control device as described in claim 11 or 12; and / or The readable storage medium as described in claim 13.