Control Method, Device, Storage Medium and Electronic Device

Abstract

The present invention provides a control method, device, storage medium, and electronic device. These methods determine initial relative posture data between a controller and a wearable extended reality device based on first relative posture data between a vehicle body and a controller and second relative posture data between the vehicle body and a wearable extended reality device. The initial relative posture data are iterated in real time to obtain real-time relative posture data of the controller relative to the wearable extended reality device. Upon receiving a user control instruction, the method generates real-time control data corresponding to the control instruction based on the real-time relative posture data. The method then sends the real-time control data to the controller to control the wearable extended reality device. Because the real-time relative posture data of the controller relative to the wearable extended reality device includes factors influencing changes in the vehicle body's posture, the controller can more accurately control the wearable extended reality device using the real-time control data, thereby enhancing the user experience.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a control method, device, storage medium, and electronic device. Background Art

[0002] XR (eXtended Reality) refers to all combined real and virtual environments and human-computer interactions generated by computer technology and wearable devices. Representative forms include augmented reality (AR), mixed reality (MR), virtual reality (VR), and cross-cutting scenarios. The level of virtuality ranges from AR with partial sensory input to fully immersive VR. A key aspect of XR is the expansion of human experience, particularly related to presence (exemplified by VR) and cognitive gain (exemplified by AR).

[0003] Taking AR as an example, augmented reality is a technology that integrates virtual information with the real world. It uses a variety of technical means such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, and sensing to simulate computer-generated virtual information such as text, images, three-dimensional models, music, and videos, and then applies them to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world.

[0004] Existing XR devices and controllers are typically used when the user is stationary. However, as technology and user needs evolve, the application scenarios of XR devices are also expanding. For example, XR devices can also be used to enhance the user experience in a moving vehicle. However, this also presents new technical challenges. For example, when the user is in a moving vehicle, the vehicle's driving state affects the user's ability to accurately control the XR device when using the controller, resulting in a poor user experience. Summary of the Invention

[0005] In view of the above problems, embodiments of the present application are proposed, which provide a control method, device, storage medium and electronic device to at least solve the above problems.

[0006] One or more embodiments of the present application provide a control method, including: determining initial relative posture data between a controller and a wearable extended reality device based on first relative posture data between a vehicle body and a controller, and second relative posture data between the vehicle body and a wearable extended reality device, wherein the controller is located in the vehicle body and is used to control the wearable extended reality device; iterating the initial relative posture data in real time to obtain real-time relative posture data of the controller relative to the wearable extended reality device; upon receiving a user's control instruction, generating real-time control data for the wearable extended reality device corresponding to the control instruction based on the real-time relative posture data; and sending the real-time control data to the controller to control the wearable extended reality device.

[0007] Optionally, the method further includes: obtaining a real-time relative angular velocity between the controller and the wearable extended reality device corresponding to the real-time relative posture data; when the real-time relative angular velocity meets a preset condition, filtering the real-time relative posture data to obtain filtered real-time relative posture data.

[0008] Optionally, based on the first relative posture data between the vehicle body and the controller, and the second relative posture data between the vehicle body and the wearable extended reality device, the initial relative posture data of the controller and the wearable extended reality device is determined, including: iterating the initial posture data of the vehicle body to obtain real-time posture data of the vehicle body; determining the first relative posture data based on the real-time posture data of the controller and the real-time posture data of the vehicle body, the real-time posture data of the controller being obtained by iterating the initial posture data of the controller by the controller; determining the initial relative posture data based on the first relative posture data and the second relative posture data.

[0009] Optionally, the method further includes: acquiring first acceleration data of the vehicle body; determining an initial rotation matrix of the vehicle body based on the first acceleration data; and transforming the initial rotation matrix of the vehicle body to obtain initial posture data of the vehicle body.

[0010] Optionally, determining the initial relative posture data between the controller and the wearable extended reality device includes: obtaining first frame data of first relative posture data; converting the first frame data of the first relative posture data to obtain second relative posture data; and determining the initial relative posture data based on the first frame data of the first relative posture data and the second relative posture data.

[0011] Optionally, the initial relative posture data is iterated in real time, including: determining the real-time relative angular velocity between the vehicle body and the controller based on the first relative posture data other than the first frame data; converting the real-time relative angular velocity to obtain the real-time relative angular velocity between the controller and the wearable extended reality device; and iterating the initial relative posture data based on the real-time relative angular velocity between the controller and the wearable extended reality device.

[0012] Optionally, when the real-time relative angular velocity meets a preset condition, the real-time relative posture data is filtered to obtain processed real-time relative posture data, including: determining the combined velocity of the real-time relative angular velocity and the size of the filtering window; when the combined velocity is less than a preset threshold, the real-time relative posture data is filtered based on the filtering window to obtain filtered real-time relative posture data.

[0013] According to another aspect of the present application, a control device is provided, including an initialization module, an iteration module, a data generation module and a sending module, wherein the initialization module is used to determine the initial relative posture data of the controller and the wearable extended reality device based on the first relative posture data between the vehicle body and the controller, and the second relative posture data between the vehicle body and the wearable extended reality device, wherein the controller is located in the vehicle body and is used to control the wearable extended reality device; the iteration module is used to iterate the initial relative posture data in real time to obtain the real-time relative posture data of the controller relative to the wearable extended reality device; the data generation module is used to generate real-time control data for the wearable extended reality device corresponding to the control instruction according to the real-time relative posture data when receiving the user's control instruction; the sending module is used to send the real-time control data to the controller to control the wearable extended reality device.

[0014] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above method.

[0015] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors; and a memory storing a program; wherein the program comprises instructions, which, when executed by the processor, cause the processor to execute the method of the above aspect.

[0016] The present application provides a control method, device, storage medium, and electronic device. The method determines initial relative posture data between a controller and a wearable extended reality device based on first relative posture data between a vehicle and a controller, and second relative posture data between the vehicle and a wearable extended reality device. The controller is located within the vehicle and is used to control the wearable extended reality device. The method iterates the initial relative posture data in real time to obtain real-time relative posture data of the controller relative to the wearable extended reality device. Upon receiving a user control command, the method generates real-time control data for the wearable extended reality device corresponding to the control command based on the real-time relative posture data. The method then sends the real-time control data to the controller to control the wearable extended reality device. The method fuses the vehicle posture data with the posture data of the controller and the wearable extended reality device, and then iterates in real time to obtain real-time relative posture data of the controller relative to the wearable extended reality device. This allows the real-time relative posture data of the controller relative to the wearable extended reality device to include factors influencing changes in the vehicle posture. Therefore, when the controller uses the real-time relative posture data and the real-time control data generated by the user's control command to control the wearable extended reality device, the control is more accurate, thereby enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A flow chart of a control method according to an exemplary embodiment of the present application;

[0019] Figure 2 A flow chart of a control method according to another exemplary embodiment of the present application;

[0020] Figure 3 A flow chart of a control method according to another exemplary embodiment of the present application;

[0021] Figure 4 A structural block diagram of a control device of an exemplary embodiment of the present application;

[0022] Figure 5 A structural block diagram of an electronic device according to an exemplary embodiment of the present application;

[0023] Description of reference numerals:

[0024] 400, control device; 401, initialization module; 402, iteration module; 403, data generation module; 404, sending module; 500, electronic device; 501, computing unit; 502, ROM; 503, RAM; 504, bus; 505, input / output interface; 506, input unit; 507, output unit; 508, storage unit; 509, communication unit. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0026] For ease of understanding, before describing the specific embodiments of the present application in detail, the application scenarios of the control method, device, storage medium and electronic device of the present application are first exemplified.

[0027] The control method of the present application can be used to control XR devices, that is, wearable extended reality devices in this application. XR refers to all real and virtual combined environments and human-computer interactions generated by computer technology and wearable devices. Representative forms include augmented reality, mixed reality and virtual reality, as well as cross-scenes between them, taking augmented reality technology as an example. Augmented reality technology is a technology that integrates virtual information with the real world. It uses a variety of technical means such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, and sensing to simulate computer-generated virtual information such as text, images, three-dimensional models, music, and videos, and then apply them to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world.

[0028] Existing XR devices and controllers are typically used when the user is stationary. However, when the user is in motion, such as in a moving vehicle, the vehicle's motion can affect the user's ability to accurately control the XR device, resulting in a poor user experience. In light of this, this application proposes a control method, device, storage medium, and electronic device that can address the various issues existing in the aforementioned prior art.

[0029] The wearable extended reality device in the embodiments of this application can be AR glasses, VR glasses, MR glasses, etc., and is not limited in the embodiments of this application. The controller in the embodiments of this application can be a wristband worn on the wrist or a ring-type controller worn on the finger. The examples in the following embodiments should not be construed as limiting the scope of protection claimed in this application.

[0030] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0031] Figure 1 This is a flow chart of a control method of an exemplary embodiment of the present application. As shown in the figure, this embodiment mainly includes the following steps:

[0032] S101: Determine initial relative posture data between the controller and the wearable extended reality device based on first relative posture data between the vehicle body and the controller and second relative posture data between the vehicle body and the wearable extended reality device.

[0033] Exemplarily, the controller is located in the vehicle body and is used to control the wearable extended reality device. The vehicle body can be the body of various vehicle equipment, such as a family car, a bus, etc. The controller can be various mobile control devices, such as a mobile phone, a handle, a ring, etc. The wearable extended reality device can be a head-mounted device, such as AR glasses, VR glasses, etc. The first relative posture data and the second relative posture data can be posture data in the form of a rotation matrix or a quaternion. The initial relative posture data of the controller and the wearable extended reality device calculated based on the first relative posture data and the second relative posture data can also be in the form of a rotation matrix or a quaternion.

[0034] It's important to note that the Euler angles used to represent attitude are the pitch, yaw, and roll angles, respectively. The pitch angle corresponds to the rotation angle around the x-axis, the yaw angle corresponds to the rotation angle around the y-axis, and the roll angle corresponds to the rotation angle around the z-axis. Each angle corresponds to a 3x3 rotation matrix. Quaternions are another form of representing attitude. Quaternions contain four components, and conversion between rotation matrices and quaternions is possible.

[0035] S102: Perform real-time iteration on the initial relative posture data to obtain real-time relative posture data of the controller relative to the wearable extended reality device.

[0036] Exemplarily, the initial relative posture data can be iterated in real time according to a preset frequency. For example, the preset frequency can be 100 Hz. The first relative posture data between the vehicle body and the controller and the second relative posture data between the vehicle body and the wearable extended reality device can be obtained according to the preset frequency. The initial relative posture data can be iterated using the first relative posture data and the second relative posture data to obtain real-time relative posture data of the controller relative to the wearable extended reality device.

[0037] It should be noted that the preset frequency can be obtained through experiments, and this embodiment does not limit this.

[0038] S103: upon receiving a manipulation instruction from the user, generating real-time manipulation data for the wearable extended reality device corresponding to the manipulation instruction according to the real-time relative posture data.

[0039] For example, a user can issue a control command by pressing a button on a controller, such as a button on a handle or ring, or by clicking a button on a controller, such as a button on a mobile phone display. Upon receiving the user's control command, real-time control data corresponding to the control command can be generated based on the real-time relative posture data, and the wearable extended reality device can be controlled based on the real-time control data.

[0040] S104: Sending real-time control data to the controller to control the wearable extended reality device.

[0041] This embodiment integrates the posture data of the vehicle body with the posture data of the controller and the wearable extended reality device, and then performs real-time iteration to obtain real-time relative posture data of the controller relative to the wearable extended reality device. As a result, the real-time relative posture data of the controller relative to the wearable extended reality device includes the factors affecting the change of the vehicle body posture. Therefore, when the controller uses the real-time relative posture data and the real-time control data generated by the user's control instructions for control, the control of the wearable extended reality device will be more accurate, thereby providing a better user experience.

[0042] Figure 2 This is a flow chart of a control method of another exemplary embodiment of the present application. This embodiment mainly shows the subsequent implementation of the above step S103. As shown in the figure, this embodiment mainly includes the following steps:

[0043] S201: Acquire a real-time relative angular velocity between the controller and the wearable extended reality device corresponding to the real-time relative posture data.

[0044] S202: When the real-time relative angular velocity meets a preset condition, filtering is performed on the real-time relative posture data to obtain filtered real-time relative posture data.

[0045] For example, the angular velocity of the controller and the vehicle can be obtained through sensors on the controller and the vehicle. Based on the angular velocity of the controller and the vehicle, the relative angular velocity of the vehicle and the controller can be determined. This relative angular velocity between the vehicle and the controller can then be converted using a rotation matrix to obtain the real-time relative angular velocity between the controller and the wearable extended reality device. Preset conditions can be obtained experimentally, such as setting an angular velocity threshold. If the real-time relative angular velocity meets the preset condition, the real-time relative posture data is filtered to obtain filtered real-time relative posture data.

[0046] This embodiment filters the real-time relative posture data to remove abnormal real-time relative posture data between the controller and the wearable extended reality device due to the influence of vehicle body shaking and hand shaking, so as to ensure that the real-time control data generated based on the real-time relative posture data is more accurate and reliable, making the user's control more precise.

[0047] In a specific implementation, when the real-time relative angular velocity meets a preset condition, the real-time relative posture data is filtered to obtain processed real-time relative posture data, including: determining the combined velocity of the real-time relative angular velocity and the size of the filtering window; if the combined velocity is less than a preset threshold, the real-time relative posture data is filtered based on the filtering window to obtain filtered real-time relative posture data.

[0048] For example, the resultant velocity v of the real-time relative angular velocity can be calculated using the following formula using the projection data x, y, and z of the real-time relative angular velocity on the x-axis, y-axis, and z-axis: According to a preset threshold, the real-time relative posture data of the controller and the wearable augmented reality device corresponding to the combined velocity v less than the preset threshold is determined. The size of the filter window can be set as needed, and this embodiment does not limit this. For example, the size of the filter window is N frames. According to the size of the filter window, the real-time relative posture data of the controller and the wearable augmented reality device determined above is The pose data for the N-1 frames after that is averaged and used as the filtered real-time relative pose data between the controller and the wearable XRD device. If the combined velocity v does not meet the above conditions, no filtering is required.

[0049] In this specific implementation, by setting a threshold, the real-time relative posture data affected by vehicle body shaking and hand shaking is screened out, and the real-time relative posture data is filtered to ensure that the obtained real-time relative posture data is more accurate and reliable.

[0050] Figure 3This is a flow chart of a control method of another exemplary embodiment of the present application. This embodiment mainly shows a specific implementation of the above step S101. As shown in the figure, this embodiment mainly includes the following steps:

[0051] S301 , iterating the initial posture data of the vehicle body to obtain the real-time posture data of the vehicle body.

[0052] S302 : Determine first relative posture data based on the real-time posture data of the controller and the real-time posture data of the vehicle body, wherein the real-time posture data of the controller is obtained by iterating the initial posture data of the controller by the controller.

[0053] S303: Determine initial relative posture data between the controller and the wearable extended reality device based on the first relative posture data and the second relative posture data.

[0054] For example, the initial posture data of the controller and the initial posture data of the vehicle body can be determined according to the sensors of the controller and the vehicle body, respectively, such as the inertial measurement unit (IMU). The IMU is a device for measuring the three-axis posture angle (or angular rate) and acceleration of an object. Usually, an IMU is equipped with a three-axis gyroscope and three-direction accelerometers to measure the angular velocity and acceleration of an object in three-dimensional space, and thereby calculate the posture of the object. Then, according to the real-time data from the sensors of the controller and the vehicle body, the initial posture data of the controller and the initial posture data of the vehicle body are iterated to obtain the real-time posture data Q of the controller. cute and the vehicle's real-time posture data Q car In the iterative process, the controller's real-time posture data Q can be processed by the Extended Kalman Filter (EKF) algorithm. cute and the vehicle's real-time posture data Q car Among them, the controller's real-time posture data Q cute Obtained from the controller, the controller's initial posture is iterated in the controller to obtain the controller's real-time posture data Q cute .

[0055] According to the controller's real-time posture data Q cute and the vehicle's real-time posture data Q car , the first relative posture data can be calculated using the following method

[0056] This specific implementation iterates the controller's initial posture data and the vehicle's initial posture data to obtain real-time controller posture data and vehicle posture data. This data is then used to determine first relative posture data between the vehicle and the controller, and thus initial relative posture data between the controller and the wearable extended reality device. This fusion of the vehicle's posture data and the controller's posture data ensures that the resulting initial relative posture data between the controller and the wearable extended reality device includes the impact of the vehicle's posture on the controller's posture, ensuring accurate and reliable control of the wearable extended reality device by the user using the controller in the vehicle.

[0057] In a specific implementation, initial relative posture data is determined based on the first relative posture data and the second relative posture data, and the initial relative posture data between the controller and the wearable extended reality device is determined, including: obtaining first frame data of the first relative posture data; converting the first frame data of the first relative posture data to obtain second relative posture data; and determining the initial relative posture data based on the first frame data and the second relative posture data of the first relative posture data.

[0058] Exemplarily, the first frame data of the first relative posture data is obtained The first relative posture data can be in the form of quaternion, using the first frame data Calculate the second relative posture data Second relative posture data It can be in quaternion form and can be calculated as: Then according to the first frame data of the first relative posture data and the second relative posture data Determine the initial relative posture data The calculation method can be:

[0059] This specific implementation method converts the first frame data of the first relative posture data between the vehicle body and the controller to obtain the second relative posture data between the vehicle body and the wearable extended reality device, and then determines the initial relative posture data between the controller and the wearable extended reality device based on the first frame data of the first relative posture data and the second relative posture data, thereby realizing the conversion from the relative posture between the vehicle body and the controller, the relative posture between the vehicle body and the wearable extended reality device, to the relative posture between the controller and the wearable extended reality device.

[0060] In one specific implementation, initial relative posture data between the controller and the wearable extended reality device is iterated in real time, including: determining the real-time relative angular velocity between the vehicle body and the controller based on first relative posture data other than the first frame data; converting the real-time relative angular velocity to obtain the real-time relative angular velocity between the controller and the wearable extended reality device; and iterating the initial relative posture data based on the real-time relative angular velocity between the controller and the wearable extended reality device.

[0061] Exemplarily, the real-time relative angular velocity of the vehicle body and the controller is determined based on the first relative posture data of the vehicle body and the controller except for the first frame data, for example, the first relative posture data of the vehicle body and the controller The real-time relative angular velocity between the vehicle and the controller is calculated at the i-th and i+1-th frames Here, i is an integer greater than or equal to 1, and the calculation method can be:

[0062] Real-time relative angular velocity between the vehicle and the controller Can be converted to real-time relative angular velocity between the controller and wearable augmented reality device The calculation method can be: in, The second relative posture data between the vehicle body and the wearable extended reality device The rotation matrix form of .

[0063] Based on the real-time relative angular velocity between the controller and the wearable extended reality device, the initial relative posture data is iterated. The calculation method can be:

[0064] This specific implementation method determines the real-time relative angular velocity between the vehicle body and the controller based on the first relative posture data other than the first frame data, converts the real-time relative angular velocity to obtain the real-time relative angular velocity between the controller and the wearable extended reality device, and based on the real-time relative angular velocity between the controller and the wearable extended reality device, it iterates the initial relative posture data to obtain the real-time relative posture data of the controller relative to the wearable extended reality device.

[0065] In a specific implementation, first acceleration data of the vehicle body is obtained; based on the first acceleration data, an initial rotation matrix of the vehicle body is determined; and the initial rotation matrix of the vehicle body is transformed to obtain initial posture data of the vehicle body.

[0066] For example, the first acceleration data of the vehicle body can be obtained from the sensor of the vehicle body, and the initial rotation matrix R of the vehicle body can be set. car Among them, the initial rotation matrix R of the vehicle body car is a 3*3 rotation matrix that satisfies R carT *[0;0;1] is equal to the normalized value of the vehicle's acceleration, and the rotation matrix R car The (2, 1) term is equal to 0, that is, the Y axis of the vehicle coordinate system is facing the front of the vehicle. Similarly, in the controller, the second acceleration data of the controller can be obtained according to the controller's sensor to set the controller's initial rotation matrix R cute is a 3*3 rotation matrix that satisfies R cute T *[0;0;1] is equal to the normalized value of the controller's acceleration, and the rotation matrix R cute The (2, 1) term is equal to 0, meaning the Y axis of the controller coordinate system points directly forward of the controller. By transforming the vehicle's initial rotation matrix, we can obtain the vehicle's initial posture data. Similarly, by transforming the controller's initial rotation matrix within the controller, we can obtain the controller's initial posture data. For example, the controller's initial rotation matrix and the vehicle's initial rotation matrix can be converted into quaternion form. Quaternion form avoids the universal lock problem associated with rotation matrix form, and quaternion form expresses all postures using only four components, making it more compact than matrix form.

[0067] This specific implementation method determines the initial rotation matrix of the vehicle body through the first acceleration data of the vehicle body, and then obtains the initial posture data of the vehicle body, so as to subsequently fuse the posture of the controller and the posture of the vehicle body.

[0068] Figure 4 This is a structural block diagram of a control device according to an exemplary embodiment of the present application.

[0069] The control device 400 of this embodiment may be installed in a control device, wherein the control device may be adapted to perform a task of controlling a wearable augmented reality device.

[0070] As shown in the figure, the control device 400 of this embodiment mainly includes: an initialization module 401 , an iteration module 402 , a data generation module 403 and a sending module 404 .

[0071] Among them, the initialization module 401 is used to determine the initial relative posture data between the controller and the wearable extended reality device based on the first relative posture data between the vehicle body and the controller, and the second relative posture data between the vehicle body and the wearable extended reality device. The controller is located in the vehicle body and is used to control the wearable extended reality device; the iteration module 402 is used to iterate the initial relative posture data in real time to obtain the real-time relative posture data of the controller relative to the wearable extended reality device; the data generation module 403 is used to generate real-time control data for the wearable extended reality device corresponding to the control instruction according to the real-time relative posture data when receiving the user's control instruction; the sending module 404 is used to send the real-time control data to the controller to control the wearable extended reality device.

[0072] This embodiment fuses the vehicle's posture data with the posture data of the controller and the wearable XR device, and then performs real-time iteration to obtain real-time relative posture data of the controller relative to the wearable XR device. This data, therefore, incorporates factors influencing changes in the vehicle's posture. Consequently, when the controller uses this real-time relative posture data and real-time control data generated by the user's control instructions to control the wearable XR device, it can more accurately control the wearable XR device, thereby enhancing the user experience. Furthermore, by calculating and transmitting real-time control data to the controller through the control device of this embodiment, the controller can reduce the controller's data processing load, making the controller design more compact and reducing the computing power requirements of the controller.

[0073] In addition, the control device 400 of the embodiment of the present application can also be used to implement other steps in the aforementioned control method embodiments, and has the beneficial effects of the corresponding method step embodiments, which will not be repeated here.

[0074] The exemplary embodiments of the present application further provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the methods of the various embodiments of the present application.

[0075] An exemplary embodiment of the present application further provides an electronic device, comprising: one or more processors; and a memory storing a program; wherein the program comprises instructions, which, when executed by the processor, cause the processor to execute the method of each embodiment of the present application.

[0076] refer to Figure 5, a block diagram of an electronic device 500 that can serve as a server or client of the present application will now be described, which is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0077] like Figure 5 As shown, the electronic device 500 may further include a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the device 500 may also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0078] Multiple components within electronic device 500 are connected to I / O interface 505, including an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. Input unit 506 can be any type of device capable of inputting information into electronic device 500. Input unit 506 can receive input numeric or character information and generate key input signals related to user settings and / or function control of the electronic device. Output unit 507 can be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 508 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 509 allows electronic device 500 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0079] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the methods of the aforementioned embodiments may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. In some embodiments, the computing unit 501 may be configured to perform the aforementioned method in any other appropriate manner (e.g., by means of firmware).

[0080] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0081] It should be noted that in the description of this application, the terms "first" and "second" are used only to facilitate the description of different components or names, and should not be understood to indicate or imply a sequential relationship, relative importance, or implicitly specify the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include at least one of such features.

[0082] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit this application.

[0083] It should be noted that although the specific embodiments of the present application are described in detail in conjunction with the accompanying drawings, this should not be construed as limiting the scope of protection of the present application. Within the scope described by the claims, various modifications and variations that can be made by those skilled in the art without creative effort still fall within the scope of protection of the present application.

[0084] The examples of the embodiments of the present application are intended to concisely illustrate the technical features of the embodiments of the present application so that those skilled in the art can intuitively understand the technical features of the embodiments of the present application, and are not intended to serve as improper limitations on the embodiments of the present application.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A control method, characterized in that: include: determining initial relative posture data between the controller and the wearable extended reality device based on first relative posture data between the vehicle body and the controller and second relative posture data between the vehicle body and the wearable extended reality device, wherein the controller is located in the vehicle body and is used to control the wearable extended reality device; Iterating the initial relative posture data in real time to obtain real-time relative posture data of the controller relative to the wearable extended reality device; Upon receiving a manipulation instruction from the user, generating real-time manipulation data for the wearable extended reality device corresponding to the manipulation instruction according to the real-time relative posture data; The real-time control data is sent to the controller to control the wearable extended reality device.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a real-time relative angular velocity between the controller and the wearable extended reality device corresponding to the real-time relative posture data; When the real-time relative angular velocity meets a preset condition, the real-time relative posture data is filtered to obtain the filtered real-time relative posture data.

3. The method according to claim 1, characterized in that The determining, based on first relative posture data between the vehicle body and the controller and second relative posture data between the vehicle body and the wearable extended reality device, initial relative posture data between the controller and the wearable extended reality device includes: Iterating the initial posture data of the vehicle body to obtain real-time posture data of the vehicle body; determining the first relative posture data based on real-time posture data of the controller and real-time posture data of the vehicle body, wherein the real-time posture data of the controller is obtained by iterating the initial posture data of the controller by the controller; The initial relative posture data is determined based on the first relative posture data and the second relative posture data.

4. The method according to claim 3, characterized in that The method further comprises: Acquiring first acceleration data of the vehicle body; determining an initial rotation matrix of the vehicle body based on the first acceleration data; The initial rotation matrix of the vehicle body is transformed to obtain initial posture data of the vehicle body.

5. The method according to claim 3, characterized in that The determining of initial relative posture data between the controller and the wearable extended reality device includes: Acquire a first frame of data of the first relative posture data; converting the first frame data of the first relative posture data to obtain the second relative posture data; The initial relative posture data is determined based on the first frame data of the first relative posture data and the second relative posture data.

6. The method according to claim 5, characterized in that The real-time iteration of the initial relative posture data includes: determining a real-time relative angular velocity between the controller and the vehicle body based on the first relative posture data other than the first frame data; Converting the real-time relative angular velocity to obtain a real-time relative angular velocity between the controller and the wearable extended reality device; The initial relative posture data is iterated based on the real-time relative angular velocity of the controller and the wearable extended reality device.

7. The method according to claim 2, characterized in that When the real-time relative angular velocity satisfies a preset condition, filtering the real-time relative posture data to obtain the processed real-time relative posture data comprises: Determining the resultant velocity of the real-time relative angular velocity and the size of the filtering window; When the combined velocity is less than a preset threshold, filtering is performed on the real-time relative posture data based on the filtering window to obtain the filtered real-time relative posture data.

8. A control device, characterized in that: include: an initialization module, configured to determine initial relative posture data between the controller and the wearable extended reality device based on first relative posture data between the vehicle body and the controller and second relative posture data between the vehicle body and the wearable extended reality device, wherein the controller is located in the vehicle body and is configured to control the wearable extended reality device; an iteration module, configured to iterate the initial relative posture data in real time to obtain real-time relative posture data of the controller relative to the wearable extended reality device; a data generation module configured to generate, upon receiving a user's control instruction, real-time control data for the wearable extended reality device corresponding to the control instruction based on the real-time relative posture data; A sending module is used to send the real-time control data to the controller to control the wearable extended reality device.

9. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: one or more processors; as well as Memory for storing programs; The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

Images