Method and apparatus for improving interoperability accuracy

Abstract

The present application relates to computer interaction technology and mixed reality technology, and particularly relates to a method for improving interaction operation accuracy and a control device and a computer readable storage medium for implementing the method. The method for improving interaction operation accuracy according to an embodiment of the present application comprises the following steps: acquiring a first spatial orientation of a plane to which a virtual interaction interface presented by augmented reality glasses or virtual reality glasses worn by a user belongs and a second spatial orientation of an operation device used to perform an interaction operation on the virtual interaction interface; correcting the second spatial orientation based on motion data of a vehicle; determining an intersection of a straight line extending along the corrected second spatial orientation and the plane based on the first spatial orientation and the corrected second spatial orientation; and determining an operation object of the user on the virtual interaction interface based on the intersection.

Description

Technical Field

[0001] This application relates to computer interaction technology and mixed reality technology, and in particular to a method for improving the accuracy of interactive operations, as well as a control device and a computer-readable storage medium for implementing the method. Background Technology

[0002] Virtual reality (VR) and augmented reality (AR) are often collectively referred to as "mixed reality" (MR) technology. In MR technology, virtual information can interact, overlay, or merge with the real world, providing users with a rich interactive experience.

[0003] Augmented Reality (AR) technology cleverly integrates virtual information with the real world. It utilizes various technologies such as multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing to simulate and apply computer-generated text, images, 3D models, music, and videos to the real world. The two types of information complement each other, thus "enhancing" the real world. Virtual Reality (VR) technology uses real-life data and electronic signals generated by computer technology, combining these signals with various output devices to simulate the real world.

[0004] Applying AR and VR technologies to automotive scenarios presents numerous challenges. For example, when the vehicle experiences bumps or jolting while driving, the object the user wants to interact with within the virtual interface may deviate from its actual position, affecting the accuracy of the interaction. Summary of the Invention

[0005] One object of this application is to provide a method for improving the accuracy of interactive operations, as well as a control device and a computer-readable storage medium for implementing the method.

[0006] According to one aspect of this application, a method for improving the accuracy of interactive operations is provided, comprising the following steps:

[0007] Acquire a first spatial orientation of the plane to which the virtual interactive interface presented by the augmented reality glasses or virtual reality glasses worn by the user belongs, and a second spatial orientation of the operating device for performing interactive operations on the virtual interactive interface;

[0008] The second spatial orientation is corrected based on the vehicle's motion data;

[0009] The position within the virtual interactive interface pointed to by the operating device is determined based on the first spatial orientation and the corrected second spatial orientation; and

[0010] The user's target on the virtual interactive interface is determined based on the location.

[0011] Optionally, the above method further includes the following steps:

[0012] A cursor is displayed in the area corresponding to the object being operated on the virtual interactive interface.

[0013] Optionally, in the above method, the intersection of the straight line extending from the operating device along the modified second spatial orientation and the plane is determined as the position within the virtual interactive interface pointed to by the operating device.

[0014] Optionally, in the above method, the step of obtaining the first spatial orientation and the second spatial orientation includes:

[0015] Receive first motion data from the augmented reality glasses or the virtual reality glasses and receive second motion data from the operating device;

[0016] The first spatial orientation is determined based on the first motion data, and the second spatial orientation is determined based on the second motion data.

[0017] Optionally, in the above method, the step of obtaining the first spatial orientation and the second spatial orientation includes:

[0018] Receive first motion data from the augmented reality glasses or the virtual reality glasses;

[0019] The first spatial orientation is determined based on the first motion data;

[0020] The second spatial orientation, determined based on the second motion data, is received from the operating device.

[0021] Optionally, in the above method, the operating device includes one of the following: a mobile phone, a ring, and a smartwatch.

[0022] Optionally, in the above method, the step of correcting the second spatial orientation includes:

[0023] Acquire the motion data of the vehicle, including the vehicle's acceleration and angular velocity;

[0024] The change in the vehicle's third spatial orientation is determined based on the acceleration and angular velocity; and

[0025] The modified second spatial orientation is obtained by subtracting the change from the second spatial orientation.

[0026] Optionally, in the above method, the step of determining the operation object includes:

[0027] Generate a region containing intersections within the plane;

[0028] The element with the largest overlap area with the area within the virtual interactive interface is identified as the operation object.

[0029] Optionally, in the above method, the step of generating the region includes:

[0030] The size characteristics of the region are determined based on the amount of change in the third spatial orientation of the vehicle, wherein the size characteristics increase as the amount of change increases;

[0031] The region is determined based on the intersection point and the size characteristics.

[0032] Optionally, in the above method, the dimensional feature is one of the following: radius, length of rectangle, and width.

[0033] According to another aspect of this application, a control device for an electric vehicle is provided, comprising:

[0034] Sensors, including inertial measurement units;

[0035] The main control unit includes:

[0036] microprocessor;

[0037] Memory; and

[0038] The communication interface is configured to establish a communication connection with the augmented reality or virtual reality glasses worn by the user and the operating device.

[0039] The following operations are performed by running a computer program stored in the memory on the microprocessor:

[0040] Obtain a first spatial orientation of the plane to which the virtual interactive interface presented by the augmented reality glasses or the virtual reality glasses belongs, and a second spatial orientation of the operating device for performing interactive operations on the virtual interactive interface;

[0041] The second spatial orientation is corrected based on the vehicle's motion data sensed by the inertial measurement unit;

[0042] Optionally, in the above-described control device, the following operations are further performed by running a computer program stored in the memory on the microprocessor:

[0043] A cursor is displayed in the area corresponding to the object being operated on the virtual interactive interface.

[0044] Optionally, in the above control device, the intersection of the straight line extending from the operating device along the modified second spatial orientation and the plane is determined as the position within the virtual interactive interface pointed to by the operating device.

[0045] Optionally, in the above control device, the operation of acquiring the first spatial orientation and the second spatial orientation includes:

[0046] The device receives first motion data from the augmented reality glasses or the virtual reality glasses and second motion data from the operating device via the communication interface.

[0047] The first spatial orientation is determined based on the first motion data, and the second spatial orientation is determined based on the second motion data.

[0048] Optionally, in the above control device, the operation of acquiring the first spatial orientation and the second spatial orientation includes:

[0049] Receive first motion data from the augmented reality glasses or the virtual reality glasses;

[0050] The first spatial orientation is determined based on the first motion data;

[0051] The second spatial orientation, determined based on the second motion data, is received from the operating device.

[0052] Optionally, in the above control device, the operating device includes one of the following: a mobile phone, a ring, and a smartwatch.

[0053] Optionally, in the above-described control device, the operation of correcting the second spatial orientation includes:

[0054] Acquire the motion data of the vehicle, including the vehicle's acceleration and angular velocity;

[0055] The change in the vehicle's third spatial orientation is determined based on the acceleration and angular velocity; and

[0056] The modified second spatial orientation is obtained by subtracting the change from the second spatial orientation.

[0057] Optionally, in the above-described control device, the operation of determining the target object includes:

[0058] Generate a region containing intersections within the plane;

[0059] The element with the largest overlap area with the area within the virtual interactive interface is identified as the operation object.

[0060] Optionally, in the above-described control device, the operation of generating the region includes:

[0061] The size characteristics of the region are determined based on the change in the third spatial orientation of the vehicle, wherein the size characteristics increase as the change increases;

[0062] The region is determined based on the intersection point and the size characteristics.

[0063] Optionally, in the above-described control device, the dimensional feature is one of the following: radius, length of rectangle, and width.

[0064] According to another aspect of this application, a computer-readable storage medium is provided, wherein instructions are stored therein, characterized in that the method described above is implemented by means of instructions executed by a processor.

[0065] In some embodiments of this application, by correcting the spatial orientation of the operating device based on vehicle motion data, the problem of insufficient accuracy in locating the operating object within the virtual interactive interface can be overcome. In other embodiments of this application, the size characteristics of the region are determined based on the change in the vehicle's spatial orientation. This allows for a fine distinction of the degree of interference from various amplitudes of vehicle movement or bumps on the user's selection of the operating object, thereby significantly improving the accuracy of determining the operating object. Furthermore, by determining the operating object based on the overlapping area of ​​the region and the element, the "hit rate" of determining the operating object is improved, and the element most likely to be the one the user wants to operate is identified as the operating object. Attached Figure Description

[0066] The above and / or other aspects and advantages of this application will become clearer and more readily understood from the following description taken in conjunction with the accompanying drawings, in which the same or similar elements are denoted by the same reference numerals. The drawings include:

[0067] Figure 1 This is a flowchart of a method for improving the accuracy of interactive operations according to some embodiments of this application.

[0068] Figure 2 This is a flowchart of a method for correcting the spatial orientation of an operating device according to other embodiments of this application.

[0069] Figure 3 This is a flowchart of a method for determining the intersection point of a line and a plane according to some other embodiments of this application.

[0070] Figure 4 This is a flowchart of a method for determining an operation object according to some other embodiments of this application.

[0071] Figure 5 This is a structural block diagram of a control device used in an electric vehicle.

[0072] Figure 6 for Figure 5 The block diagram of the main control unit in the control device shown. Detailed Implementation

[0073] The present application will now be described more fully with reference to the accompanying drawings, which illustrate exemplary embodiments thereof. However, the present application may be implemented in various forms and should not be construed as being limited to the embodiments given herein. The foregoing embodiments are intended to make the disclosure herein complete and thorough, so as to more fully convey the scope of protection of the present application to those skilled in the art.

[0074] In this specification, terms such as “comprising” and “including” indicate that, in addition to having the units and steps that are directly and explicitly stated in the specification and claims, the technical solution of this application does not exclude the presence of other units and steps that are not directly or explicitly stated.

[0075] Unless otherwise specified, terms such as “first” and “second” do not indicate the order of units in terms of time, space, size, etc., but are merely used to distinguish between units.

[0076] In a typical mixed reality (MRT) interaction, AR or VR glasses worn by the user present a virtual interface. That is, the user can view a virtual scene simulating a real-world interface using AR or VR glasses. The user can then click or select elements within the virtual interface (e.g., including but not limited to buttons, message boxes, and menus) in various ways. In some embodiments of this application, the user uses an operating device to perform operations such as clicking, selecting, and confirming within the virtual interface they are viewing. Examples of operating devices include, but are not limited to, mobile phones, rings, and smartwatches. The presented virtual interface can be mapped to a specific plane in physical space. The user can point the operating device to a location within that specific plane (e.g., by controlling the orientation of the operating device) to locate the object they wish to operate on and perform click-like operations using components (e.g., buttons, keypads) on the operating device.

[0077] Figure 1 This is a flowchart of a method for improving the accuracy of interactive operation according to some embodiments of this application. Exemplarily, in the following description, a controller in an electric vehicle (e.g., a cockpit domain controller) is used as... Figure 1 The entity responsible for executing each step of the method shown.

[0078] Figure 1The illustrated process begins at step 110. In step 110, the controller communicates with the AR or VR glasses worn by the user to obtain the spatial orientation O1 (e.g., represented in the form of a three-dimensional vector) of the plane π to which the virtual interactive interface belongs (i.e., the plane corresponding to the interface in physical space). In some embodiments, the spatial orientation O1 is obtained by first acquiring motion data DATA_1 sensed by the AR or VR glasses using their built-in inertial measurement unit (IMU), and then determining the spatial orientation O1 based on the motion data DATA_1. The motion data DATA_1 includes, for example, the acceleration and angular velocity of the AR or VR glasses.

[0079] In step 110, the controller also communicates with the operating device to obtain the spatial orientation O2 of the operating device (e.g., represented in the form of a three-dimensional vector). Exemplarily, the spatial orientation of the operating device can be the direction of the normal vector of a specific region of the device (e.g., the front of a mobile phone, the surface of a smartwatch, or the middle region of a ring). In some embodiments, the spatial orientation O2 is obtained by first acquiring motion data DATA_2 sensed by the operating device using its built-in IMU, and then determining the spatial orientation O2 based on the motion data DATA_2. Similarly, the motion data DATA_2 includes, for example, the acceleration and angular velocity of the operating device. In other embodiments, the spatial orientation O2 can be received directly from the operating device; that is, the operation of determining the spatial orientation O2 from the motion data DATA_2 is performed locally on the operating device.

[0080] In one example, spatial orientation can be determined as follows: First, the motion data is filtered (e.g., using techniques such as Kalman filters or complementary filters) and calibrated to remove noise. Then, the object's attitude angle is estimated using angular velocity data measured by a gyroscope, which can be obtained by integrating the angular velocity data. Next, the acceleration data is compensated, i.e., the gravity component is removed from the accelerometer data. Finally, sensor fusion is performed; for example, the spatial orientation can be calculated using the previously estimated attitude angles and the gravity-compensated accelerometer data, combined with a sensor fusion algorithm (e.g., a Kalman filter or complementary filter).

[0081] After completing step 110, Figure 1 The process then proceeds to step 120. In this step, the controller corrects the spatial orientation O2 to eliminate changes in spatial orientation O2 caused by vehicle movement, thereby enabling the spatial orientation of the operating device to be more accurately aligned with the operating object that the user wishes to select within the virtual interactive interface. In some embodiments of this application, the vehicle's IMU is used to acquire vehicle motion data DATA_3, and the second spatial orientation O2 is corrected based on the motion data DATA_3.

[0082] Figure 2 This is a flowchart of a method for correcting the spatial orientation of an operating device according to other embodiments of this application.

[0083] like Figure 2 As shown, in step 210, the controller acquires motion data DATA_3, which includes the vehicle's acceleration and angular velocity, from the vehicle's IMU. The IMU typically contains an accelerometer for acquiring acceleration data and a gyroscope for acquiring angular velocity data.

[0084] Subsequently, Figure 2 The steps shown lead to step 220, in which the controller determines the spatial orientation O3(n) at the current moment based on the motion data DATA_3 (for example, it can be represented in the form of a three-dimensional vector).

[0085] Next, in step 230, the controller determines the change in the vehicle's spatial orientation, ΔO3(n). For example, it can be determined using the following formula:

[0086] ΔO3(n)=O3(n)-O3(n-1) (1)

[0087] Here, O3(n) is the spatial orientation of the vehicle at the nth time (the current time), and O3(n-1) is the spatial orientation of the vehicle at the (n-1)th time (the previous time).

[0088] After completing step 230, Figure 2 The process shown proceeds to step 240, in which the controller obtains the corrected spatial orientation O'2 of the operating device by subtracting the change obtained in step 230 from the spatial orientation O2 obtained in step 110, i.e.:

[0089] O′2=O2-ΔO3(n) (2)

[0090] exist Figure 2 In the embodiments shown, the spatial orientation of the operating equipment and the vehicle can be determined from motion data using the exemplary methods already given above.

[0091] In the Figure 2 In an improved embodiment shown, the controller may receive data from the vehicle's IMU or other devices (e.g., as an execution unit). Figure 1 Other vehicle controllers (excluding the controller shown in the steps) acquire the vehicle's current spatial orientation or the change in spatial orientation. In this case, Figure 2 Steps 210 and 220 or 210–230 in the process become unnecessary.

[0092] The operating device or its specific area can be considered as a point mass, and a straight line l can be drawn from this point along the modified spatial orientation of the operating device. Further, the intersection point P of this straight line l and the plane π to which the virtual interactive interface belongs can be determined as the position where the user intends to operate on the virtual interactive interface. Therefore, after completing step 120, the process is as follows: Figure 1 The process proceeds to step 130. In this step, the intersection point P of the straight line l extending along the spatial orientation O1 of the plane π and the plane π is determined based on the spatial orientation O1 of the plane π and the corrected spatial orientation O'2 of the operating device.

[0093] There are several ways to determine the intersection point P. Figure 3 This is a flowchart illustrating a method for determining the intersection point of a line and a plane according to other embodiments of this application. Figure 3 As shown, in step 310, the equation F1 of line l is first determined. For example, a point (x, y, z) on line l can be represented as (x, y, z) = (x0, y0, z0) + t(a, b, c), where (x0, y0, z0) is a point on line l with known coordinates (e.g., the operating device can be taken as a point with known coordinates), (a, b, c) is the direction vector of line l (which corresponds to the spatial orientation O'2 of the operating device), and t is a parameter. Subsequently... Figure 3 The process proceeds to step 320, where the equation F2 of plane π is determined. For example, it can be expressed as Ax + By + Cz + D = 0, where (A, B, C) are the normal vectors of plane π (corresponding to the spatial orientation O1 of plane π). Next, in step 330, the equation F1 of line l is substituted into the equation F2 of plane π to obtain an equation F3(t) containing only the parameter t. Equation F3(t) is then solved to determine the value of parameter t. Substituting the obtained value of parameter t into equation F1 yields the coordinates of the intersection point P.

[0094] See Figure 1 After completing step 130, the process is as follows: Figure 1 The process proceeds to step 140. In this step, the user's interaction object on the virtual interface is determined based on the intersection point P. Interaction objects such as buttons, menus, and message boxes are presented within the virtual interface as areas occupying a certain area. In some cases, the intersection point P may not fall within the area corresponding to the interaction object. In some embodiments, it can be done according to... Figure 4 The method shown is used to determine the object of operation.

[0095] Figure 4The process shown begins at step 410, in which a region A containing the intersection point P is generated within the plane π. Region A can have various shapes, such as circles and polygons. The intersection point P can be the geometric center of region A (e.g., the center of a circle, the centroid of a triangle, the center of a rectangle, etc.), but it can also be any other point within region A.

[0096] For example, in step 410, region A can be generated as follows: determine the size features f of the region to be generated (e.g., radius, length and width of a rectangle, etc.), and then generate region A in plane π based on the intersection point P and the size features f.

[0097] Optionally, the size characteristic f is not fixed; it can increase linearly or non-linearly with the change in the vehicle's spatial orientation O3, ΔO3(n). In one example, the change in size characteristic f is defined as follows:

[0098]

[0099] Here f i Let L be the size feature in the i-th dimension (e.g., the length or width of a rectangle, the radius of a circle, etc.). i Let x be the maximum size in the i-th dimension, x be the absolute value of the change in the vehicle's spatial orientation O3 ΔO3(n), and k be the maximum size in the i-th dimension. i and x i Parameters greater than 0 can be set according to application requirements.

[0100] The process then proceeds to step 420, where the overlap area between the elements within the virtual interactive interface and region A is calculated. In some cases, only one element within the virtual interactive interface may overlap with region A, while in others, multiple elements may overlap with region A. In the subsequent step 430, the element with the largest overlap area with region A is identified as the user's desired operation target.

[0101] By altering dimensional characteristics based on changes in the vehicle's spatial orientation, the degree of interference from various amplitudes of vehicle movement or bumps on the user's selection of the target object can be precisely differentiated, thereby significantly improving the accuracy of target object identification. Furthermore, compared to determining whether an intersection point falls within the target object's area, comparing overlapping areas improves the "hit rate" in identifying the target object and identifies the element most likely to be the user's desired target.

[0102] See also Figure 1After completing step 140, the process proceeds to step 150. In step 150, the controller displays a cursor on the virtual interactive interface within the area corresponding to the operation object determined in step 130. This cursor display provides visual feedback to the user performing the interactive operation regarding their selected operation object, which is beneficial for improving the user experience.

[0103] It should be noted that the bumps during vehicle movement can also cause AR and VR glasses to shake, so theoretically it seems necessary to use vehicle motion data to correct the spatial orientation of AR and VR glasses. However, after in-depth research, the inventors of this application found that simply correcting the spatial orientation of the operating device based on vehicle motion data is sufficient to overcome the problem of insufficient accuracy in locating the operating object within the virtual interactive interface; on the other hand, simply correcting the spatial orientation of AR and VR glasses does not seem to effectively improve positioning accuracy. In addition, after in-depth research, the inventors of this application also found that equipping AR glasses with a function to suppress or eliminate virtual image jitter can not only solve the dizziness when wearing AR glasses, but also help to further improve the positioning accuracy of the operating object. Chinese Patent 202211451808.2, entitled "Augmented Reality Glasses and a Method for Achieving Display Enhancement Using Augmented Reality Glasses," provides a detailed description of AR glasses with a function to suppress or eliminate virtual image jitter, which is incorporated herein by reference in its entirety.

[0104] Figure 5 This is a structural block diagram of a control device used in an electric vehicle. Figure 5 The control device 50 shown can be used to perform the above-mentioned functions. Figure 1-4 The steps of the method shown.

[0105] See Figure 5 The control device 50 includes a sensor 510 and a main control unit 520.

[0106] Sensor 510 is used to monitor the internal condition of the vehicle and the external environmental condition, including but not limited to temperature sensors, humidity sensors, inertial measurement units, and light sensors.

[0107] The main control unit 520 is the core of the control device 50. It communicates with the sensor 510, is responsible for processing the data collected by the sensor 510, and executes computer programs to implement various functions.

[0108] like Figure 6As shown, the main control unit includes a memory 521, a communication interface 522, and a processor core 523. The communication interface 522 is configured to establish communication connections with the sensor 510, the mixed reality device (e.g., AR glasses or VR glasses) 51, and the operating device 52. The above-mentioned functions are executed by running a computer program 524 stored in the memory 521 on the processor core 523. Figure 1-4 The method shown includes one or more steps.

[0109] According to another aspect of this application, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, can realize the above-described functions. Figure 1-4 The method includes one or more steps.

[0110] The term "computer-readable storage medium" as used in this application includes various types of computer storage media, and can be any available medium accessible to a general-purpose or special-purpose computer. For example, a computer-readable storage medium may include RAM, ROM, EPROM, E2PROM, registers, hard disk, removable disk, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device, or any other temporary or non-temporary medium capable of carrying or storing desired program code units having the form of instructions or data structures and accessible by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Combinations of the above should also be included within the scope of protection of computer-readable storage media. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0111] Those skilled in the art will understand that the various illustrative logic blocks, modules, circuits, and algorithm steps described herein can be implemented as electronic hardware, computer software, or a combination of both.

[0112] To demonstrate the interchangeability between hardware and software, various illustrative components, blocks, modules, circuits, and steps have been generally described above according to their functionality. Whether such functionality is implemented in hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art can implement the described functionality in varying ways for specific applications; however, such implementation decisions should not be construed as departing from the scope of this application.

[0113] Although only some specific embodiments of this application have been described, those skilled in the art will understand that this application can be implemented in many other forms without departing from its spirit and scope. Therefore, the examples and embodiments shown are to be considered illustrative rather than restrictive, and various modifications and substitutions may be made without departing from the spirit and scope of this application as defined by the appended claims.

[0114] The embodiments and examples presented herein are provided to best illustrate embodiments according to the present technology and its particular applications, thereby enabling those skilled in the art to implement and use the application. However, those skilled in the art will understand that the above description and examples are provided for ease of illustration and example only. The descriptions presented are not intended to cover all aspects of the application or to limit the application to the precise forms disclosed.

Claims

1. A method for improving the accuracy of interactive operations, comprising the following steps performed by a control device: Acquire a first spatial orientation of the plane to which the virtual interactive interface presented by the augmented reality glasses or virtual reality glasses worn by the user belongs, and a second spatial orientation of the operating device for performing interactive operations on the virtual interactive interface; The second spatial orientation is corrected based on the vehicle's motion data; The position within the virtual interactive interface pointed to by the operating device is determined based on the first spatial orientation and the corrected second spatial orientation. as well as The user's target on the virtual interactive interface is determined based on the location. Specifically, the intersection of the straight line extending from the operating device along the modified second spatial orientation and the plane is determined as the position within the virtual interactive interface pointed to by the operating device. The steps for obtaining the first spatial orientation and the second spatial orientation include: The first spatial orientation is determined based on first motion data received from the augmented reality glasses or the virtual reality glasses, and the second spatial orientation is determined based on second motion data received from the operating device, or the second spatial orientation is determined based on the second motion data received from the operating device. The step of correcting the second spatial orientation includes: Acquire the motion data of the vehicle, including the vehicle's acceleration and angular velocity; The change in the vehicle's third spatial orientation is determined based on the acceleration and angular velocity; and The modified second spatial orientation is obtained by subtracting the change amount from the second spatial orientation. The step of determining the operation object includes: Generate a region containing the intersection points within the plane; The element with the largest overlap area with the area within the virtual interactive interface is identified as the operation object.

2. The method of claim 1, further comprising the following steps: A cursor is displayed in the area corresponding to the object being operated on the virtual interactive interface.

3. The method as described in claim 1, wherein, The operating device includes one of the following: mobile phone, ring, and smartwatch.

4. The method of claim 1, wherein, The steps for generating the region include: The size characteristics of the region are determined based on the amount of change in the third spatial orientation of the vehicle, wherein the size characteristics increase as the amount of change increases; The region is determined based on the intersection point and the size characteristics.

5. The method of claim 4, wherein, The dimensional feature is one of the following: radius, length of rectangle, and width.

6. A control device for an electric vehicle, comprising: Sensors, including inertial measurement units; The main control unit includes: microprocessor; Memory; and The communication interface is configured to establish a communication connection with the augmented reality or virtual reality glasses worn by the user and the operating device. The following operations are performed by running a computer program stored in the memory on the microprocessor: Acquire a first spatial orientation of the plane to which the virtual interactive interface presented by the augmented reality glasses or virtual reality glasses worn by the user belongs, and a second spatial orientation of the operating device for performing interactive operations on the virtual interactive interface; The second spatial orientation is corrected based on the vehicle's motion data; The position within the virtual interactive interface pointed to by the operating device is determined based on the first spatial orientation and the corrected second spatial orientation; and The user's target on the virtual interactive interface is determined based on the location. Specifically, the intersection of the straight line extending from the operating device along the modified second spatial orientation and the plane is determined as the position within the virtual interactive interface pointed to by the operating device. The operations for obtaining the first spatial orientation and the second spatial orientation include: The first spatial orientation is determined based on first motion data received from the augmented reality glasses or the virtual reality glasses, and the second spatial orientation is determined based on second motion data received from the operating device, or the second spatial orientation is determined based on the second motion data received from the operating device. The operation of correcting the second spatial orientation includes: Acquire the motion data of the vehicle, including the vehicle's acceleration and angular velocity; The change in the vehicle's third spatial orientation is determined based on the acceleration and angular velocity; and The modified second spatial orientation is obtained by subtracting the change amount from the second spatial orientation. The operation of determining the operation object includes: Generate a region containing the intersection points within the plane; The element with the largest overlap area with the area within the virtual interactive interface is identified as the operation object.

7. The control device of claim 6 further performs the following operations by running a computer program stored in the memory on the microprocessor: A cursor is displayed in the area corresponding to the object being operated on the virtual interactive interface.

8. The control device as claimed in claim 6, wherein, The operating device includes one of the following: mobile phone, ring, and smartwatch.

9. The control device as claimed in claim 7, wherein, The operations for generating the region include: The size characteristics of the region are determined based on the amount of change in the third spatial orientation of the vehicle, wherein the size characteristics increase as the amount of change increases; The region is determined based on the intersection point and the size characteristics.

10. The control device as claimed in claim 9, wherein, The dimensional feature is one of the following: radius, length of rectangle, and width.

11. A computer-readable storage medium storing instructions, characterized in that, The method described in any one of claims 1-5 is implemented by having the processor execute the instructions.

Images