Equipment control method and device, equipment and storage medium

Abstract

The invention provides an equipment control method and device, equipment and a storage medium, and belongs to the field of equipment control, and the method comprises the steps: obtaining the relative position information of head-mounted display equipment and control equipment, and receiving target IMU data transmitted by the control equipment; according to the relative position information and the target IMU data, determining a target moving track of the control device relative to the head-mounted display device when the control device performs device control; and according to the target movement track, generating a control instruction for controlling the head-mounted display device, and executing the control instruction to control the head-mounted display device. According to the method and the device, the control instruction of the head-mounted display equipment can be accurately generated based on the target moving track, and the head-mounted display equipment can be accurately controlled by executing the control instruction, so that the control efficiency and accuracy of the head-mounted display equipment are greatly improved.

Description

Technical Field

[0001] The present application relates to the field of device control technology, and in particular to a device control method, apparatus, device and storage medium. Background Art

[0002] At present, the mainstream interactive control solutions for augmented reality (AR) devices and virtual reality (VR) devices include touch screen, keyboard / mouse, remote control and gesture control, etc. These methods can solve the problems of most interactive scenarios, but there are still many defects in the current interaction. For example, when users touch with their hands, there are a series of problems such as high false touch rate and low recognition rate.

[0003] Therefore, how to improve the accuracy and efficiency of head-mounted display device control is an urgent problem to be solved. Summary of the invention

[0004] The main purpose of this application is to provide a device control method, apparatus, device and storage medium, aiming to improve the efficiency and accuracy of head-mounted display device control.

[0005] In a first aspect, the present application provides a device control method, which is applied to a head mounted display device, wherein the head mounted display device is communicatively connected to a control device, and the device control method comprises the following steps:

[0006] Acquire the relative position information of the head mounted display device and the control device, and receive the target IMU data transmitted by the control device;

[0007] Determining, according to the relative position information and the target IMU data, a target movement trajectory of the control device relative to the head mounted display device when performing device control;

[0008] A control instruction for controlling the head mounted display device is generated according to the target movement trajectory, and the control instruction is executed to control the head mounted display device.

[0009] In some embodiments, determining, based on the relative position information and the target IMU data, a target movement trajectory of the control device relative to the head mounted display device when performing device control includes:

[0010] Determine, according to the relative position information and the target IMU data, the position coordinates of the control device relative to the head mounted display device at each time point of device control, to obtain a plurality of relative position coordinates;

[0011] According to the multiple relative position coordinates, a target movement trajectory of the control device relative to the head mounted display device when performing device control is determined.

[0012] In some embodiments, determining, based on the multiple relative position coordinates, a target movement trajectory of the control device relative to the head mounted display device when performing device control includes:

[0013] The plurality of relative position coordinates are connected in sequence based on the order of each time point to obtain the target movement trajectory.

[0014] In some embodiments, the obtaining the relative position information of the head mounted display device and the control device includes:

[0015] Acquire first IMU data of the head mounted display device and second IMU data of the control device;

[0016] Relative position information of the head mounted display device and the control device is determined according to the first IMU data and the second IMU data.

[0017] In some embodiments, determining the relative position information of the head mounted display device and the control device according to the first IMU data and the second IMU data includes:

[0018] Get the Earth's magnetic field direction and gravitational acceleration;

[0019] Determine first coordinate information and first angle information of the head mounted display device relative to the world coordinates according to the magnetic field direction, the gravitational acceleration and the first IMU data;

[0020] Determine second coordinate information and second angle information of the control device relative to the world coordinates according to the magnetic field direction, the gravitational acceleration and the second IMU data;

[0021] Relative position information of the head mounted display device and the control device is determined according to the first coordinate information, the first angle information, the second coordinate information, and the second angle information.

[0022] In some embodiments, the relative position information includes relative distance information and relative posture information; and determining the relative position information of the head mounted display device and the control device according to the first coordinate information, the first angle information, the second coordinate information, and the second angle information includes:

[0023] Calculate the relative distance between the head mounted display device and the control device according to the first coordinate information and the second coordinate information to obtain relative distance information;

[0024] The relative posture of the head mounted display device and the control device is calculated according to the first angle information and the second angle information to obtain relative posture information.

[0025] In some embodiments, generating a control instruction for controlling the head mounted display device according to the target movement trajectory includes:

[0026] Obtaining a mapping relationship table between preset movement trajectories and control instructions;

[0027] The control instruction matching the target movement trajectory is queried from the mapping relationship table.

[0028] In a second aspect, the present application further provides a device control apparatus, the device control apparatus comprising an acquisition module, a determination module and a control module, wherein:

[0029] The acquisition module is used to acquire relative position information between the head mounted display device and the control device, and receive target IMU data transmitted by the control device;

[0030] The determination module is used to determine a target movement trajectory of the control device relative to the head mounted display device when performing device control according to the relative position information and the target IMU data;

[0031] The control module is used to generate a control instruction for controlling the head mounted display device according to the target movement trajectory, and execute the control instruction to control the head mounted display device.

[0032] In a third aspect, the present application also provides a head-mounted display device, wherein the controller includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the device control method as described above are implemented.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the device control method as described above are implemented.

[0034] The present application provides a device control method, apparatus, device and storage medium. The present application obtains the relative position information of a head-mounted display device and a control device, and receives the target IMU data transmitted by the control device; then, according to the relative position information and the target IMU data, determines the target movement trajectory of the control device relative to the head-mounted display device when performing device control; then, according to the target movement trajectory, generates a control instruction to control the head-mounted display device, and executes the control instruction to control the head-mounted display device. In the present application, through the target IMU data of the control device, the relative position information of the head-mounted display device and the control device, the target movement trajectory of the control device relative to the head-mounted display device when the control action for controlling the head-mounted display device is triggered can be accurately known, and then based on the target movement trajectory, the control instruction of the head-mounted display device can be accurately generated, and the control instruction can be executed to accurately control the head-mounted display device, which greatly improves the efficiency and accuracy of the control of the head-mounted display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 A schematic diagram of a flow chart of a device control method provided in an embodiment of the present application;

[0037] Figure 2 for Figure 1 A schematic diagram of a sub-step flow chart of a device control method in FIG.

[0038] Figure 3 A schematic block diagram of a device control apparatus provided in an embodiment of the present application;

[0039] Figure 4 for Figure 3 A schematic block diagram of a submodule of a device control device in;

[0040] Figure 5 A schematic block diagram of the structure of a head mounted display device provided in an embodiment of the present application.

[0041] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0044] At present, the mainstream interactive control solutions for augmented reality (AR) devices and virtual reality (VR) devices include touch screen, keyboard / mouse, remote control and gesture control, etc. These methods can solve the problems of most interactive scenarios, but there are still many defects in the current interaction. For example, when users touch with their hands, there are a series of problems such as high false touch rate and low recognition rate.

[0045] To solve the above problems, the present application provides a device control method, apparatus, device and storage medium. The device control method includes obtaining relative position information of the head-mounted display device and the control device, and receiving target IMU data transmitted by the control device; determining the target movement trajectory of the control device relative to the head-mounted display device when performing device control according to the relative position information and the target IMU data; generating a control instruction to control the head-mounted display device according to the target movement trajectory, and executing the control instruction to control the head-mounted display device.

[0046] Among them, the device control method can be applied to head-mounted display devices, which include augmented reality (AR) glasses, AR helmets, mixed reality (MR) glasses, MR helmets, virtual reality (VR) glasses and other devices.

[0047] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0048] Please refer to Figure 1 , Figure 1 A schematic flow chart of a device control method provided in an embodiment of the present application.

[0049] like Figure 1 As shown, the device control method includes steps S101 to S103.

[0050] Step S101: Obtain relative position information between the head mounted display device and the control device, and receive target IMU data transmitted by the control device.

[0051] Among them, the present application is applied to a head-mounted display device, the head-mounted display device establishes a communication connection with a control device, and the control device is provided with an inertial measurement unit (IMU) sensor, that is, the control device can collect IMU data through the IMU sensor, and the control device can be selected according to actual conditions, and the embodiment of the present invention does not make specific restrictions on this. For example, the control device can be a stylus and a touch ring, etc. Among them, the communication method between the head-mounted display device and the control device can be set according to actual conditions, and the embodiment of the present invention does not make specific restrictions on this. For example, the communication method can be Bluetooth communication and ANT+, etc.

[0052] In some embodiments, Figure 2 As shown, step S101 includes sub-steps S1011 to S1012.

[0053] Sub-step S1011, obtaining first IMU data of the head mounted display device and second IMU data of the control device.

[0054] The head mounted display device is provided with an IMU sensor, which is used to collect IMU data of the head mounted display device. The control device is provided with a device control key, which is used to receive a device control instruction triggered by the wearer so that the control device can collect IMU data. The device control key includes but is not limited to a physical key and a virtual key. By receiving the device control instruction triggered by the wearer, the position of the head mounted display device and the control device can be accurately calibrated.

[0055] In some embodiments, when the control device obtains the device control instruction, it turns on the IMU sensor, collects the current IMU data of the control device through the IMU sensor, obtains the second IMU data, and sends the second IMU data to the head-mounted display device, so that the head-mounted display device receives the second IMU data sent by the control device. By receiving the second IMU data sent by the control device, the accuracy of calculating the relative position information of the head-mounted display device and the control device can be improved.

[0056] In some embodiments, the IMU data is collected by an IMU sensor provided in the head mounted display device to obtain the first IMU data. The IMU sensor of the head mounted display device can accurately obtain the first IMU data, which can improve the accuracy of calculating the relative position information between the head mounted display device and the control device.

[0057] Sub-step S1012: determining relative position information of the head mounted display device and the control device according to the first IMU data and the second IMU data.

[0058] The relative position information includes relative distance information and relative posture information. The relative distance information is the position information of the relative distance between the head-mounted display device and the control device, and the relative posture information is the pitch angle and / or yaw angle information of the relative posture between the head-mounted display device and the control device.

[0059] In some embodiments, the magnetic field direction and gravitational acceleration of the earth are obtained; the first coordinate information and first angle information of the head-mounted display device relative to the world coordinates are determined according to the magnetic field direction, gravitational acceleration and the first IMU data; the second coordinate information and second angle information of the control device relative to the world coordinates are determined according to the magnetic field direction, gravitational acceleration and the second IMU data; the relative position information of the head-mounted display device and the control device is determined according to the first coordinate information, the first angle information, the second coordinate information and the second angle information. By using the magnetic field direction and gravitational acceleration of the earth as a medium, the relative position information between the head-mounted display device and the control device can be accurately calculated, which greatly improves the efficiency and accuracy of page control.

[0060] In some embodiments, the method of determining the first coordinate information and the first angle information of the head mounted display device relative to the world coordinates according to the magnetic field direction, the gravitational acceleration and the first IMU data can be: according to the magnetic field direction, the gravitational acceleration, the acceleration data and the angular velocity data in the first IMU data, the positioning calculation is performed relative to the world coordinate system to obtain the first coordinate information and the first angle information of the head mounted display device relative to the world coordinates. By calculating the magnetic field direction, the gravitational acceleration and the first IMU data, the first coordinate information and the first angle information of the head mounted display device relative to the world coordinates can be accurately obtained.

[0061] In some embodiments, according to the magnetic field direction, the gravity acceleration and the second IMU data, the second coordinate information and the second angle information of the control device relative to the world coordinates can be determined by: performing positioning calculation relative to the world coordinate system according to the magnetic field direction, gravity acceleration, acceleration data and angular velocity data in the second IMU data, and obtaining the second coordinate information and the second angle information of the control device relative to the world coordinates. By calculating the magnetic field direction, gravity acceleration and the second IMU data, the second coordinate information and the second angle information of the control device relative to the world coordinates can be accurately obtained.

[0062] In some embodiments, the relative distance between the head-mounted display device and the control device is calculated based on the first coordinate information and the second coordinate information to obtain relative distance information; the relative posture of the head-mounted display device and the control device is calculated based on the first angle information and the second angle information to obtain relative posture information.

[0063] In some embodiments, the relative distance between the head-mounted display device and the control device is calculated based on the first coordinate information and the second coordinate information. The relative distance information can be obtained by calculating the relative position distance between the head-mounted display device and the control device in the spatial coordinates based on the first coordinate information and the second coordinate information to obtain the relative distance information.

[0064] In some embodiments, the relative posture of the head-mounted display device and the control device is calculated based on the first angle information and the second angle information. The relative posture information can be obtained by calculating the relative pitch angle and / or yaw angle of the head-mounted display device and the control device in the spatial coordinates based on the first angle information and the second angle information to obtain the relative posture information.

[0065] In some embodiments, target IMU data transmitted by a control device is received, where the target IMU data is IMU data collected in real time by the control device when a device control instruction is triggered.

[0066] Step S102: determining a target movement trajectory of the control device relative to the head mounted display device when performing device control according to the relative position information and the target IMU data.

[0067] The target movement trajectory is the movement trajectory of the control device relative to the head mounted display device in space after the control device triggers the device control instruction.

[0068] In some embodiments, the position coordinates of the control device relative to the head mounted display device at each time point during device control are determined based on the relative position information and the target IMU data, and a plurality of relative position coordinates are obtained; based on the plurality of relative position coordinates, the target movement trajectory of the control device relative to the head mounted display device during device control is determined. Based on the relative position information and the target IMU data, the target movement trajectory of the control device relative to the head mounted display device during device control can be accurately determined.

[0069] In some embodiments, the position coordinates of the control device relative to the head-mounted display device at each moment in the virtual page control are determined based on the relative position information and the target IMU data, and a method for obtaining multiple relative position coordinates can be: according to the target IMU data, the IMU data of the control device at each moment is determined, and based on the IMU data and relative position information at each moment, the position coordinates of the control device relative to the head-mounted display device at each moment are determined with the head-mounted display device as the center, to obtain multiple relative position coordinates.

[0070] In some embodiments, the method of determining the target movement trajectory of the control device relative to the head mounted display device when controlling the device according to the multiple relative position coordinates may be: connecting the multiple relative position coordinates in sequence based on the order of each time point to obtain the target movement trajectory. By connecting the respective relative position coordinates, the target movement trajectory of the control device relative to the head mounted display device when controlling the device can be accurately obtained.

[0071] Step S103: generating a control instruction for controlling the head mounted display device according to the target movement trajectory, and executing the control instruction to control the head mounted display device.

[0072] Among them, the movement trajectory includes single-click movement trajectory, double-click movement trajectory, upward movement trajectory, left movement trajectory, right movement trajectory, downward movement trajectory, circle movement trajectory and cross movement trajectory, etc.

[0073] In some embodiments, a mapping relationship table between preset movement trajectories and control instructions is obtained; and the control instructions matching the target movement trajectory are queried from the mapping relationship table. The mapping relationship table is established in advance based on the movement trajectory and the control instructions. The mapping relationship table can be established according to actual conditions. The embodiment of the present invention does not specifically limit this. The control instructions can be accurately queried through the mapping relationship table.

[0074] It should be noted that the control instruction of the single-click movement trajectory is to open the application, the control instruction of the double-click movement trajectory is to exit the application, the control instruction of the upward movement trajectory is to switch applications or switch to the previous page or slide the interface up or switch to the previous song, the control instruction of the downward movement trajectory is to switch applications or switch to the next page or slide the interface down or switch to the next song, the control instruction of the right-swipe movement trajectory is to switch applications or switch to the previous page or slide the interface up, the control instruction of the left-swipe movement trajectory is to switch applications or switch to the next page or slide the interface down, and the control instruction of the circle movement trajectory is to start the preset application.

[0075] In some embodiments, the target movement trajectory of the control device relative to the head-mounted display device when controlling the device is determined based on the relative position information and the target IMU data, and a mapping relationship table between the preset movement trajectory and the control instruction is obtained; the control instruction matching the target movement trajectory is queried from the mapping relationship table, and the control instruction is executed to control the head-mounted display device. By executing the control instruction, the head-mounted display device can be accurately controlled, which greatly improves the user experience.

[0076] Exemplarily, relative position information of a head-mounted display device and the control device is obtained, and target IMU data transmitted by the control device is received; based on the relative position information and the target IMU data, it is determined that the target movement trajectory of the control device relative to the head-mounted display device when controlling the device is a circular movement trajectory; a mapping relationship table between preset movement trajectories and control instructions is obtained; the control instruction matching the circular movement trajectory is queried from the mapping relationship table as an instruction to enter a WeChat application program, and the instruction to enter the WeChat application program is executed to control the head-mounted display device to start WeChat, and the WeChat interface page is displayed on the virtual display interface of the head-mounted display device.

[0077] The device control method provided in the above embodiment obtains the relative position information of the head-mounted display device and the control device, and receives the target IMU data transmitted by the control device; then, based on the relative position information and the target IMU data, determines the target movement trajectory of the control device relative to the head-mounted display device when performing device control; then, based on the target movement trajectory, generates a control instruction to control the head-mounted display device, and executes the control instruction to control the head-mounted display device. In the present application, through the target IMU data of the control device, the relative position information of the head-mounted display device and the control device, it is possible to accurately know the target movement trajectory of the control device relative to the head-mounted display device when the control action for controlling the head-mounted display device is triggered, and then based on the target movement trajectory, it is possible to accurately generate the control instruction of the head-mounted display device, and execute the control instruction to accurately control the head-mounted display device, which greatly improves the efficiency and accuracy of the control of the head-mounted display device.

[0078] See also Figure 3 , Figure 3 A schematic block diagram of a device control apparatus provided in an embodiment of the present application.

[0079] like Figure 3 As shown, the device control apparatus 200 includes an acquisition module 210, a determination module 220 and a control module 230, wherein:

[0080] The acquisition module 210 is used to acquire relative position information between the head mounted display device and the control device, and receive target IMU data transmitted by the control device;

[0081] The determination module 220 is used to determine the target movement trajectory of the control device relative to the head mounted display device when performing device control according to the relative position information and the target IMU data;

[0082] The control module 230 is used to generate a control instruction for controlling the head mounted display device according to the target movement trajectory, and execute the control instruction to control the head mounted display device.

[0083] In some embodiments, the determining module 220 is further configured to:

[0084] Determine, according to the relative position information and the target IMU data, the position coordinates of the control device relative to the head mounted display device at each time point during device control, to obtain a plurality of relative position coordinates;

[0085] According to the multiple relative position coordinates, a target movement trajectory of the control device relative to the head mounted display device when performing device control is determined.

[0086] In some embodiments, the determining module 220 is further configured to:

[0087] The plurality of relative position coordinates are connected in sequence based on the order of each time point to obtain the target movement trajectory.

[0088] In some embodiments, the acquisition module 210 includes an acquisition submodule 211 and a determination submodule 212, wherein:

[0089] The acquisition submodule 211 is used to acquire the first IMU data of the head mounted display device and the second IMU data of the control device;

[0090] The determination submodule 212 is used to determine the relative position information of the head mounted display device and the control device according to the first IMU data and the second IMU data.

[0091] In some embodiments, the determining submodule 212 is further configured to:

[0092] Get the Earth's magnetic field direction and gravitational acceleration;

[0093] Determine first coordinate information and first angle information of the head mounted display device relative to the world coordinates according to the magnetic field direction, the gravitational acceleration and the first IMU data;

[0094] Determine second coordinate information and second angle information of the control device relative to the world coordinates according to the magnetic field direction, the gravitational acceleration and the second IMU data;

[0095] Relative position information of the head mounted display device and the control device is determined according to the first coordinate information, the first angle information, the second coordinate information, and the second angle information.

[0096] In some embodiments, the determining submodule 212 is further configured to:

[0097] Calculate the relative distance between the head mounted display device and the control device according to the first coordinate information and the second coordinate information to obtain relative distance information;

[0098] The relative posture of the head mounted display device and the control device is calculated according to the first angle information and the second angle information to obtain relative posture information.

[0099] In some embodiments, the page control module 230 is further used to:

[0100] Obtaining a mapping relationship table between preset movement trajectories and control instructions;

[0101] The control instruction matching the target movement trajectory is queried from the mapping relationship table.

[0102] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-mentioned equipment control device can refer to the corresponding process in the aforementioned equipment control method embodiment, and will not be repeated here.

[0103] See also Figure 5 , Figure 5 A schematic block diagram of the structure of a head mounted display device provided in an embodiment of the present application.

[0104] like Figure 5 As shown, the head mounted display device 300 includes an IMU sensor 302, a processor 303 and a memory 304 connected via a system bus 301, wherein the memory 304 may include a storage medium and an internal memory.

[0105] The storage medium may store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor 303 may execute any device control method.

[0106] The processor 303 is used to provide computing and control capabilities to support the operation of the entire head-mounted display device.

[0107] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any device control method.

[0108] Those skilled in the art will understand that Figure 5 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the controller to which the scheme of the present application is applied. The specific controller may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] It should be understood that the processor 303 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0110] In one embodiment, the processor 303 is used to run a computer program stored in the memory to implement the following steps:

[0111] Acquire the relative position information of the head mounted display device and the control device, and receive the target IMU data transmitted by the control device;

[0112] Determining, according to the relative position information and the target IMU data, a target movement trajectory of the control device relative to the head mounted display device when performing device control;

[0113] A control instruction for controlling the head mounted display device is generated according to the target movement trajectory, and the control instruction is executed to control the head mounted display device.

[0114] In one embodiment, when the processor 303 determines the target movement trajectory of the control device relative to the head mounted display device when performing device control according to the relative position information and the target IMU data, it is used to implement:

[0115] Determine, according to the relative position information and the target IMU data, the position coordinates of the control device relative to the head mounted display device at each time point during device control, to obtain a plurality of relative position coordinates;

[0116] According to the multiple relative position coordinates, a target movement trajectory of the control device relative to the head mounted display device when performing device control is determined.

[0117] In one embodiment, when the processor 303 implements the step of determining, according to the multiple relative position coordinates, a target movement trajectory of the control device relative to the head mounted display device when performing device control, the processor 303 is configured to implement:

[0118] The plurality of relative position coordinates are connected in sequence based on the order of each time point to obtain the target movement trajectory.

[0119] In one embodiment, when the processor 303 implements the acquiring of the relative position information between the head mounted display device and the control device, it is configured to implement:

[0120] Acquire first IMU data of the head mounted display device and second IMU data of the control device;

[0121] Relative position information of the head mounted display device and the control device is determined according to the first IMU data and the second IMU data.

[0122] In one embodiment, when the processor 303 determines the relative position information of the head mounted display device and the control device according to the first IMU data and the second IMU data, it is used to implement:

[0123] Get the Earth's magnetic field direction and gravitational acceleration;

[0124] Determine first coordinate information and first angle information of the head mounted display device relative to the world coordinates according to the magnetic field direction, the gravitational acceleration and the first IMU data;

[0125] Determine second coordinate information and second angle information of the control device relative to the world coordinates according to the magnetic field direction, the gravitational acceleration and the second IMU data;

[0126] Relative position information of the head mounted display device and the control device is determined according to the first coordinate information, the first angle information, the second coordinate information, and the second angle information.

[0127] In one embodiment, the relative position information includes relative distance information and relative posture information; when the processor 303 determines the relative position information of the head mounted display device and the control device according to the first coordinate information, the first angle information, the second coordinate information, and the second angle information, it is used to implement:

[0128] Calculate the relative distance between the head mounted display device and the control device according to the first coordinate information and the second coordinate information to obtain relative distance information;

[0129] The relative posture of the head mounted display device and the control device is calculated according to the first angle information and the second angle information to obtain relative posture information.

[0130] In one embodiment, when the processor 303 generates a control instruction for controlling the head mounted display device according to the target movement trajectory, it is configured to implement:

[0131] Obtaining a mapping relationship table between preset movement trajectories and control instructions;

[0132] The control instruction matching the target movement trajectory is queried from the mapping relationship table.

[0133] It should be noted that technicians in the relevant field can clearly understand that for the convenience and brevity of description, the specific working process of the head-mounted display device described above can refer to the corresponding process in the aforementioned device control method embodiment, and will not be repeated here.

[0134] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the device control method of the present application.

[0135] The computer-readable storage medium may be an internal storage unit of the head-mounted display device described in the foregoing embodiment, such as a hard disk or memory of the head-mounted display device. The computer-readable storage medium may be non-volatile or volatile. The computer-readable storage medium may also be an external storage device of the head-mounted display device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the head-mounted display device.

[0136] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0137] It should also be understood that the term "and / or" used in the present specification refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0138] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application.

Claims

1. A device control method, characterized in that: Applied to a head mounted display device, the head mounted display device is communicatively connected with a control device, and the method comprises: Acquire the relative position information of the head mounted display device and the control device, and receive the target IMU data transmitted by the control device; Determining, according to the relative position information and the target IMU data, a target movement trajectory of the control device relative to the head mounted display device when performing device control; A control instruction for controlling the head mounted display device is generated according to the target movement trajectory, and the control instruction is executed to control the head mounted display device.

2. The device control method according to claim 1, characterized in that: Determining, according to the relative position information and the target IMU data, a target movement trajectory of the control device relative to the head mounted display device when performing device control, includes: Determine, according to the relative position information and the target IMU data, the position coordinates of the control device relative to the head mounted display device at each time point during device control, to obtain a plurality of relative position coordinates; According to the multiple relative position coordinates, a target movement trajectory of the control device relative to the head mounted display device when performing device control is determined.

3. The device control method according to claim 2, characterized in that: The step of determining, based on the plurality of relative position coordinates, a target movement trajectory of the control device relative to the head mounted display device when performing device control comprises: The plurality of relative position coordinates are connected in sequence based on the order of each time point to obtain the target movement trajectory.

4. The device control method according to claim 1, characterized in that: The obtaining the relative position information of the head mounted display device and the control device includes: Acquire first IMU data of the head mounted display device and second IMU data of the control device; Relative position information of the head mounted display device and the control device is determined according to the first IMU data and the second IMU data.

5. The device control method according to claim 4, characterized in that: The determining, according to the first IMU data and the second IMU data, relative position information of the head mounted display device and the control device includes: Get the Earth's magnetic field direction and gravitational acceleration; Determine first coordinate information and first angle information of the head mounted display device relative to the world coordinates according to the magnetic field direction, the gravitational acceleration and the first IMU data; Determine second coordinate information and second angle information of the control device relative to the world coordinates according to the magnetic field direction, the gravitational acceleration and the second IMU data; Relative position information of the head mounted display device and the control device is determined according to the first coordinate information, the first angle information, the second coordinate information, and the second angle information.

6. The device control method according to claim 5, characterized in that: The relative position information includes relative distance information and relative posture information; and determining the relative position information of the head mounted display device and the control device according to the first coordinate information, the first angle information, the second coordinate information, and the second angle information includes: Calculate the relative distance between the head mounted display device and the control device according to the first coordinate information and the second coordinate information to obtain relative distance information; The relative posture of the head mounted display device and the control device is calculated according to the first angle information and the second angle information to obtain relative posture information.

7. The device control method according to claim 1, characterized in that: The step of generating a control instruction for controlling the head mounted display device according to the target movement trajectory includes: Obtaining a mapping relationship table between preset movement trajectories and control instructions; The control instruction matching the target movement trajectory is searched from the mapping relationship table.

8. A device control device, characterized in that: The device control apparatus comprises an acquisition module, a determination module and a control module, wherein: The acquisition module is used to acquire relative position information between the head mounted display device and the control device, and receive target IMU data transmitted by the control device; The determination module is used to determine a target movement trajectory of the control device relative to the head mounted display device when performing device control according to the relative position information and the target IMU data; The control module is used to generate a control instruction for controlling the head mounted display device according to the target movement trajectory, and execute the control instruction to control the head mounted display device.

9. A head mounted display device, characterized in that: The head mounted display device comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the device control method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the device control method according to any one of claims 1 to 7 are implemented.

Images