Manipulation device, head-mounted display device, control method, and storage medium

Abstract

The application discloses a control method of a control device, a control method of a head-mounted display device, the control device, the head-mounted display device and a storage medium. The control device is used for pairing operation with the head-mounted display device. The control method of the control device comprises the following steps: collecting a first image; determining first position information of the head-mounted display device according to a first image recognition result of a first identification element in the first image, the first identification element being arranged on the head-mounted display device; and sending the first position information to the head-mounted display device, so that the head-mounted display device displays an image according to the first position information and second position information, the second position information being position information of the control device. The application aims to improve the operation precision of the control device and the head-mounted display device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic equipment, and in particular to a control method of a control device, a control method of a head-mounted display device, the control device, the head-mounted display device, and a storage medium. BACKGROUND

[0002] The head-mounted display device (for example, a virtual reality device, an augmented reality device, etc.) needs to be used with a control device such as a handle in many application scenarios. During use, the control device displays a corresponding feature image on the head-mounted display device, and the feature image moves with the movement of the control device. The user performs operations based on the image displayed on the head-mounted display device.

[0003] However, the head-mounted display device generally detects the position information of the control device based on a detection device arranged on the body. During display of the image, the head-mounted display device separately displays the feature image based on the positioning information of the control device. This easily leads to inaccurate display of the position of the feature image on the head-mounted display device, and affects the operation precision of the control device and the head-mounted display device. SUMMARY

[0004] The main purpose of the present application is to provide a control method of a control device, a control method of a head-mounted display device, the control device, the head-mounted display device, and a storage medium, which aims to improve the operation precision of the control device and the head-mounted display device.

[0005] To achieve the above purpose, the present application provides a control method of a control device, which is used for paired operation with a head-mounted display device. The control method of the control device comprises the following steps:

[0006] acquiring a first image;

[0007] determining first position information of the head-mounted display device according to a first image recognition result of a first identification element in the first image, the first identification element being arranged on the head-mounted display device;

[0008] sending the first position information to the head-mounted display device, so that the head-mounted display device displays an image according to the first position information and second position information, the second position information being position information of the control device.

[0009] Optionally, the step of determining the first position information of the control device according to the first image recognition result of the first identification element in the first image comprises:

[0010] identifying a first target image of the first identification element in the first image;

[0011] determining first image position information of the first target image in the first image, the first image recognition result comprising the first image position information;

[0012] determining, according to the first image position information, position information of the head-mounted display device relative to the control device as the first position information.

[0013] Optionally, the step of collecting the first image comprises:

[0014] receiving the second position information sent by the head-mounted display device;

[0015] collecting the first image according to the second position information;

[0016] wherein the second position information is position information of the control device relative to the head-mounted display device.

[0017] Optionally, the step of collecting the first image according to the second position information comprises:

[0018] determining a rotation angle of a first camera on the control device according to the second position information;

[0019] controlling the first camera to rotate according to the rotation angle so as to make the first camera face the head-mounted display device;

[0020] acquiring an image collected by the first camera as the first image.

[0021] In addition, in order to achieve the above-mentioned purpose, the present application further provides a control method of a head-mounted display device, the head-mounted display device being used for paired operation with the control device according to any one of the above-mentioned embodiments, the control method of the head-mounted display device comprising:

[0022] collecting a second image;

[0023] determining second position information of the control device according to a second image recognition result of a second identification member in the second image, the second identification member being arranged on the control device;

[0024] receiving the first position information sent by the control device;

[0025] displaying an image according to the first position information and the second position information.

[0026] Optionally, the step of determining the second position information of the control device according to the second image recognition result of the second identification member in the second image comprises:

[0027] identifying a second target image of the second identification member in the second image;

[0028] determining second image position information of the second target image in the second image, the second image recognition result comprising the second image position information;

[0029] determining, according to the second image position information, position information of the control device relative to the head-mounted display device as the second position information.

[0030] Optionally, before the step of receiving the first position information sent by the control device, the method further comprises:

[0031] sending the second position information to the control device, so that the control device detects the first position information based on the second position information.

[0032] In addition, in order to achieve the above object, the present application further provides a control device, which comprises:

[0033] a first camera;

[0034] a second identification member;

[0035] a first control device, the camera being connected to the first control device, the first control device comprising a memory, a processor, and a control program of the control device stored in the memory and executable on the processor, the control program of the control device being executed by the processor to implement the steps of the control method of the control device according to any one of the above.

[0036] Optionally, the second identification member is an infrared light emitting member.

[0037] Optionally, the number of the second identification members is more than one.

[0038] In addition, in order to achieve the above object, the present application further provides a head-mounted display device, which comprises:

[0039] a second camera;

[0040] a first identification member;

[0041] a second control device, the camera being connected to the second control device, the second control device comprising a memory, a processor, and a control program of the head-mounted display device stored in the memory and executable on the processor, the control program of the head-mounted display device being executed by the processor to implement the steps of the control method of the head-mounted display device according to any one of the above.

[0042] Optionally, the first identification member is an infrared light emitting member.

[0043] Optionally, the number of the first identification members is more than one.

[0044] Optionally, the second camera is more than one in number.

[0045] In addition, in order to achieve the above object, the application further provides a storage medium, wherein the storage medium stores a control program of an air conditioner, and the control program of the air conditioner implements the steps of the control method of the air conditioner according to any one of the above when executed by a processor.

[0046] The control method of the operation device provided by the application determines the first position information of the head-mounted display device based on the image recognition of the first identification member on the head-mounted display device in the collected first image, and sends the first position information to the head-mounted display device, so that the head-mounted display device can perform image display in combination with the first position information and the second position information instead of performing display based on the second position information of the operation device alone, which can effectively increase the position data used by the head-mounted display device in the image display process, ensure the image display accuracy, and enable the user to achieve accurate operation based on the image, thereby effectively improving the operation precision of the operation device and the head-mounted display device. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The figure is a schematic diagram of the hardware structure related to the positioning operation of the operation device and the head-mounted display device according to an embodiment of the application.

[0048] Figure 2 The figure is a flowchart of the control method of the operation device according to an embodiment of the application.

[0049] Figure 3 The figure is a flowchart of the control method of the operation device according to another embodiment of the application.

[0050] Figure 4 The figure is a flowchart of the control method of the operation device according to still another embodiment of the application.

[0051] Figure 5 The figure is a flowchart of the control method of the head-mounted display device according to an embodiment of the application.

[0052] Figure 6 The figure is a flowchart of the control method of the head-mounted display device according to another embodiment of the application.

[0053] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are merely intended to explain the application, and are not intended to limit the application.

[0055] The embodiment of the present application provides a control device 1 which is used in pairing operation with a head-mounted display device 2. In the embodiment, the control device 1 is a handle. In other embodiments, the control device 1 can also be other user-operated remote sensing devices, such as a smart watch, a game gun, etc.

[0056] With reference to Figure 1 The control device 1 comprises a body, a first camera 11 arranged on the body, a second identification element 12 arranged on the body, and a first control device 13. The first camera 11 is connected with the first control device 13, and the first control device 13 can acquire image data collected by the first camera 11.

[0057] In the embodiment, the first camera 11 is installed on the body through a rotating mechanism, the rotating mechanism is connected with the first control device 13, and the first control device 13 can control the rotating mechanism to operate so as to rotate the first camera 11 relative to the body. Specifically, the first camera 11 can rotate 360 degrees relative to the body under the driving of the rotating mechanism. In other embodiments, the first camera 11 can also be fixed on the body. Alternatively, in other embodiments, the number of the first camera 11 can be multiple, and the multiple cameras are fixed on the body and have different shooting directions.

[0058] In the embodiment, the first camera 11 is a camera for collecting infrared images.

[0059] In the embodiment, the second identification element 12 is an infrared light emitting element, such as an infrared LED, etc. In the embodiment, the number of the second identification element 12 is more than one, and the more than one second identification element 12 is fixed on the body. Since the infrared light is non-visible light and has the penetrating property, unnecessary visible light can be avoided to interfere with the user's vision, and the second identification element 12 can be arranged inside the body without being externally arranged, so that the effective positioning is ensured and the influence on the user's visual sense is reduced.

[0060] In other embodiments, the second identification element 12 can also be a module with a preset color on the body or other parts on the body which are easy to identify, such as a fluorescent strip pasted on the body, etc.

[0061] Further, the control device 1 further comprises a first wireless module 14 which is used for pairing connection with the head-mounted display device 2. In the embodiment, the first wireless module 14 is a Bluetooth module.

[0062] In the embodiment of the present application, with reference to Figure 1The first control device 13 of the control device 1 includes a processor 1001 (for example, a CPU), a memory 1002, a timer 1003, and the like. The components in the first control device 13 are connected through a communication bus. The memory 1002 can be a high-speed RAM memory or a stable memory (for example, a disk memory). The memory 1002 can alternatively be a storage device independent of the aforementioned processor 1001.

[0063] Those skilled in the art can understand that Figure 1 The device structure shown in the foregoing embodiments does not constitute a limitation on the device, and the device can include more or fewer components than those shown in the drawings, or some components can be combined, or different components can be arranged.

[0064] As shown in Figure 1 The memory 1002 as a storage medium can include a control program of the control device 1. In the device shown in Figure 1 The processor 1001 can be used to call the control program of the control device 1 stored in the memory 1002 and perform the related step operations of the control method of the control device 1 in the following embodiments.

[0065] The application further provides a control method of a control device, applied to the control device.

[0066] Referring to Figure 2 , an embodiment of the control method of the control device is provided. In the embodiment, the control method of the air conditioner includes:

[0067] Step S10, acquiring a first image;

[0068] The first image is specifically acquired through a first camera on the control device. In the embodiment, the first image is an infrared image.

[0069] Step S10 is performed when the control device and the head-mounted display device are in a paired state. In the paired state, the control device and the head-mounted display device establish a Bluetooth communication connection.

[0070] Step S20, determining first position information of the head-mounted display device according to a first image recognition result of a first identification element in the first image, the first identification element being arranged on the head-mounted display device;

[0071] Specifically, the position, shape, and / or color of the image corresponding to the first identification element in the first image are identified to obtain the first image recognition result. The first position information is determined according to the first image recognition result.

[0072] In the embodiment, the first position information is relative position information of the head-mounted display device relative to the control device. In other embodiments, the first position information can also be position information of the head-mounted display device in a world coordinate system.

[0073] In step S30, the first position information is sent to the head-mounted display device, so that the head-mounted display device displays an image according to the first position information and second position information, the second position information being position information of the control device.

[0074] Specifically, the first position information is sent to the head-mounted display device based on a Bluetooth communication connection between the control device and the head-mounted display device.

[0075] In the embodiment, the second position information is information detected by the head-mounted display device through a detection module of the head-mounted display device. In other embodiments, the second position information can also be information obtained by the head-mounted display device from an external device of the head-mounted display device and the control device.

[0076] When there are more than one control device paired and connected to the head-mounted display device, each control device can send corresponding first position information to the head-mounted display device, the head-mounted display device can obtain first position information of all control devices connected thereto, and the head-mounted display device displays an image according to all received first position information and second position information. For example, when the more than one control device includes a left handle and a right handle, the first position information of the left handle is defined as M1, and the first position information of the right handle is defined as M2. Then, the left handle determines M1 according to the above manner and sends M1 to the head-mounted display device, the right handle determines M2 according to the above manner and sends M2 to the head-mounted display device, the head-mounted display device receives M1 and M2, and displays an image according to M1, M2 and the second position information.

[0077] The control method of the control device proposed in the embodiment can effectively increase position data used in the image display process of the head-mounted display device, ensure image display accuracy, so that the user can achieve precise control based on the image seen, and effectively improve the operation precision of the control device and the head-mounted display device.

[0078] Further, in the above embodiment, when the fine control function on the head-mounted display device is turned on, step S10 is executed; when the fine control function on the head-mounted display device is turned off, step S10 is prohibited from being executed, and then the head-mounted display device displays the image according to the second position information alone.

[0079] Further, based on the above embodiment, another embodiment of the control method of the air conditioner is provided.

[0080] In the embodiment, with reference to Figure 3 , the step S20 comprises:

[0081] Step S21, identifying a first target image of the first marker in the first image;

[0082] In the embodiment, the first image is an infrared image, and the first marker is an infrared light-emitting element, so that a sub-image in an image region with a luminance in a preset luminance interval in the infrared image is identified as the first target image.

[0083] In other embodiments, the first image can also be a visible light image, so that a sub-image with image features matching the first marker in the first image is identified as the first target image according to the shape features and / or color features of the first marker.

[0084] Step S22, determining first image position information of the first target image in the first image, and the first image recognition result comprises the first image position information;

[0085] Specifically, a first image coordinate system of the first camera of the control device can be established in advance, and a first image coordinate of the first image in the first image coordinate system is determined, so that the first image coordinate is the first image position information.

[0086] Step S23, determining position information of the head-mounted display device relative to the control device as the first position information according to the first image position information.

[0087] A first space coordinate system is established in advance with a preset base point on the control device as the origin, a first coordinate conversion relationship between the first space coordinate system and the first image coordinate system is established in advance, and the first image coordinate is converted into a first space coordinate in the first space coordinate system based on the first coordinate conversion relationship, so that the first space coordinate is the first position information.

[0088] The first position information represents the direction and / or distance of the head-mounted display device relative to the control device.

[0089] In the embodiment, the position information of the head-mounted display device relative to the operation device is determined based on the image position of the first marker in the first image as the first position information, so that the head-mounted display device can display images based on the relative position information between the head-mounted display device and the operation device, and the image display accuracy is further improved, and the operation precision of the operation device and the head-mounted display device is further improved.

[0090] Further, based on any of the above embodiments, another embodiment of the control method of the air conditioner is provided. In the embodiment, referring to Figure 4 , step S10 comprises:

[0091] Step S11, receiving the second position information sent by the head-mounted display device; wherein the second position information is the position information of the operation device relative to the head-mounted display device;

[0092] In the embodiment, the second position information is detected by the head-mounted display device.

[0093] Step S12, acquiring the first image according to the second position information.

[0094] According to the second position information, the shooting direction of the camera on the operation device is controlled or the target image in the image shot by the camera is extracted to obtain the first image.

[0095] In the embodiment, according to the second position information, the rotation angle of the first camera on the operation device is determined, and according to the rotation angle, the first camera is controlled to rotate to face the head-mounted display device, and the image acquired by the first camera is obtained as the first image.

[0096] Specifically, the current shooting direction of the first camera is determined as a reference direction, the first direction of the head-mounted display device relative to the operation device is determined according to the second position information, the included angle between the first direction and the reference direction is determined as the rotation angle, the first camera is controlled to rotate according to the rotation angle, and when the shooting direction of the first camera is collinear with the first direction, the current image acquired by the first camera is obtained as the first image.

[0097] In the embodiment, the second position information detected by the head-mounted display device is not only used for the display of the image itself, but also used for the regulation and control of the first image acquisition process of the operation device, so that the image information of the head-mounted display device and the corresponding first marker can be contained in the acquired first image, and the first position information can be accurately identified based on the first image. Wherein, the rotation of the first camera is controlled based on the second position information, so as to realize the accurate regulation and control of the image acquisition range of the first camera, and to further improve the effectiveness of the first image acquisition and the accuracy of the first position information positioning.

[0098] In other embodiments, the target image acquisition range can also be determined according to the second position information, and a sub-image in a panoramic image captured by the camera on the control device and located in the target image acquisition range is taken as the first image. Alternatively, a target camera among the plurality of cameras on the control device can be determined according to the second position information, and an image captured by the target camera is taken as the first image.

[0099] The embodiment of the present application also provides a head-mounted display device 2. The head-mounted display device 2 is used in pairing operation with the control device. The head-mounted display device 2 can include head-mounted display glasses or a display helmet, etc. The head-mounted display device 2 is a device based on virtual reality technology display, or the head-mounted display device 2 is a device based on augmented reality technology display.

[0100] In the embodiment of the present application, with reference to Figure 1 The head-mounted display device 2 includes a display body, a second camera 21 arranged on the display body, a first identification element 22 arranged on the display body, and a second control device 23. The second camera 21 is connected with the second control device 23, and the second control device 23 can acquire image data captured by the second camera 21.

[0101] In the embodiment, the number of the second cameras 21 is more than one, and the plurality of second cameras 21 are arranged in a transverse direction of the display body. In the embodiment, the second camera 21 is an infrared camera.

[0102] In the embodiment, the first identification element 22 is an infrared light emitting element, such as an infrared LED, etc. In the embodiment, the number of the first identification elements 22 is more than one, and the plurality of first identification elements 22 are fixed on the display body. Since the infrared light is non-visible light and has penetrating property, unnecessary visible light interference to the user's vision can be avoided, and the first identification element 22 can be arranged inside the display body without external arrangement, so as to ensure effective positioning and reduce the influence on the user's visual experience.

[0103] In other embodiments, the first identification element 22 can also be a module with a preset color on the display body or other components on the display body which are easy to be recognized, such as a fluorescent strip pasted on the display body, etc.

[0104] Further, the head-mounted display device 2 also includes a second wireless module 24, which is used for pairing connection with the head-mounted display device 2. In the embodiment, the second wireless module 24 is a Bluetooth module.

[0105] In the embodiment of the present application, with reference to Figure 1The second control device 23 of the head-mounted display device 2 includes a processor 1001 (for example, a CPU), a memory 1002, a timer 1003, and the like. The components in the second control device 23 are connected through a communication bus. The memory 1002 can be a high-speed RAM memory, or a stable memory (non-volatile memory) such as a disk memory. The memory 1002 can alternatively be a storage device independent of the aforementioned processor 1001.

[0106] Those skilled in the art can understand that Figure 1 The device structure shown in the foregoing embodiments does not constitute a limitation on the device, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0107] As shown in Figure 1 The memory 1002 as a storage medium can include a control program of the head-mounted display device 2. In Figure 1 In the device shown, the processor 1001 can be used to invoke the control program of the head-mounted display device 2 stored in the memory 1002, and perform the relevant step operations of the control method of the head-mounted display device 2 in the following embodiments.

[0108] Further, based on any of the foregoing embodiments, an embodiment of the control method of the head-mounted display device is provided. In this embodiment, referring to Figure 5 The control method of the head-mounted display device includes the following steps:

[0109] Step S100, acquiring a second image;

[0110] The second image is specifically acquired through a second camera on the control device. In this embodiment, the second image is an infrared image.

[0111] Step S100 is performed when the control device and the head-mounted display device are in a paired state. In the paired state, the control device and the head-mounted display device establish a Bluetooth communication connection.

[0112] Step S200, determining second position information of the control device according to a second image recognition result of a second identification element in the second image, the second identification element being arranged on the control device;

[0113] Specifically, the position, shape, and / or color of the image corresponding to the second identification element in the second image are identified to obtain the second image recognition result. The first position information is determined according to the second image recognition result.

[0114] In this embodiment, the second position information is a relative position of the control device relative to the head-mounted display device.

[0115] The second position information can also be the position information of the control device in the world coordinate system.

[0116] At step S300, the first position information sent by the control device is received.

[0117] Specifically, the first position information can be detected by the control device after receiving the second position information, or can be synchronously detected by the control device during the detection of the second position information.

[0118] At step S400, an image is displayed according to the first position information and the second position information.

[0119] Specifically, the head-mounted display device can pre-establish a display coordinate system of a display interface. Based on this, the target image required to be displayed by the head-mounted display device and the relative position relationship between the feature image corresponding to the control device in the display coordinate system can be determined according to the first position information and the second position information. The coordinate set of the target image and the feature image in the display coordinate system is determined based on the relative position relationship, and the image is displayed on the display interface of the head-mounted display device based on the coordinate set.

[0120] Specifically, the head-mounted display device can determine the position information of the control device relative to the head-mounted display device according to the first position information, obtain third position information, and display the image according to the third position information and the second position information.

[0121] The embodiment provides a control method of a head-mounted display device, realizes the bidirectional positioning of the head-mounted display device and the control device, and the head-mounted display device no longer displays the image based on only the second position information of the control device in the process of image display, but can display the image in combination with the first position information and the second position information. The position data used in the process of image display of the head-mounted display device can be effectively increased, the image display accuracy is ensured, the user can realize accurate control based on the image seen, and the operation precision of the control device and the head-mounted display device is effectively improved.

[0122] Further, in the above embodiment, when the fine control function on the head-mounted display device is turned on

[0123] , step S100 is executed; when the fine control function on the head-mounted display device is turned off, step S100 is prohibited from being executed, and the head-mounted display device displays the image based on only the second position information.

[0124] Further, based on any one of the above embodiments, another embodiment of the control method of the head-mounted display device is provided. Figure 6 , step S200 includes:

[0125] Step S201, identifying a second target image of the second marker in the second image;

[0126] In the embodiment, the second image is an infrared image, and the second marker is an infrared light-emitting element. Then, a sub-image in an image region with a luminance in a preset luminance interval in the infrared image is identified as the second target image.

[0127] In other embodiments, the second image can also be a visible light image. Then, a sub-image with image features matching the second marker in the second image can be identified as the second target image according to shape features and / or color features of the second marker.

[0128] Step S202, determining second image position information of the second target image in the second image, and the second image identification result includes the second image position information.

[0129] Specifically, a second image coordinate system of the second camera of the head-mounted display device can be established in advance, and a second image coordinate of the second image in the second image coordinate system can be determined. Then, the second image coordinate is the second image position information.

[0130] Step S203, determining position information of the control device relative to the head-mounted display device as the second position information according to the second image position information.

[0131] A second space coordinate system can be established in advance with a preset base point on the head-mounted display device as an origin. A second coordinate conversion relationship between the second space coordinate system and the second image coordinate system can be established in advance. The second image coordinate can be converted into a second space coordinate in the second space coordinate system based on the second coordinate conversion relationship. Then, the second space coordinate is the second position information.

[0132] The second position information represents a direction and / or distance of the control device relative to the head-mounted display device.

[0133] In the embodiment, the position information of the control device relative to the head-mounted display device is determined based on the image position of the second marker in the second image as the second position information. Then, the head-mounted display device can perform image display based on the relative position information with the control device, which is beneficial to further improve the image display accuracy, thereby further improving the operation precision of the control device and the head-mounted display device.

[0134] Further, based on any of the above embodiments, another embodiment of a control method of the head-mounted display device is provided. In the embodiment, after step S200 and before step S300, the method further includes: sending the second position information to the control device, so that the control device detects the first position information based on the second position information.

[0135] After receiving the second location information, the control device can acquire the first image, determine the first location information, and send it in accordance with the manner mentioned in the above-mentioned control method embodiment.

[0136] In this embodiment, the second position information detected by the head-mounted display device is not only used for displaying its own image, but also to assist the control device in detecting the first position information, thereby improving the control device's ability to recognize the first position information.

[0137] The accuracy of the first position information is improved to further enhance the precision of the operation of the control device and the head-mounted display.

[0138] Furthermore, when more than one control device is paired and connected to the head-mounted display device, each control device can send its corresponding first position information to the head-mounted display device, which can then...

[0139] The head-mounted display device acquires the first position information of all connected control devices and displays an image based on the received first and second position information. For example, when more than one control device includes a left handle and a right handle, the first position information of the left handle is defined as M1, the first position information of the right handle is defined as M2, the second position information of the left handle is defined as M3, and the first position information of the right handle is defined as M4. The head-mounted display device can determine M3 and M4 in the above manner and send M3 to the left handle.

[0140] The left handle sends M4 to the right handle, the left handle acquires the first image based on M3 and determines the corresponding M1, the right handle acquires the first image based on M4 and determines the corresponding M2, the left handle sends M1 to the head-mounted display device, the right handle sends M2 to the head-mounted display device, and after receiving M1 and M2, the head-mounted display device displays the image based on M1, M2, M3 and M4.

[0141] To better understand the determination of the first and second location information mentioned in the embodiments of this application

[0142] During the process, both the left and right handles are paired and connected to the head-mounted display device. The following is a specific example to illustrate this:

[0143] (1) The camera on the head-mounted display device captures the positions of the infrared LEDs on the two handles (the second identifier on the first control device and the second identifier on the second control device), sets the spatial coordinates of the head-mounted display device itself to (0,0,0), and identifies the position of the infrared LED on the left handle as (x1,y1,z1), and the right handle...

[0144] The infrared LED position of the handle is (x2, y2, z2). The head-mounted display device sends the acquired position data 5 (x1, y1, z1), (x2, y2, z2) to the left handle and the right handle through the Bluetooth communication channel.

[0145] (2) The left handle and the right handle receive the position data transmitted by the head-mounted display device, and the control devices of the left handle and the right handle control the stepping motors to rotate the cameras to the direction of the head-mounted display device through the acquired position data (x1, y1, z1), (x2, y2, z2), until the visual angle range of the cameras on the handles covers the infrared LED (i.e. the first identification element on the head-mounted display device) of the head-mounted display device.

[0146] 0(3) After the handle performs the rotation action of the camera, the handle sets its own coordinates as (0, 0, 0), and the cameras on the left handle and the right handle identify the coordinates of the infrared LED of the head-mounted display device as (x3, y3, z3) and (x4, y4, z4) respectively, and the data is transmitted to the head-mounted display device through the Bluetooth communication channel, and the head-mounted display device converts it into position data (x5, y5, z5) and (x6, y6, z6) with the head-mounted display device as the origin.

[0147] (4) The head-mounted display device integrates the four groups of data (x1, y1, z1), (x2, y2, z2), (x5, y5, z5), (x6, y6, z6), and the position information data volume is effectively increased, that is, the positioning is more accurate.

[0148] In addition, the embodiment of the present application also provides a storage medium, and the storage medium stores a control program of an air conditioner. When the control program of the air conditioner is executed by a processor, the related steps of any one of the embodiments of the control method of the air conditioner are realized.

[0149] It should be noted that in this paper, the terms "including", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitation, the element defined by the statement "including a" does not exclude the existence of another same element in the process, method, article or system including the element.

[0150] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0151] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, can also be through hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application essentially or say the part of the prior art contribution can be embodied in the form of software products, the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disc, optical disc) as described above, including a number of instructions to make a terminal device (may be a mobile phone, computer, server, air conditioner, or network equipment, etc.) executes the method described in various embodiments of the present application.

[0152] The above is only the preferred embodiment of the present application, not therefore limit the patent scope of the present application, any equivalent structure or equivalent flow transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A control method of a manipulation device, characterized by, The control method of the control device for pairing operation with the head-mounted display device comprises the following steps: receiving second position information sent by the head-mounted display device; collecting a first image according to the second position information; determining first position information of the head-mounted display device according to a first image recognition result of a first identification element in the first image, the first identification element being arranged on the head-mounted display device; sending the first position information to the head-mounted display device, so that the head-mounted display device displays an image according to the first position information and second position information, the second position information being position information of the control device relative to the head-mounted display device.

2. The control method of the manipulation device according to claim 1, characterized by, The step of determining the first position information of the head-mounted display device according to the first image recognition result of the first identification element in the first image comprises: identifying a first target image of the first identification element in the first image; determining first image position information of the first target image in the first image, the first image recognition result comprising the first image position information; determining position information of the head-mounted display device relative to the control device as the first position information according to the first image position information.

3. The control method of the manipulation device according to claim 1, characterized by, The step of collecting the first image according to the second position information comprises: determining a rotation angle of a first camera on the control device according to the second position information; controlling the first camera to rotate according to the rotation angle so that the first camera faces the head-mounted display device; obtaining an image collected by the first camera as the first image.

4. A control method of a head-mounted display device, characterized by, The control method of the head-mounted display device for pairing operation with the control device as claimed in any one of claims 1 to 3 comprises: collecting a second image; determining second position information of the control device according to a second image recognition result of a second identification element in the second image, the second identification element being arranged on the control device; receiving the first position information sent by the control device; displaying an image according to the first position information and the second position information; Before the step of receiving the first position information sent by the control device, the method further comprises: sending the second position information to the control device, so that the control device detects the first position information based on the second position information.

5. The control method of the head-mounted display device according to claim 4, wherein The step of determining the second position information of the control device according to the second image recognition result of the second identification element in the second image comprises: identifying a second target image of the second identification element in the second image; determining second image position information of the second target image in the second image, the second image recognition result comprising the second image position information; determining position information of the control device relative to the head-mounted display device as the second position information according to the second image position information.

6. A steering device characterized by comprising: The control device comprises: a first camera; a second identification element, the second identification element being an infrared light-emitting element; A first control device, the camera is connected with the first control device, the first control device includes: memory, processor and storage on the memory and can be run on the processor control program of the control device, the control program of the control device is executed by the processor to realize the steps of the control method of the control device as claimed in any one of claims 1 to 3.

7. A head-mounted display device, comprising: The head-mounted display device comprises: A second camera; A first identification element; A second control device, the camera is connected with the second control device, the second control device includes: memory, processor and storage on the memory and can be run on the processor control program of the head-mounted display device, the control program of the head-mounted display device is executed by the processor to realize the steps of the control method of the head-mounted display device as claimed in any one of claims 4 to 5.

8. A storage medium, characterized by The storage medium has a control program stored thereon, and the control program, when executed by a processor, implements the steps of the control method according to any one of claims 1 to 5.

Images