Control method and apparatus for near-to-eye display device, and device and storage medium

Abstract

A control method for a near-to-eye display device. The method comprises: acquiring attitude data of a near-to-eye display device (S101); on the basis of the attitude data of the near-to-eye display device, determining a target operation object in a human-machine interaction interface of the near-to-eye display device (S102); and on the basis of operation information received by the near-to-eye display device, executing a corresponding operation on the target operation object (S103). Also provided are an apparatus, a device and a storage medium.

Description

Control method, device, equipment and storage medium for near-eye display device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 15, 2023, with application number 202311739489X, and invention name “Control method, device and equipment for near-eye display device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of human-computer interaction technology, and in particular to a control method, apparatus, device, and storage medium for a near-eye display device. Background Art

[0003] Currently, when using a near-eye display device, users need to raise their arms to control the device. For example, when a user controls a near-eye display device using a mobile phone, a controller, or other device, the user can raise the phone or controller and perform corresponding operations on the phone or controller to control the near-eye display device. In this case, the user's control of the near-eye display device is not convenient. As the user holds the phone or controller for an extended period of time, the user's arm becomes fatigued, resulting in a poor user experience with the near-eye display device.

[0004] Summary of the Invention

[0005] The main purpose of this application is to provide a control method, device, equipment and storage medium for a near-eye display device, aiming to solve the technical problem of poor user interaction experience with the near-eye display device due to insufficient convenience of user control of the near-eye display device.

[0006] In a first aspect, the present application provides a method for controlling a near-eye display device, comprising:

[0007] Obtaining posture data of a near-eye display device;

[0008] determining a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device;

[0009] According to the operation information received by the near-eye display device, a corresponding operation is performed on the target operation object.

[0010] In a second aspect, the present application further provides a control device for a near-eye display device, the control device comprising:

[0011] An acquisition module, used to obtain posture data of a near-eye display device;

[0012] an operation object determination module, configured to determine a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device;

[0013] An operation module is used to perform a corresponding operation on the target operation object according to the operation information received by the near-eye display device.

[0014] In a third aspect, the present application further provides a near-eye display device, the near-eye display device comprising a memory and a processor;

[0015] The memory is used to store computer programs;

[0016] The processor is configured to execute the computer program and implement the following steps when executing the computer program:

[0017] Obtaining posture data of a near-eye display device;

[0018] determining a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device;

[0019] According to the operation information received by the near-eye display device, a corresponding operation is performed on the target operation object.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the following steps are implemented:

[0021] Obtaining posture data of a near-eye display device;

[0022] determining a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device;

[0023] According to the operation information received by the near-eye display device, a corresponding operation is performed on the target operation object.

[0024] The present application provides a control method, apparatus, device and storage medium for a near-eye display device. The control method includes: obtaining posture data of the near-eye display device; determining a target operation object in a human-computer interaction interface of the near-eye display device based on the posture data of the near-eye display device; and performing corresponding operations on the target operation object based on the operation information received by the near-eye display device.

[0025] By obtaining the posture data of the near-eye display device, the near-eye display device can determine the user's operational requirements for the human-computer interaction interface of the near-eye display device, such as determining the target operation object in the human-computer interaction interface. When the target operation object is determined, the corresponding operation can be performed on the target operation object according to the operation information received by the near-eye display device, thereby realizing the control of the near-eye display device. In the process of realizing the near-eye display device, since the near-eye display device is worn on the user's head, the user can adjust the posture of the near-eye display device by changing the posture of the head. In response to the change in the posture of the near-eye display device, the posture data of the near-eye display device obtained will also change. Then, by obtaining the posture data of the near-eye display device, the near-eye display device is controlled, which is conducive to improving the control convenience of the near-eye display device. For example, the user does not need to raise his arm for a long time to realize the control of the near-eye display device, so the user is less likely to feel arm fatigue, which is conducive to improving the user's interactive experience with the near-eye display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0027] FIG1 is a schematic flow chart of a method for controlling a near-eye display device according to an embodiment of the present application;

[0028] FIG2 is a schematic diagram of a user's head posture according to an embodiment of the present application;

[0029] FIG3 is a schematic diagram of a human-computer interaction interface of a near-eye display device according to an embodiment of the present application;

[0030] FIG4 is a schematic diagram of a human-computer interaction interface of a near-eye display device according to another embodiment of the present application;

[0031] FIG5 is a schematic block diagram of a control device for a near-eye display device provided in an embodiment of the present application;

[0032] FIG6 is a schematic block diagram of the structure of a near-eye display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0035] Embodiments of the present application provide a control method, apparatus, device, and storage medium for a near-eye display device.

[0036] The control method of the near-eye display device can be applied to near-eye display devices. Near-eye display devices may include augmented reality (AR) glasses, virtual reality (VR) glasses, AR helmets, VR helmets, etc., without limitation.

[0037] The control method of the near-eye display device can also be applied to a server, which can be a separate server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0038] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0039] Please refer to Figure 1, which is a flow chart of a control method for a near-eye display device provided in an embodiment of the present application. It should be noted that the control method for a near-eye display device provided in an embodiment of the present application can be used for a near-eye display device. Of course, it can also be used for a server. For example, the server can obtain the posture data of the near-eye display device from the near-eye display device to determine the target operation object in the human-computer interaction interface of the near-eye display device. Exemplarily, the server can, for example, send the target operation object to the near-eye display device, so that the near-eye display device can further perform corresponding operations on the target operation object based on the received operation information; of course, it is not limited to this and is not limited here.

[0040] As shown in FIG1 , the control method of the near-eye display device includes steps S101 to S103 .

[0041] S101: Acquire posture data of a near-eye display device.

[0042] For example, the posture data of the near-eye display device can be determined using sensor data collected by a sensor provided in the near-eye display device. For example, the sensor data can be used as the posture data of the near-eye display device to determine the posture of the near-eye display device based on the posture data of the near-eye display device.

[0043] In some embodiments, the three degrees of freedom (3DOF) rotation data of the near-eye display device can be obtained to use the 3DOF rotation data as the posture data of the near-eye display device. As shown in Figure 2, the near-eye display device is worn on the user's head, and the sensor data obtained by the near-eye display device can be the user's head 3DOF rotation data. The user's head 3DOF rotation data can also be called the user's head posture data, and the user's head posture data can be used to indicate the user's head posture. For example, when the user's head rotates around the horizontal axis (X-axis) corresponding to the near-eye display device, the near-eye display device can obtain the rotation angle data when the head rotates, such as the pitch angle (Pitch). When the user's head rotates around the vertical axis (Y-axis) corresponding to the near-eye display device, the near-eye display device can obtain the rotation angle data when the head rotates, such as the yaw angle (Yaw). When the user's head rotates around the vertical axis (Z-axis) corresponding to the near-eye display device, the near-eye display device can obtain the rotation angle data when the head rotates, such as the roll angle (Roll). When the near-eye display device obtains at least one rotation angle data of Pitch, Yaw and Roll, it can use the obtained at least one rotation angle data as the posture data of the near-eye display device for subsequent control of the near-eye display device.

[0044] S102: Determine a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device.

[0045] For example, the user can interact with the near-eye display device by adjusting the posture of the near-eye display device. In some embodiments, the near-eye display device can, for example, display the human-computer interaction interface of the near-eye display device to the user, and the user can interact with the human-computer interaction interface by adjusting the posture of the near-eye display device. The human-computer interaction interface may include at least one preset operation object. The preset operation object may include at least one of a virtual operation object and a real operation object. Virtual operation objects may include icons, buttons, lists, texts, etc., which are not limited here. For example, the user can click, rotate, move, edit, delete, etc. on the virtual operation object through the near-eye display device, which are not limited here. Real operation objects may include objects in the real scene outside the near-eye display device. For example, the user can add virtual special effects, modify virtual special effects, delete virtual special effects, etc. to the real operation object through the near-eye display device, which are not limited here. Taking AR glasses as an example of a near-eye display device, due to the optical properties of the lenses of AR glasses, such as the fact that the lenses of AR glasses can pass through the light provided by the real scene outside, when the AR glasses display the human-computer interaction interface to the user's eyes, the user can still see the real scene outside, so the human-computer interaction interface of the near-eye display device can also include real operation objects.

[0046] Based on this, when a user needs to interact with the human-computer interaction interface of a near-eye display device, they can determine the target operation object in the human-computer interaction interface by adjusting the posture of the near-eye display device. For example, the posture of the near-eye display device will change with the changes in the posture of the user's head. In response to the change in the posture of the near-eye display device, the acquired posture data of the near-eye display device will also change accordingly. Then, based on the posture data of the near-eye display device, the target operation object in the human-computer interaction interface of the near-eye display device can be determined.

[0047] For example, the position of the virtual cursor in the human-computer interaction interface is adjusted according to the posture data; when the distance between the position of the virtual cursor in the human-computer interaction interface and the position of the preset operation object in the human-computer interaction interface is reduced to less than or equal to the preset distance threshold, the preset operation object is determined to be the target operation object.

[0048] In some embodiments, a virtual cursor may be displayed in the human-computer interaction interface. The virtual cursor may be used to indicate the user's current interaction position on the human-computer interaction interface. When the user's head rotates around the Y-axis corresponding to the near-eye display device, such as rotating to the left, based on the acquired posture data of the near-eye display device, it can be determined that the posture of the near-eye display device is translated to the left, and the near-eye display device can translate the position of the virtual cursor in the human-computer interaction interface to the left. When the user's head rotates around the X-axis corresponding to the near-eye display device, such as rotating downward, based on the acquired posture data of the near-eye display device, it can be determined that the posture of the near-eye display device is moving downward, and the near-eye display device can move the position of the virtual cursor in the human-computer interaction interface downward. When the user's head rotates around the Z-axis corresponding to the near-eye display device, such as rotating diagonally upward, based on the acquired posture data of the near-eye display device, it can be determined that the posture of the near-eye display device is moving diagonally upward, and the near-eye display device can move the position of the virtual cursor in the human-computer interaction interface diagonally upward. Of course, this is not limited to the above. For example, the near-eye display device may also move the position of the virtual cursor in the human-computer interaction interface horizontally to the right, upward, or diagonally downward based on the gesture data, and this is not limited here. In the process of adjusting the position of the virtual cursor in the human-computer interaction interface based on the gesture data, the rate of change of the virtual cursor position, such as the movement speed of the virtual cursor in the human-computer interaction interface, may be pre-set or user-set, and this is not limited here.

[0049] Take the preset operation objects in the human-computer interaction interface including preset operation objects 1 to preset operation objects 4 as an example.

[0050] As shown in Figure 3, preset operation objects 1 to 4 are arranged from left to right in the human-computer interaction interface, and preset operation objects 1 to 4 are horizontally aligned in the human-computer interaction interface. The initial position of the virtual cursor in the human-computer interaction interface is, for example, a position between preset operation object 2 and preset operation object 3. For example, the virtual cursor can move from the initial position to the left to the position of any one of preset operation object 1 and preset operation object 2 in the human-computer interaction interface in response to the user's head rotating to the left. Of course, this is not limited to this. The virtual cursor can also move from the initial position to the right to the position of any one of preset operation object 3 and preset operation object 4 in the human-computer interaction interface in response to the user's head rotating to the right. This is not limited here.

[0051] As shown in FIG4 , preset operation objects 1 and 2, and preset operation objects 3 and 4 are horizontally aligned in the human-computer interaction interface, while preset operation objects 1 and 3 are vertically aligned, and preset operation objects 2 and 4 are vertically aligned. Preset operation object 1 is located above preset operation object 3, and preset operation object 2 is located above preset operation object 4. The initial position of the virtual cursor in the human-computer interaction interface is, for example, the midpoint between preset operation objects 1 and 4. For example, the virtual cursor can move upward from its initial position to a position between preset operation objects 1 and 2 in response to an upward rotation of the user's head, or can move downward from its initial position to a position between preset operation objects 3 and 4 in response to a downward rotation of the user's head. For another example, the virtual cursor can move diagonally upward from its initial position to the position of either preset operation object 1 or preset operation object 2 in response to an upward rotation of the user's head, or can move diagonally downward from its initial position to the position of either preset operation object 3 or preset operation object 4 in response to a downward rotation of the user's head. Of course, it is not limited to this. The virtual cursor can be translated to the left from the initial position to the position between the preset operation object 1 and the preset operation object 3 in response to the user's head rotating to the left, or it can be translated to the right from the initial position to the position between the preset operation object 2 and the preset operation object 4 in response to the user's head rotating to the right. There is no limitation here.

[0052] For example, in the process of adjusting the position of the virtual cursor in the human-computer interaction interface according to the posture data, the distance between the position of the virtual cursor and the position of the preset operation object in the human-computer interaction interface will change. The position of the virtual cursor can be determined based on the center point of the virtual cursor or one of the corner points of the virtual cursor. The position of the preset operation object can be determined based on the center point of the preset operation object or one of the corner points of the preset operation object, and the position determination method of different preset operation objects in the same human-computer interaction interface is the same. Of course, it is not limited to this. The position determination method of the virtual cursor and the preset operation object can be pre-set or set by the user, which is not limited here.

[0053] When determining the distance between the position of the virtual cursor and the position of the preset operation object in the human-computer interaction interface, the user's operation requirements for the preset operation object in the human-computer interaction interface can be determined based on the distance between the position of the virtual cursor and the position of the preset operation object in the human-computer interaction interface.

[0054] For example, a preset operation object in a human-computer interaction interface includes a preset operation object 1 and a preset operation object 2, with preset operation object 1 located to the left of preset operation object 2 and horizontally aligned with preset operation object 2. The initial position of the virtual cursor in the human-computer interaction interface can be between preset operation object 1 and preset operation object 2. When the virtual cursor adjusts its position in the human-computer interaction interface to the left based on the gesture data, the virtual cursor gradually approaches preset operation object 1 and gradually moves away from preset operation object 2. This is equivalent to the distance between the position of the virtual cursor in the human-computer interaction interface and the position of preset operation object 1 in the human-computer interaction interface decreasing, while the distance between the position of the virtual cursor in the human-computer interaction interface and the position of preset operation object 2 in the human-computer interaction interface increasing. When the distance between the position of the virtual cursor in the human-computer interaction interface and the position of preset operation object 1 decreases to less than or equal to a preset position threshold, it can be determined that the user has an operation requirement for preset operation object 1, and preset operation object 1 can be determined as the target operation object. Of course, the preset operation objects in the human-computer interaction interface are not limited to preset operation object 1 and preset operation object 2, and the positions of the preset operation objects in the human-computer interaction interface are not limited thereto and are not limited here.

[0055] In an exemplary embodiment, the initial position of the virtual cursor in the human-computer interaction interface may be closer to one of the preset operation object 1 and the preset operation object 2. For example, the distance between the initial position of the virtual cursor in the human-computer interaction interface and the position of the preset operation object 1 in the human-computer interaction interface is greater than a preset distance threshold, and the initial position of the virtual cursor in the human-computer interaction interface is less than or equal to the preset distance threshold. By reducing the distance between the virtual cursor and the preset operation object and making the distance between the virtual cursor and the preset operation object less than or equal to the preset distance threshold as the basis for determining that the preset operation object is the target operation object, it is possible to avoid the situation where the user directly determines that the preset operation object 2 has an operation requirement based on the distance between the virtual cursor and the preset operation object 2 being less than or equal to the preset distance threshold, thereby directly determining that the preset operation object 2 is the target operation object, while the preset operation object that the user actually needs to operate is preset operation object 1. Based on this, the determination of whether the preset operation object is the target operation object is based on the changing trend of the distance between the position of the virtual cursor in the human-computer interaction interface and the position of the preset operation object in the human-computer interaction interface and the preset distance threshold, which is conducive to improving the accuracy of the near-eye display device in determining the target operation object. Based on the improvement of the accuracy of determining the target operation object, the target operation object can be subsequently used for controlling the near-eye display device, which is conducive to subsequently improving the control accuracy of the near-eye display device.

[0056] In some embodiments, when the distance between the position of the virtual cursor in the human-computer interaction interface and the positions of multiple preset operation objects in the human-computer interaction interface is reduced to less than or equal to a preset distance threshold, the preset operation object with the smallest distance from the position of the virtual cursor is selected from the multiple preset operation objects whose positions are reduced to less than or equal to the preset distance threshold as the target operation object.

[0057] For example, as shown in FIG4 , during the process of the virtual cursor moving diagonally upward and leftward from its initial position at a small angle, the distances between the virtual cursor and both preset operation object 1 and preset operation object 3 can be reduced to less than or equal to a preset distance threshold. In this case, the near-eye display device can select the preset operation object closest to the virtual cursor from preset operation object 1 and preset operation object 3 as the target operation object. For example, if the distance between the virtual cursor and preset operation object 1 is less than the distance between the virtual cursor and preset operation object 3, preset operation object 1 can be determined to be the target operation object.

[0058] In some embodiments, when the distance between the position of the virtual cursor in the human-computer interaction interface and the positions of multiple preset operation objects in the human-computer interaction interface is reduced to less than or equal to a preset distance threshold, and the distance between the position and the position of the virtual cursor is reduced to less than or equal to the preset distance threshold and the distance between each of the multiple preset operation objects and the virtual cursor is the same, a prompt message is output; based on the feedback information corresponding to the prompt message, the target operation object is determined from the multiple preset operation objects whose distance between the position and the position of the virtual cursor is reduced to less than or equal to the preset distance threshold.

[0059] For example, as shown in Figure 4, when the virtual cursor begins to translate to the right from its initial position, the distances between the virtual cursor and both preset operation objects 2 and 4 can decrease to less than or equal to a preset distance threshold. Furthermore, the distances between the virtual cursor and both preset operation objects 2 and 4 can be the same. Based on this, the near-eye display device can output prompt information to the user. In one exemplary embodiment, the prompt information can be used to instruct the user to determine a target operation object from among multiple preset operation objects whose distances from the virtual cursor's position decrease to less than or equal to the preset distance threshold. For example, the prompt information can instruct the user to determine a target operation object from among preset operation objects 2 and 4. If the near-eye display device receives feedback information corresponding to the prompt information, the target operation object can be determined based on the feedback information. For example, if the user determines that preset operation object 4 is the target operation object based on the prompt information, the feedback information corresponding to the prompt information can carry identification information of preset operation object 4. The near-eye display device can then determine that preset operation object 4 is the target operation object based on the identification information of preset operation object 4 carried in the feedback information. This is of course not limiting and is not intended to be limiting herein.

[0060] In this way, the target operation object in the human-computer interaction interface of the near-eye display device can be determined based on the posture data of the near-eye display device. For example, the position of the virtual cursor in the human-computer interaction interface can be adjusted based on the posture data of the near-eye display device. When the distance between the position of the preset operation object in the human-computer interaction interface and the position of the virtual cursor is reduced to less than or equal to the preset distance threshold, the preset operation object is determined as the target operation object, which is conducive to improving the convenience of the near-eye display device in determining the target operation object.

[0061] In some implementations, a virtual cursor in the human-computer interaction interface is adsorbed onto the target operation object.

[0062] For example, after determining a target operation object in a human-computer interaction interface of a near-eye display device, a virtual cursor in the human-computer interaction interface may be adsorbed onto the target operation object.

[0063] When the target operation object determined by the near-eye display device is a button, the virtual cursor can be adsorbed onto the button so that the user can perform subsequent operations on the button.

[0064] When the target operation object determined by the near-eye display device is a character or word of the text, the virtual cursor can be adsorbed onto the character or word of the text so that the user can subsequently edit, select, locate, etc. the text based on the character or word of the text.

[0065] In the case where the target operation object determined by the near-eye display device is a function list, the virtual cursor can be adsorbed onto the function list so that the user can subsequently select a corresponding function based on the function list.

[0066] When the target operation object determined by the near-eye display device is a real operation object, the virtual cursor can be adsorbed on the real operation object so that the user can subsequently add, modify or delete corresponding virtual special effects for the real operation object.

[0067] By attaching the virtual cursor to the target operation object, the user can be informed of the preset operation object as the target operation object through the position of the virtual cursor. The user can then perform subsequent operations on the target operation object according to the indication of the virtual cursor, or continue to adjust the posture of the near-eye display device to reselect the target operation object.

[0068] In some embodiments, the target operation object can be determined based on the distance between the virtual cursor and the position of the preset operation object in the human-computer interaction interface being reduced to less than or equal to a preset distance value threshold. When the user wears a near-eye display device on his head and the user adjusts the posture of the near-eye display device by rotating his head, by adsorbing the virtual cursor in the human-computer interaction interface onto the target operation object, the user can interact with the target operation object without continuing to rotate his head. Based on this, by adsorbing the virtual cursor in the human-computer interaction interface onto the target operation object, it is beneficial to reduce the amplitude of the movement required when the user adjusts the posture of the near-eye display device, thereby improving the convenience of the user's adjustment of the posture of the near-eye display device. Moreover, by adsorbing the virtual cursor onto the target operation object, the position of the virtual cursor in the human-computer interaction interface can be changed from a position where the distance between the virtual cursor and the target operation object is less than or equal to the preset distance threshold to the position of the target operation object in the human-computer interaction interface, thereby saving the time required to adjust the position of the virtual cursor in the human-computer interaction interface and improving the efficiency of the near-eye display device in adjusting the position of the virtual cursor. Accordingly, based on the improvement of the near-eye display device's efficiency in adjusting the position of the virtual cursor, the near-eye display device can more conveniently determine the target operation object from one or more preset operation objects in the human-computer interaction interface, which is conducive to improving the near-eye display device's efficiency in selecting the target operation object.

[0069] For example, after the virtual cursor in the human-computer interaction interface is adsorbed onto the target operation object, the near-eye display device may further adjust the display form, display effect, etc. of the virtual cursor when it is adsorbed onto the target operation object.

[0070] For example, the display form of the virtual cursor when it is attached to the target operation object is determined according to the object type of the target operation object.

[0071] In the case that different target operation objects have different object types, the display forms of the virtual cursor when it is attached to different target operation objects may be different.

[0072] For example, if the target operation object includes a virtual operation object and a real operation object, the virtual operation object is a virtual element and the real operation object is a real element, and the object type of the virtual operation object is different from the object type of the real operation object.

[0073] In the case where the target operation object is a virtual operation object, when the near-eye display device adsorbs the virtual cursor on the virtual operation object, the display form of the virtual cursor can be reflected as being adsorbed on the surface of the virtual operation object, that is, the virtual cursor can be presented on the surface. At this time, the virtual cursor can also be called an on-surface cursor. For example, the same virtual operation object may include multiple surfaces, and the multiple surfaces included in the virtual operation object are all available for the user to perform subsequent operations. By adsorbing the virtual cursor on the surface of the virtual operation object, the user can be indicated that the current operation object is the surface to which the virtual cursor is currently adsorbed. According to the indication of the virtual cursor, the user can operate on the surface to which the virtual cursor is currently adsorbed in the virtual operation object, such as adjusting the shape of the surface, adding virtual special effects, etc., or reselect other surfaces of the virtual operation object or any other operation objects for subsequent operations, which are not limited here.

[0074] When the target operation object is a real operation object, when the near-eye display device attaches the virtual cursor to the real operation object, the virtual cursor can be displayed as being attached to the sphere corresponding to the real operation object, that is, the virtual cursor can appear on the sphere. In this case, the virtual cursor can also be referred to as a sphere-on-sphere cursor. The sphere corresponding to the real operation object can be a virtual effect added to the real operation object in the human-computer interaction interface. The sphere corresponding to the real operation object can cover the position range corresponding to the real operation object in the human-computer interaction interface, without limitation. For example, because the real operation object is an object located in the external real scene, the user cannot change the position of the real operation object in the external real scene through the near-eye display device, such as rotating or moving the real operation object. However, the user can add virtual special effects to the real operation object in the human-computer interaction interface. By attaching the virtual cursor to the sphere corresponding to the real operation object, the user can be informed that the current operation object is the entire real operation object. Based on the indication of the virtual cursor, the user can operate on the entire real operation object or reselect another operation object for subsequent operations, without limitation.

[0075] In this way, the object types of different target operation objects can be indicated to the user by the different display forms of the virtual cursor when it is adsorbed on target operation objects of different object types. Take the example of different target operation objects including virtual operation objects and real operation objects. If the display form of the virtual cursor when it is adsorbed on a target operation object is adsorbed on the surface of the target operation object, the user can determine that the target operation object is a virtual operation object. If the display form of the virtual cursor when it is adsorbed on another target operation object is adsorbed on the sphere corresponding to the target operation object, the user can determine that the target operation object is a real operation object. The user can distinguish the object type of the target operation object by the display form of the virtual cursor, and then when the user subsequently operates the target operation object, the appropriate operation can be selected based on the object type of the target operation object, which is conducive to improving the user's recognition convenience and operation convenience of the target operation object.

[0076] For example, the display effect when the virtual cursor is adsorbed on the target operation object is determined according to the interactive control information of the target operation object.

[0077] Exemplarily, the interactive control information of the target operation object may be used to indicate whether the target operation object is an interactive control such as a label icon, a button, a list, an application icon, etc., which can be interacted with by the user.

[0078] Take the target operation object as a label icon as an example. The label icon may include an icon or a label (tab) involved in the user interface (UI) design of the human-computer interaction interface. The background color of the label icon may change with the change of the background color of the human-computer interaction interface where the label icon is located, which is equivalent to the label icon being a label icon without a background. Based on this, in the case where the target operation object adsorbed by the virtual cursor is a label icon, a highlighted display effect can be added to at least one of the virtual cursor and the label icon to indicate to the user that the label icon is the target operation object, and to distinguish the label icon as the target operation object from the preset operation objects other than the target operation object in the human-computer interaction interface. The virtual cursor with a highlighted display effect can be presented as the outline color of the virtual cursor being changed to a highlighted color, or as a highlighted aperture being added to the tip of the virtual cursor, and so on, without limitation here. The label icon with a highlighted display effect can be presented as the outline color of the label icon being changed to a highlighted color, or as a highlighted aperture being added to the inside of the label icon, and so on, without limitation here.

[0079] Take the case where the target operation object is any one of a button and a list as an example. The background color of a button may not change with the change of the background color of the human-computer interaction interface where the button is located, and the background color of a list may not change with the change of the background color of the human-computer interaction interface where the list is located, which is equivalent to a button being a button with a background, and a list being a list with a background. Based on this, when the target operation object adsorbed by the virtual cursor is any one of a button and a list, an elevated display effect can be added to the virtual cursor to indicate to the user that the button or any one of the lists is the target operation object, and to distinguish the button or list as the target operation object from the preset operation objects other than the target operation object in the human-computer interaction interface. When the virtual cursor with an elevated display effect is adsorbed on a button, the button can be slightly pressed or bounced up. When the virtual cursor with an elevated display effect is adsorbed on a list, the background color of the list can be slightly changed, and so on. There is no limitation here.

[0080] For example, if the target operation object is an application icon, a floating display effect can be added to the virtual cursor to indicate to the user that the application icon is the target operation object, and to distinguish the application icon as the target operation object from the preset operation objects other than the target operation object in the human-computer interaction interface. When the virtual cursor with the floating display effect is attached to the application icon, the application icon can be moved along a preset movement path, or the area of ​​the application icon can be increased, etc., without limitation here.

[0081] Of course, the display effect when the virtual cursor is adsorbed on the target operation object is not limited to this. The association between the interactive control information corresponding to the target operation object and the display effect when the virtual cursor is adsorbed on the target operation object can be pre-set or set by the user, and there is no restriction here.

[0082] In this way, when the interactive control information corresponding to the target operation object is different, the display effect of the virtual cursor when it is attached to different target operation objects can also be different. Based on the different display effects of the virtual cursor, the user can determine the object type of the target operation object currently attached to the virtual cursor, or determine the interactive control information corresponding to the target operation object currently attached to the virtual cursor, so as to facilitate subsequent operation on the target operation object, which helps improve the user's identification and operation convenience of the target operation object.

[0083] In some embodiments, when the near-eye display device receives a call instruction corresponding to the virtual cursor, the virtual cursor is displayed in the human-computer interaction interface.

[0084] For example, the near-eye display device may determine whether a call instruction corresponding to the virtual cursor has been received based on at least one of a user operation on the near-eye display device and a user operation on a device with which the near-eye display device has established a communication connection. The call instruction corresponding to the virtual cursor may be pre-set or user-set, and is not limited here.

[0085] For example, the user can send a call instruction corresponding to the virtual cursor to the near-eye display device through a touch operation, a press operation, etc. on the control device corresponding to the near-eye display device. A communication connection is established between the control device and the near-eye display device. The communication connection may include a Bluetooth connection, a wireless communication (WiFi) connection, etc., which are not limited here. The control device may include a smart ring, a smart bracelet, and the like, which are not limited here. For example, when the near-eye display device receives a first call signal transmitted by the control device, it determines that a call instruction corresponding to the virtual cursor is received to display the virtual cursor in the human-computer interaction interface.

[0086] Taking the control device as a smart ring as an example, the user can press or touch the smart ring once to cause the smart ring to transmit a first call signal to the near-eye display device. When the near-eye display device receives the first call signal transmitted by the smart ring, it can determine that it has received a call instruction corresponding to the virtual cursor. Based on the received call instruction, the virtual cursor can be displayed in the human-computer interaction interface. Based on the virtual cursor displayed in the human-computer interaction interface, the user can adjust the position of the virtual cursor in the human-computer interaction interface by adjusting the posture of the near-eye display device.

[0087] Of course, it is not limited to this. The user can send a call instruction corresponding to the virtual cursor to the near-eye display device through touch operations, press operations, etc. on the near-eye display device. For example, a touch area or a press area can be set on the device body of the near-eye display device. Taking the near-eye display device as an eyewear device such as AR glasses and VR glasses as an example, the device body of the near-eye display device may include temples, frames, etc., which are not limited here. The near-eye display device can respond to the preset operation information received by the device body of the near-eye display device to determine that a call instruction corresponding to the virtual cursor is received, so as to display the virtual cursor in the human-computer interaction interface.

[0088] Taking the near-eye display device as an eyewear device such as AR glasses and VR glasses, the near-eye display device includes temples with touch areas provided on the temples as an example. The user can press or touch the temples of the near-eye display device once, so that the near-eye display device determines that a call instruction corresponding to a virtual cursor has been received based on the received press operation or touch operation. Based on the received call instruction, the virtual cursor can be displayed in the human-computer interaction interface. Based on the virtual cursor displayed in the human-computer interaction interface, the user can adjust the position of the virtual cursor in the human-computer interaction interface by adjusting the posture of the near-eye display device.

[0089] In other embodiments, the user may also send a call instruction corresponding to the virtual cursor to the near-eye display device via voice. For example, the user may say "call virtual cursor," "display virtual cursor," or other voice instructions to the near-eye display device to call the virtual cursor. Based on the user's voice, the near-eye display device may determine that it has received a call instruction corresponding to the virtual cursor. Based on the received call instruction, the virtual cursor may be displayed in the human-computer interaction interface, without limitation.

[0090] In this way, when the near-eye display device receives a call instruction corresponding to the virtual cursor, the virtual cursor is displayed in the human-computer interaction interface. Then, when the user selects the target operation object from the preset operation objects in the human-computer interaction interface, the user can drive the posture change of the near-eye display device by changing the head posture, thereby changing the posture data of the near-eye display device. In response to the change of the posture data of the near-eye display device, the position of the virtual cursor in the human-computer interaction interface can be adjusted. When the user needs to call the virtual cursor, he can choose to call the virtual cursor on his own, which is conducive to improving the convenience of calling the virtual cursor by the near-eye display device. In addition, when the near-eye display device does not receive a call instruction corresponding to the virtual cursor, it can not display the virtual cursor in the human-computer interaction interface to avoid additional increase in the display energy consumption of the human-computer interaction interface, which is conducive to saving energy consumption of the near-eye display device.

[0091] In some embodiments, within a preset time period after the virtual cursor is displayed in the human-computer interaction interface, if it is determined that the posture data of the near-eye display device has not changed, the virtual cursor is hidden in the human-computer interaction interface.

[0092] For example, since the user can adjust the posture of the near-eye display device by changing the posture of the head, the posture data of the near-eye display device obtained in response to the change in the posture of the near-eye display device will also change. Based on this, after the virtual cursor is displayed in the human-computer interaction interface, the near-eye display device can still continuously obtain the posture data of the near-eye display device to determine whether the user has an operation demand for the virtual cursor. For example, within a preset time period after the virtual cursor is displayed in the human-computer interaction interface, the posture data of the near-eye display device continues to be obtained. Among them, the preset time period can be pre-set or set by the user, and is not limited here.

[0093] Within a preset time period after the virtual cursor is displayed in the human-computer interaction interface, if it is determined that the posture data of the near-eye display device has not changed, it can be determined that the user has not operated the virtual cursor for a long time, which is equivalent to determining that the user has no need to operate the virtual cursor. Based on this, the virtual cursor is hidden in the human-computer interaction interface to avoid additional display energy consumption of the human-computer interaction interface, which is beneficial to saving energy consumption of the near-eye display device. In addition, the near-eye display device can hide the virtual cursor in the human-computer interaction interface based on the duration of the virtual cursor being in the display state and the near-eye display device not changing for a preset time period, which is beneficial to improving the convenience of the near-eye display device to stop calling the virtual cursor.

[0094] Of course, this is not limited to the above. The user can also choose whether to stop calling the virtual cursor. For example, the user can send a stop call instruction corresponding to the virtual cursor to the near-eye display device, and the near-eye display device can hide the virtual cursor in the human-computer interaction interface based on the received stop call instruction. The stop call instruction corresponding to the virtual cursor can be pre-set or set by the user, and there is no limitation here.

[0095] For example, the user may send a stop call instruction corresponding to the virtual cursor to the near-eye display device by performing a touch operation, a press operation, etc. on a control device corresponding to the near-eye display device.

[0096] For example, using a smart ring as the control device, a user can press or touch the smart ring twice in succession, causing it to transmit a second call signal to the near-eye display device. Upon receiving the second call signal, the near-eye display device can determine that it has received a stop call instruction corresponding to the virtual cursor. Based on the received stop call instruction, the virtual cursor can be hidden from the human-computer interaction interface.

[0097] For example, the user may also send a stop call instruction corresponding to the virtual cursor to the near-eye display device through a touch operation, a press operation, etc. on the near-eye display device.

[0098] Taking a near-eye display device including AR glasses, VR glasses, and other eyewear devices, where the near-eye display device includes temples with touch areas provided thereon as an example, a user can press or touch the temples of the near-eye display device twice in succession, so that the near-eye display device can determine, based on the two consecutive press operations or two consecutive touch operations, that a stop call instruction corresponding to a virtual cursor has been received. Based on the received stop call instruction, the virtual cursor can be hidden in the human-computer interaction interface.

[0099] In other embodiments, the user may also send a stop call instruction corresponding to the virtual cursor to the near-eye display device via voice. For example, the user may say "stop calling the virtual cursor" or "hide the virtual cursor," etc., to instruct the near-eye display device to stop calling the virtual cursor. Based on the user's voice, the near-eye display device may determine that it has received a stop call instruction corresponding to the virtual cursor. Based on the received stop call instruction, the virtual cursor may be hidden in the human-computer interaction interface, without limitation.

[0100] In this way, the user can choose whether to stop calling the virtual cursor, which is conducive to improving the flexibility of the near-eye display device in calling or stopping the virtual cursor.

[0101] In some implementations, the display size information of a preset operation object in the human-computer interaction interface is adjusted according to a preset device posture change range.

[0102] For example, when a user wears a near-eye display device and changes the posture of the near-eye display device by turning their head, the rotation angle of the user's head rotation has a certain rotation angle range, and accordingly, the posture of the near-eye display device can also have a certain change range, such as a preset device posture change range. Based on this, before the user determines the target operation object in the human-computer interaction interface of the near-eye display device by adjusting the posture of the near-eye display device, the display size information of the preset operation object in the human-computer interaction interface can be adjusted according to the change range of the near-eye display device's posture, such as the preset device posture change range.

[0103] For example, a near-eye display device can display a preset operation object in a human-computer interaction interface with a larger display size. In this case, the preset operation object displayed with the larger display size can fill the entire human-computer interaction interface. However, if the preset operation object is displayed with a larger display size, if the user needs to change the posture of the near-eye display device to determine the target operation object in the near-eye display device's human-computer interaction interface, the user may need to make excessive head posture changes to trigger the near-eye display device's posture change, resulting in poor convenience for the near-eye display device to determine the target operation object. Based on this, the display size information of the preset operation object can be reduced. After the display size information is reduced, the preset operation object can be arranged only in a portion of the human-computer interaction interface. By reducing the display size information of the preset operation object, the preset operation object can be concentrated in a portion of the human-computer interaction interface, and the user does not need to make excessive head posture changes to trigger the near-eye display device's posture change. The user can more easily select or operate the target operation object from the human-computer interaction interface, which helps improve the convenience of the near-eye display device in determining the target operation object and improve the convenience of the user's control of the near-eye display device.

[0104] For example, a near-eye display device can display a preset operation object in a human-computer interaction interface at a smaller display size. For example, if the user temporarily does not need to operate a preset operation object in the human-computer interaction interface, the preset operation object can be reduced to a smaller display size. However, if the preset operation object is displayed at a smaller display size, and the user needs to change the posture of the near-eye display device to determine the target operation object in the human-computer interaction interface, the user may need to make a small head posture change to trigger the near-eye display device to change its posture, resulting in poor convenience for the near-eye display device to determine the target operation object. Based on this, the display size information of the preset operation object can be increased. After the display size information is increased, the preset operation object can be arranged in a portion of the human-computer interaction interface. By increasing the display size information of the preset operation object, the preset operation object can be dispersed in the human-computer interaction interface, eliminating the need for the user to make small head posture changes to trigger the near-eye display device to change its posture. This makes it easier for the user to select or operate the target operation object in the human-computer interaction interface, which helps improve the convenience of the near-eye display device in determining the target operation object and improves the convenience of the user's control of the near-eye display device.

[0105] S103: Perform a corresponding operation on the target operation object according to the operation information received by the near-eye display device.

[0106] In some embodiments, the near-eye display device can determine whether operation information is received based on at least one of the user's operation on the near-eye display device and the user's operation on a device that has established a communication connection with the near-eye display device, so as to perform a corresponding operation on the target operation object when the operation information is received.

[0107] For example, obtaining posture data of a near-eye display device includes: obtaining posture data of the near-eye display device in response to a change in the head posture of a user wearing the near-eye display device. Performing a corresponding operation on a target operation object based on operation information received by the near-eye display device includes: performing a corresponding operation on the target operation object based on operation information received by a control device corresponding to the near-eye display device; the control device and the near-eye display device have established a communication connection.

[0108] When a user is wearing a near-eye display device, if the user's head posture changes, it can be determined that the user wants to control the near-eye display device by changing the head posture. Based on this, the posture data of the near-eye display device can be obtained in response to the change in the head posture of the user wearing the near-eye display device. For example, when the user's head posture changes, the posture of the near-eye display device will change, and the posture data of the near-eye display device will also change accordingly, then the posture data of the near-eye display device when the head posture of the user wearing the near-eye display device changes can be obtained. Based on the posture data of the near-eye display device when the head posture of the user wearing the near-eye display device changes, the target operation object in the human-computer interaction interface of the near-eye display device is determined, so that the near-eye display device can operate the target operation object.

[0109] For example, the user can determine the operation information of the target operation object through the touch operation, press operation and other operation information of the control device corresponding to the near-eye display device. The control device can be a smart ring, smart bracelet and other devices that have established a communication connection with the near-eye display device, and is not limited here. Taking the control device as a smart ring as an example, when the near-eye display device determines the target operation object in the human-computer interaction interface, if the user touches or presses the smart ring, the near-eye display device can respond to the user's touch operation or press operation on the smart ring to determine that the operation information of the target operation object has been received. The near-eye display device can then perform the corresponding operation on the target operation object based on the received operation information.

[0110] For example, according to the operation information received by the device body of the near-eye display device, a corresponding operation is performed on the target operation object.

[0111] For example, a user can determine operation information on a target operation object through operation information such as touch operation, pressing operation, etc. on a near-eye display device. The near-eye display device may include a device body. For example, if the near-eye display device includes glasses such as AR glasses and VR glasses, and the device body of the near-eye display device includes temples, when the near-eye display device determines the target operation object in the human-computer interaction interface, if the user touches or presses the temples, the near-eye display device can respond to the user's touch operation or pressing operation on the temples of the near-eye display device to determine that the operation information on the target operation object has been received. The near-eye display device can then perform corresponding operations on the target operation object based on the received operation information.

[0112] For example, when the control device corresponding to the near-eye display device does not meet the preset control conditions and receives operation information triggered in the near-eye display device, the corresponding operation is performed on the target operation object according to the operation information triggered in the near-eye display device; the preset control conditions include at least one of the control device receiving the operation information and the power of the control device being greater than or equal to a preset power threshold.

[0113] In some embodiments, when the target operation object is determined, the near-eye display device may preferentially perform the corresponding operation on the target operation object according to the operation information received by the control device corresponding to the near-eye display device. However, when the control device corresponding to the near-eye display device, such as a smart ring, a smart wristband, etc., is lost, the user cannot operate the control device corresponding to the near-eye display device, and the control device corresponding to the near-eye display device cannot receive the operation information. Moreover, when the power of the control device corresponding to the near-eye display device, such as a smart ring, a smart wristband, etc., is too low, if the operation on the target operation object is performed according to the user's operation on the control device, it is easy to cause the power of the smart ring, smart wristband, etc. to be exhausted, making the user temporarily unable to use the smart ring, smart wristband, etc., thereby affecting the user's experience of using the smart ring, smart wristband, etc.

[0114] Based on this, the near-eye display device needs to switch the method of triggering the near-eye display device to operate the target operation object, such as performing corresponding operations on the target operation object based on the operation information received from the control device corresponding to the near-eye display device, and switching to performing corresponding operations on the target operation object based on the operation information received from the device body of the near-eye display device.

[0115] For example, a preset control condition can be set for the control device corresponding to the near-eye display device. The preset control condition includes at least one of the control device receiving operation information and the power of the control device being greater than or equal to a preset power threshold. When the control device meets the control condition, the corresponding operation can be performed on the target operation object based on the operation information received by the control device. When the control device does not meet the preset control condition, it can be switched to performing the corresponding operation on the target operation object based on the operation information received by the device body of the near-eye display device, that is, switching to performing the corresponding operation on the target operation object based on the operation information triggered by the user in the near-eye display device. Among them, the preset power threshold can be pre-set or set by the user, and is not limited here.

[0116] It is understandable that since the near-eye display device and the control device have established a communication connection, the near-eye display device can obtain the power of the control device. When the control device and the near-eye display device are disconnected, the near-eye display device cannot receive the operation information sent by the control device. Therefore, if the near-eye display device does not receive the operation information sent by the control device for a period of time, it can be considered that the near-eye display device has not received the operation information.

[0117] Take the example of a smart ring as the control device corresponding to the near-eye display device. For example, when the smart ring is lost, that is, the smart ring does not receive the operation information, if the device body of the near-eye display device receives the operation information, then the corresponding operation can be performed on the target operation object according to the operation information received by the device body of the near-eye display device. For another example, when the power of the smart ring is less than a preset power threshold, if the device body of the near-eye display device receives the operation information, then the corresponding operation can be performed on the target operation object according to the operation information received by the device body of the near-eye display device. For another example, when the smart ring receives the operation information and the power of the smart ring is less than the preset power threshold, if the device body of the near-eye display device also receives the operation information, then the corresponding operation can be performed on the target operation object according to the operation information received by the device body of the near-eye display device.

[0118] Of course, the preset control conditions are not limited to this. For example, the preset control conditions can be set to include at least one of the following: the control device receives operation information, the power of the control device is greater than or equal to a preset power threshold, and the control device is not in power saving mode. For example, when the smart ring receives operation information and the smart ring is in power saving mode, if the device body of the near-eye display device also receives operation information, then the corresponding operation can be performed on the target operation object based on the operation information received by the device body of the near-eye display device. The preset control conditions can be pre-set or set by the user, and are not limited here.

[0119] Based on switching the method of triggering the near-eye display device to operate the target operation object, when the control device corresponding to the near-eye display device is lost or has low battery, the near-eye display device can still perform corresponding operations on the target operation object based on the operation information received by the device body of the near-eye display device.

[0120] Of course, this is not limited to this. After determining the target operation object, the near-eye display device can also determine whether the operation information is received by detecting the user's voice information, gesture information, etc., so as to perform corresponding operations on the target operation object when the operation information is received. There is no limitation here.

[0121] In this way, the near-eye display device can trigger the execution of corresponding operations on the target operation object based on the operation information received by the control device corresponding to the near-eye display device, and can also trigger the execution of corresponding operations on the target operation object based on the operation information received by the device body of the near-eye display device. In addition, when the control device corresponding to the near-eye display device is lost or has low battery, the method of triggering the near-eye display device to operate the target operation object can be switched, which is conducive to improving the convenience of the near-eye display device in operating the target operation object, and is conducive to improving the operational flexibility of the near-eye display device on the target operation object.

[0122] In some embodiments, when the target operation object is an icon, the corresponding operation performed by the near-eye display device on the application icon may include displaying the interactive interface corresponding to the icon, such as displaying the interactive interface corresponding to the icon on the upper layer of the human-computer interaction interface, or switching the human-computer interaction interface to the interactive interface corresponding to the icon, etc., which are not limited here. When the target operation object is text, the corresponding operation performed by the near-eye display device on the text may include selecting, editing, etc., which are not limited here. When the target operation object is a real operation object, the corresponding operation performed by the near-eye display device on the real operation object may include selecting virtual special effects to be added for the real operation object, modifying the virtual special effects of the real operation object, etc., which are not limited here.

[0123] In an exemplary embodiment, as shown in FIG3 , when the virtual cursor moves to the left to a preset operation object 1 in response to a change in the posture of the near-eye display device, the user can press or touch the confirmation key on the smart ring to send operation information for the preset operation object 1 to the near-eye display device. The near-eye display device can then perform a corresponding operation on the preset operation object 1 based on the received operation information. Of course, this is not limited to this. For example, the confirmation key can include a physical button, a touch area, a gesture, etc., which are not limited here.

[0124] In this way, according to the operation information received by the near-eye display device, the corresponding operation is performed on the target operation object. The user does not need to raise his arm for a long time to operate the target operation object in the human-computer interaction interface of the near-eye display device. The user is less likely to feel arm fatigue, which is beneficial to improving the convenience of operating the target operation object of the near-eye display device and improving the convenience of controlling the near-eye display device, thereby improving the user's interactive experience with the near-eye display device.

[0125] The above-mentioned embodiment provides a method for controlling a near-eye display device, which includes: obtaining posture data of the near-eye display device; determining a target operation object in a human-computer interaction interface of the near-eye display device based on the posture data of the near-eye display device; and performing a corresponding operation on the target operation object based on the operation information received by the near-eye display device. Through the obtained posture data of the near-eye display device, the near-eye display device can determine the user's operation requirements for the human-computer interaction interface of the near-eye display device, such as determining the target operation object in the human-computer interaction interface. When the target operation object is determined, the corresponding operation can be performed on the target operation object based on the operation information received by the near-eye display device, thereby achieving control of the near-eye display device. During the process of controlling the near-eye display device, since the near-eye display device is worn on the user's head, the user can adjust the posture of the near-eye display device by changing the posture of the head. In response to the change in the posture of the near-eye display device, the posture data of the near-eye display device obtained will also change. Then, based on the obtained posture data of the near-eye display device, subsequent control operations of the near-eye display device are performed, which is conducive to improving the control convenience of the near-eye display device. For example, users do not need to raise their arms for a long time to control the near-eye display device, so they are less likely to feel arm fatigue, which is conducive to improving the user's interactive experience with the near-eye display device.

[0126] Please refer to Figure 5, which is a schematic block diagram of a control device for a near-eye display device provided in an embodiment of the present application. The control device for the near-eye display device can be configured in the near-eye display device or a server to execute the aforementioned control method for the near-eye display device.

[0127] As shown in FIG5 , the control device of the near-eye display device includes: an acquisition module 110 , an operation object determination module 120 and an operation module 130 .

[0128] The acquisition module 110 is configured to acquire posture data of a near-eye display device.

[0129] The operation object determination module 120 is configured to determine a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device.

[0130] The operation module 130 is configured to perform a corresponding operation on the target operation object according to the operation information received by the near-eye display device.

[0131] Exemplarily, the operation object determination module 120 includes a position adjustment submodule and a determination submodule.

[0132] The position adjustment submodule is used to adjust the position of the virtual cursor in the human-computer interaction interface according to the posture data.

[0133] The determination submodule is configured to determine that the preset operation object is a target operation object when the distance between the position of the virtual cursor in the human-computer interaction interface and the position of the preset operation object in the human-computer interaction interface is reduced to less than or equal to a preset distance threshold.

[0134] Exemplarily, the control device of the near-eye display device further includes a cursor display submodule.

[0135] The cursor display submodule is configured to display the virtual cursor in the human-computer interaction interface when the near-eye display device receives a call instruction corresponding to the virtual cursor.

[0136] Exemplarily, the control device of the near-eye display device further includes a cursor hiding submodule.

[0137] The cursor hiding submodule is configured to hide the virtual cursor in the human-computer interaction interface within a preset time period after the virtual cursor is displayed in the human-computer interaction interface if it is determined that the posture data of the near-eye display device has not changed.

[0138] Exemplarily, the control device of the near-eye display device further includes a cursor adsorption submodule.

[0139] The adsorption submodule is used to adsorb the virtual cursor in the human-computer interaction interface onto the target operation object.

[0140] Exemplarily, the control device of the near-eye display device further includes a display form determination submodule and a display effect determination submodule.

[0141] A display form determination submodule, configured to determine, according to the object type of the target operation object, a display form of the virtual cursor when it is adsorbed on the target operation object;

[0142] The display effect determination submodule is used to determine the display effect when the virtual cursor is adsorbed on the target operation object according to the interactive control information of the target operation object.

[0143] Exemplarily, the control device of the near-eye display device further includes an interface adjustment submodule.

[0144] The interface adjustment submodule is used to adjust the display size information of the preset operation object in the human-computer interaction interface according to the preset device posture change range.

[0145] Exemplarily, the acquisition module 110 includes a first acquisition submodule.

[0146] The first acquisition submodule is configured to acquire posture data of the near-eye display device in response to a change in a head posture of a user wearing the near-eye display device.

[0147] The operation module 130 includes a first operation submodule.

[0148] The first operation submodule is configured to perform a corresponding operation on the target operation object according to operation information received by a control device corresponding to the near-eye display device; the control device establishes a communication connection with the near-eye display device.

[0149] Exemplarily, the operation module 130 includes a second operation submodule.

[0150] The second operation sub-module is used to perform a corresponding operation on the target operation object according to the operation information triggered in the near-eye display device when the control device corresponding to the near-eye display device does not meet the preset control conditions and receives the operation information triggered in the near-eye display device; the preset control conditions include at least one of the control device receiving the operation information and the power of the control device being greater than or equal to a preset power threshold.

[0151] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0152] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0153] For example, the above methods and apparatuses may be implemented as a computer program that can be run on a near-eye display device. For example, the near-eye display device may include VR glasses, AR glasses, VR helmets, AR helmets, etc., without limitation.

[0154] Please refer to FIG. 6 , which is a schematic block diagram of the structure of a near-eye display device provided in an embodiment of the present application.

[0155] As shown in Figure 6, the near-eye display device includes a memory and a processor. The memory and the processor may be connected via a system bus, and the memory may include a storage medium and an internal memory.

[0156] The storage medium may store an operating system and a computer program. When the computer program is executed, the processor may execute any method for controlling a near-eye display device.

[0157] The processor is used to provide computing and control capabilities to support the operation of the entire near-eye display device.

[0158] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any control method of the near-eye display device.

[0159] Those skilled in the art will understand that the structure shown in Figure 6 is merely a block diagram of a portion of the structure related to the scheme of the present application, and does not constitute a limitation on the near-eye display device to which the scheme of the present application is applied. The specific near-eye display device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0160] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0161] In one embodiment, the processor is configured to execute a computer program and implement the following steps when executing the computer program:

[0162] Obtaining posture data of a near-eye display device;

[0163] determining a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device;

[0164] According to the operation information received by the near-eye display device, a corresponding operation is performed on the target operation object.

[0165] It should be noted that, those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the control of the near-eye display device described above can refer to the corresponding process in the aforementioned embodiment of the control method of the near-eye display device, and will not be repeated here.

[0166] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the control method of the near-eye display device of the present application.

[0167] The computer-readable storage medium may be an internal storage unit of the near-eye display device described in the aforementioned embodiment, such as a hard disk or memory of the near-eye display device. The computer-readable storage medium may also be an external storage device of the near-eye display device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the near-eye display device.

[0168] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0169] It should also be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0170] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for controlling a near-eye display device, comprising: Obtaining posture data of a near-eye display device; Determining a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device; According to the operation information received by the near-eye display device, a corresponding operation is performed on the target operation object.

2. The control method according to claim 1, wherein: The step of determining a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device comprises: According to the gesture data, adjusting the position of the virtual cursor in the human-computer interaction interface; When the distance between the position of the virtual cursor in the human-computer interaction interface and the position of a preset operation object in the human-computer interaction interface is reduced to be less than or equal to a preset distance threshold, the preset operation object is determined to be a target operation object.

3. The control method according to claim 2, wherein: The control method further comprises: When the near-eye display device receives a call instruction corresponding to the virtual cursor, the virtual cursor is displayed in the human-computer interaction interface.

4. The control method according to claim 3, wherein: The control method further comprises: Within a preset time period after the virtual cursor is displayed in the human-computer interaction interface, if it is determined that the posture data of the near-eye display device has not changed, the virtual cursor is hidden in the human-computer interaction interface.

5. The control method according to claim 1, wherein: After determining a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device, the control method further includes: Adsorbing the virtual cursor in the human-computer interaction interface onto the target operation object.

6. The control method according to claim 5, wherein: After the virtual cursor in the human-computer interaction interface is adsorbed onto the target operation object, the control method further includes: Determining, according to the object type of the target operation object, a display form of the virtual cursor when it is adsorbed on the target operation object; and / or The display effect when the virtual cursor is adsorbed on the target operation object is determined according to the interactive control information of the target operation object.

7. The control method according to claim 1, wherein: Before determining the target operation object in the human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device, the control method further includes: According to a preset device posture change range, the display size information of the preset operation object in the human-computer interaction interface is adjusted.

8. The control method according to claim 1, wherein: The acquiring of the posture data of the near-eye display device includes: In response to a change in a head posture of a user wearing the near-eye display device, acquiring posture data of the near-eye display device; The performing a corresponding operation on the target operation object according to the operation information received by the near-eye display device includes: According to the operation information received by the control device corresponding to the near-eye display device, a corresponding operation is performed on the target operation object; and a communication connection is established between the control device and the near-eye display device.

9. The control method according to claim 1, wherein: The performing a corresponding operation on the target operation object according to the operation information received by the near-eye display device includes: When the control device corresponding to the near-eye display device does not meet the preset control condition and receives the operation information triggered in the near-eye display device, performing the corresponding operation on the target operation object according to the operation information triggered in the near-eye display device; The preset control condition includes at least one of the following: the control device receives operation information, and the power of the control device is greater than or equal to a preset power threshold.

10. A control device for a near-eye display device, wherein: The control device comprises: An acquisition module, used to acquire posture data of a near-eye display device; An operation object determination module, used to determine a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device; An operation module is used to perform a corresponding operation on the target operation object according to the operation information received by the near-eye display device.

11. The control device according to claim 10, wherein: The operation object determination module includes a position adjustment submodule and a determination submodule; The position adjustment submodule is used to adjust the position of the virtual cursor in the human-computer interaction interface according to the posture data; The determination submodule is used to determine that the preset operation object is the target operation object when the distance between the position of the virtual cursor in the human-computer interaction interface and the position of the preset operation object in the human-computer interaction interface is reduced to less than or equal to a preset distance threshold.

12. The control device according to claim 11, wherein: The control device also includes a cursor display submodule; The cursor display submodule is used to display the virtual cursor in the human-computer interaction interface when the near-eye display device receives a call instruction corresponding to the virtual cursor.

13. The control device according to claim 12, wherein: The control device also includes a cursor hiding submodule; The cursor hiding submodule is used to hide the virtual cursor in the human-computer interaction interface within a preset time period after the virtual cursor is displayed in the human-computer interaction interface if it is determined that the posture data of the near-eye display device has not changed.

14. The control device according to claim 10, wherein: The control device also includes a cursor adsorption submodule; The cursor adsorption submodule is used to adsorb the virtual cursor in the human-computer interaction interface onto the target operation object.

15. The control device according to claim 14, wherein: The control device also includes a display form determination submodule and a display effect determination submodule; The display form determination submodule is used to determine the display form of the virtual cursor when it is adsorbed on the target operation object according to the object type of the target operation object; The display effect determination submodule is used to determine the display effect when the virtual cursor is adsorbed on the target operation object according to the interactive control information of the target operation object.

16. The control method according to claim 10, wherein: The control device also includes an interface adjustment submodule; The interface adjustment submodule is used to adjust the display size information of the preset operation object in the human-computer interaction interface according to the preset device posture change range.

17. The control device according to claim 10, wherein: The acquisition module includes a first acquisition submodule; The first acquisition submodule is used to acquire the posture data of the near-eye display device in response to the change of the head posture of the user wearing the near-eye display device; The operation module includes a first operation submodule; The first operation submodule is used to perform a corresponding operation on the target operation object according to the operation information received by the control device corresponding to the near-eye display device; the control device establishes a communication connection with the near-eye display device.

18. The control device according to claim 10, wherein: The operation module includes a second operation submodule; The second operation submodule is used to perform a corresponding operation on the target operation object according to the operation information triggered in the near-eye display device when the control device corresponding to the near-eye display device does not meet the preset control conditions and receives the operation information triggered in the near-eye display device; the preset control conditions include at least one of the control device receiving the operation information and the power of the control device being greater than or equal to a preset power threshold.

19. A near-eye display device, characterized in that: The near-eye display device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the following steps when executing the computer program: Obtaining posture data of a near-eye display device; Determining a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device; According to the operation information received by the near-eye display device, a corresponding operation is performed on the target operation object.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the following steps are implemented: Obtaining posture data of a near-eye display device; Determining a target operation object in a human-computer interaction interface of the near-eye display device according to the posture data of the near-eye display device; According to the operation information received by the near-eye display device, a corresponding operation is performed on the target operation object.

Images