Control method and device of near-to-eye display equipment, equipment and storage medium

Abstract

The invention relates to the technical field of equipment control, and provides a control method and device of near-eye display equipment, equipment and a storage medium, which are applied to the near-eye display equipment, and the near-eye display equipment is in communication connection with wearable equipment. The control method comprises the steps that when a first preset operation on the wearable device is detected, the near-eye display device is controlled to display a preset interaction interface corresponding to the wearable device; the preset interaction interface comprises at least one preset interaction object; determining a target interaction object from at least one preset interaction object according to the equipment posture of the wearable equipment; and controlling the near-to-eye display device to execute a function corresponding to the target interaction object, so as to improve the operation convenience of the wearable device and the control convenience of the near-to-eye display device.

Description

Technical Field

[0001] The present application relates to the field of device control technology, and in particular to a control method, apparatus, device and storage medium for a near-eye display device. Background Art

[0002] At present, when a user wears a near-eye display device to use the near-eye display device, the user can control the near-eye display device with the help of a wearable device, such as a finger ring, a bracelet, etc. However, in the related art, the near-eye display device's own display interface is generally interacted with through a wearable device, which easily causes the user to perform cumbersome operations on the wearable device in coordination with the near-eye display device's own display interface, resulting in poor convenience in operating the wearable device and poor convenience in controlling the near-eye display device. Summary of the invention

[0003] The main purpose of the present application is to provide a control method, apparatus, device and storage medium for a near-eye display device, aiming to solve the technical problems of poor operation convenience of the wearable device and poor control convenience of the near-eye display device due to the need for users to cooperate with the display interface of the near-eye display device itself to perform cumbersome operations on the wearable device.

[0004] In a first aspect, the present application provides a control method for a near-eye display device, which is applied to a near-eye display device, wherein the near-eye display device is communicatively connected to a wearable device; the control method comprises:

[0005] When a first preset operation on the wearable device is detected, controlling the near-eye display device to display a preset interaction interface corresponding to the wearable device; the preset interaction interface includes at least one preset interaction object;

[0006] Determining a target interaction object from the at least one preset interaction object according to the device posture of the wearable device;

[0007] Control the near-eye display device to execute the function corresponding to the target interactive object.

[0008] In a second aspect, the present application provides a control device for a near-eye display device, the control device comprising:

[0009] A first control module, configured to control a near-eye display device to display a preset interaction interface corresponding to the wearable device when a first preset operation on the wearable device is detected; the preset interaction interface includes at least one preset interaction object;

[0010] An object determination module, configured to determine a target interaction object from the at least one preset interaction object according to a device posture of the wearable device;

[0011] The second control module is used to control the near-eye display device to execute the function corresponding to the target interactive object.

[0012] In a third aspect, the present application provides a near-eye display device, the near-eye display device comprising a memory and a processor;

[0013] The memory is used to store computer programs;

[0014] The processor is used to execute the computer program and implement the steps of the control method of the near-eye display device as described above when executing the computer program.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the control method of the near-eye display device as described above are implemented.

[0016] The present application provides a control method, apparatus, device and storage medium for a near-eye display device, which are applied to the near-eye display device, wherein the near-eye display device is communicatively connected with a wearable device; the control method comprises: when a first preset operation on the wearable device is detected, controlling the near-eye display device to display a preset interaction interface corresponding to the wearable device; the preset interaction interface comprises at least one preset interaction object; according to the device posture of the wearable device, determining a target interaction object from at least one preset interaction object; and controlling the near-eye display device to execute a function corresponding to the target interaction object.

[0017] When the near-eye display device displays a preset interactive interface, since the near-eye display device is communicatively connected with the wearable device and the preset interactive interface is adapted to the wearable device, in response to changes in the device posture of the wearable device, the near-eye display device can perform corresponding interactive operations with the preset interactive interface to achieve control of the functions performed by the near-eye display device, thereby helping to improve the convenience of operating the wearable device and improving the convenience of controlling the near-eye display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 is a flow chart of a control method of a near-eye display device provided in an embodiment of the present application;

[0020] Figure 2It is a schematic diagram of a preset interactive interface involved in the first embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a preset interactive interface involved in another embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a preset interactive interface involved in yet another embodiment of the present application;

[0023] Figure 5 is a schematic diagram of a preset interactive interface involved in yet another embodiment of the present application;

[0024] Figure 6 is a schematic block diagram of a control device for a near-eye display device provided in an embodiment of the present application;

[0025] Figure 7 It is a schematic block diagram of a near-eye display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0028] The embodiments of the present application provide a control method, device, equipment and storage medium for a near-eye display device. The control method of the near-eye display device can be applied to a near-eye display device. The near-eye display device may include augmented reality (AR) glasses, mixed reality (MR) glasses, AR helmets, MR helmets, etc., without limitation.

[0029] The near-eye display device can be connected to a wearable device for communication. The wearable device may include a smart ring, a smart bracelet, a smart watch, etc., which are not limited here.

[0030] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0031] See also Figure 1 , Figure 1 It is a flow chart of a control method of a near-eye display device provided in an embodiment of the present application. It should be noted that the control method of a near-eye display device provided in an embodiment of the present application can be used in a near-eye display device, which is not limited here.

[0032] like Figure 1 As shown, the control method of the near-eye display device includes steps S101 to S103.

[0033] S101. When a first preset operation on a wearable device is detected, control a near-eye display device to display a preset interaction interface corresponding to the wearable device; the preset interaction interface includes at least one preset interaction object.

[0034] When a communication connection is established between the near-eye display device and the wearable device, a user wearing the near-eye display device and the wearable device can control the near-eye display device by operating the wearable device.

[0035] For example, the user can perform a first preset operation on the wearable device. When the wearable device detects the user's first preset operation, the wearable device can feed back to the near-eye display device based on the communication connection between the near-eye display device and the wearable device, and the near-eye display device can detect the first preset operation on the wearable device. The first preset operation may include a first button operation, a first touch operation, and the like. For example, when the near-eye display device displays the initial display interface, if the user performs a button operation on a button set on the wearable device, the wearable device can determine that the first button operation is detected. For another example, when the near-eye display device displays the initial display interface, if the user performs a touch operation on the touch area set on the wearable device, the wearable device can determine that the first touch operation is detected. The first preset operation can be pre-set or set by the user, which is not limited here.

[0036] Accordingly, when the first preset operation on the wearable device is detected, the near-eye display device may determine that the user needs to interact with the near-eye display device through the wearable device, and the near-eye display device may display the preset interaction interface corresponding to the wearable device. The preset interaction interface includes at least one preset interaction object, and based on the preset interaction interface displayed by the near-eye display device, the user can interact with at least one preset interaction object through the wearable device. The preset interaction object included in the preset interaction interface may be determined according to the initial display interface of the near-eye display device. For example, when the user edits a document through the near-eye display device, the initial display interface may include a document editing interface. Accordingly, the preset interaction object may include document insertion identification, page setting identification, review setting identification, and other identification information corresponding to functions related to document editing, which is not limited here. For example, when the user plays a game through the near-eye display device, the initial display interface may include a game interface. Accordingly, when the user controls the near-eye display device through the wearable device, the preset interaction object may include mobile control identification, game screen setting identification, game interaction setting identification, and other identification information corresponding to functions related to the game, which is not limited here. The identification information may include text identification, icon identification, etc., which is not limited here.

[0037] In an exemplary embodiment, the near-eye display device can detect whether the wearable device is in an on state based on a communication connection with the wearable device. When it is determined that the wearable device is in an on state, if the near-eye display device detects a first preset operation on the wearable device, the preset interaction interface corresponding to the wearable device can be displayed.

[0038] Based on the fact that the preset interaction interface is the preset interaction interface corresponding to the wearable device, the preset interaction interface can be designed to determine the arrangement of the preset interaction objects in conjunction with the usage of the wearable device, so that the preset interaction interface can be adapted to the wearable device, thereby improving the convenience of operating the wearable device and improving the convenience of controlling the near-eye display device.

[0039] In some implementations, the interface shape of the preset interactive interface is the same as the shape of the wearable device.

[0040] For example, the shape of the wearable device may be determined according to the shape of one of the cross sections of the wearable device. For example, the wearable device includes a smart ring. Figure 2As shown, when one of the cross sections of the smart ring is a circular ring, it can be determined that the shape of the wearable device is a circular ring. Accordingly, the interface shape of the preset interaction interface can be set to a circular ring according to the shape of the wearable device. When the preset interaction interface includes at least one preset interaction object, the at least one preset interaction object can be respectively located in different preset direction ranges of the preset interaction interface, and the preset interaction object has a corresponding preset direction range.

[0041] When the interface shape of the preset interaction interface is the same as the shape of the wearable device, when the device posture of the wearable device changes subsequently, the near-eye display device can determine whether the direction indicated by the device posture of the wearable device is within the preset direction range corresponding to the preset interaction object.

[0042] like Figure 2 As shown, the preset interactive objects may include interactive objects 1 to 4. The preset direction range corresponding to interactive object 1 is 0 degrees to 90 degrees, the preset direction range corresponding to interactive object 2 is 90 degrees to 180 degrees, the preset direction range corresponding to interactive object 3 is 180 degrees to 270 degrees, and the preset direction range corresponding to interactive object 4 is 270 degrees to 360 degrees. Taking the wearable device as a smart ring as an example, when the user changes the posture of the finger wearing the wearable device, such as changing the direction of the finger, the device posture of the wearable device will change accordingly. For example, the direction indicated by the device posture will change in response to the change in the direction of the finger. Based on this, the near-eye display device can determine whether the direction indicated by the device posture is within the preset direction range corresponding to one of the preset interactive objects of interactive objects 1 to interactive objects 4, so as to subsequently determine the target interactive object.

[0043] When the direction indicated by the device posture is within the direction range corresponding to the preset interaction object, the near-eye display device can subsequently determine the target interaction object based on the preset interaction object.

[0044] In this way, when the interface shape of the preset interaction interface is the same as the shape of the wearable device, the user can subsequently change the device posture of the wearable device according to the preset direction range of the preset interaction object in the preset interaction interface, thereby realizing interaction with the near-eye display device. Based on this, the direction indicated by the device posture can correspond to the preset direction range of the preset interaction object, which is conducive to improving the convenience of operating the wearable device and improving the convenience of controlling the near-eye display device.

[0045] Exemplarily, the preset interaction interface includes a first interaction interface and a second interaction interface.

[0046] In some embodiments, when a first preset operation on the wearable device is detected, the near-eye display device is controlled to display a first interaction interface; when the direction indicated by the device posture is within a preset direction range corresponding to the first interaction object in the first interaction interface, and a second preset operation on the wearable device is detected, the near-eye display device is controlled to display a second interaction interface; the second interaction interface includes a second interaction object corresponding to the first interaction object.

[0047] For example, the first interactive object included in the first interactive interface may have a primary function. The second interactive object included in the second interactive interface may have a secondary function. The secondary function may be a refined function of the primary function. The secondary functions corresponding to different first interactive objects may be different, and accordingly, the second interactive objects corresponding to different first interactive objects may also be different. The preset interactive object included in the preset interactive interface may be determined according to the initial display interface of the near-eye display device. For example, when a user edits a document through a near-eye display device, the initial display interface may include a document editing interface. Accordingly, when a user controls the near-eye display device through a wearable device, the primary function of the first interactive object included in the first interactive interface may include a document insert function, a document page setting function, a document review setting function, and other functions related to document editing, which are not limited here. Taking the first function including the document insert function as an example, the secondary function of the second interactive object may include a table insert function, a picture insert function, a screenshot insert function, a shape insert function, and other refined functions related to the document insert function, which are not limited here. For example, when a user plays a game through a near-eye display device, the initial display interface may include a game interface. Accordingly, when a user controls a near-eye display device through a wearable device, the primary functions of the first interactive object may include mobile control functions, game screen setting functions, game interaction setting functions, and other game-related functions. Taking the game interaction setting function as an example, the secondary functions of the second interactive object may include manual selection of interactive object mode, automatic selection of interactive object mode, and other detailed functions related to game interaction settings, which are not limited here.

[0048] If the near-eye display device displays all first interactive objects and all second interactive objects at the same time, then as the number of first interactive objects and / or second interactive objects increases, the preset direction ranges corresponding to each of the first interactive objects and the second interactive objects will correspondingly become smaller, resulting in poor operating convenience of the wearable device and poor control convenience of the near-eye display device.

[0049] Based on this, when the near-eye display device detects the first preset operation on the wearable device, it can first control the near-eye display device to display the first interaction interface. The first interaction interface may include multiple first interaction objects. In the case where the second interaction object is an extended function of the corresponding first interaction object, the near-eye display device reduces the number of preset interaction objects in the preset interaction interface by preferentially displaying the first interaction object, which is conducive to simplifying the interface displayed by the near-eye display device.

[0050] Accordingly, the near-eye display device can subsequently determine whether it is necessary to display the second interaction interface corresponding to the first interaction object based on the operation of the wearable device on the first interaction object, thereby displaying the second interaction interface according to the user's needs, which is conducive to improving the convenience of interaction with the near-eye display device.

[0051] For example, when displaying the first interactive interface, the near-eye display device can obtain the device posture of the wearable device and determine whether the direction indicated by the device posture is within the preset direction range corresponding to the first interactive object. If the direction indicated by the device posture is within the preset direction range corresponding to the first interactive object, and the second preset operation on the wearable device is detected, the near-eye display device can determine that the user needs to interact with the second interactive object corresponding to the first interactive object, so that the near-eye display device can display the second interactive interface. The second interactive interface includes the second interactive object corresponding to the first interactive object.

[0052] The second preset operation may include a duration in which the posture of the device remains unchanged that is greater than or equal to a preset duration threshold, a second button operation, a second touch operation, and the like, which are not limited here. For example, when the near-eye display device displays the first interactive interface, if the user performs a button operation on a button set on the wearable device, the wearable device can determine that a second button operation is detected. For another example, when the near-eye display device displays the first interactive interface, if the user performs a click touch operation on a touch area set on the wearable device, the wearable device can determine that a second touch operation is detected. The second preset operation may be pre-set or may be set by the user, which is not limited here. Based on the fact that the display interface of the near-eye display device corresponding to the first preset operation is the initial display interface, the display interface of the near-eye display device corresponding to the second preset operation is the first interactive interface, and the display interfaces of the near-eye display devices corresponding to the first preset operation and the second preset operation are different, the first preset operation and the second preset operation may be set to the same operation or different operations, which are not limited here.

[0053] In some implementations, the second interaction interface is displayed within an extended position range corresponding to the first interaction interface.

[0054] In the case where the second interactive object is an extended function of the first interactive object, the near-eye display device can determine the extended position range corresponding to the first interactive interface based on the position range of the first interactive interface. Accordingly, the near-eye display device can display the second interactive interface within the extended position range corresponding to the first interactive interface to indicate the association relationship between the first interactive interface and the second interactive interface.

[0055] For example, the near-eye display device may determine the extended position range corresponding to the first interactive interface according to the interface shape of the first interactive interface.

[0056] like Figure 3 As shown, when the interface shape of the first interactive interface is a circular ring, the near-eye display device can determine the extended position range corresponding to the first interactive interface according to the ring width of the ring corresponding to the first interactive interface. For example, the large radius corresponding to the extended position range is obtained by adding the ring width of the ring to the large radius of the ring corresponding to the first interactive interface. Correspondingly, the near-eye display device can determine the large radius of the ring corresponding to the first interactive interface as the small radius corresponding to the extended position range. Based on this, the extended position can be obtained according to the large radius and the small radius corresponding to the extended position range. Correspondingly, the near-eye display device can display the second interactive interface within the extended position range. Of course, the method for determining the extended position range is not limited to this. For example, when the interface shape of the first interactive interface is a rectangle, the near-eye display device can determine the extended position range corresponding to the first interactive interface according to the side length of the rectangle, and so on, which is not limited here.

[0057] In this way, by displaying the second interaction interface within the extended position range corresponding to the first interaction interface, the user can clearly understand the association relationship between the first interaction interface and the second interaction interface when viewing the first interaction interface and the second interaction interface displayed by the near-eye display device, and thus can interact with the corresponding first interaction object or second interaction object based on the association relationship, which is conducive to improving the convenience of interaction with the preset interaction objects.

[0058] In some implementations, the second interaction interface is displayed superimposed on the first interaction interface.

[0059] Since the near-eye display device displays the second interaction interface based on the direction indicated by the device posture of the wearable device being within the preset direction range corresponding to the first interaction object in the first interaction interface, and detects the second preset operation on the wearable device, the near-eye display device can determine that the user needs to interact with the second interaction interface, but does not need to interact with the first interaction interface temporarily. Based on this, the near-eye display device can hide the first interaction interface while displaying the second interaction interface.

[0060] like Figure 4As shown, the near-eye display device can display the second interactive interface on top of the first interactive interface, so that the first interactive interface can be blocked by the second interactive interface, thereby achieving a hiding effect.

[0061] In this way, by superimposing and displaying the second interaction interface on the first interaction interface, the near-eye display device can hide the first interaction interface while displaying the second interaction interface, so as to simplify the interface displayed by the near-eye display device.

[0062] Of course, the present invention is not limited thereto. After controlling the near-eye display device to display the second interactive interface, the near-eye display device may also hide the second interactive interface.

[0063] In some embodiments, when a hiding instruction for the second interaction interface is detected, the near-eye display device is controlled to switch the second interaction interface to the first interaction interface.

[0064] In the case where the user is not familiar with the second interactive object included in the second interactive interface, the user can control the near-eye display device to display the second interactive interface by changing the device posture of the wearable device so that the direction indicated by the device posture is within the preset direction range corresponding to the first interactive object in the first interactive interface, and performing a second preset operation on the wearable device. In the case where the user views the second interactive object included in the second interactive interface, if the second interactive object does not contain the object that the user needs to interact with, the user needs to control the near-eye display device to return to displaying the first interactive interface.

[0065] For example, a user may send a hide instruction for the second interaction interface to the near-eye display device through a wearable device. The near-eye display device may then control the near-eye display device to switch the second interaction interface to the first interaction interface in response to the detected hide instruction for the second interaction interface.

[0066] like Figure 3 or Figure 4 As shown, the second interaction interface also includes a hidden identifier corresponding to the second interaction interface. The hidden identifier can be pre-set or set by the user, which is not limited here. When the direction indicated by the device posture of the wearable device is within the preset direction range corresponding to the hidden identifier, and a second preset operation on the wearable device is detected, the near-eye display device can determine that a hidden instruction for the second interaction interface is detected. Of course, it is not limited to this. When the position of the preset interaction identifier corresponding to the wearable device is within the preset position range corresponding to the hidden identifier, and a second preset operation on the wearable device is detected, the near-eye display device can determine that a hidden instruction for the second interaction interface is detected, which is not limited here. In response to detecting the hidden instruction for the second interaction interface, the near-eye display device can switch the second interaction interface to the first interaction interface to hide the second interaction interface.

[0067] In this way, when a hiding instruction for the second interactive interface is detected, the near-eye display device is controlled to switch the second interactive interface to the first interactive interface. The near-eye display device can determine the user's need to display or hide at least one of the first interactive interface and the second interactive interface based on the device posture of the wearable device and the detected second preset operation on the wearable device, combined with the first interactive interface or the second interactive interface currently displayed by the near-eye display device. This is conducive to improving the convenience of displaying the preset interactive interface and improving the convenience of switching the preset interactive interface, thereby helping to improve the convenience of controlling the near-eye display device.

[0068] In some embodiments, the near-eye display device itself may have an initial display interface. In the case where the near-eye display device does not need to be controlled by the wearable device, the near-eye display device may display the initial display interface.

[0069] For example, when the first preset operation on the wearable device is not detected, the near-eye display device is controlled to display an initial display interface.

[0070] Of course, it is not limited to this. In the case of controlling the near-eye display device to display the preset interactive interface corresponding to the wearable device, if the near-eye display device detects the third preset operation on the wearable device, the preset interactive interface can be switched to the initial display interface. The third preset operation includes a third button operation, a third touch operation, and the like, which are not limited here. For example, in the case where the near-eye display device displays the preset interactive interface, if the user performs two button operations on the button set on the wearable device, the wearable device can determine that the third button operation is detected. For another example, in the case where the near-eye display device displays the preset interactive interface, if the user performs a return touch operation on the touch area set on the wearable device, the wearable device can determine that the third touch operation is detected. The third preset operation can be pre-set or set by the user, which is not limited here. Based on the display interface of the near-eye display device corresponding to the first preset operation is the initial display interface, the display interface of the near-eye display device corresponding to the third preset operation is the preset interactive interface, and the display interfaces of the near-eye display devices corresponding to the first preset operation and the third preset operation are different, then the first preset operation and the third preset operation can be set to the same operation or different operations, which are not limited here. Based on the fact that the display interface of the near-eye display device corresponding to the second preset operation is the first interactive interface, the display interfaces of the near-eye display devices corresponding to the second preset operation and the third preset operation can be the same, then the second preset operation and the third preset operation can be set to different operations to avoid conflicts in controlling the functions executed by the near-eye display device.

[0071] S102: Determine a target interaction object from at least one preset interaction object according to a device posture of the wearable device.

[0072] For example, since the wearable device can be worn at a preset part of the user, the device posture of the wearable device will change in response to a change in the posture of the preset part of the user.

[0073] Take the wearable device as a smart ring as an example. When the posture of the finger wearing the smart ring changes, the device posture of the smart ring also changes accordingly. For example, when the initial state of the finger is pointing directly in front of the user, if the user straightens the finger and lifts the finger in the upper left direction or the user drives the finger to lift in the upper left direction with the wrist, the device posture of the smart ring will move to the upper left direction accordingly. If the user straightens the finger and puts it down in the lower right direction or the user drives the finger to put it down in the lower right direction with the wrist, the device posture of the smart ring will move to the lower right direction accordingly, and so on.

[0074] When a first preset operation on the wearable device is detected, the near-eye display device may determine that the user needs to interact with the preset interaction interface. In response to a change in the device posture of the wearable device, the near-eye display device may change the position of the wearable device corresponding to the preset interaction interface according to the change in the device posture of the wearable device. The position of the wearable device corresponding to the preset interaction interface allows the near-eye display device to determine a target interaction object from at least one preset interaction object included in the preset interaction interface.

[0075] Take the wearable device as a smart ring as an example. Figure 2 , when the initial state of the finger wearing the smart ring is pointing directly in front of the user, the initial position of the smart ring corresponding to the preset interaction interface is, for example, at the center point of the preset interaction interface. When the device posture of the smart ring moves to the upper left direction, the position of the smart ring corresponding to the preset interaction interface also moves from the initial position to the upper left direction accordingly. For another example, when the device posture of the smart ring moves to the lower right direction, the position of the smart ring corresponding to the preset interaction interface also moves from the initial position to the lower right direction accordingly, and so on.

[0076] In the case where the preset interaction interface includes at least one preset interaction object, the at least one preset interaction object may be respectively within different preset position ranges of the preset interaction interface. Based on this, the near-eye display device may determine the position of the wearable device corresponding to the preset interaction interface according to the device posture of the wearable device. Accordingly, the near-eye display device may determine whether the position corresponding to the wearable device is within the preset position range corresponding to one of the preset interaction objects. In the case where the position corresponding to the wearable device is within the preset position range corresponding to one of the preset interaction objects, the near-eye display device may determine the target interaction object based on the preset interaction object. The target interaction object may be used by the near-eye display device to subsequently determine the function to be executed.

[0077] Take the wearable device as a smart ring as an example. Figure 2 , in the process that the position corresponding to the smart ring responds to the change of the device posture of the smart ring, the position corresponding to the smart ring, for example, moves from the initial position to the upper left direction to the preset position range corresponding to the interactive object 1, then the near-eye display device can determine the target interactive object based on the interactive object 1. In an exemplary embodiment, the near-eye display device can directly determine the interactive object 1 as the target interactive object, or the near-eye display device can also determine the interactive object 1 as the target interactive object in combination with the detected second preset operation on the wearable device. By analogy, if the position corresponding to the smart ring moves from the initial position to the lower right direction to the preset range corresponding to the interactive object 3, for example, the near-eye display device can determine the target interactive object based on the interactive object 3.

[0078] Of course, the initial position of the smart ring is not limited to the center point of the preset interaction interface. For example, after the position corresponding to the smart ring moves to the preset position range corresponding to the interaction object 1, if the user changes the device posture of the smart ring again to move the position corresponding to the smart ring from the preset position range corresponding to the preset interaction object 1 to the preset position range corresponding to the interaction object 2, the near-eye display device can change the initial position of the smart ring corresponding to the preset interaction interface to be within the preset position range corresponding to the interaction object 1, and there is no limitation here.

[0079] In some implementations, a preset interaction identifier corresponding to the wearable device is displayed on a preset interaction interface; the position of the preset interaction identifier is used to indicate an interaction position corresponding to the device posture of the wearable device on the preset interaction interface.

[0080] like Figure 5 As shown, when the preset interaction mark is displayed on the preset interaction interface, the user can view the preset interaction mark. Accordingly, the user can determine the interaction position with the preset interaction interface according to the position of the preset interaction mark, thereby determining the interaction position corresponding to the device posture of the wearable device in the preset interaction interface.

[0081] For example, in response to a change in the device posture of the wearable device, the position of the preset interaction identifier is changed; when the position of the preset interaction identifier is within a preset position range corresponding to one of the preset interaction objects, and a second preset operation on the wearable device is detected, the preset interaction object corresponding to the preset position range in which the position is located is determined to be the target interaction object.

[0082] When the posture of the wearable device changes, the near-eye display device may change the position of the preset interaction mark according to the change information of the device posture. For example, the change information of the device posture may indicate the change direction and change distance of the wearable device, and the near-eye display device may change the position of the preset interaction mark according to the change direction and change distance.

[0083] Accordingly, when the position of the preset interaction identifier is within the preset position range corresponding to one of the preset interaction objects and a second preset operation on the wearable device is detected, the near-eye display device can determine that the user needs to interact with the preset interaction object. The near-eye display device can then determine the preset interaction object as the target interaction object for the subsequent near-eye display device to perform the corresponding function.

[0084] In some embodiments, when the direction indicated by the device posture is within a preset direction range corresponding to one of the preset interaction objects and a second preset operation on the wearable device is detected, the preset interaction object corresponding to the preset direction range in which the direction is located is determined to be the target interaction object.

[0085] For example, the preset interactive object may be within a preset direction range of the preset interactive interface, and the preset interactive object corresponds to the preset direction range.

[0086] like Figure 2 As shown, the preset interactive objects include interactive objects 1 to 4. The preset direction range corresponding to interactive object 1 is from 0 to 90 degrees, the preset direction range corresponding to interactive object 2 is from 90 to 180 degrees, the preset direction range corresponding to interactive object 3 is from 180 to 270 degrees, and the preset direction range corresponding to interactive object 4 is from 270 to 360 degrees.

[0087] When the near-eye display device detects that the device posture of the wearable device has changed, the near-eye display device can determine the direction indicated by the wearable device. Accordingly, when the direction indicated by the device posture is determined, the near-eye display device can determine whether the direction is within the direction range corresponding to one of the preset interactive objects.

[0088] When the posture of the finger wearing the smart ring changes, the device posture of the smart ring also changes accordingly. For example, when the initial state of the finger is pointing directly in front of the user, if the user straightens the finger and lifts the finger in the upper left direction or the user drives the finger to lift in the upper left direction through the wrist, the device posture of the smart ring will move to the upper left direction accordingly. If the user straightens the finger and puts it down in the lower right direction or the user drives the finger to put it down in the lower right direction through the wrist, the device posture of the smart ring will move to the lower right direction accordingly, and so on.

[0089] For example, the wearable device is a smart ring, and when the finger wearing the smart ring points to the left, the direction indicated by the device posture of the smart ring is 0 degrees. Figure 2 As shown, when the user changes the posture of the finger wearing the smart ring to point to the upper left, the direction indicated by the device posture of the smart ring is correspondingly changed to the upper left direction, such as changing to any one of 0 to 90 degrees. Since interactive object 1 is in the upper left area of ​​the preset interactive interface, the preset direction range corresponding to interactive object 1 is within 0 to 90 degrees, and the near-eye display device can determine that the direction indicated by the device posture is within the preset direction range corresponding to interactive object 1. For another example, when the user changes the posture of the finger wearing the smart ring to point to the upper right, the direction indicated by the device posture of the smart ring is correspondingly changed to the upper right direction, such as changing to any one of 90 to 180 degrees. Since interactive object 2 is in the upper right area of ​​the preset interactive interface, the preset direction range corresponding to interactive object 2 is 90 to 180 degrees, and the near-eye display device can determine that the direction indicated by the device posture is within the preset direction range corresponding to interactive object 2. And so on. When the direction indicated by the device posture is within the preset direction range corresponding to each of the two adjacent preset interactive objects, the near-eye display device can determine the target interactive object from the two adjacent preset interactive objects based on the change trajectory of the direction indicated by the device posture. For example, when the direction indicated by the device posture is 0 degrees, the direction indicated by the device posture is within the preset direction range corresponding to each of interactive object 1 and interactive object 4. If the change trajectory of the direction indicated by the device posture changes from 90 degrees to 0 degrees, the near-eye display device can determine the target interactive object based on interactive object 1. If the change trajectory of the direction indicated by the device posture changes from 270 degrees to 0 degrees (i.e., 360 degrees), the near-eye display device can determine the target interactive object based on interactive object 4. Of course, it is not limited to this. For example, the near-eye display device can also determine the target interactive object from multiple preset interactive objects based on the change in the direction indicated by the subsequent device posture, which is not limited here.

[0090] In the case where the direction indicated by the device posture is within the preset direction range corresponding to one of the preset interactive objects, the near-eye display device can determine the target interactive object based on the preset interactive object. Accordingly, in order to prevent the user from mistakenly changing the device posture of the wearable device and causing the near-eye display device to mistakenly identify the target interactive object, the near-eye display device can further determine whether a second preset operation of the wearable device is detected when it is determined that the direction indicated by the device posture is within the preset direction range corresponding to one of the preset interactive objects, so as to determine the target interactive object by combining the preset direction range and the second preset operation. For example, in the case where the direction indicated by the device posture is within the preset direction range corresponding to one of the preset interactive objects, and the second preset operation on the wearable device is detected, the near-eye display device can determine that the user needs to interact with the preset interactive object, and then the preset interactive object can be determined as the target interactive object.

[0091] In this way, the near-eye display device can integrate the device posture of the wearable device and the detected second preset operation on the wearable device to determine the target interaction object from at least one preset interaction object, which is conducive to improving the accuracy of determining the target interaction object, thereby facilitating the subsequent improvement of the convenience of controlling the near-eye display device.

[0092] S103: Control the near-eye display device to execute a function corresponding to the target interactive object.

[0093] For example, the near-eye display device may establish an association relationship between a preset interactive object and a corresponding function. When the near-eye display device determines a target interactive object, the near-eye display device may determine the function corresponding to the target interactive object based on the association relationship, thereby executing the function corresponding to the target interactive object.

[0094] In this way, when controlling the near-eye display device to execute the function corresponding to the target interactive object, the near-eye display device can execute the corresponding function required by the user, which is conducive to improving the convenience of controlling the near-eye display device.

[0095] The control method of the near-eye display device provided in the above embodiment includes: when a first preset operation on the wearable device is detected, controlling the near-eye display device to display a preset interaction interface corresponding to the wearable device; the preset interaction interface includes at least one preset interaction object; according to the device posture of the wearable device, determining a target interaction object from at least one preset interaction object; controlling the near-eye display device to execute the function corresponding to the target interaction object.

[0096] When the near-eye display device displays a preset interactive interface, since the near-eye display device is communicatively connected to the wearable device and the preset interactive interface is adapted to the wearable device, the near-eye display device can perform corresponding interactive operations with the preset interactive interface in response to changes in the device posture of the wearable device, which is beneficial to improving the convenience of operating the wearable device and improving the convenience of controlling the near-eye display device.

[0097] See also Figure 6 , Figure 6 1 is a schematic block diagram of a control device for a near-eye display device provided in an embodiment of the present application. The control device for the near-eye display device can be configured in the near-eye display device to execute the control method for the near-eye display device described above. The near-eye display device can include AR glasses, MR glasses, AR helmets, MR helmets, etc., which are not limited here.

[0098] The near-eye display device can be connected to a wearable device for communication. The wearable device may include a smart ring, a smart bracelet, a smart watch, etc., which are not limited here.

[0099] like Figure 6 As shown, the control device of the near-eye display device includes a first control module 110 , an object determination module 120 and a second control module 130 .

[0100] The first control module 110 is used to control the near-eye display device to display a preset interaction interface corresponding to the wearable device when a first preset operation on the wearable device is detected; the preset interaction interface includes at least one preset interaction object.

[0101] The object determination module 120 is used to determine a target interaction object from the at least one preset interaction object according to the device posture of the wearable device.

[0102] The second control module 130 is used to control the near-eye display device to execute the function corresponding to the target interactive object.

[0103] Exemplarily, the interface shape of the preset interactive interface is the same as the shape of the wearable device.

[0104] Exemplarily, the object determination module 120 includes a first determination submodule.

[0105] The first determination submodule is used to determine that the preset interaction object corresponding to the preset direction range where the direction is located is a target interaction object when the direction indicated by the device posture is within a preset direction range corresponding to one of the preset interaction objects and a second preset operation on the wearable device is detected.

[0106] Exemplarily, the preset interaction interface includes a first interaction interface and a second interaction interface.

[0107] The first control module 110 includes a first display submodule and a second display submodule.

[0108] The first display submodule is used to control the near-eye display device to display a first interactive interface when a first preset operation on the wearable device is detected.

[0109] The second display submodule is used to control the near-eye display device to display a second interaction interface when the direction indicated by the device posture is within the preset direction range corresponding to the first interaction object in the first interaction interface and a second preset operation on the wearable device is detected; the second interaction interface includes a second interaction object corresponding to the first interaction object.

[0110] Exemplarily, the second display submodule includes a third display submodule and a fourth display submodule.

[0111] The third display submodule is used to display the second interaction interface within the extended position range corresponding to the first interaction interface.

[0112] The fourth display submodule is used to display the second interactive interface superimposed on the first interactive interface.

[0113] Exemplarily, the control device further includes a hidden display submodule.

[0114] The hidden display submodule is used to control the near-eye display device to switch the second interactive interface to the first interactive interface when a hiding instruction for the second interactive interface is detected.

[0115] Exemplarily, the control device also includes an identification display submodule.

[0116] The logo display submodule is used to display the preset interaction logo corresponding to the wearable device on the preset interaction interface; the position of the preset interaction logo is used to indicate the interaction position corresponding to the device posture of the wearable device on the preset interaction interface.

[0117] The object determination module 120 includes a posture change submodule and a second determination submodule.

[0118] The posture changing submodule is used to change the position of the preset interaction mark in response to the change of the device posture of the wearable device.

[0119] The second determination submodule is used to determine that the preset interaction object corresponding to the preset position range of the position is the target interaction object when the position of the preset interaction identifier is within the preset position range corresponding to one of the preset interaction objects and a second preset operation on the wearable device is detected.

[0120] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0121] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0122] Exemplarily, the above-mentioned method and apparatus may be implemented in the form of a computer program, which may be run on a near-eye display device or a server to control the near-eye display device. Exemplarily, the near-eye display device may include AR glasses, MR glasses, AR helmets, MR helmets, and the like, which are not limited here. Wearable devices may include smart rings, smart bracelets, smart watches, and the like. The server may be a separate server, or it may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms.

[0123] See also Figure 7 , Figure 7 It is a schematic block diagram of the structure of a near-eye display device provided in an embodiment of the present application.

[0124] like Figure 7 As shown, the near-eye display device includes a memory and a processor. The memory and the processor may be connected via a system bus, and the memory may include a storage medium and an internal memory.

[0125] The storage medium may store an operating system and a computer program. When the computer program is executed, the processor may execute any method for controlling a near-eye display device.

[0126] The processor is used to provide computing and control capabilities to support the operation of the entire near-eye display device.

[0127] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any control method of the near-eye display device.

[0128] Those skilled in the art will understand that Figure 7 The structure shown in the figure is merely a block diagram of a portion of the structure related to the scheme of the present application, and does not constitute a limitation on the near-eye display device to which the scheme of the present application is applied. The specific near-eye display device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0129] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0130] In one embodiment, the processor is used to execute a computer program and implement the following steps when executing the computer program:

[0131] When a first preset operation on the wearable device is detected, controlling the near-eye display device to display a preset interaction interface corresponding to the wearable device; the preset interaction interface includes at least one preset interaction object;

[0132] Determining a target interaction object from the at least one preset interaction object according to the device posture of the wearable device;

[0133] Control the near-eye display device to execute the function corresponding to the target interactive object.

[0134] It should be noted that technicians in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of controlling the near-eye display device described above can refer to the corresponding process in the aforementioned near-eye display device control method embodiment, and will not be repeated here.

[0135] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the control method of the near-eye display device of the present application.

[0136] The computer-readable storage medium may be an internal storage unit of the near-eye display device described in the aforementioned embodiment, such as a hard disk or memory of the near-eye display device. The computer-readable storage medium may also be an external storage device of the near-eye display device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the near-eye display device.

[0137] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0138] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0139] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for controlling a near-eye display device, characterized in that: Applied to a near-eye display device, the near-eye display device is communicatively connected with a wearable device; the control method comprises: When a first preset operation on the wearable device is detected, controlling the near-eye display device to display a preset interaction interface corresponding to the wearable device; the preset interaction interface includes at least one preset interaction object; Determining a target interaction object from the at least one preset interaction object according to the device posture of the wearable device; Control the near-eye display device to execute the function corresponding to the target interactive object.

2. The control method according to claim 1, characterized in that: The interface shape of the preset interactive interface is the same as the shape of the wearable device.

3. The control method according to claim 1, characterized in that: The step of determining a target interaction object from the at least one preset interaction object according to the device posture of the wearable device includes: When the direction indicated by the device posture is within a preset direction range corresponding to one of the preset interaction objects and a second preset operation on the wearable device is detected, the preset interaction object corresponding to the preset direction range in which the direction is located is determined as the target interaction object.

4. The control method according to claim 1, characterized in that: The preset interaction interface includes a first interaction interface and a second interaction interface; When a first preset operation on the wearable device is detected, controlling a near-eye display device corresponding to the wearable device to display a preset interaction interface includes: When a first preset operation on the wearable device is detected, controlling the near-eye display device to display a first interactive interface; When the direction indicated by the device posture is within the preset direction range corresponding to the first interactive object in the first interactive interface, and a second preset operation on the wearable device is detected, the near-eye display device is controlled to display a second interactive interface; the second interactive interface includes a second interactive object corresponding to the first interactive object.

5. The control method according to claim 4, characterized in that: The controlling the near-eye display device to display a second interactive interface includes: Displaying a second interactive interface within the extended position range corresponding to the first interactive interface; or, The second interaction interface is displayed superimposed on the first interaction interface.

6. The control method according to claim 4, characterized in that: After controlling the near-eye display device to display the second interactive interface, the control method further includes: When a hiding instruction for the second interactive interface is detected, the near-eye display device is controlled to switch the second interactive interface to the first interactive interface.

7. The control method according to any one of claims 1 to 5, characterized in that: After controlling the near-eye display device corresponding to the wearable device to display a preset interactive interface, the control method further includes: Displaying a preset interaction mark corresponding to the wearable device on the preset interaction interface; the position of the preset interaction mark is used to indicate the interaction position corresponding to the device posture of the wearable device on the preset interaction interface; The step of determining a target interaction object from the plurality of preset interaction objects according to the device posture of the wearable device includes: In response to a change in the device posture of the wearable device, changing the position of the preset interaction mark; When the position of the preset interaction identifier is within a preset position range corresponding to one of the preset interaction objects and a second preset operation on the wearable device is detected, the preset interaction object corresponding to the preset position range where the position is located is determined as the target interaction object.

8. A control device for a near-eye display device, characterized in that: The control device comprises: A first control module, configured to control a near-eye display device to display a preset interaction interface corresponding to the wearable device when a first preset operation on the wearable device is detected; the preset interaction interface includes a plurality of preset interaction objects; An object determination module, configured to determine a target interaction object from the plurality of preset interaction objects according to a device posture of the wearable device; The second control module is used to control the near-eye display device to execute the function corresponding to the target interactive object.

9. A near-eye display device, characterized in that: The near-eye display device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the steps of the control method of the near-eye display device according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for controlling a near-eye display device according to any one of claims 1 to 7 are implemented.

Images