Display method and device of near-to-eye display equipment, equipment and storage medium

Abstract

The invention relates to the technical field of interaction, and provides a display method and device of near-eye display equipment, equipment and a storage medium, and the method comprises the steps: displaying an interaction identifier in a display interface of the near-eye display equipment; in response to an interaction action of the user on the near-to-eye display equipment, adjusting a display position of the interaction identifier on the display interface; and according to the change of the display position of the interaction identifier, displaying a movement track corresponding to the interaction identifier on a display interface so as to improve the display effect of the near-to-eye display equipment on the interaction identifier.

Description

Technical Field

[0001] This application relates to the field of interaction technologies, and in particular, to a display method, device, equipment, and storage medium for a near-eye display device. Background Art

[0002] Near-eye display devices have an interactive motion tracking function. For example, a near-eye display device can perform eye movement tracking, head movement tracking, and so on on a user. When performing interactive motion tracking on a near-eye display device, an interaction identifier can be displayed in the display interface of the near-eye display device. However, in related technologies, the interaction identifiers displayed by near-eye display devices often exist in the form of instantaneous points, which makes it easy for the interaction identifiers to be unnoticed or ignored by users, thereby resulting in poor display effects of the interaction identifiers in near-eye display devices. Summary of the Invention

[0003] The main purpose of this application is to provide a display method, device, equipment, and storage medium for a near-eye display device, aiming to solve the technical problem that in the process of performing interactive motion tracking on a near-eye display device, due to the fact that the interaction identifiers displayed by the near-eye display device are not easily noticed or ignored by users, the display effect of the interaction identifiers in the near-eye display device is poor.

[0004] In a first aspect, this application provides a display method for a near-eye display device, including:

[0005] Displaying an interaction identifier in the display interface of the near-eye display device;

[0006] Responding to an interaction action of a user on the near-eye display device, and adjusting the display position of the interaction identifier in the display interface;

[0007] Displaying a motion trajectory corresponding to the interaction identifier on the display interface according to the change in the display position of the interaction identifier.

[0008] In a second aspect, this application provides a display device for a near-eye display device, where the display device includes:

[0009] An identifier display module, configured to display an interaction identifier in the display interface of the near-eye display device;

[0010] A position adjustment module, configured to respond to an interaction action of a user on the near-eye display device, and adjust the display position of the interaction identifier in the display interface;

[0011] A trajectory display module, configured to display a motion trajectory corresponding to the interaction identifier on the display interface according to the change in the display position of the interaction identifier.

[0012] In a third aspect, the present application provides a near-eye display device, which includes a memory and a processor;

[0013] The memory is used to store a computer program;

[0014] The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the display method of the near-eye display device as described above.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the display method of the near-eye display device as described above are implemented.

[0016] The present application provides a display method, device, equipment and storage medium for a near-eye display device. The display method includes: displaying an interaction identifier in a display interface of the near-eye display device; in response to an interaction action of a user on the near-eye display device, adjusting a display position of the interaction identifier in the display interface; and displaying a motion trajectory corresponding to the interaction identifier on the display interface according to a change in the display position of the interaction identifier.

[0017] When an interaction identifier is displayed in the display interface of the near-eye display device, by responding to an interaction action of a user on the near-eye display device and adjusting the display position of the interaction identifier in the display interface, the near-eye display device can track the interaction action, so that the adjustment of the display position of the interaction identifier by the near-eye display device adapts to the interaction requirements of the user for the near-eye display device. Correspondingly, when the display position of the interaction identifier changes, a motion trajectory corresponding to the interaction identifier can be displayed on the display interface. The motion trajectory corresponding to the interaction identifier can assist the user in viewing the interaction identifier. Based on the display of the interaction identifier and the motion trajectory corresponding to the interaction identifier by the near-eye display device, the user can better and more conveniently view the interaction identifier displayed on the display interface, which is beneficial to improving the display effect of the interaction identifier by the near-eye display device. Correspondingly, when the near-eye display device displays the motion trajectory corresponding to the interaction identifier, the motion trajectory corresponding to the interaction identifier can smoothly show the change in the display position of the interaction identifier on the display interface, which is beneficial to reducing the situation where the user has a sense of dizziness caused by inaccurate tracking or tracking delay of the interaction identifier, and thus is beneficial to improving the user's interaction experience with the near-eye display device. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 is a schematic flowchart of a display method for a near-eye display device provided by an embodiment of the present application;

[0020] Figure 2 is a flowchart block diagram for determining a motion trajectory corresponding to an interaction identifier according to an embodiment of the present application;

[0021] Figure 3 is a schematic block diagram of a display device for a near-eye display device provided by an embodiment of the present application;

[0022] Figure 4 is a schematic block diagram of a near-eye display device provided by an embodiment of the present application. Detailed implementation manners

[0023] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0024] The flowcharts shown in the accompanying drawings are only illustrative, and do not necessarily include all the content and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can also be decomposed, combined, or partially merged, so the actual execution order may change according to the actual situation.

[0025] An embodiment of the present application provides a display method, device, equipment, and storage medium for a near-eye display device. The display method of the near-eye display device can be applied to a near-eye display device. The near-eye display device can include augmented reality (AR) glasses, virtual reality (VR) glasses, mixed reality (MR) glasses, AR helmets, VR helmets, MR helmets, etc., which are not limited herein. The display method of the near-eye display device can also be applied to a server, which can be a separate server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0026] Next, some embodiments of the present application will be described in detail in conjunction with the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0027] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a display method for a near-eye display device provided by an embodiment of the present application. It should be noted that the display method for the near-eye display device provided by the embodiments of the present application can be used in a near-eye display device or in a server, and is not limited herein.

[0028] As Figure 1 shown, the display method for the near-eye display device includes steps S101 to S103.

[0029] S101. Display an interaction identifier in the display interface of the near-eye display device.

[0030] The near-eye display device has a display function. The near-eye display device can display a visually visible display interface by calling relevant components. Taking the near-eye display device as an AR glasses as an example, the lens of the AR glasses can be used as the display medium of the near-eye display device to present a display interface visible to the human eye. Taking the near-eye display device including a VR helmet as an example, the goggles of the VR helmet can be used as the display medium of the near-eye display device to present a display interface visible to the human eye. Of course, it is not limited thereto and is not limited herein.

[0031] The near-eye display device can display an interaction identifier in the display interface. The interaction identifier can have different display forms. For example, the interaction identifier can include a cursor. The cursor can exist in the form of a preset shape, a static graph, a dynamic graph, etc., and is not limited herein.

[0032] The interaction identifier displayed in the display interface of the near-eye display device can be used as the interaction basis for the user with respect to the display interface.

[0033] In some embodiments, the display position of the interaction identifier on the display interface can be used to indicate the interaction position when the user operates on the display interface. When the user views the interaction identifier in the display interface, the user can determine the interaction position of the user with respect to the display interface of the near-eye display device according to the display position of the interaction identifier in the display interface. Accordingly, the user determines whether subsequent interaction actions need to be performed based on the interaction position with respect to the display interface.

[0034] For example, an object to be interacted with may also be displayed in the display interface of the near-eye display device. The object to be interacted with may include at least one of an application identifier corresponding to an application, a system component corresponding to the display interface, a message prompt identifier, etc., which is not limited herein. When the user has an interaction requirement for the object to be interacted with in the display interface, if the display position of the interaction identifier is not within the position range corresponding to the object to be interacted with in the display interface, the user can determine that the interaction position of the user with respect to the display interface of the near-eye display device is not within the position range corresponding to the object to be interacted with. Furthermore, the user can determine that subsequent interaction actions on the near-eye display device are still required to adjust the display position of the interaction identifier. Correspondingly, if the display position of the interaction identifier is within the position range corresponding to the object to be interacted with, the user can determine that the interaction position of the user with respect to the display interface of the near-eye display device is already within the position range corresponding to the object to be interacted with. Furthermore, the user can subsequently perform corresponding interaction operations on the object to be interacted with according to the user's interaction requirement for the object to be interacted with, so as to control the near-eye display device to execute the function corresponding to the object to be interacted with.

[0035] Based on this, when an interaction identifier is displayed in the display interface of the near-eye display device, the user can interact with the display interface according to the interaction identifier, which is beneficial to improving the interaction convenience between the user and the near-eye display device. Correspondingly, the interaction identifier displayed in the display interface of the near-eye display device can intuitively indicate the user's interaction position on the display interface to the user, which is beneficial to improving the convenience of determining the interaction position, and further beneficial to improving the user's interaction experience with the near-eye display device.

[0036] S102. In response to the user's interaction action on the near-eye display device, adjust the display position of the interaction identifier in the display interface.

[0037] The near-eye display device may have an interactive motion tracking function. For example, the near-eye display device can track the interaction actions made by the user on the near-eye display device. Exemplarily, the interaction actions may include eye movement actions, manual actions, head movement actions, etc., which is not limited herein. When the near-eye display device tracks the user's interaction actions, the near-eye display device can perform eye movement tracking, manual action tracking, head movement tracking, etc. on the user, which is not limited herein.

[0038] In some embodiments, the near-eye display device can adjust the display position of the interaction identifier in the display interface in response to the user's interaction action on the near-eye display device. The display position of the interaction identifier in the display interface may include three-dimensional coordinates or two-dimensional coordinates, which is not limited herein.

[0039] Take the interactive action including eye movement as an example. When the near-eye display device detects that the user's eyes move to the left, the near-eye display device can adjust the display position of the interactive identifier on the display interface to the left in response to the user's eye movement. When the near-eye display device detects that the user's eyes move to the lower right, the near-eye display device can adjust the display position of the interactive identifier on the display interface to the lower right in response to the user's eye movement. And so on. Of course, the interactive action is not limited to eye movement and is not restricted here.

[0040] In some embodiments, when the user has an interactive need for the object to be interacted with in the display interface, if the display position of the interactive identifier is not within the position range corresponding to the object to be interacted with in the display interface, the user can make corresponding interactive actions on the near-eye display device according to the display position of the interactive identifier and the position range corresponding to the object to be interacted with, so that the interactive identifier gradually approaches the object to be interacted with until the display position of the interactive identifier is within the position range corresponding to the object to be interacted with.

[0041] For example, when the position range corresponding to the object to be interacted with is on the right side of the display position of the interactive identifier, the user can make an interactive action to indicate that the interactive identifier moves to the right on the near-eye display device, such as the user's eyes moving to the right, the user's hand moving to the right, the user's head moving to the right, etc., which is not restricted here. The near-eye display device can then adjust the display position of the interactive identifier on the display interface to the right in response to the detected interactive action. If the display position of the interactive identifier is still not within the position range corresponding to the object to be interacted with, the user can continue to make corresponding interactive actions on the near-eye display device according to the display position of the interactive identifier and the position range corresponding to the object to be interacted with until the display position of the interactive identifier is within the position range corresponding to the object to be interacted with. Correspondingly, when the display position of the interactive identifier is within the position range corresponding to the object to be interacted with, the user can control the near-eye display device to execute the function corresponding to the object to be interacted with through the interactive operation with the object to be interacted with. The interactive operation can include blinking, gazing, eye movement after gazing, preset voice, preset gesture, etc., which is not restricted here. Take the object to be interacted with including the application identifier corresponding to the application program as an example. When the near-eye display device detects that the display position of the interactive identifier is within the position range corresponding to the application identifier and detects the interactive operation on the application identifier, the near-eye display device can, for example, display the application interface of the application program on the display interface. Of course, it is not limited to this and is not restricted here.

[0042] When the near-eye display device adjusts the display position of the interaction identifier on the display interface in response to a user's interaction action on the near-eye display device, the near-eye display device can determine the user's interaction requirement for the interaction identifier based on the user's interaction action on the near-eye display device, so as to adjust the display position of the interaction identifier, which is beneficial to improving the convenience of adjusting the display position of the interaction identifier by the near-eye display device, and further beneficial to improving the user's interaction experience with the near-eye display device.

[0043] S103. Display the movement trajectory corresponding to the interaction identifier on the display interface according to the change in the display position of the interaction identifier.

[0044] For example, when the display position of the interaction identifier changes, the near-eye display device can display the movement trajectory corresponding to the interaction identifier on the display interface. Correspondingly, in response to each change in the display position of the interaction identifier, the near-eye display device can update the movement trajectory corresponding to the interaction identifier displayed on the display interface.

[0045] In some embodiments, the near-eye display device can determine whether the display position of the interaction identifier changes based on the current display position of the interaction identifier and the historical display position of the interaction identifier. Correspondingly, when the near-eye display device determines that the display position of the interaction identifier changes, it can display the movement trajectory corresponding to the interaction identifier on the display interface according to the change in the display position of the interaction identifier. Take the current display position of the interaction identifier at the current time as position 1, the historical display position at historical time 1 as position 2, and historical time 1 is within the preset time period corresponding to the current time as an example. When position 1 and position 2 are different display positions, the near-eye display device can determine that the display position of the interaction identifier changes. Based on this, the near-eye display device can determine the movement trajectory corresponding to the interaction identifier according to position 1 and position 2. For example, the movement trajectory corresponding to the interaction identifier can include a shape pointing from position 2 to position 1. Another example is that the movement trajectory corresponding to the interaction identifier can include a non-pointing shape corresponding to position 1 and position 2. Of course, the movement trajectory corresponding to the interaction identifier is not limited to this, and no limitation is made here.

[0046] In some other embodiments, the near-eye display device may determine whether the display position of the interaction identifier has changed based on the historical display position of the interaction identifier. Accordingly, when the display position of the interaction identifier changes, the near-eye display device may display the motion trajectory corresponding to the interaction identifier on the display interface based on the historical display position of the interaction identifier. Taking the current display position of the interaction identifier at the current time as position 1, the historical display position at historical time 1 as position 2, and the historical display position at historical time 2 as position 3, and both historical time 1 and historical time 2 are within the preset time period corresponding to the current time as an example. When position 2 and position 3 are different display positions, regardless of whether position 1 is the same as position 2 or position 1 is the same as position 3, the near-eye display device can determine that the display position of the interaction identifier has changed. Based on this, the near-eye display device may determine the motion trajectory corresponding to the interaction identifier according to position 2 and position 3. For example, when historical time 2 is earlier than historical time 1, the motion trajectory corresponding to the interaction identifier may include a shape pointing from position 3 to position 2. For example, when historical time 1 is earlier than historical time 2, the motion trajectory corresponding to the interaction identifier may include a shape pointing from position 2 to position 3. For another example, the motion trajectory corresponding to the interaction identifier may include a non-directional shape corresponding to position 2 and position 3. Of course, the motion trajectory corresponding to the interaction identifier is not limited to this and is not restricted here.

[0047] After determining the motion trajectory corresponding to the interaction identifier, the near-eye display device may display the motion trajectory corresponding to the interaction identifier on the display interface.

[0048] For example, when the user has an interaction requirement for the interaction object, the user can determine whether it is necessary to adjust the interaction action by combining the position range corresponding to the interaction object to be interacted with, the movement trajectory corresponding to the interaction identifier, and the display position of the interaction identifier. When it is determined that the interaction action needs to be adjusted, the user can adjust at least one of the movement direction and the movement distance corresponding to the interaction action. For example, when the display position of the interaction identifier is between the position range corresponding to the interaction object to be interacted with and the movement trajectory corresponding to the interaction identifier, and the display position of the interaction identifier is not within the position range corresponding to the interaction object to be interacted with, the user can determine that the interaction identifier is gradually approaching the interaction object under the action of the movement direction indicated by the interaction action, and then can adjust the movement distance indicated by the interaction action. For another example, when the display position of the interaction identifier is outside the position range corresponding to the interaction object to be interacted with and the movement trajectory corresponding to the interaction identifier, and the display position of the interaction identifier is not within the position range corresponding to the interaction object to be interacted with, the user can determine that the interaction identifier is gradually moving away from the interaction object under the action of the movement direction indicated by the interaction action, and then can adjust the movement direction indicated by the interaction action. Of course, it is not limited to this, and no limitation is made here. Based on this, the movement trajectory corresponding to the interaction identifier can be used as a basis for the user to adjust the interaction action on the near-eye display device, so the movement trajectory corresponding to the interaction identifier can prompt the user to adjust the interaction action on the near-eye display device, which is beneficial to improving the convenience of the near-eye display device in prompting the user to adjust the interaction action, and further beneficial to improving the user's interaction experience with the near-eye display device. Moreover, since the movement trajectory corresponding to the interaction identifier is associated with the change of the display position of the interaction identifier, the movement trajectory corresponding to the interaction identifier can assist the user to better and more accurately view the interaction identifier displayed in the display interface, which is beneficial to improving the display effect of the interaction identifier by the near-eye display device.

[0049] In an exemplary embodiment, the near-eye display device can call functions related to the display of the interaction identifier, the adjustment of the display position of the interaction identifier, and the display of the movement trajectory corresponding to the interaction identifier inside the rendering engine, so that the display process of the interaction identifier, the adjustment process of the display position of the interaction identifier, and the display process of the movement trajectory corresponding to the interaction identifier on the near-eye display device can be embodied as a complete rendering pipeline. Then, the calculations involved in the display process of the interaction identifier, the adjustment process of the display position of the interaction identifier, and the display process of the movement trajectory corresponding to the interaction identifier on the near-eye display device can be efficiently executed on a Graphics Processing Unit (GPU), which is beneficial to improving the tracking accuracy of the interaction action by the near-eye display device and reducing the tracking latency of the interaction action by the near-eye display device.

[0050] When displaying the movement trajectory corresponding to the interaction identifier on the display interface according to the change in the display position of the interaction identifier, the near-eye display device only needs to obtain the display positions related to the change in the display position of the interaction identifier, and then it can generate and display the movement trajectory corresponding to the interaction identifier. This is beneficial to improving the convenience of determining the movement trajectory corresponding to the interaction identifier and reducing the hardware requirements for determining the movement trajectory corresponding to the interaction identifier, thereby reducing the system cost. Since the display position of the interaction identifier can be adjusted in response to the user's interaction actions on the near-eye display device, the movement trajectory corresponding to the interaction identifier can reflect the trajectory of the user's interaction actions on the near-eye display device, which is beneficial to improving the convenience of tracking the interaction actions by the near-eye display device. Correspondingly, based on the improvement in the convenience of determining the movement trajectory corresponding to the interaction identifier and the convenience of tracking the interaction actions by the near-eye display device, the movement trajectory corresponding to the interaction identifier can smoothly show the change in the display position of the interaction identifier on the display interface, which is beneficial to reducing the situation where the user experiences dizziness caused by inaccurate tracking or tracking delay of the interaction identifier, and thus is beneficial to improving the user's interaction experience with the near-eye display device.

[0051] In some embodiments, obtain the current display position of the interaction identifier on the display interface; determine the position change information corresponding to the interaction action; perform a position transformation process on the current display position according to the position change information to obtain the target display position of the interaction identifier; and display the interaction identifier at the target display position.

[0052] The near-eye display device can obtain the current display position of the interaction identifier on the display interface. When the near-eye display device detects the user's interaction actions on the near-eye display device, the near-eye display device can determine the position change information corresponding to the interaction action.

[0053] For example, during the process of tracking the interaction action, the near-eye display device can obtain the positions before and after the user makes the interaction action. Taking the interaction action including a hand movement as an example, during the process of tracking the hand movement, the near-eye display device can determine the positions of the user's hands in the space where the near-eye display device is located before and after the user makes the hand movement. Taking the interaction action including a head movement as an example, during the process of tracking the head movement, the near-eye display device can determine the positions of the user's head in the space where the near-eye display device is located before and after the user makes the head movement. And so on.

[0054] Accordingly, the near-eye display device can determine the moving distance and moving direction corresponding to the interaction action based on the positions before and after the user makes the interaction action, that is, determine the position change information corresponding to the interaction action. Taking the interaction action including a manual action as an example, the near-eye display device can determine the moving distance and moving direction corresponding to the manual action based on the positions of the user's hand in the space where the near-eye display device is located before and after the user makes the manual action. Taking the interaction action including a head movement as an example, the near-eye display device can determine the moving distance and moving direction corresponding to the head movement based on the positions of the user's head in the space where the near-eye display device is located before and after the user makes the head movement. And so on.

[0055] In the case of determining the position change information corresponding to the interaction action, the near-eye display device can perform a position transformation process on the current display position based on the position change information to obtain the target display position of the interaction identifier, and then display the interaction identifier at the target display position. For example, the near-eye display device can control the interaction identifier to move a corresponding distance from the current display position in the moving direction corresponding to the interaction action to obtain the target display position of the interaction identifier. Among them, the distance that the near-eye display device controls the interaction identifier to move can be determined according to the moving distance corresponding to the interaction action. For example, the distance that the near-eye display device controls the interaction identifier to move can be in a proportional relationship with the moving distance corresponding to the interaction action.

[0056] Taking the interaction action including a manual action, and the position change information corresponding to the manual action including the hand moving upward by a distance of 1 as an example, the near-eye display device can control the interaction identifier to move upward by a distance of 2 from the current display position based on the position change information corresponding to the manual action to obtain the target display position of the interaction identifier. Among them, the distance 2 is in a proportional relationship with the distance 1.

[0057] Taking the interaction action including an eye movement, and the position change information corresponding to the eye movement including the eye moving rightward by a distance of 3 as an example, the near-eye display device can control the interaction identifier to move rightward by a distance of 4 from the current display position based on the position change information corresponding to the eye movement to obtain the target display position of the interaction identifier. Among them, the distance 4 is in a proportional relationship with the distance 3.

[0058] Of course, the interaction actions and the position change information corresponding to the interaction actions are not limited to this, and no limitation is made here.

[0059] In the process of adjusting the display position of the interaction identifier on the display interface, the near-eye display device performs a position transformation process on the current display position of the interaction identifier based on the position change information corresponding to the interaction action, obtains the target display position of the interaction identifier, and then displays the interaction identifier at the target display position, which is beneficial to improving the convenience of adjusting the display position of the interaction identifier. Correspondingly, if the display position of the interaction identifier by the near-eye display device is adjusted based on the position change information corresponding to the interaction action, the adjustment of the display position of the interaction identifier by the near-eye display device can be adapted to the user's position adjustment requirement for the interaction identifier, and thus is beneficial to improving the user's interaction experience with the near-eye display device.

[0060] In some embodiments, according to the moving distance and moving direction indicated by the position change information, a position transformation process is performed on the current display position to obtain the predicted display position of the interaction identifier; according to the interface reference position of the display interface, a position transformation process is performed on the predicted display position to obtain the target display position of the interaction identifier.

[0061] In the process of the near-eye display device adjusting the display position of the interaction identifier on the display interface in response to the user's interaction action on the near-eye display device, the device posture of the near-eye display device may also change in response to the user's interaction action. Taking the interaction action as a head movement action as an example, since the near-eye display device can be worn on the user's head, when the user makes actions such as turning the head, tilting the head forward, or tilting the head backward, the device posture of the near-eye display device will change in response to the user's head movement action. Correspondingly, the position range corresponding to the display interface of the near-eye display device in the space where the near-eye display device is located will change in response to the change of the device posture of the near-eye display device. For example, when the user makes a head-turning action, the position range corresponding to the display interface of the near-eye display device in the space where the near-eye display device is located will rotate following the rotation of the user's head, so that the near-eye display device can continuously display the display interface within the user's visual field.

[0062] In the case where the device posture of the near-eye display device may change following the user's interaction action, considering improving the determination accuracy of the target display position of the interaction identifier, it is necessary to determine the target display position of the interaction identifier in combination with the interface reference position of the display interface. The interface reference position of the display interface is determined according to the position involved in the display interface displayed by the near-eye display device. In the case where the position range corresponding to the display interface displayed by the near-eye display device in the space where the near-eye display device is located changes following the change of the device posture of the near-eye display device, the interface reference position of the display interface will correspondingly change. The interface reference position includes, for example, the position of the interface center point of the display interface, and there is no limitation here.

[0063] For example, the near-eye display device may perform a position transformation process on the current display position of the interaction identifier according to the moving distance and moving direction indicated by the position change information corresponding to the interaction action, so as to obtain the predicted display position of the interaction identifier.

[0064] In an exemplary embodiment, the near-eye display device may move the current display position of the interaction identifier by a corresponding distance in the moving direction indicated by the position change information to obtain the predicted display position of the interaction identifier. Among them, the distance that the interaction identifier moves in the moving direction is determined according to the moving distance indicated by the position change information. For example, the distance that the interaction identifier moves in the moving direction may be in a proportional relationship with the moving distance indicated by the position change information. The predicted display position of the interaction identifier is equivalent to the target display position of the interaction identifier when the device pose of the near-eye display device does not change. However, when the device pose of the near-eye display device changes, the predicted display position cannot correctly reflect the target display position of the interaction identifier.

[0065] Based on this, the near-eye display device may determine the target display position of the interaction identifier by combining the predicted display position of the interaction identifier and the interface reference position of the display interface.

[0066] For example, the interface reference position of the display interface changes with the change of the device pose of the near-eye display device, so the interface reference position of the display interface is in the display interface coordinate system corresponding to the display interface after the device pose of the near-eye display device changes. The predicted display position of the interaction identifier is equivalent to being in the display interface coordinate system corresponding to the display interface before the device pose of the near-eye display device changes. The near-eye display device may perform a position transformation process on the predicted display position of the interaction identifier by combining the interface reference position of the display interface, so as to change the interaction identifier from the display interface coordinate system corresponding to the display interface before the device pose changes to the display interface coordinate system corresponding to the display interface after the device pose changes, and obtain the target display position of the interaction identifier.

[0067] When performing a position transformation process on the current display position according to the moving distance and moving direction indicated by the position change information to obtain the predicted display position of the interaction identifier, and performing a position transformation process on the predicted display position according to the interface reference position of the display interface to obtain the target display position of the interaction identifier, the influence of the device pose change of the near-eye display device on the target display position of the interaction identifier can be comprehensively considered during the process of determining the target display position of the interaction identifier, which is beneficial to improving the determination accuracy of the target display position of the interaction identifier. Correspondingly, based on the improvement of the determination accuracy of the target display position of the interaction identifier by the near-eye display device, it is beneficial to improve the tracking accuracy of the interaction action of the near-eye display device, and further beneficial to improving the interaction experience of the user with the near-eye display device.

[0068] In some embodiments, a rotation matrix corresponding to a predicted display position is determined according to a reference normal vector corresponding to an interface reference position of a display interface and a normal vector corresponding to the predicted display position; position transformation processing is performed on the predicted display position according to the rotation matrix, a translation matrix corresponding to the predicted display position, and a scaling matrix corresponding to the predicted display position to obtain a target display position.

[0069] For example, in the process of determining the target display position of an interaction identifier by combining the predicted display position of the interaction identifier and the interface reference position of the display interface, a near-eye display device can obtain the normal vector corresponding to the predicted display position and the reference normal vector corresponding to the interface reference position. Accordingly, the near-eye display device can determine the angle and rotation axis between the normal vector corresponding to the display position and the reference normal vector corresponding to the interface reference position based on the normal vector corresponding to the predicted display position and the reference normal vector corresponding to the interface reference position, and then determine the rotation matrix corresponding to the predicted display position based on the angle and rotation axis between the normal vector and the reference normal vector. The rotation matrix corresponding to the predicted display position can be used to compensate for the rotation change amount corresponding to the device posture of the near-eye display device, such as the change generated when the device posture of the near-eye display device responds to the rotation of the user's head.

[0070] In an exemplary embodiment, the near-eye display device can obtain an array of vertices corresponding to the interface reference position. Taking the array of vertices including an array of quadrilateral vertices (quadVertices) as an example. The quadVertices array can include the vertex coordinates of each vertex of the quadrilateral corresponding to the interface reference position and the texture coordinates of the quadrilateral corresponding to the interface reference position. The vertex coordinates can be used to indicate the position of the quadrilateral corresponding to the interface reference position in the space where the near-eye display device is located. The texture coordinates can be used to indicate how the texture involved in the quadrilateral corresponding to the interface reference position is mapped onto the quadrilateral. Among them, the vertex coordinates and texture coordinates corresponding to the interface reference position can be set as a unit rectangle. The near-eye display device can calculate the initial normal vector of the unit rectangle by using vector cross product and dot product. The initial normal vector is obtained by normalizing the reference normal vector corresponding to the interface reference position. Accordingly, when the normal vector corresponding to the predicted display position is obtained, the near-eye display device can normalize the normal vector corresponding to the predicted display position to obtain a normalized normal vector. In the case of determining the initial normal vector and the normalized normal vector, the initial normal vector and the normalized normal vector can be compared to determine the angle and rotation axis between the two, and then the rotation matrix corresponding to the predicted display position can be determined based on the angle and rotation axis between the initial normal vector and the normalized normal vector.

[0071] Based on this, the determination of the reference normal vector corresponding to the interface reference position, such as the calculation of the initial normal vector, can be used to correct the display position of the interaction identifier once, so as to improve the accuracy of determining the target display position of the interaction identifier subsequently.

[0072] For example, in the case of determining the rotation matrix corresponding to the predicted display position, the near-eye display device can combine the rotation matrix corresponding to the predicted display position, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position to perform a position transformation process on the predicted display position, so as to change the interaction identifier from the display interface coordinate system corresponding to the display interface before the device pose change to the display interface coordinate system corresponding to the display interface after the device pose change, and obtain the target display position of the interaction identifier.

[0073] Exemplarily, the translation matrix corresponding to the predicted display position can be determined according to the interaction action of the user on the near-eye display device. Taking the interaction action as a head movement action as an example, when the user moves the head forward, the head of the user undergoes a displacement. Correspondingly, the device pose of the near-eye display device can change in response to the displacement of the user's head. Based on this, the near-eye display device can determine the translation matrix corresponding to the predicted display position according to the displacement of the user's head. The translation matrix corresponding to the predicted display position can be used to compensate for the translational change amount corresponding to the device pose of the near-eye display device, such as the change caused by the translational movement of the user's head in response to the device pose of the near-eye display device.

[0074] Exemplarily, the scaling matrix corresponding to the predicted display position can be determined according to the distance between the interaction identifier and the display interface. For example, the near-eye display device can adjust at least one of the display distance of the interaction identifier and the display distance of the display interface.

[0075] The near-eye display device can adjust at least one of the display distance of the interaction identifier and the display distance of the display interface by adjusting the device parameters of the relevant components. The relevant components can include the lens module of the near-eye display device, such as liquid crystal lenses, mechanical moving lenses, etc. Taking the near-eye display device adjusting the display distance of the interaction identifier as an example. If the focal length of the lens module of the near-eye display device is changeable, the near-eye display device can dynamically adjust the lens focal length of the lens module, so that the light path related to the display of the interaction identifier changes, and then the user can perceive that the display distance of the interaction identifier changes. For example, the interaction identifier is at different depths before and after the change of the focal length of the lens module, and the near-eye display device can realize the adjustment of the display distance of the interaction identifier. Of course, the method of adjusting the display distance of the interaction identifier is not limited to this, and is not restricted here.

[0076] The near-eye display device can perform a scaling process on at least one of the interaction identifier and the display interface in response to a preset scaling operation of the user on the near-eye display device. The preset scaling operation can include a scaling gesture, a scaling voice, etc., which are not limited herein. The scaling gesture can include a gesture for indicating to enlarge the size of at least one of the interaction identifier and the display interface and a gesture for indicating to reduce the size of at least one of the interaction identifier and the display interface. The scaling voice can include at least one of a voice for indicating to enlarge the size of the interaction identifier, a voice for indicating to enlarge the size of the display interface, as well as a voice for indicating to reduce the size of the interaction identifier, a voice for indicating to reduce the size of the display interface. And so on.

[0077] In an exemplary embodiment, when the near-eye display device detects a preset scaling operation, it can adjust the display distance of at least one of the interaction identifier and the display interface so that the user perceives a change (enlargement or reduction) in at least one of the size of the interaction identifier and the size of the display interface. For example, when the operation control in response to the preset scaling operation reduces the display distance of the interaction identifier, it is equivalent to the distance between the interaction identifier and the user's eye retina becoming smaller, and the user can perceive that the size of the interaction identifier becomes larger. Another example is that when the operation control in response to the preset scaling operation increases the display distance of the interaction identifier, it is equivalent to the distance between the interaction identifier and the user's eye retina becoming larger, and the user can perceive that the size of the interaction identifier becomes smaller. And so on.

[0078] In another exemplary embodiment, when the near-eye display device detects a preset scaling operation, it can adjust at least one of the size of the interaction identifier and the size of the display interface. For example, when the detected preset scaling operation is used to indicate enlarging the size of the interaction identifier, the size of the interaction identifier can be enlarged. Another example is that when the detected preset scaling operation is used to indicate reducing the size of the interaction identifier, the size of the interaction identifier can be reduced. And so on.

[0079] When a near-eye display device detects a change in the display distance of at least one of the interaction identifier and the display interface presented to the user, it can determine the distance between the interaction identifier and the display interface. Taking the case where the display distance of the interaction identifier changes, the display distance of the display interface does not change, and the display interface does not occlude the interaction identifier, that is, the display distance of the interaction identifier is less than the display distance of the display interface as an example, when the display distance of the interaction identifier becomes smaller, it is equivalent to the distance between the interaction identifier and the user's eye retina becoming smaller. Then, when the distance between the display interface and the user's eye retina remains unchanged, it can be determined that the distance between the interaction identifier and the display interface increases. When the display distance of the interaction identifier becomes larger, it is equivalent to the distance between the interaction identifier and the user's eye retina becoming larger. Then, when the distance between the display interface and the user's eye retina remains unchanged, it can be determined that the distance between the interaction identifier and the display interface increases.

[0080] Based on this, the near-eye display device can determine the scaling matrix corresponding to the predicted display position according to the distance between the interaction identifier and the display interface. The scaling matrix corresponding to the predicted display position can be used to adjust the display ratio of the interaction identifier. When the size of the interaction identifier is enlarged, the user can view a more obvious interaction identifier, which is beneficial to improving the user's viewing experience of the interaction identifier. When the size of the interaction identifier is reduced, the occlusion of other objects (such as objects to be operated) in the display interface by the interaction identifier can be reduced.

[0081] In an exemplary embodiment, when determining the predicted display position of the interaction identifier and the normal vector corresponding to the predicted display position, the graphics to be rendered corresponding to the predicted display position can be determined according to the predicted display position and the normal vector corresponding to the predicted display position. The graphics to be rendered can be polygons, such as triangles, quadrilaterals, etc., which are not limited here. When determining the rotation matrix, translation matrix, and scaling matrix corresponding to the predicted display position, the position transformation process can be performed on each vertex of the graphics to be rendered according to the rotation matrix, translation matrix, and scaling matrix corresponding to the predicted display position to obtain the target display positions of each vertex of the image to be rendered. Accordingly, the target display position of the interaction identifier can be determined according to the target display positions of each vertex of the image to be rendered.

[0082] Based on this, the determination of the rotation matrix, translation matrix, and scaling matrix corresponding to the predicted display position can be used to perform a secondary correction on the display position of the interaction identifier to determine the target display position of the interaction identifier, which is beneficial to improving the accuracy of determining the target display position of the interaction identifier.

[0083] When determining the rotation matrix corresponding to the predicted display position based on the reference normal vector corresponding to the interface reference position of the display interface and the normal vector corresponding to the predicted display position, and performing a position transformation process on the predicted display position according to the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position to obtain the target display position, the interaction identifier can be changed from the display interface coordinate system corresponding to the display interface before the device pose change to the display interface coordinate system corresponding to the display interface after the device pose change, which is beneficial to improving the determination accuracy of the target display position of the interaction identifier. Thus, the target display position of the interaction identifier can more accurately reflect the interaction position of the user with the display interface of the near-eye display device, which is beneficial to improving the user's device interaction experience.

[0084] By performing a position transformation process on the predicted display position according to the rotation matrix, the translation matrix, and the scaling matrix corresponding to the predicted display position, the display position of the interaction identifier can be smoothly transitioned from the current display position to the target display position, which is beneficial to achieving a smooth transition of the display position of the interaction identifier during the position switching process. Thereby, the probability of the user experiencing dizziness when viewing the display interface can be reduced, and the user's device interaction experience can be improved.

[0085] In some embodiments, the interaction identifier is subjected to texture filling processing according to the target display position and the first preset texture information to obtain the processed interaction identifier.

[0086] The processed interaction identifier is displayed at the target display position.

[0087] For example, when determining the target display position of the interaction identifier, the near-eye display device can display the interaction position at the target display position. Correspondingly, the near-eye display device can perform texture filling processing on the interaction identifier so that the interaction identifier can have corresponding textures. For example, the near-eye display device can perform texture filling processing on the interaction identifier according to the target display position and the first preset texture information to obtain the processed interaction identifier. When the processed interaction identifier is obtained, the near-eye display device can display the processed interaction identifier at the target display position. The first preset texture information may include a first preset color and / or a first preset pattern, etc., which is not limited herein.

[0088] In an exemplary embodiment, when the near-eye display device determines the graphics to be rendered corresponding to the predicted display position of the interaction identifier and determines the target display position of the interaction identifier based on the target display positions of the vertices of the graphics to be rendered, the first preset texture information can be mapped onto the graphics to be rendered by means of external texture binding and texture object construction, so as to perform texture filling processing on the graphics to be rendered. After performing texture filling processing on the graphics to be rendered, the processed interaction identifier can be obtained. Wherein, the processed interaction identifier and the graphics to be rendered may have different shapes or the same shape, which is not limited herein.

[0089] When the processed interaction identifier is displayed at the target display position, the user can view the processed interaction identifier, which is beneficial to improving the display diversity of the interaction identifier by the near-eye display device.

[0090] In some embodiments, a plurality of historical display positions of the interaction identifier are obtained; position offset processing is performed on the historical display positions according to a preset offset parameter to obtain the processed historical display positions; and the motion trajectory corresponding to the interaction identifier is determined and displayed according to the plurality of processed historical display positions.

[0091] Exemplarily, the near-eye display device can store a plurality of historical display positions of the interaction identifier in the form of a queue. For example, the near-eye display device can record the display position of the interaction identifier into a first-in-first-out double-ended queue in the rendering engine to obtain the historical display position queue corresponding to the interaction identifier. The historical display position queue can maintain the historical display positions of the interaction identifier in real time according to a preset maximum capacity. The preset maximum capacity can include 10, 20, 50, 100, etc. The preset maximum capacity can be determined according to the computing power of the GPU of the near-eye display device, which is not limited herein. Correspondingly, the near-eye display device can obtain a plurality of historical display positions of the interaction identifier from the historical display position queue. By storing the historical display positions of the interaction identifier, the near-eye display device can construct the motion trajectory corresponding to the interaction identifier based on the historical display positions, which is beneficial to improving the convenience of determining the motion trajectory corresponding to the interaction identifier.

[0092] For example, the number of historical display positions obtained by the near-eye display device can be determined according to the moving speed of the user's interaction actions with the near-eye display device. When the moving speed of the interaction action changes, the number of historical display positions of the interaction identifier obtained by the near-eye display device can be dynamically changed. For example, when the moving speed of the interaction action is faster, the number of historical display positions of the interaction identifier obtained by the near-eye display device is more. When the moving speed of the interaction action is slower, the number of historical display positions of the interaction identifier obtained by the near-eye display device is less. Different moving speeds of the interaction action can correspond one-to-one to different preset acquisition numbers of the historical display positions of the interaction identifier. In an exemplary embodiment, the minimum acquisition number of the historical display positions of the interaction identifier by the near-eye display device is, for example, 10, and the maximum acquisition number is, for example, 100. Among them, in response to the change in the moving speed of the interaction action, the acquisition number of the historical display positions of the interaction identifier by the near-eye display device can be dynamically changed between 10 and 100. Of course, it is not limited to this, and no limitation is made here.

[0093] The historical display positions stored in the historical display position queue corresponding to the interaction identifier can be cleared in response to a preset clearing instruction. The preset clearing instruction can be pre-set or set by the user himself / herself, and no limitation is made here. For example, when the near-eye display device receives a shutdown instruction, it can determine that a preset clearing instruction for the historical display position queue is detected, and then can clear the historical display positions stored in the historical display position queue. Another example is that when the duration of the near-eye display device in the standby state is greater than or equal to a preset time threshold, it can determine that a preset clearing instruction for the historical display position queue is detected, and then can clear the historical display positions stored in the historical display position queue. Of course, it is not limited to this, and no limitation is made here.

[0094] In an exemplary embodiment, as Figure 2 shown, the processes involved in the near-eye display device determining the movement trajectory corresponding to the interaction identifier include: obtaining the current display position of the interaction identifier on the display interface; recording the current display position of the interaction identifier into the historical display position queue; determining whether the historical display position queue includes historical display positions other than the current display position; if so, determining the movement trajectory corresponding to the interaction identifier according to the historical display positions included in the historical display position queue; if not, continuing to obtain the current display position of the interaction identifier on the display interface.

[0095] For example, in a case where the historical display position queue does not include historical display positions other than the current display position, it can be determined that the display position of the interaction identifier has not changed, and there is no need to display the movement trajectory corresponding to the interaction identifier on the display interface of the near-eye display device. Another example is that in a case where the current display position is recorded in the historical display position queue, the current display position can also be used as the historical display position included in the historical display position queue to determine the movement trajectory corresponding to the interaction identifier, which is not limited herein.

[0096] When multiple historical display positions of the interaction identifier are obtained, the near-eye display device can perform position offset processing on each historical display position according to a preset offset parameter to obtain the processed historical display position corresponding to each historical display position.

[0097] For example, taking the historical display position including the coordinate in the horizontal direction (x coordinate) and the coordinate in the vertical direction (y coordinate) as an example. The preset offset parameter can be used to indicate offset processing of at least one of the x coordinate and the y coordinate of the historical display position. For example, the preset offset parameter can be used to indicate increasing or decreasing the x coordinate of the historical display position by a preset multiple. Another example is that the preset offset parameter can be used to indicate increasing or decreasing the y coordinate of the historical display position by a preset multiple. The preset multiple includes, for example, 1 time, 2 times, 10 times, and so on. Of course, the preset offset parameter and the preset multiple are not limited to this, and are not limited herein.

[0098] When the historical display position is offset according to a preset offset parameter to obtain the processed historical display position, the number of processed historical display positions corresponding to the same historical display position may include at least one. Taking the historical display position including historical display position 1 and historical display position 2, and the preset offset parameter including offset parameter 1 and offset parameter 2 as an example. The near-eye display device may offset historical display position 1 according to offset parameter 1 to obtain the processed historical display position 1a. The near-eye display device may offset historical display position 2 according to offset parameter 1 to obtain the processed historical display position 2a. The near-eye display device may also offset historical display position 2 according to offset parameter 2 to obtain the processed historical display position 2b. Based on this, it can be determined that the number of processed historical display positions corresponding to historical display position 1 is one, and the number of processed historical display positions corresponding to historical display position 2 is two. When determining the motion trajectory corresponding to the interaction identifier according to the processed historical display position, the near-eye display device may, for example, connect the processed historical display position 1a and the processed historical display position 2a, and connect historical display position 1 and historical display position 2, so that historical display position 1 and historical display position 2 can be extended into a path with a certain width, thereby facilitating the determination of the motion trajectory corresponding to the interaction identifier. Of course, this is not limited to this, and no restrictions are imposed here.

[0099] In an exemplary embodiment, when the near-eye display device obtains multiple historical display positions of the interaction identifier, it can determine the motion trajectory corresponding to the interaction identifier based on the multiple historical display positions. For example, when the shape of the interaction identifier is a triangle, the triangle trajectory corresponding to the interaction identifier can be determined based on the multiple historical display positions. Correspondingly, the near-eye display device can perform position offset processing on every two consecutive historical display positions among the multiple historical display positions according to a preset offset parameter to obtain the processed historical display positions corresponding to each of the multiple historical display positions, and then determine the motion trajectory corresponding to the interaction identifier based on the processed historical display positions corresponding to each of the multiple historical display positions. For example, every two consecutive historical display positions can be used to determine a line segment. When position offset processing is performed on every two consecutive historical display positions to obtain the processed historical display positions corresponding to each of the multiple historical display positions, every two consecutive processed historical display positions among the multiple processed historical display positions can be used to determine another line segment. The near-eye display device can synthesize the two line segments corresponding to every two consecutive historical display positions before and after the position offset processing, expand the multiple historical display positions into a path with a certain width, and then determine the motion trajectory corresponding to the interaction identifier. When the near-eye display device determines the motion trajectory corresponding to the interaction identifier, it can display the motion trajectory corresponding to the interaction identifier based on the processed historical display positions corresponding to each of the multiple historical display positions.

[0100] When the near-eye display device combines multiple historical display positions of the interaction identifier and a predicted offset parameter to determine the motion trajectory corresponding to the interaction identifier and displays the motion trajectory corresponding to the interaction identifier, the display interface of the near-eye display device can continuously display the motion trajectory corresponding to the interaction identifier, and the motion trajectory corresponding to the interaction identifier can be used to assist the user in viewing the interaction identifier, which is beneficial to improving the display effect of the interaction identifier on the near-eye display device. Correspondingly, when the amplitude of the interaction action of the user on the near-eye display device is relatively large, the display of the motion trajectory corresponding to the interaction identifier can smoothly show the change in the display position of the interaction identifier on the display interface, which is beneficial to reducing the situation where the user has a sense of dizziness caused by inaccurate tracking or tracking delay of the interaction identifier, and thus is beneficial to improving the user's interaction experience with the near-eye display device.

[0101] In some embodiments, texture filling processing is performed on the motion trajectory corresponding to the interaction identifier according to the multiple processed historical display positions and the second preset texture information to obtain the processed motion trajectory. The processed motion trajectory is displayed at the multiple processed historical display positions.

[0102] For example, when the near-eye display device determines the motion trajectory corresponding to the interaction identifier, the near-eye display device may perform texture filling processing on the motion trajectory corresponding to the interaction identifier so that the motion trajectory corresponding to the interaction identifier can have corresponding textures. For example, the near-eye display device may perform texture filling processing on the motion trajectory corresponding to the interaction identifier according to multiple processed historical display positions and second preset texture information to obtain a processed motion trajectory. When the processed motion trajectory is obtained, the near-eye display device can display the processed motion trajectory at multiple processed historical display positions. The second preset texture information includes, for example, a second preset color, a second preset pattern, etc. The second preset texture information may be different from the first preset texture information or the same as the first preset texture information, which is not limited herein.

[0103] In an exemplary embodiment, the second preset texture information may include a second preset color. The second preset color includes, for example, light blue or light green. Then, the processed motion trajectory may be a motion trajectory filled with light blue or light green. Of course, the processed motion trajectory is not limited to this, which is not limited herein.

[0104] When performing texture filling processing on the motion trajectory corresponding to the interaction identifier according to multiple processed historical display positions and the second preset texture information to obtain a processed motion trajectory, it is beneficial to improve the convenience of texture filling for the motion trajectory corresponding to the interaction identifier. Correspondingly, when the processed motion trajectory is displayed at multiple processed historical display positions, the user can view the processed motion trajectory, which is beneficial to improving the display diversity of the motion trajectory corresponding to the interaction identifier by the near-eye display device.

[0105] The display method of the near-eye display device provided in the above embodiment includes: displaying an interaction identifier in the display interface of the near-eye display device; in response to a user's interaction action on the near-eye display device, adjusting the display position of the interaction identifier in the display interface; and displaying a motion trajectory corresponding to the interaction identifier on the display interface according to the change in the display position of the interaction identifier.

[0106] When an interaction identifier is displayed on the display interface of a near-eye display device, by responding to a user's interaction action on the near-eye display device and adjusting the display position of the interaction identifier on the display interface, the near-eye display device can track the interaction action, so that the adjustment of the display position of the interaction identifier by the near-eye display device adapts to the user's interaction requirements for the near-eye display device. Correspondingly, when the display position of the interaction identifier changes, a motion trajectory corresponding to the interaction identifier can be displayed on the display interface. The motion trajectory corresponding to the interaction identifier can assist the user in viewing the interaction identifier. Based on the display of the interaction identifier and the motion trajectory corresponding to the interaction identifier by the near-eye display device, the user can better and more conveniently view the interaction identifier displayed on the display interface, which is beneficial to improving the display effect of the interaction identifier by the near-eye display device. Correspondingly, when the near-eye display device displays the motion trajectory corresponding to the interaction identifier, the motion trajectory corresponding to the interaction identifier can smoothly show the change in the display position of the interaction identifier on the display interface, which is beneficial to reducing the situation where the user experiences dizziness caused by inaccurate tracking or tracking delay of the interaction identifier, and thus is beneficial to improving the user's interaction experience with the near-eye display device.

[0107] Please refer to Figure 3 , Figure 3 FIG. is a schematic block diagram of a display device of a near-eye display device provided by an embodiment of the present application. The display device of the near-eye display device can be configured in the near-eye display device or the server and is used to execute the foregoing display method of the near-eye display device. The near-eye display device may include an AR glasses, a VR glasses, an MR glasses, an AR helmet, a VR helmet, an MR helmet, etc., which are not limited herein. The server may be a separate server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0108] As Figure 3 shown, the display device of the near-eye display device includes an identifier display module 110, a position adjustment module 120, and a trajectory display module 130.

[0109] The identifier display module 110 is configured to display an interaction identifier on the display interface of the near-eye display device;

[0110] The position adjustment module 120 is configured to respond to a user's interaction action on the near-eye display device and adjust the display position of the interaction identifier on the display interface;

[0111] The trajectory display module 130 is configured to display a motion trajectory corresponding to the interaction identifier on the display interface according to the change in the display position of the interaction identifier.

[0112] Exemplarily, the position adjustment module 120 includes a current position acquisition sub-module, a position change determination sub-module, a position transformation sub-module, and an identification display sub-module.

[0113] The current position acquisition sub-module is configured to acquire the current display position of the interaction identifier on the display interface.

[0114] The position change determination sub-module is configured to determine the position change information corresponding to the interaction action.

[0115] The position transformation sub-module is configured to perform a position transformation process on the current display position according to the position change information to obtain the target display position of the interaction identifier.

[0116] The identification display sub-module is configured to display the interaction identifier at the target display position.

[0117] Exemplarily, the position transformation sub-module includes a position prediction sub-module and a target position determination sub-module.

[0118] The position prediction sub-module is configured to perform a position transformation process on the current display position according to the moving distance and moving direction indicated by the position change information to obtain the predicted display position of the interaction identifier.

[0119] The target position determination sub-module is configured to perform a position transformation process on the predicted display position according to the interface reference position of the display interface to obtain the target display position of the interaction identifier.

[0120] Exemplarily, the target position determination sub-module includes a matrix determination sub-module and a first position transformation sub-module.

[0121] The matrix determination sub-module is configured to determine the rotation matrix corresponding to the predicted display position according to the reference normal vector corresponding to the interface reference position of the display interface and the normal vector corresponding to the predicted display position.

[0122] The first position transformation sub-module is configured to perform a position transformation process on the predicted display position according to the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position to obtain the target display position.

[0123] Exemplarily, the position adjustment module 120 further includes an identification texture filling sub-module.

[0124] The identification texture filling sub-module is configured to perform a texture filling process on the interaction identifier according to the target display position and the first preset texture information to obtain the processed interaction identifier.

[0125] The identification display sub-module includes a first identification display sub-module.

[0126] The first identification display sub-module is used to display the processed interaction identification at the target display position.

[0127] Exemplarily, the trajectory display module 130 includes a historical position acquisition sub-module, a position offset sub-module, and a trajectory determination sub-module.

[0128] The historical position acquisition sub-module is used to acquire multiple historical display positions of the interaction identification;

[0129] The position offset sub-module is used to perform position offset processing on the historical display positions according to preset offset parameters to obtain processed historical display positions;

[0130] The trajectory determination sub-module is used to determine and display the motion trajectory corresponding to the interaction identification according to multiple processed historical display positions.

[0131] Exemplarily, the trajectory display module 130 further includes a trajectory texture filling sub-module.

[0132] The trajectory texture filling sub-module is used to perform texture filling processing on the motion trajectory corresponding to the interaction identification according to multiple processed historical display positions and second preset texture information to obtain a processed motion trajectory;

[0133] The trajectory determination sub-module includes a trajectory display sub-module.

[0134] The trajectory display sub-module is used to display the processed motion trajectory at multiple processed historical display positions.

[0135] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described device and each module and unit can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0136] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0137] Exemplarily, the above method and device can be implemented in the form of a computer program, which can run on a near-eye display device or a server to control the near-eye display device. Exemplarily, the near-eye display device can include AR glasses, VR glasses, MR glasses, AR helmets, VR helmets, MR helmets, etc., without limitation here. The server can be a single server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0138] Please refer to Figure 4 , Figure 4 which is a schematic block diagram of the structure of a near-eye display device provided by an embodiment of the present application.

[0139] As Figure 4 shown, the near-eye display device includes a memory and a processor. Among them, the memory and the processor can be connected through a system bus, and the memory can include a storage medium and an internal memory.

[0140] The storage medium can store an operating system and a computer program. When the computer program is executed, the processor can be made to execute any display method of the near-eye display device.

[0141] The processor is used to provide computing and control capabilities to support the operation of the entire near-eye display device.

[0142] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can be made to execute any display method of the near-eye display device.

[0143] Those skilled in the art can understand thatFigure 4 The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the near-eye display device to which the solution of this application is applied. Specifically, the near-eye display device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0144] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0145] Among them, in one embodiment, the processor is used to execute a computer program and implement the following steps when executing the computer program:

[0146] Display an interaction identifier in the display interface of the near-eye display device;

[0147] In response to a user's interaction action on the near-eye display device, adjust the display position of the interaction identifier in the display interface;

[0148] According to the change in the display position of the interaction identifier, display the movement trajectory corresponding to the interaction identifier on the display interface.

[0149] It should be noted that those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the display of the near-eye display device described above can refer to the corresponding process in the embodiment of the display method of the near-eye display device described above, and will not be repeated here.

[0150] The embodiment of this application also provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor may refer to each embodiment of the display method of the near-eye display device of this application.

[0151] Among them, the computer-readable storage medium may be an internal storage unit of the near-eye display device described in the foregoing embodiments, such as the hard disk or memory of the near-eye display device. The computer-readable storage medium may also be an external storage device of the near-eye display device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the near-eye display device.

[0152] It should be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. As used in the specification of this application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0153] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations. It should be noted that in this document, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or system comprising the element.

[0154] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments. The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A display method for a near-eye display device, characterized in that: include: Displaying an interactive identifier in a display interface of a near-eye display device; In response to a user's interactive action on the near-eye display device, adjusting a display position of the interactive marker on the display interface; According to the change of the display position of the interaction mark, the motion track corresponding to the interaction mark is displayed on the display interface.

2. The display method according to claim 1, characterized in that: The adjusting the display position of the interaction mark on the display interface in response to the user's interaction action on the near-eye display device includes: Acquire the current display position of the interaction mark on the display interface; Determining position change information corresponding to the interactive action; According to the position change information, the current display position is subjected to position change processing to obtain a target display position of the interactive identifier; The interactive mark is displayed at the target display position.

3. The display method according to claim 2, characterized in that: The performing position transformation processing on the current display position according to the position change information to obtain the target display position of the interactive identifier includes: According to the moving distance and the moving direction indicated by the position change information, the current display position is subjected to position transformation processing to obtain a predicted display position of the interactive identifier; The predicted display position is transformed according to the interface reference position of the display interface to obtain the target display position of the interaction identifier.

4. The display method according to claim 3, characterized in that: The performing position transformation processing on the predicted display position according to the interface reference position of the display interface to obtain the target display position of the interaction identifier includes: Determining a rotation matrix corresponding to the predicted display position according to a reference normal vector corresponding to the interface reference position of the display interface and a normal vector corresponding to the predicted display position; The predicted display position is transformed according to the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position to obtain a target display position.

5. The display method according to claim 2, characterized in that: After performing position transformation processing on the current display position according to the position change information to obtain the target display position of the interactive identifier, the method further includes: According to the target display position and the first preset texture information, the interactive mark is subjected to texture filling processing to obtain a processed interactive mark; The step of displaying the interactive identifier at the target display position includes: The processed interaction mark is displayed at the target display position.

6. The display method according to any one of claims 1 to 5, characterized in that: The step of displaying the motion track corresponding to the interaction mark on the display interface according to the change of the display position of the interaction mark includes: Acquire multiple historical display positions of the interaction identifier; Performing position shift processing on the historical display position according to a preset shift parameter to obtain a processed historical display position; According to the plurality of processed historical display positions, a motion track corresponding to the interaction identifier is determined and displayed.

7. The display method according to claim 6, characterized in that: After determining the motion track corresponding to the interaction mark according to the plurality of processed historical display positions, the method further includes: According to the plurality of processed historical display positions and the second preset texture information, a texture filling process is performed on the motion trajectory corresponding to the interaction identifier to obtain a processed motion trajectory; The step of displaying the motion track corresponding to the interaction identifier according to the plurality of processed historical display positions includes: The processed motion trajectory is displayed at a plurality of the processed history display positions.

8. A display device of a near-eye display device, characterized in that: The display device comprises: A logo display module, used to display an interactive logo in a display interface of a near-eye display device; A position adjustment module, configured to adjust a display position of the interaction mark on the display interface in response to a user's interaction action on the near-eye display device; The trajectory display module is used to display the motion trajectory corresponding to the interaction mark on the display interface according to the change of the display position of the interaction mark.

9. A near-eye display device, characterized in that: The near-eye display device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the steps of the display method of the near-eye display device according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the display method of the near-eye display device according to any one of claims 1 to 7 are implemented.

Images