Display method and apparatus for near-eye display device, and device and storage medium

Abstract

A display method and apparatus for a near-eye display device, and a device and a storage medium. The method comprises: displaying an interaction indicator on a display interface of a near-eye display device (S101); in response to an interaction action of a user on the near-eye display device, adjusting a display position of the interaction indicator on the display interface (S102); and on the basis of a change in the display position of the interaction indicator, displaying, on the display interface, a motion trajectory corresponding to the interaction indicator (S103).

Description

Display methods, devices, equipment and storage media for near-eye display devices

[0001] This application claims priority to Chinese Patent Application No. 2025103618822, filed on March 25, 2025, entitled “Display Method, Apparatus, Device and Storage Medium for Near-Eye Display Device”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of interactive technology, and in particular to a display method, apparatus, device and storage medium for a near-eye display device. Background Technology

[0003] Near-eye display devices (NIVs) feature interactive motion tracking capabilities. For example, NIVs can track user eye movements, head movements, and so on. During interactive motion tracking, the display interface of a NIV can show corresponding interactive indicators. However, in related technologies, the interactive indicators displayed by NIVs are often instantaneous points, which can easily lead to the interactive indicators being difficult for users to notice or ignore, resulting in poor display quality. Summary of the Invention

[0004] The main objective of this application is to provide a display method, apparatus, device, and storage medium for a near-eye display device, aiming to solve the technical problem that the display effect of interactive symbols on the near-eye display device is poor because the interactive symbols displayed by the near-eye display device are not easily noticed or ignored by the user during interactive motion tracking.

[0005] In a first aspect, this application provides a display method for a near-eye display device, comprising:

[0006] Display interactive icons in the display interface of near-eye display devices;

[0007] In response to the user's interaction with the near-eye display device, the display position of the interaction icon on the display interface is adjusted;

[0008] Based on the change in the display position of the interactive icon, the motion trajectory corresponding to the interactive icon is displayed on the display interface.

[0009] Secondly, this application provides a display device for a near-eye display device, the display device comprising:

[0010] The signage display module is used to display interactive signs on the display interface of near-eye display devices;

[0011] A position adjustment module is used to adjust the display position of the interaction icon on the display interface in response to the user's interaction with the near-eye display device.

[0012] The trajectory display module is used to display the motion trajectory corresponding to the interactive identifier on the display interface according to the change in the display position of the interactive identifier.

[0013] Thirdly, this application provides a near-eye display device, which includes a memory and a processor;

[0014] The memory is used to store computer programs;

[0015] The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the display method for the near-eye display device as described above.

[0016] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the display method for a near-eye display device as described above.

[0017] This application provides a display method, apparatus, device, and storage medium for a near-eye display device. The display method includes: displaying an interactive symbol on the display interface of the near-eye display device; adjusting the display position of the interactive symbol on the display interface in response to a user's interactive action on the near-eye display device; and displaying the motion trajectory corresponding to the interactive symbol on the display interface according to the change in the display position of the interactive symbol.

[0018] When displaying interactive icons on the interface of a near-eye display device, adjusting the display position of the interactive icons in response to user interactions allows the near-eye display device to track these interactions, ensuring the adjustment of the icon's position matches the user's interaction needs. Consequently, when the display position of the interactive icon changes, its corresponding motion trajectory can be displayed on the interface. This motion trajectory helps the user see the interactive icon. Displaying the interactive icon and its corresponding motion trajectory on the near-eye display device allows users to see the interactive icon more easily and conveniently, thus improving the display effect. Furthermore, when displaying the motion trajectory of the interactive icon, the trajectory smoothly reflects changes in the icon's position on the interface, reducing dizziness caused by inaccurate or delayed tracking, thereby enhancing the user's interactive experience. Attached Figure Description

[0019] Figure 1 is a flowchart illustrating a display method for a near-eye display device according to an embodiment of this application;

[0020] Figure 2 is a flowchart illustrating the process of determining the motion trajectory corresponding to the interactive identifier in an embodiment of this application;

[0021] Figure 3 is a schematic block diagram of a display device for a near-eye display device provided in an embodiment of this application;

[0022] Figure 4 is a schematic block diagram of a near-eye display device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0025] This application provides a display method, apparatus, device, and storage medium for a near-eye display device. The display method for this near-eye display device can be applied to near-eye display devices. Near-eye display devices may include augmented reality (AR) glasses, virtual reality (VR) glasses, mixed reality (MR) glasses, AR headsets, VR headsets, MR headsets, etc., and are not limited thereto. The display method for this near-eye display device can also be applied to a server, which can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0026] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] Please refer to Figure 1, which is a schematic flowchart of a display method for a near-eye display device according to an embodiment of this application. It should be noted that the display method for a near-eye display device provided in this embodiment can be used in a near-eye display device or a server, and is not limited thereto.

[0028] As shown in Figure 1, the display method of the near-eye display device includes steps S101 to S103.

[0029] S101. Display interactive icons in the display interface of the near-eye display device.

[0030] Near-eye display devices possess display functionality. By utilizing relevant components, they can display a visually perceptible interface. For example, in the case of AR glasses, the lenses of the AR glasses can serve as the display medium, presenting a visually perceptible interface. Similarly, in the case of VR headsets, the goggles of the VR headset can act as the display medium, presenting a visually perceptible interface. However, these are not exhaustive and are not limited to here.

[0031] Near-eye display devices can display interactive icons on the display interface. Interactive icons can have different display formats. For example, interactive icons can include a cursor. The cursor can exist in the form of a preset shape, a static image, an animated image, etc., without limitation.

[0032] Interactive icons displayed on the display interface of near-eye display devices can serve as a basis for users to interact with the display interface.

[0033] In some implementations, the display position of the interaction indicator on the display interface can be used to indicate the user's interaction position when operating the display interface. When the user sees the interaction indicator on the display interface, the user can determine their interaction position with the near-eye display device's display interface based on the display position of the interaction indicator. Accordingly, the user determines whether further interactive actions are needed based on the interaction position on the display interface.

[0034] For example, the display interface of a near-eye display device can also display an interactive object. The interactive object can include at least one of the following: an application identifier corresponding to an application, a system component corresponding to the display interface, a message notification identifier, etc., without limitation. When a user has an interactive need for the interactive object on the display interface, if the display position of the interactive identifier is not within the corresponding position range of the interactive object on the display interface, the user can determine that their interaction position on the near-eye display device's display interface is not within the corresponding position range of the interactive object. Therefore, the user can determine that they will need to adjust the display position of the interactive identifier through subsequent interactions with the near-eye display device. Conversely, if the display position of the interactive identifier is within the corresponding position range of the interactive object, the user can determine that their interaction position on the near-eye display device's display interface is within the corresponding position range of the interactive object. Therefore, the user can subsequently perform corresponding interactive operations on the interactive object according to their interaction needs to control the near-eye display device to execute the function corresponding to the interactive object.

[0035] Therefore, displaying interactive icons on the display interface of a near-eye display device allows users to interact with the interface based on these icons, thus improving the ease of interaction between the user and the near-eye display device. Correspondingly, the interactive icons displayed on the near-eye display device's interface can intuitively indicate the user's interaction position on the display interface, improving the user's ability to determine the interaction position and consequently enhancing the user's interactive experience with the near-eye display device.

[0036] S102. In response to the user's interaction with the near-eye display device, adjust the display position of the interaction indicator on the display interface.

[0037] Near-eye display devices can have interactive motion tracking capabilities. For example, a near-eye display device can track user interactions with the device. Exemplary interactions may include eye movements, hand movements, head movements, etc., without limitation. When a near-eye display device tracks user interactions, it can perform eye tracking, hand movement tracking, head movement tracking, etc., without limitation.

[0038] In some implementations, the near-eye display device can adjust the display position of the interaction indicator on the display interface in response to user interaction with the near-eye display device. The display position of the interaction indicator on the display interface can include three-dimensional coordinates or two-dimensional coordinates, and is not limited thereto.

[0039] Taking eye-tracking actions as an example, when a near-eye display device detects a user's eye moving to the left, it can respond by adjusting the display position of the interactive icon to the left. Similarly, when it detects a user's eye moving to the lower right, it can adjust the display position of the interactive icon to the lower right. And so on. Of course, interactive actions are not limited to eye-tracking actions, and this will not be discussed further.

[0040] In some implementations, when a user has an interactive need for an interactive object on the display interface, if the display position of the interactive indicator is not within the corresponding position range of the interactive object on the display interface, the user can perform corresponding interactive actions on the near-eye display device based on the display position of the interactive indicator and the corresponding position range of the interactive object, so that the interactive indicator gradually moves closer to the interactive object until the display position of the interactive indicator is within the corresponding position range of the interactive object.

[0041] For example, if the location of the object to be interacted with is to the right of the display position of the interactive indicator, the user can perform an interactive action on the near-eye display device to instruct the interactive indicator to move to the right. This could include moving the user's eyes, hand, or head to the right, etc., without limitation. The near-eye display device can then respond to the detected interactive action by adjusting the display position of the interactive indicator on the display interface to the right. If the display position of the interactive indicator is still not within the location range corresponding to the object to be interacted with, the user can continue to perform corresponding interactive actions on the near-eye display device based on the display position of the interactive indicator and the location range corresponding to the object to be interacted with, until the display position of the interactive indicator is within the location range corresponding to the object to be interacted with. Accordingly, when the display position of the interactive indicator is within the location range corresponding to the object to be interacted with, the user can control the near-eye display device to perform the function corresponding to the object to be interacted with through interactive operations. Interactive operations can include blinking, gazing, eye movements after gazing, preset voice, preset gestures, etc., without limitation. Taking an application identifier corresponding to an application as an example, if the near-eye display device detects that the display position of the interaction identifier is within the position range corresponding to the application identifier, and detects an interaction operation on the application identifier, the near-eye display device can, for example, display the application interface on the display screen. Of course, it is not limited to this, and no limitation is made here.

[0042] When a near-eye display device adjusts the display position of interactive icons in response to user interaction actions, the near-eye display device can determine the user's interaction needs for the interactive icons based on the user's interaction actions, and adjust the display position of the interactive icons accordingly. This improves the ease of adjusting the display position of interactive icons on the near-eye display device, and thus enhances the user's interactive experience with the near-eye display device.

[0043] S103. Based on the change in the display position of the interactive symbol, display the motion trajectory corresponding to the interactive symbol on the display interface.

[0044] For example, when the display position of the interactive symbol changes, the near-eye display device can display the motion trajectory corresponding to the interactive symbol on the display interface. Accordingly, in response to each change in the display position of the interactive symbol, the near-eye display device can update the motion trajectory corresponding to the interactive symbol displayed on the display interface.

[0045] In some implementations, the near-eye display device can determine whether the display position of the interactive icon has changed based on its current display position and its historical display position. Accordingly, if the near-eye display device determines that the display position of the interactive icon has changed, it can display the corresponding motion trajectory of the interactive icon on the display interface based on this change. For example, consider the current display position of the interactive icon at the current time as position 1, and its historical display position at historical time 1 as position 2, where historical time 1 falls within a preset time period corresponding to the current time. If position 1 and position 2 are different display positions, the near-eye display device can determine that the display position of the interactive icon has changed. Based on this, the near-eye display device can determine the motion trajectory corresponding to the interactive icon based on position 1 and position 2. For example, the motion trajectory corresponding to the interactive icon may include a shape pointing from position 2 to position 1. Alternatively, the motion trajectory corresponding to the interactive icon may include non-directional shapes corresponding to positions 1 and 2. Of course, the motion trajectory corresponding to the interactive icon is not limited to these examples, and is not restricted here.

[0046] In other embodiments, the near-eye display device can determine whether the display position of the interactive icon has changed based on its historical display position. Accordingly, if the display position of the interactive icon has changed, the near-eye display device can display the corresponding motion trajectory of the interactive icon on the display interface based on its historical display position. For example, consider the current display position of the interactive icon at the current time as position 1, its historical display position at historical time 1 as position 2, and its historical display position at historical time 2 as position 3, where both historical time 1 and historical time 2 are within a preset time period corresponding to the current time. Even if positions 2 and 3 are different display positions, the near-eye display device can determine that the display position of the interactive icon has changed regardless of whether positions 1 and 2 or positions 1 and 3 are the same. Based on this, the near-eye display device can determine the motion trajectory corresponding to the interactive icon based on positions 2 and 3. For example, if historical time 2 is earlier than historical time 1, the motion trajectory corresponding to the interactive icon may include a shape pointing from position 3 to position 2. Similarly, if historical time 1 is earlier than historical time 2, the motion trajectory corresponding to the interactive icon may include a shape pointing from position 2 to position 3. For example, the motion trajectory corresponding to the interactive identifier can include the non-directional shapes corresponding to positions 2 and 3. Of course, the motion trajectory corresponding to the interactive identifier is not limited to this, and no restriction is made here.

[0047] Once the motion trajectory corresponding to the interactive identifier is determined, the near-eye display device can display the motion trajectory corresponding to the interactive identifier on the display interface.

[0048] For example, when a user has an interaction need with an interactive object, the user can determine whether the interaction needs adjustment by considering the location range of the interactive object, the movement trajectory of the interactive indicator, and the display position of the interactive indicator. If it is determined that the interaction needs adjustment, the user can adjust at least one of the movement direction or movement distance of the interactive action. For instance, if the display position of the interactive indicator is between the location range of the interactive object and the movement trajectory of the interactive indicator, but not within the location range of the interactive object, the user can determine that the interactive indicator is gradually moving closer to the interactive object under the influence of the movement direction indicated by the interactive action, and can therefore adjust the movement distance of the interactive action indicator. Similarly, if the display position of the interactive indicator is outside the location range of the interactive object and the movement trajectory of the interactive indicator, and not within the location range of the interactive object, the user can determine that the interactive indicator is gradually moving away from the interactive object under the influence of the movement direction indicated by the interactive action, and can therefore adjust the movement direction of the interactive action indicator. Of course, this is not a limitation and is not set forth here. Based on this, the motion trajectory corresponding to the interactive icon can serve as a basis for users to adjust their interactive actions on the near-eye display device. This motion trajectory can prompt users to adjust their interactive actions, improving the ease with which the near-eye display device prompts users to adjust their interactive actions, thereby enhancing the user's interactive experience. Furthermore, since the motion trajectory corresponding to the interactive icon is related to changes in the icon's display position, it can help users better and more accurately view the interactive icons displayed on the interface, thus improving the display effect of the interactive icons on the near-eye display device.

[0049] In one exemplary embodiment, the near-eye display device can, within its rendering engine, call functions related to the display of interactive markers, the adjustment of the display position of interactive markers, and the display of the motion trajectory corresponding to the interactive markers. This allows the near-eye display device to represent the display process of interactive markers, the adjustment process of the display position of interactive markers, and the display process of the motion trajectory corresponding to the interactive markers as a complete rendering pipeline. Consequently, the calculations involved in the display process of interactive markers, the adjustment process of the display position of interactive markers, and the display process of the motion trajectory corresponding to the interactive markers can be executed efficiently on the graphics processing unit (GPU). This, in turn, helps to improve the tracking accuracy of interactive actions by the near-eye display device and reduce the tracking latency of interactive actions by the near-eye display device.

[0050] When displaying the motion trajectory of an interactive symbol on the screen based on changes in its display position, the near-eye display device only needs to acquire the display position related to the change in the interactive symbol's display position to generate and display the corresponding motion trajectory. This improves the ease of determining the motion trajectory and reduces the hardware requirements for determining it, thereby lowering system costs. Since the display position of the interactive symbol can adjust in response to the user's interaction with the near-eye display device, the corresponding motion trajectory can reflect the trajectory of the user's interaction, further improving the ease of tracking interaction actions. Correspondingly, based on the ease of determining the motion trajectory and the improved ease of tracking interaction actions, the motion trajectory can smoothly display changes in the interactive symbol's display position on the screen. This helps reduce dizziness experienced by users due to inaccurate or delayed tracking of the interactive symbol, thus improving the user's interactive experience with the near-eye display device.

[0051] In some implementations, the current display position of the interactive identifier on the display interface is obtained; the position change information corresponding to the interactive action is determined; the current display position is transformed according to the position change information to obtain the target display position of the interactive identifier; and the interactive identifier is displayed at the target display position.

[0052] Near-eye display devices can obtain the current display position of the interactive identifier on the display interface. When the near-eye display device detects a user's interactive action, it can determine the positional change information corresponding to the interactive action.

[0053] For example, when tracking interactive actions, near-eye display devices can acquire the user's position before and after performing the action. Taking manual actions as an example, the device can determine the position of the user's hand within the space occupied by the display device before and after the action. Similarly, when tracking head movements, the device can determine the position of the user's head within the space occupied by the display device before and after the action. And so on.

[0054] Accordingly, near-eye display devices can determine the movement distance and direction corresponding to an interactive action based on the user's position before and after the action, i.e., they can determine the positional change information corresponding to the interactive action. Taking manual actions as an example, the near-eye display device can determine the movement distance and direction of a manual action based on the position of the user's hand within the space occupied by the near-eye display device before and after the action. Similarly, taking head movements as an example, the near-eye display device can determine the movement distance and direction of a head movement based on the position of the user's head within the space occupied by the near-eye display device before and after the action. And so on.

[0055] Given the positional change information corresponding to the interactive action, the near-eye display device can perform position transformation processing on the current display position based on the positional change information to obtain the target display position of the interactive icon, and then display the interactive icon at the target display position. For example, the near-eye display device can control the interactive icon to move a corresponding distance from the current display position according to the movement direction corresponding to the interactive action to obtain the target display position of the interactive icon. The distance the near-eye display device controls the interactive icon to move can be determined based on the movement distance corresponding to the interactive action. For example, the distance the near-eye display device controls the interactive icon to move can be directly proportional to the movement distance corresponding to the interactive action.

[0056] Taking an interactive action that includes a manual action, and the corresponding positional change information for the manual action including moving the hand upwards by a distance of 1, as an example, the near-eye display device can control the interactive icon to move upwards by a distance of 2 from its current display position based on the positional change information corresponding to the manual action, thus obtaining the target display position of the interactive icon. Here, distance 2 is directly proportional to distance 1.

[0057] Taking an interactive action that includes eye movements, and the corresponding positional change information for the eye movements including a distance of 3 to the right, as an example, the near-eye display device can control the interactive icon to move a distance of 4 to the right from its current display position based on the positional change information corresponding to the eye movements, thus obtaining the target display position of the interactive icon. Here, distance 4 is directly proportional to distance 3.

[0058] Of course, interactive actions and the corresponding positional change information are not limited to this, and no restrictions are imposed here.

[0059] In adjusting the display position of interactive icons on the screen, the near-eye display device uses the position change information corresponding to the interactive action as a basis to perform position transformation processing on the current display position of the interactive icon, obtain the target display position of the interactive icon, and then display the interactive icon at the target display position. This improves the ease of adjusting the display position of the interactive icon. Accordingly, since the display position of the interactive icon on the near-eye display device is adjusted based on the position change information corresponding to the interactive action, the adjustment of the display position of the interactive icon on the near-eye display device can adapt to the user's needs for adjusting the position of the interactive icon, thereby improving the user's interactive experience on the near-eye display device.

[0060] In some implementations, the current display position is transformed based on the movement distance and direction indicated by the position change information to obtain the predicted display position of the interactive icon; the predicted display position is then transformed based on the interface reference position of the display interface to obtain the target display position of the interactive icon.

[0061] In response to user interactions, near-eye display devices adjust the display position of interactive icons on the screen, and the device's posture also changes accordingly. Taking head movements as an example, since near-eye display devices are worn on the user's head, their posture changes when the user turns, tilts, or tilts their head back. Consequently, the position of the display interface within the space occupied by the near-eye display changes with the device's posture. For instance, when a user turns their head, the position of the display interface within the space rotates with the user's head, ensuring the display remains continuously visible within the user's field of vision.

[0062] When the orientation of a near-eye display device may change in response to user interactions, to improve the accuracy of determining the target display position of interactive icons, it is necessary to combine the interface reference position of the display interface to determine the target display position of the interactive icons. The interface reference position of the display interface is determined based on the location of the display interface on the near-eye display device. As the position range of the display interface in the space occupied by the near-eye display device changes with the orientation of the near-eye display device, the interface reference position will also change accordingly. The interface reference position may include, for example, the position of the center point of the display interface, but is not limited here.

[0063] For example, near-eye display devices can perform position transformation processing on the current display position of the interactive sign based on the movement distance and direction indicated by the position change information corresponding to the interactive action, so as to obtain the predicted display position of the interactive sign.

[0064] In one exemplary embodiment, the near-eye display device can move the current display position of the interactive icon a corresponding distance according to the movement direction indicated by the position change information to obtain the predicted display position of the interactive icon. The distance the interactive icon moves in the movement direction is determined based on the movement distance indicated by the position change information. For example, the distance the interactive icon moves in the movement direction can be directly proportional to the movement distance indicated by the position change information. The predicted display position of the interactive icon is equivalent to the target display position of the interactive icon when the device orientation of the near-eye display device remains unchanged. However, if the device orientation of the near-eye display device changes, the predicted display position may not accurately reflect the target display position of the interactive icon.

[0065] Based on this, near-eye display devices can determine the target display position of interactive signs by combining the predicted display position of the interactive sign with the interface reference position of the display interface.

[0066] For example, the interface reference position of the display interface changes with the device posture of the near-eye display device. Therefore, the interface reference position is located in the display interface coordinate system corresponding to the display interface after the device posture changes. The predicted display position of the interactive icon is equivalent to being located in the display interface coordinate system corresponding to the display interface before the device posture changes. The near-eye display device can combine the interface reference position to perform a position transformation on the predicted display position of the interactive icon, changing the interactive icon from the display interface coordinate system corresponding to the display interface before the device posture change to the display interface coordinate system corresponding to the display interface after the device posture change, thus obtaining the target display position of the interactive icon.

[0067] By performing position transformation processing on the current display position based on the movement distance and direction indicated by the position change information to obtain the predicted display position of the interactive icon, and by performing position transformation processing on the predicted display position based on the interface reference position to obtain the target display position of the interactive icon, the influence of device posture changes of the near-eye display device on the target display position of the interactive icon can be comprehensively considered during the determination of the target display position of the interactive icon. This helps to improve the accuracy of determining the target display position of the interactive icon. Accordingly, the improved accuracy of determining the target display position of the interactive icon based on the near-eye display device helps to improve the tracking accuracy of the near-eye display device for interactive actions, thereby improving the user's interactive experience with the near-eye display device.

[0068] In some implementations, a rotation matrix corresponding to the predicted display position is determined based on the reference normal vector corresponding to the interface reference position of the display interface and the normal vector corresponding to the predicted display position; the predicted display position is then transformed based on the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position to obtain the target display position.

[0069] For example, when determining the target display position of an interactive sign by combining the predicted display position of the interactive sign with the interface reference position of the display interface, the near-eye display device can obtain the normal vector corresponding to the predicted display position and the reference normal vector corresponding to the interface reference position. Accordingly, the near-eye display device can determine the angle and rotation axis between the normal vector corresponding to the display position and the reference normal vector corresponding to the interface reference position based on the normal vector corresponding to the predicted display position and the reference normal vector corresponding to the interface reference position. Then, based on the angle and rotation axis between the normal vector and the reference normal vector, it can determine the rotation matrix corresponding to the predicted display position. The rotation matrix corresponding to the predicted display position can be used to compensate for rotational changes in the device posture of the near-eye display device, such as changes in the device posture in response to the user's head rotation.

[0070] In one exemplary embodiment, the near-eye display device can acquire a vertex array corresponding to the interface reference position. For example, the vertex array may include a quadrilateral (quadVertices) array. The quadVertices array may include the vertex coordinates of each vertex of the quadrilateral corresponding to the interface reference position and the texture coordinates of the quadrilateral corresponding to the interface reference position. The vertex coordinates can be used to indicate the position of the quadrilateral corresponding to the interface reference position in the space where the near-eye display device is located. The texture coordinates can be used to indicate how the texture involved in the quadrilateral corresponding to the interface reference position is mapped onto the quadrilateral. The vertex coordinates and texture coordinates corresponding to the interface reference position can be set as a unit rectangle. The near-eye display device can calculate the initial normal vector of the unit rectangle using vector cross product and dot product. This initial normal vector is essentially obtained by normalizing the reference normal vector corresponding to the interface reference position. Accordingly, when the normal vector corresponding to the predicted display position is obtained, the near-eye display device can normalize the normal vector corresponding to the predicted display position to obtain a normalized normal vector. Given the initial normal vector and the normalized normal vector, the initial normal vector and the normalized normal vector can be compared to determine the angle between them and the axis of rotation. Then, based on the angle between the initial normal vector and the normalized normal vector and the axis of rotation, the rotation matrix corresponding to the predicted display position can be determined.

[0071] Based on this, the determination of the reference normal vector corresponding to the interface reference position, such as the calculation of the initial normal vector, can be used to correct the display position of the interactive icon, so as to improve the accuracy of determining the target display position of the interactive icon in the future.

[0072] For example, given a rotation matrix corresponding to the predicted display position, a near-eye display device can combine the rotation matrix, translation matrix, and scaling matrix corresponding to the predicted display position to perform a position transformation on the predicted display position. This transforms the interactive identifier from the display interface coordinate system corresponding to the display interface before the device pose change to the display interface coordinate system corresponding to the display interface after the device pose change, thus obtaining the target display position of the interactive identifier.

[0073] For example, the translation matrix corresponding to the predicted display position can be determined based on the user's interaction with the near-eye display device. Taking head movement as an example, when the user moves their head forward, their head shifts. Correspondingly, the device posture of the near-eye display device can change in response to the user's head displacement. Based on this, the near-eye display device can determine the translation matrix corresponding to the predicted display position based on the displacement of the user's head. The translation matrix corresponding to the predicted display position can be used to compensate for the translational changes in the device posture of the near-eye display device, such as the changes in the device posture caused by the user's head movement.

[0074] For example, the scaling matrix corresponding to the predicted display position can be determined based on the distance between the interactive icon and the display interface. For instance, a near-eye display device can adjust at least one of the display distances of the interactive icon and the display interface.

[0075] Near-eye display devices can adjust at least one of the display distances: the display distance of interactive signs and the display distance of the display interface. These related devices can include the lens module of the near-eye display device, such as a liquid crystal lens or a mechanically movable lens. Taking the adjustment of the display distance of interactive signs as an example, since the focal length of the lens module in a near-eye display device is variable, the device can dynamically adjust the focal length to change the light path related to the display of the interactive sign, thus allowing the user to perceive a change in the display distance. For example, if the interactive sign is at different depths before and after the focal length of the lens module changes, the near-eye display device can adjust the display distance of the interactive sign. Of course, the methods for adjusting the display distance of interactive signs are not limited to this, and are not specified here.

[0076] The near-eye display device can respond to a user's preset zoom operation on the near-eye display device by zooming in or out on at least one of the interactive icons and the display interface. The preset zoom operation may include zoom gestures, zoom voice commands, etc., without limitation. Zoom gestures may include gestures indicating zooming in or out on the size of at least one of the interactive icons and the display interface, and gestures indicating zooming out on the size of at least one of the interactive icons and the display interface. Zoom voice commands may include at least one voice command indicating zooming in on the size of the interactive icon, at least one voice command indicating zooming in on the size of the display interface, and at least one voice command indicating zooming out on the size of the interactive icon, and at least one voice command indicating zooming out on the size of the display interface. And so on.

[0077] In one exemplary embodiment, upon detecting a preset zoom operation, the near-eye display device can adjust the display distance of at least one of an interactive icon and a display interface, so that the user perceives a change (enlargement or reduction) in at least one of the sizes of the interactive icon and the display interface. For example, if the display distance of the interactive icon is reduced in response to a preset zoom operation, it is equivalent to a decrease in the distance between the interactive icon and the user's retina, and the user will perceive the size of the interactive icon as larger. Conversely, if the display distance of the interactive icon is increased in response to a preset zoom operation, it is equivalent to a increase in the distance between the interactive icon and the user's retina, and the user will perceive the size of the interactive icon as smaller. And so on.

[0078] In another exemplary embodiment, when a preset scaling operation is detected, the near-eye display device can adjust at least one of the size of the interactive icon and the size of the display interface. For example, if the detected preset scaling operation indicates that the size of the interactive icon should be increased, the size of the interactive icon can be increased. As another example, if the detected preset scaling operation indicates that the size of the interactive icon should be decreased, the size of the interactive icon can be decreased. And so on.

[0079] When a near-eye display device detects a change in the display distance of at least one of an interactive icon or a display interface presented to the user, it can determine the distance between the interactive icon and the display interface. For example, if the display distance of the interactive icon changes while the display distance of the display interface remains unchanged, and the display interface does not obstruct the interactive icon (i.e., the display distance of the interactive icon is less than the display distance of the display interface), then a decrease in the display distance of the interactive icon is equivalent to a decrease in the distance between the interactive icon and the user's retina. Therefore, if the distance between the display interface and the user's retina remains unchanged, the distance between the interactive icon and the display interface can be determined to have increased. Conversely, an increase in the display distance of the interactive icon is equivalent to an increase in the distance between the interactive icon and the user's retina. Therefore, if the distance between the display interface and the user's retina remains unchanged, the distance between the interactive icon and the display interface can be determined to have increased.

[0080] Based on this, near-eye display devices can determine the scaling matrix corresponding to the predicted display position based on the distance between the interactive icon and the display interface. This scaling matrix can be used to adjust the display ratio of the interactive icon. Enlarging the interactive icon makes it more visible to the user, improving the viewing experience. Reducing the interactive icon's size minimizes its obstruction of other objects (such as the object to be interacted with) on the display interface.

[0081] In one exemplary embodiment, given the predicted display position of the interactive identifier and the corresponding normal vector, the graphic to be rendered corresponding to the predicted display position can be determined based on the predicted display position and the corresponding normal vector. The graphic to be rendered can be a polygon, such as a triangle, quadrilateral, etc., and is not limited thereto. Given the predicted rotation matrix, translation matrix, and scaling matrix, the positions of each vertex of the graphic to be rendered can be transformed based on these matrices to obtain the target display positions of each vertex of the image to be rendered. Accordingly, the target display position of the interactive identifier can be determined based on the target display positions of each vertex of the image to be rendered.

[0082] Based on this, the determination of the rotation matrix, translation matrix, and scaling matrix corresponding to the predicted display position can be used to make secondary corrections to the display position of the interactive logo, so as to determine the target display position of the interactive logo, which helps to improve the accuracy of determining the target display position of the interactive logo.

[0083] By determining the rotation matrix corresponding to the predicted display position based on the reference normal vector corresponding to the interface reference position and the normal vector corresponding to the predicted display position, and then performing a position transformation on the predicted display position based on the rotation matrix, the translation matrix, and the scaling matrix corresponding to the predicted display position to obtain the target display position, the coordinate system of the interactive identifier can be changed from the display interface coordinate system corresponding to the display interface before the device pose change to the display interface coordinate system corresponding to the display interface after the device pose change. This helps improve the accuracy of determining the target display position of the interactive identifier. Therefore, the target display position of the interactive identifier can more accurately reflect the user's interaction position with the display interface of the near-eye display device, thus improving the user's device interaction experience.

[0084] By performing position transformation processing on the predicted display position based on the rotation, translation, and scaling matrices corresponding to the predicted display position, the display position of the interactive icon can smoothly transition from the current display position to the target display position. This facilitates a smooth transition during position switching, reducing the likelihood of users experiencing dizziness when viewing the display interface and improving the user's device interaction experience.

[0085] In some implementations, the interactive identifier is texture-filled based on the target display position and the first preset texture information to obtain the processed interactive identifier.

[0086] Display the processed interactive identifier at the target display location.

[0087] For example, given a determined target display location for an interactive icon, the near-eye display device can display the interactive icon at that location. Accordingly, the near-eye display device can perform texture filling processing on the interactive icon to give it a corresponding texture. For instance, the near-eye display device can perform texture filling processing on the interactive icon based on the target display location and first preset texture information to obtain a processed interactive icon. Once the processed interactive icon is obtained, the near-eye display device can display it at the target display location. The first preset texture information may include a first preset color and / or a first preset pattern, etc., and is not limited thereto.

[0088] In one exemplary embodiment, when the near-eye display device determines the graphic to be rendered corresponding to the predicted display position of the interactive identifier, and determines the target display position of the interactive identifier based on the target display positions of each vertex of the graphic to be rendered, it can map first preset texture information onto the graphic to be rendered through external texture binding and texture object construction to perform texture filling processing on the graphic to be rendered. After texture filling processing on the graphic to be rendered, the processed interactive identifier can be obtained. The processed interactive identifier and the graphic to be rendered can be of different shapes or the same shape; this is not limited here.

[0089] When the processed interactive identifier is displayed at the target display location, the user can view the processed interactive identifier, which helps to improve the display diversity of interactive identifiers on near-eye display devices.

[0090] In some implementations, multiple historical display positions of the interactive identifier are obtained; the historical display positions are offset according to a preset offset parameter to obtain the processed historical display positions; and the motion trajectory corresponding to the interactive identifier is determined and displayed based on the multiple processed historical display positions.

[0091] For example, near-eye display devices can use a queue to store multiple historical display positions of interactive icons. Specifically, the near-eye display device can record the display positions of interactive icons in a first-in-first-out (FIFO) double-ended queue within the rendering engine, creating a historical display position queue corresponding to the interactive icon. This historical display position queue can be maintained in real-time according to a preset maximum capacity. The preset maximum capacity can include 10, 20, 50, 100, etc., and can be determined based on the computing power of the near-eye display device's GPU, without limitation. Accordingly, the near-eye display device can obtain multiple historical display positions of interactive icons from this historical display position queue. By storing the historical display positions of interactive icons, the near-eye display device can construct the motion trajectory corresponding to the interactive icon based on the historical display positions, which improves the ease of determining the motion trajectory corresponding to the interactive icon.

[0092] For example, the number of historical display positions acquired by the near-eye display device can be determined based on the movement speed of the user's interaction with the near-eye display device. As the movement speed of the interaction changes, the number of historical display positions of the interaction identifier acquired by the near-eye display device can dynamically change. For instance, the faster the movement speed of the interaction, the more historical display positions of the interaction identifier acquired by the near-eye display device. The slower the movement speed of the interaction, the fewer historical display positions of the interaction identifier acquired by the near-eye display device. Different movement speeds of the interaction can correspond one-to-one with different preset numbers of historical display positions acquired for the interaction identifier. In an exemplary embodiment, the minimum number of historical display positions acquired by the near-eye display device for the interaction identifier is, for example, 10, and the maximum number is, for example, 100. The number of historical display positions acquired by the near-eye display device for the interaction identifier can dynamically change between 10 and 100 in response to changes in the movement speed of the interaction. However, this is not a limitation and is not intended to restrict the scope of the invention.

[0093] The historical display positions stored in the historical display position queue corresponding to the interaction identifier can be cleared in response to a preset clear command. The preset clear command can be pre-set or user-defined; there are no restrictions here. For example, when a near-eye display device receives a power-off command, it can determine that a preset clear command for the historical display position queue has been detected, and thus clear the historical display positions stored in the queue. As another example, if the near-eye display device remains in standby mode for a duration greater than or equal to a preset time threshold, it can determine that a preset clear command for the historical display position queue has been detected, and thus clear the historical display positions stored in the queue. Of course, these are not the only possibilities; there are no restrictions here.

[0094] In an exemplary implementation, as shown in FIG2, the process of determining the motion trajectory corresponding to the interactive identifier in the near-eye display device includes: obtaining the current display position of the interactive identifier on the display interface; recording the current display position of the interactive identifier in the historical display position queue; determining whether the historical display position queue includes historical display positions other than the current display position; if yes, determining the motion trajectory corresponding to the interactive identifier based on the historical display positions included in the historical display position queue; if no, continuing to obtain the current display position of the interactive identifier on the display interface.

[0095] For example, if the historical display position queue does not include historical display positions other than the current display position, it can be determined that the display position of the interaction identifier has not changed, and therefore there is no need to display the motion trajectory corresponding to the interaction identifier on the display interface of the near-eye display device. As another example, if the current display position is recorded in the historical display position queue, the current display position can also be included as a historical display position in the historical display position queue to determine the motion trajectory corresponding to the interaction identifier; this is not restricted.

[0096] When multiple historical display positions of the interactive identifier are obtained, the near-eye display device can perform position offset processing on each historical display position according to the preset offset parameters to obtain the processed historical display position corresponding to each historical display position.

[0097] For example, consider a historical display position that includes both horizontal (x-coordinate) and vertical (y-coordinate) coordinates. A preset offset parameter can be used to indicate an offset of at least one of the x-coordinate and y-coordinate of the historical display position. For instance, the preset offset parameter can indicate increasing or decreasing the x-coordinate of the historical display position by a preset factor. Similarly, the preset offset parameter can indicate increasing or decreasing the y-coordinate of the historical display position by a preset factor. Preset factors include, for example, 1x, 2x, 10x, etc. Of course, the preset offset parameter and preset factor are not limited to these and are not restricted here.

[0098] When a historical display position is offset according to a preset offset parameter to obtain a processed historical display position, the number of processed historical display positions corresponding to the same historical display position can include at least one. Taking a historical display position including historical display position 1 and historical display position 2, and preset offset parameters including offset parameter 1 and offset parameter 2 as an example: The near-eye display device can offset historical display position 1 according to offset parameter 1 to obtain a processed historical display position 1a. The near-eye display device can offset historical display position 2 according to offset parameter 1 to obtain a processed historical display position 2a. The near-eye display device can further offset historical display position 2 according to offset parameter 2 to obtain a processed historical display position 2b. Based on this, it can be determined that the number of processed historical display positions corresponding to historical display position 1 is one, and the number of processed historical display positions corresponding to historical display position 2 is two. When determining the motion trajectory corresponding to the interactive identifier based on the processed historical display position, the near-eye display device can, for example, connect the processed historical display position 1a with the processed historical display position 2a, and connect historical display position 1 with historical display position 2, so that historical display position 1 and historical display position 2 can be expanded into a path with a certain width, thereby enabling the determination of the motion trajectory corresponding to the interactive identifier. Of course, it is not limited to this, and no limitation is made here.

[0099] In one exemplary embodiment, when a near-eye display device acquires multiple historical display positions of an interactive icon, it can determine the motion trajectory corresponding to the interactive icon based on these multiple historical display positions. For example, if the interactive icon is triangular in shape, a triangular trajectory corresponding to the interactive icon can be determined based on the multiple historical display positions. Accordingly, the near-eye display device can perform position offset processing on every two consecutive historical display positions among the multiple historical display positions according to a preset offset parameter, obtaining processed historical display positions corresponding to each of the multiple historical display positions, and then determine the motion trajectory corresponding to the interactive icon based on the processed historical display positions corresponding to each of the multiple historical display positions. For example, every two consecutive historical display positions can be used to determine a line segment. When performing position offset processing on every two consecutive historical display positions to obtain processed historical display positions corresponding to each of the multiple historical display positions, every two consecutive processed historical display positions among the multiple processed historical display positions can be used to determine another line segment. The near-eye display device can combine the two line segments corresponding to each two consecutive historical display positions before and after the position offset processing, expanding the multiple historical display positions into a path with a certain width, thereby determining the motion trajectory corresponding to the interactive icon. When the motion trajectory corresponding to the interactive symbol is determined, the near-eye display device can display the motion trajectory corresponding to the interactive symbol based on the processed historical display positions corresponding to multiple historical display positions.

[0100] When a near-eye display device combines multiple historical display positions of interactive icons with predicted offset parameters to determine the motion trajectory of the interactive icon and displays this trajectory, the display interface of the near-eye display device can continuously display the motion trajectory of the interactive icon. This motion trajectory can assist the user in viewing the interactive icon, thus improving the display effect of the interactive icon on the near-eye display device. Correspondingly, when the user's interaction with the near-eye display device involves a large amplitude, the display of the motion trajectory of the interactive icon can smoothly show the change in the display position of the interactive icon on the display interface. This helps reduce the dizziness experienced by users due to inaccurate tracking or tracking delay of the interactive icon, thereby improving the user's interactive experience with the near-eye display device.

[0101] In some implementations, the motion trajectory corresponding to the interactive identifier is texture-filled based on multiple processed historical display positions and second preset texture information to obtain the processed motion trajectory.

[0102] The processed motion trajectory is displayed at multiple processed historical display positions.

[0103] For example, when a near-eye display device determines the motion trajectory corresponding to an interactive icon, it can perform texture filling processing on the motion trajectory to give it a corresponding texture. For instance, the near-eye display device can perform texture filling processing on the motion trajectory based on multiple processed historical display positions and second preset texture information to obtain a processed motion trajectory. With the processed motion trajectory obtained, the near-eye display device can then display it at multiple processed historical display positions. The second preset texture information may include, for example, a second preset color or a second preset pattern. The second preset texture information can be different from or the same as the first preset texture information; this is not limited here.

[0104] In one exemplary embodiment, the second preset texture information may include a second preset color, such as light blue or light green, and the processed motion trajectory may be a motion trajectory filled with light blue or light green. Of course, the processed motion trajectory is not limited to this, and is not restricted here.

[0105] By performing texture filling on the motion trajectory corresponding to the interactive icon based on multiple processed historical display positions and second preset texture information, the ease of texture filling for the motion trajectory corresponding to the interactive icon is improved. Correspondingly, when the processed motion trajectory is displayed at multiple processed historical display positions, the user can view the processed motion trajectory, which helps to improve the display diversity of the motion trajectory corresponding to the interactive icon on near-eye display devices.

[0106] The display method for a near-eye display device provided in the above embodiments includes: displaying an interactive identifier in the display interface of the near-eye display device; adjusting the display position of the interactive identifier in the display interface in response to the user's interactive action on the near-eye display device; and displaying the motion trajectory corresponding to the interactive identifier on the display interface according to the change in the display position of the interactive identifier.

[0107] When displaying interactive icons on the interface of a near-eye display device, adjusting the display position of the interactive icons in response to user interactions allows the near-eye display device to track these interactions, ensuring the adjustment of the icon's position matches the user's interaction needs. Consequently, when the display position of the interactive icon changes, its corresponding motion trajectory can be displayed on the interface. This motion trajectory helps the user see the interactive icon. Displaying the interactive icon and its corresponding motion trajectory on the near-eye display device allows users to see the interactive icon more easily and conveniently, thus improving the display effect. Furthermore, when displaying the motion trajectory of the interactive icon, the trajectory smoothly reflects changes in the icon's position on the interface, reducing dizziness caused by inaccurate or delayed tracking, thereby enhancing the user's interactive experience.

[0108] Please refer to Figure 3, which is a schematic block diagram of a display device for a near-eye display device according to an embodiment of this application. The display device can be configured within a near-eye display device or a server to execute the aforementioned display method for the near-eye display device. The near-eye display device may include AR glasses, VR glasses, MR glasses, AR helmets, VR helmets, MR helmets, etc., and is not limited thereto. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0109] As shown in Figure 3, the display device of the near-eye display device includes an identification display module 110, a position adjustment module 120, and a trajectory display module 130.

[0110] The sign display module 110 is used to display interactive signs in the display interface of the near-eye display device;

[0111] The position adjustment module 120 is used to adjust the display position of the interaction mark on the display interface in response to the user's interaction with the near-eye display device.

[0112] The trajectory display module 130 is used to display the motion trajectory corresponding to the interactive identifier on the display interface according to the change in the display position of the interactive identifier.

[0113] For example, the position adjustment module 120 includes a current position acquisition submodule, a position change determination submodule, a position transformation submodule, and an identification display submodule.

[0114] The current position acquisition submodule is used to acquire the current display position of the interaction identifier on the display interface.

[0115] The position change determination submodule is used to determine the position change information corresponding to the interactive action.

[0116] The position transformation submodule is used to perform position transformation processing on the current display position according to the position change information to obtain the target display position of the interactive identifier.

[0117] The identifier display submodule is used to display the interactive identifier at the target display location.

[0118] For example, the position transformation submodule includes a position prediction submodule and a target position determination submodule.

[0119] The location prediction submodule is used to perform location transformation processing on the current display position based on the movement distance and movement direction indicated by the location change information, so as to obtain the predicted display position of the interactive identifier.

[0120] The target location determination submodule is used to perform position transformation processing on the predicted display location based on the interface reference location of the display interface to obtain the target display location of the interactive identifier.

[0121] For example, the target position determination submodule includes a matrix determination submodule and a first position transformation submodule.

[0122] The matrix determination submodule is used to determine the rotation matrix corresponding to the predicted display position based on the reference normal vector corresponding to the interface reference position of the display interface and the normal vector corresponding to the predicted display position.

[0123] The first position transformation submodule is used to perform position transformation processing on the predicted display position according to the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position to obtain the target display position.

[0124] For example, the position adjustment module 120 also includes an identification texture fill submodule.

[0125] The identifier texture filling submodule is used to perform texture filling processing on the interactive identifier according to the target display position and the first preset texture information to obtain the processed interactive identifier.

[0126] The identifier display submodule includes the first identifier display submodule.

[0127] The first identifier display submodule is used to display the processed interactive identifier at the target display position.

[0128] For example, the trajectory display module 130 includes a historical location acquisition submodule, a location offset submodule, and a trajectory determination submodule.

[0129] The historical location acquisition submodule is used to acquire multiple historical display locations of the interaction identifier;

[0130] The position offset submodule is used to perform position offset processing on the historical display position according to the preset offset parameters to obtain the processed historical display position;

[0131] The trajectory determination submodule is used to determine and display the motion trajectory corresponding to the interactive identifier based on multiple processed historical display positions.

[0132] For example, the trajectory display module 130 also includes a trajectory texture filling submodule.

[0133] The trajectory texture filling submodule is used to perform texture filling processing on the motion trajectory corresponding to the interactive identifier based on multiple processed historical display positions and second preset texture information to obtain the processed motion trajectory.

[0134] The trajectory determination submodule includes the trajectory display submodule.

[0135] The trajectory display submodule is used to display the processed motion trajectory at multiple processed historical display positions.

[0136] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the above-described apparatus and its modules and units can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0137] The method of this application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0138] For example, the above-described method and apparatus can be implemented as a computer program that can run on a near-eye display device or a server to control the near-eye display device. For example, the near-eye display device may include AR glasses, VR glasses, MR glasses, AR helmets, VR helmets, MR helmets, etc., without limitation. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0139] Please refer to Figure 4, which is a schematic block diagram of the structure of a near-eye display device provided in an embodiment of this application.

[0140] As shown in Figure 4, the near-eye display device includes a memory and a processor. The memory and processor can be connected via a system bus, and the memory may include a storage medium and internal memory.

[0141] The storage medium can store the operating system and computer programs. When the computer program is executed, it enables the processor to perform any display method of the near-eye display device.

[0142] The processor provides computing and control capabilities to support the operation of the entire near-eye display device.

[0143] Internal memory provides an environment for the execution of computer programs stored in storage media. When these computer programs are executed by the processor, the processor can execute any display method of a near-eye display device.

[0144] Those skilled in the art will understand that the structure shown in Figure 4 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the near-eye display device to which the present application is applied. A specific near-eye display device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0145] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other convertible logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0146] In one embodiment, the processor is configured to execute a computer program and, when executing the computer program, perform the following steps:

[0147] Display interactive icons in the display interface of near-eye display devices;

[0148] In response to the user's interaction with the near-eye display device, the display position of the interaction icon on the display interface is adjusted;

[0149] Based on the change in the display position of the interactive icon, the motion trajectory corresponding to the interactive icon is displayed on the display interface.

[0150] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the near-eye display device described above can be referred to the corresponding process in the aforementioned near-eye display device display method embodiments, and will not be repeated here.

[0151] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method implemented can be referred to in various embodiments of the display method of the near-eye display device of this application.

[0152] The computer-readable storage medium can be an internal storage unit of the near-eye display device described in the foregoing embodiments, such as the hard disk or memory of the near-eye display device. Alternatively, the computer-readable storage medium can be an external storage device of the near-eye display device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the near-eye display device.

[0153] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A display method for a near-eye display device, comprising: Display interactive icons in the display interface of near-eye display devices; In response to the user's interaction with the near-eye display device, the display position of the interaction icon on the display interface is adjusted; Based on the change in the display position of the interactive icon, the motion trajectory corresponding to the interactive icon is displayed on the display interface.

2. The display method according to claim 1, wherein, The step of adjusting the display position of the interaction indicator on the display interface in response to a user's interaction with the near-eye display device includes: Obtain the current display position of the interaction identifier on the display interface; Determine the position change information corresponding to the interactive action; Based on the position change information, the current display position is transformed to obtain the target display position of the interactive identifier; The interactive identifier is displayed at the target display location.

3. The display method according to claim 2, wherein, The step of performing a position transformation process on the current display position based on the position change information to obtain the target display position of the interactive identifier includes: Based on the movement distance and direction indicated by the position change information, the current display position is transformed to obtain the predicted display position of the interactive identifier; Based on the interface reference position of the display interface, the predicted display position is transformed to obtain the target display position of the interactive identifier.

4. The display method according to claim 3, wherein, The step of performing a position transformation process on the predicted display position based on the interface reference position of the display interface to obtain the target display position of the interactive identifier includes: Based on the reference normal vector corresponding to the interface reference position of the display interface and the normal vector corresponding to the predicted display position, determine the rotation matrix corresponding to the predicted display position; Based on the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position, the predicted display position is transformed to obtain the target display position.

5. The display method according to claim 4, wherein, The display method further includes: Based on the predicted display position and the normal vector corresponding to the predicted display position, determine the image to be rendered corresponding to the predicted display position; The step of performing a position transformation on the predicted display position based on the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position to obtain the target display position includes: Based on the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position, position transformation processing is performed on each vertex of the image to be rendered to obtain the target display position of each vertex of the image to be rendered. The target display position of the interactive identifier is determined based on the target display position of each vertex of the image to be rendered.

6. The display method according to claim 2, wherein, After performing a position transformation process on the current display position based on the position change information to obtain the target display position of the interactive identifier, the method further includes: Based on the target display position and the first preset texture information, the interactive identifier is subjected to texture filling processing to obtain the processed interactive identifier; Displaying the interactive identifier at the target display location includes: The processed interactive identifier is displayed at the target display location.

7. The display method according to claim 1, wherein, The step of displaying the motion trajectory corresponding to the interactive identifier on the display interface based on the change in the display position of the interactive identifier includes: Obtain multiple historical display positions of the interaction identifier; The historical display position is offset according to the preset offset parameter to obtain the processed historical display position; Based on the multiple processed historical display positions, the motion trajectory corresponding to the interactive identifier is determined and displayed.

8. The display method according to claim 7, wherein, After determining the motion trajectory corresponding to the interactive identifier based on multiple processed historical display positions, the method further includes: Based on the multiple processed historical display positions and the second preset texture information, the motion trajectory corresponding to the interactive identifier is texture-filled to obtain the processed motion trajectory. The step of displaying the motion trajectory corresponding to the interactive identifier based on multiple processed historical display positions includes: The processed motion trajectory is displayed at multiple historical display positions after processing.

9. The display method according to claim 1, wherein, The step of displaying the motion trajectory corresponding to the interactive identifier on the display interface based on the change in the display position of the interactive identifier includes: Obtain the current display position of the interaction identifier on the display interface; Record the current display position to the historical display position queue; When the historical display position queue includes historical display positions other than the current display position, the motion trajectory corresponding to the interaction identifier is determined based on the historical display positions included in the historical display position queue.

10. A display device for a near-eye display apparatus, the display device comprising: The signage display module is used to display interactive signs on the display interface of near-eye display devices; A position adjustment module is used to adjust the display position of the interaction icon on the display interface in response to the user's interaction with the near-eye display device. The trajectory display module is used to display the motion trajectory corresponding to the interactive identifier on the display interface according to the change in the display position of the interactive identifier.

11. A near-eye display device, the near-eye display device comprising a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, perform the following steps: Display interactive icons in the display interface of near-eye display devices; In response to the user's interaction with the near-eye display device, the display position of the interaction icon on the display interface is adjusted; Based on the change in the display position of the interactive icon, the motion trajectory corresponding to the interactive icon is displayed on the display interface.

12. The near-eye display device according to claim 11, wherein, When the processor adjusts the display position of the interaction identifier on the display interface in response to the user's interaction with the near-eye display device, it performs the following steps: Obtain the current display position of the interaction identifier on the display interface; Determine the position change information corresponding to the interactive action; Based on the position change information, the current display position is transformed to obtain the target display position of the interactive identifier; The interactive identifier is displayed at the target display location.

13. The near-eye display device according to claim 12, wherein, When the processor performs the position transformation processing on the current display position based on the position change information to obtain the target display position of the interactive identifier, it implements the following steps: Based on the movement distance and direction indicated by the position change information, the current display position is transformed to obtain the predicted display position of the interactive identifier; Based on the interface reference position of the display interface, the predicted display position is transformed to obtain the target display position of the interactive identifier.

14. The near-eye display device according to claim 13, wherein, When the processor performs the position transformation process on the predicted display position based on the interface reference position of the display interface to obtain the target display position of the interactive identifier, it implements the following steps: Based on the reference normal vector corresponding to the interface reference position of the display interface and the normal vector corresponding to the predicted display position, determine the rotation matrix corresponding to the predicted display position; Based on the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position, the predicted display position is transformed to obtain the target display position.

15. The near-eye display device according to claim 14, wherein, When the processor executes the computer program, it performs the following steps: Based on the predicted display position and the normal vector corresponding to the predicted display position, determine the image to be rendered corresponding to the predicted display position; The step of performing a position transformation on the predicted display position based on the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position to obtain the target display position includes: Based on the rotation matrix, the translation matrix corresponding to the predicted display position, and the scaling matrix corresponding to the predicted display position, position transformation processing is performed on each vertex of the image to be rendered to obtain the target display position of each vertex of the image to be rendered. The target display position of the interactive identifier is determined based on the target display position of each vertex of the image to be rendered.

16. The near-eye display device according to claim 12, wherein, After executing the position transformation process based on the position change information to obtain the target display position of the interactive identifier, the processor performs the following steps: Based on the target display position and the first preset texture information, the interactive identifier is subjected to texture filling processing to obtain the processed interactive identifier; When the processor executes the step of displaying the interactive identifier at the target display location, it performs the following steps: The processed interactive identifier is displayed at the target display location.

17. The near-eye display device according to claim 11, wherein, When the processor executes the step of displaying the motion trajectory corresponding to the interaction identifier on the display interface based on the change in the display position of the interaction identifier, it performs the following steps: Obtain multiple historical display positions of the interaction identifier; The historical display position is offset according to the preset offset parameter to obtain the processed historical display position; Based on the multiple processed historical display positions, the motion trajectory corresponding to the interactive identifier is determined and displayed.

18. The near-eye display device according to claim 17, wherein, After executing the step of determining the motion trajectory corresponding to the interactive identifier based on multiple processed historical display positions, the processor performs the following steps: Based on the multiple processed historical display positions and the second preset texture information, the motion trajectory corresponding to the interactive identifier is texture-filled to obtain the processed motion trajectory. The step of displaying the motion trajectory corresponding to the interactive identifier based on multiple processed historical display positions includes: The processed motion trajectory is displayed at multiple historical display positions after processing.

19. The near-eye display device according to claim 11, wherein, When the processor executes the step of displaying the motion trajectory corresponding to the interaction identifier on the display interface based on the change in the display position of the interaction identifier, it performs the following steps: Obtain the current display position of the interaction identifier on the display interface; Record the current display position to the historical display position queue; When the historical display position queue includes historical display positions other than the current display position, the motion trajectory corresponding to the interaction identifier is determined based on the historical display positions included in the historical display position queue.

20. A computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, performing the following steps: Display interactive icons in the display interface of near-eye display devices; In response to the user's interaction with the near-eye display device, the display position of the interaction icon on the display interface is adjusted; Based on the change in the display position of the interactive icon, the motion trajectory corresponding to the interactive icon is displayed on the display interface.

Images