Display method, device, equipment and storage medium of near-eye display device

Abstract

The application relates to the technical field of display, and provides a display method and device of a near-eye display equipment, equipment and a storage medium, the method comprises the following steps: acquiring a first environment image obtained by the near-eye display equipment shooting the environment; determining an object in the first environment image; acquiring preset information corresponding to the object in the first environment image; determining part of the preset information corresponding to the first environment image as to-be-displayed information; and displaying the to-be-displayed information according to a first area of the object corresponding to the to-be-displayed information in the first environment image, so as to improve the display effect of the to-be-displayed information by the near-eye display equipment.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a display method, apparatus, device and storage medium for a near-eye display device. Background Technology

[0002] Currently, when extracting and processing information, such as photo translation and object recognition, it is generally necessary to preview the extracted information. Users can then process the previewed information, such as filtering or selecting information, for subsequent display of the processed information.

[0003] However, due to limitations such as battery size and heat dissipation in near-eye display devices, if the near-eye display device is used to preview the extracted information, it is prone to problems such as information display lag, slow information display, and overheating, resulting in poor information display effect. Summary of the Invention

[0004] The main objective of this application is to provide a display method, apparatus, device, and storage medium for a near-eye display device, aiming to solve the technical problem of poor information display effect caused by the limitations of battery size and heat dissipation in near-eye display devices.

[0005] In a first aspect, this application provides a display method for a near-eye display device, the display method comprising:

[0006] Acquire a first environmental image of the surrounding environment captured by a near-eye display device;

[0007] Identify the objects in the first environment image;

[0008] Obtain preset information corresponding to the object in the first environmental image;

[0009] The preset information corresponding to the first environmental image is determined to be the information to be displayed.

[0010] The information to be displayed is displayed according to the first region of the object corresponding to the information to be displayed in the first environment image.

[0011] Secondly, this application provides a display device for a near-eye display device, the display device comprising:

[0012] The image acquisition module is used to acquire a first environmental image of the environment captured by the near-eye display device.

[0013] The first determining module is used to determine objects in the first environmental image;

[0014] The information acquisition module is used to acquire preset information corresponding to the object in the first environmental image;

[0015] The second determining module is used to determine that a portion of the preset information corresponding to the first environmental image is information to be displayed.

[0016] The display module is used to display the information to be displayed in a first region of the first environment image based on the object corresponding to the information to be displayed.

[0017] Thirdly, this application provides a near-eye display device, which includes a memory and a processor;

[0018] The memory is used to store computer programs;

[0019] The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the display method for the near-eye display device as described above.

[0020] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the display method for a near-eye display device as described above.

[0021] This application provides a display method, apparatus, device, and storage medium for a near-eye display device. The display method includes: acquiring a first environmental image of the environment captured by the near-eye display device; determining an object in the first environmental image; acquiring preset information corresponding to the object in the first environmental image; determining a portion of the preset information corresponding to the first environmental image as information to be displayed; and displaying the information to be displayed according to a first region in the first environmental image corresponding to the object to be displayed.

[0022] Upon acquiring a first environmental image, the near-eye display device can identify objects within that image. Correspondingly, the near-eye display device can acquire preset information corresponding to those objects. As the number of objects in the first environmental image increases, the amount of preset information corresponding to those objects also increases. If the near-eye display device displays all preset information when the amount is excessive, it may experience slow display, stuttering, and overheating, resulting in poor display quality. Therefore, the near-eye display device can select a portion of the preset information from all available preset information to alleviate its display burden. Accordingly, the near-eye display device can display the information based on a first region in the first environmental image corresponding to the object being displayed, allowing the user to clearly understand the relationship between the displayed information and the object. This reduction in display burden and improved intuitiveness in understanding the relationship between the displayed information and the object enhances the display quality of the near-eye display device. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic flowchart of a display method for a near-eye display device provided in an embodiment of this application;

[0025] Figure 2 This is a schematic block diagram of a first environmental image according to an embodiment of this application;

[0026] Figure 3 This is a schematic block diagram of a first environmental image related to another embodiment of this application;

[0027] Figure 4 This is a schematic flowchart of a display method for a near-eye display device according to an embodiment of this application;

[0028] Figure 5 This is a schematic block diagram of a display device for a near-eye display apparatus provided in an embodiment of this application;

[0029] Figure 6 This is a schematic block diagram of a near-eye display device provided in an embodiment of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0032] This application provides a display method, apparatus, device, and storage medium for a near-eye display device. The display method for this near-eye display device can be applied to near-eye display devices. Near-eye display devices may include virtual reality (VR) glasses, augmented reality (AR) glasses, mixed reality (MR) glasses, VR headsets, AR headsets, MR headsets, etc., and are not limited thereto. The display method for this near-eye display device can also be applied to a server, which can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0033] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0034] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a display method for a near-eye display device according to an embodiment of this application. It should be noted that the display method for a near-eye display device provided in this embodiment can be used in a near-eye display device or a server, and is not limited thereto.

[0035] like Figure 1 As shown, the display method of the near-eye display device includes steps S101 to S105.

[0036] S101. Obtain the first environmental image of the environment captured by the near-eye display device.

[0037] For example, near-eye display devices can have a shooting function. For instance, a near-eye display device can be equipped with a shooting module, which can enable the near-eye display device to take pictures; there are no restrictions on this.

[0038] A near-eye display device can capture an image of its surroundings to obtain a first environmental image. Accordingly, the near-eye display device can acquire this first environmental image. The first environmental image may include one or more objects, without limitation. Objects in the first environmental image may include physical objects, text, etc., without limitation. The first environmental image can be used by the near-eye display device to determine information to be displayed. For example, the near-eye display device can perform object recognition on the first environmental image to obtain object information corresponding to the objects in the first environmental image. The near-eye display device can determine the object information corresponding to the objects as preset information corresponding to the first environmental image for subsequent determination of information to be displayed. As another example, the near-eye display device can translate text in the first environmental image to obtain translation information corresponding to the text in the first environmental image. The near-eye display device can determine the translation information corresponding to the text as preset information corresponding to the first environmental image for subsequent determination of information to be displayed, without limitation.

[0039] In some implementations, when a user's preset operation on the near-eye display device is detected, a first environmental image of the environment captured by the near-eye display device is acquired.

[0040] For example, preset operations may include maintaining the device pose of the near-eye display device unchanged for a duration greater than or equal to a preset duration threshold, button operations on the near-eye display device, touch operations on the near-eye display device, voice instructions, gesture information, etc. Preset operations can be pre-set or user-defined, and are not limited here.

[0041] Since the near-eye display device is worn on the user's head, its posture remains unchanged as long as the user's head remains stationary. For example, if the user wearing the near-eye display device gazes at an object in the environment in which the near-eye display device is located for a duration greater than or equal to a preset duration threshold, the near-eye display device can determine that the duration for which its posture remains unchanged is greater than or equal to the preset duration threshold, and thus determine that a preset operation by the user on the near-eye display device has been detected.

[0042] For example, near-eye display devices can be equipped with a pose sensor, such as an inertial measurement unit (IMU), to acquire the device's pose; this is not a limitation. The near-eye display device can determine, based on the pose data acquired by the IMU sensor, whether the duration for which the device's pose remains unchanged is greater than or equal to a preset duration threshold. For instance, if, within a preset time period closest to the current time, the pose changes between adjacent pose data are all less than or equal to a preset pose change threshold, then the duration for which the near-eye display device's pose remains unchanged is greater than or equal to the preset duration threshold; where the preset time period is greater than or equal to the preset duration threshold. Accordingly, the near-eye display device can also determine that the IMU sensor is in a stable state.

[0043] In one exemplary embodiment, the near-eye display device can detect whether the IMU sensor is in a stable state. For example, the near-eye display device can detect whether the IMU sensor is in a stable state in real time, or at preset time intervals. The preset time interval may include 80 milliseconds, 90 milliseconds, 100 milliseconds, etc., and is not limited thereto. During the process of the near-eye display device acquiring a first environmental image, if the IMU sensor changes from a stable state to an unstable state, the near-eye display device can stop acquiring the first environmental image. Correspondingly, if the IMU sensor changes from an unstable state to a stable state, the near-eye display device can resume acquiring the first environmental image.

[0044] For example, a near-eye display device may have preset buttons. These preset buttons may include an OK button, a Tab key, etc., and are not limited to this. When the near-eye display device detects a user's operation on a preset button, the near-eye display device can determine that the user has performed a preset operation on the near-eye display device.

[0045] For example, a touch area can be provided on a near-eye display device. When the near-eye display device detects a user's preset touch operation on the touch area, the near-eye display device can determine that a preset operation has been detected. The preset touch operation can include single-touch operations, double-touch operations, long-touch operations, etc., and is not limited thereto.

[0046] For example, a near-eye display device can determine that a user has performed a preset operation on the near-eye display device if the user speaks a preset voice message and / or makes a preset gesture. The preset voice message and preset gesture can be pre-set or set by the user, and there is no restriction on this.

[0047] If a user's preset operation on the near-eye display device is detected, the near-eye display device can respond to the detected preset operation by acquiring a first environmental image for subsequent determination of the information to be displayed.

[0048] Thus, based on the settings of different types of preset operations, the near-eye display device can determine the user's display requirements for the information to be displayed according to the different types of preset operations performed by the user on the near-eye display device. Accordingly, the near-eye display device can acquire a first environmental image in response to the user's display requirements for the information to be displayed, so as to subsequently determine and display the information to be displayed. This improves the ease of acquiring the first environmental image and the ease of determining the information to be displayed by the near-eye display device.

[0049] S102. Determine the objects in the first environment image.

[0050] Once a first ambient image is acquired, the near-eye display device can identify objects within that image. These objects may include physical objects, text, etc., and are not limited thereto.

[0051] Taking a supermarket as an example, the environment in which the near-eye display device is located. A supermarket may display different types of objects. Correspondingly, the first environmental image acquired by the near-eye display device may include different types of objects. The near-eye display device can identify the objects in the first environmental image as objects. For example, upon acquiring the first environmental image, the near-eye display device can determine that the user has a need for object recognition in the first environmental image. In response to the user's need for object recognition in the first environmental image, the near-eye display device can subsequently perform object recognition processing on the objects in the first environmental image, so that the near-eye display device can subsequently determine the information to be displayed.

[0052] Taking a highway as an example, the environment in which the near-eye display device is located. Different types of traffic signs can be installed on the highway. Correspondingly, the first environmental image acquired by the near-eye display device can include different types of traffic signs. The near-eye display device can identify the different types of traffic signs included in the first environmental image as objects. For example, text can be displayed on the traffic signs. Upon acquiring the first environmental image, the near-eye display device can determine that the user has a translation requirement for the first environmental image. In response to the user's translation requirement for the first environmental image, the near-eye display device can subsequently translate the text associated with the traffic signs in the first environmental image, so that the near-eye display device can subsequently determine the information to be displayed.

[0053] Thus, when the near-eye display device identifies objects in the first environmental image, those objects can be used to subsequently determine the information to be displayed.

[0054] S103. Obtain the preset information corresponding to the object in the first environment image.

[0055] In some implementations, when an object is identified in the first environmental image, the near-eye display device can perform information extraction processing on the object to determine preset information corresponding to the object. For example, the information extraction processing may include object recognition processing, optical character recognition (OCR) processing, natural language processing (NLP), etc., and is not limited thereto.

[0056] For example, near-eye display devices can perform object recognition processing to obtain preset information corresponding to the object. The preset information may include, for example, the object's name, a brief description of the object, instructions for use of the object, etc., and is not limited here.

[0057] For example, near-eye display devices can also perform OCR or NLP processing on objects to obtain preset information corresponding to the objects. Preset information may include, for example, translation information of the objects, text information of the objects, etc., without limitation.

[0058] For example, the same object can correspond to one or more preset information.

[0059] For example, in a supermarket, different text can be set on different packaging areas of an object, such as the object's name, a brief description, and so on. Near-eye display devices can extract information from the text set on each packaging area to determine the preset information corresponding to the object. Based on this, the different text set on different packaging areas can correspond to multiple different preset information for that object; that is, the same object can correspond to multiple preset information.

[0060] For example, a traffic sign may display a traffic diagram. A near-eye display device can extract information from the traffic diagram on the traffic sign to determine the preset information corresponding to the object. Based on this, a traffic diagram on a traffic sign can correspond to a preset piece of information on that traffic sign; that is, the same object can correspond to a single set of preset information.

[0061] Thus, upon acquiring the preset information corresponding to the object in the first environmental image, the near-eye display device can use this preset information as the basis for determining the information to be displayed. For example, the near-eye display device can select a portion of the preset information corresponding to the object as the information to be displayed. When the near-eye display device subsequently displays the information to be displayed, since the number of information to be displayed is less than the number of preset information, the display pressure on the near-eye display device can be alleviated, avoiding issues such as slow information display, display stuttering, and overheating, thereby improving the display effect of the information to be displayed.

[0062] S104. Determine that the preset information corresponding to the first environmental image is the information to be displayed.

[0063] For example, in order to alleviate the information display pressure of near-eye display devices, near-eye display devices can select some preset information from all preset information corresponding to the first environmental image as the information to be displayed.

[0064] In some implementations, based on the size of the object corresponding to the preset information, a portion of the preset information corresponding to the first environment image is determined as the information to be displayed; the size of the object corresponding to the information to be displayed is larger than the size of the object corresponding to the other preset information.

[0065] For example, the size of an object may include at least one of the object's width and the object's height in the first environment image, without limitation.

[0066] Near-eye display devices can determine whether there is a need to display the preset information corresponding to a given object based on the size of the object. For example, the closer the near-eye display device is to the object, the larger the object's size will be in the first ambient image. The larger the object's size in the first ambient image, the higher the likelihood that the user will see the object, and thus the user's need to display the preset information corresponding to that object will also increase. Based on this, the near-eye display device can determine whether the user needs to display the preset information corresponding to the largest object in the first ambient image, and then determine the preset information corresponding to the largest object as the information to be displayed. The largest object can include one or more, and correspondingly, the number of preset information items corresponding to the largest object can also include one or more, without limitation. When the object corresponding to the information to be displayed is the largest, the size of the object corresponding to the information to be displayed is larger than the size of the objects corresponding to the other preset information items.

[0067] like Figure 2As shown, the objects in the first environmental image include objects 1 to 3. In the first environmental image, the size of object 1 is smaller than the size of object 2, and the size of object 1 is larger than the size of object 3. Based on this, the near-eye display device can determine the preset information corresponding to object 2 as the information to be displayed, since the size of object 2 is larger than the sizes of the other objects, such as object 1 and object 3.

[0068] In this way, when determining the information to be displayed, the near-eye display device can determine whether the user has the need to display the preset information corresponding to different objects based on the size of the objects corresponding to the preset information. Then, it can select some preset information from the preset information corresponding to the objects as the information to be displayed, which helps to reduce the number of information to be displayed, thereby relieving the information display pressure of the near-eye display device and improving the display effect of the near-eye display device on the information to be displayed.

[0069] In some implementations, preset information corresponding to objects within a preset range of the first environmental image is determined as the information to be displayed, and the preset range is a local range of the first environmental image.

[0070] Near-eye display devices can determine whether a user has a need to display the preset information corresponding to the object based on the range of the object in the first environmental image.

[0071] For example, near-eye display devices can be set with a preset range, which can be pre-set or set by the user, without any restrictions.

[0072] In one exemplary embodiment, the near-eye display device can output a range selection prompt before determining that a portion of preset information corresponding to the first ambient image is the information to be displayed; and determine the preset range based on the feedback information corresponding to the range selection prompt. For example, the near-eye display device can output the range selection prompt in the form of text, voice, etc., to prompt the user to determine the preset range. Accordingly, in response to the range selection prompt output by the near-eye display device, the user can select a local range as the preset range from the overall range of the first ambient image through gestures, voice, touch, etc. For example, the range selection prompt information can be used to indicate the correspondence between each of the multiple local ranges included in the first ambient image and different numbers, so the user can determine the feedback information for the range selection prompt information by making gestures indicating different numbers or speaking voice indicating different numbers. As another example, when the near-eye display device outputs the range selection prompt information, the user can select a local range as the preset range of the first ambient image by performing a touch operation on the near-eye display device, such as a long touch and move operation. Of course, the method of determining the preset range is not limited to this, and is not restricted here. The preset range allows the near-eye display device to select a portion of the preset information from all preset information corresponding to the first environmental image as the information to be displayed.

[0073] When an object is within a preset range of the first environmental image, the near-eye display device can determine that the user has a need to display preset information corresponding to the object within the preset range of the first environmental image, and then determine the preset information corresponding to the object within the preset range as the information to be displayed.

[0074] like Figure 3 As shown, the objects include objects 1 to 3. In the first environmental image, objects 1 and 2 are both within a preset range of the first environmental image, while object 3 is not within the preset range of the first environmental image. Therefore, the near-eye display device can determine that the preset information corresponding to each of objects 1 and 2 is the information to be displayed.

[0075] In this way, when determining the information to be displayed, the near-eye display device can determine whether the user has the need to display the preset information corresponding to different objects based on the preset range of the first environmental image. Then, it can select a portion of the preset information corresponding to the object as the information to be displayed, which helps to reduce the amount of information to be displayed, thereby relieving the information display pressure on the near-eye display device and improving the display effect of the near-eye display device on the information to be displayed.

[0076] For example, a near-eye display device can determine the information to be displayed by combining the size of objects in a first environmental image with a preset range of the first environmental image.

[0077] In some implementations, within a preset range of the first environmental image, the preset information corresponding to the largest object is determined as the information to be displayed; the preset range is a local range of the first environmental image.

[0078] comprehensive Figure 2 as well as Figure 3 It can be seen that the objects include objects 1 to 3. Since the size of object 1 is smaller than the size of object 2 and the size of object 3 is larger than the size of object 3 in the first environmental image, and both object 1 and object 2 are within the preset range of the first environmental image, the near-eye display device can determine that the preset information corresponding to object 2 is the information to be displayed.

[0079] Thus, in the process of determining the partial preset information corresponding to the first environmental image as the information to be displayed, the near-eye display device can flexibly select partial information from all the preset information corresponding to the first environmental image as the information to be displayed based on at least one of the object size and the preset range of the first environmental image, which helps to improve the flexibility and convenience of determining the information to be displayed.

[0080] S105. Display the information to be displayed in the first region of the first environment image according to the object corresponding to the information to be displayed.

[0081] When the information to be displayed is determined, the near-eye display device can acquire a first region of the object corresponding to the information in a first ambient image, and display the information based on the first region of the object in the first ambient image. The first region can serve as the basis for displaying the information. For example, the first region can correspond to the position and size of the object corresponding to the information in the environment where the near-eye display device is located. The near-eye display device can display the information in the environment where the near-eye display device is located by determining the first position and first size of the first region in the environment where the near-eye display device is located, thereby indicating the association between the information to be displayed and the object corresponding to the information. For example, the first region can also correspond to the position and size of the object corresponding to the information in the first ambient image. The near-eye display device can display the information in the first ambient image by determining the second position and second size of the first region in the first ambient image, thereby indicating the association between the information to be displayed and the object corresponding to the information.

[0082] For example, a near-eye display device can adjust the size of a first region of an object in a first environmental image.

[0083] In some implementations, when the size of the shortest side of the first region corresponding to the information to be displayed is less than a preset size threshold, the first region is expanded to obtain a processed first region; wherein the size of the shortest side of the processed first region is greater than or equal to the preset size threshold; and the information to be displayed is displayed according to the processed first region.

[0084] For example, if the shortest side of the first region is less than a preset size threshold, the first size of the first region in the environment where the near-eye display device is located, and the second size of the first region in the first environmental image, are generally also small. When displaying information based on the first region, the information may appear within a small area of ​​the environment where the near-eye display device is located, or within a small area of ​​the first environmental image. This can easily lead to the user being unable to clearly see the information, resulting in poor display performance of the information on the near-eye display device. Therefore, if the shortest side of the first region is less than the preset size threshold, the near-eye display device can perform size expansion processing on the first region until the shortest side of the processed first region is greater than or equal to the preset size threshold. For example, if it is determined that a portion of the preset information corresponding to the first environmental image is the information to be displayed, the near-eye display device can perform size expansion processing on the first region based on the fact that the shortest side of the first region corresponding to the information to be displayed is less than the preset size threshold, to obtain the processed first region. For example, if the preset information corresponding to a portion of objects within a preset range of the first environmental image is determined to be the information to be displayed, the near-eye display device can expand the size of the first region based on the fact that the shortest side of the first region corresponding to the information to be displayed is less than a preset size threshold, to obtain the processed first region. This is not limited in scope. The preset size threshold can be pre-set or user-defined. For example, the preset size threshold may include 40 pixels, 45 pixels, 50 pixels, 55 pixels, 60 pixels, etc. Of course, the preset size threshold is not limited to these; the unit of measurement for the preset size threshold may also include millimeters, centimeters, etc., without limitation.

[0085] The processed first region can be used by the near-eye display device as a basis for subsequently displaying the information to be displayed, thereby improving the display effect of the information. For example, the processed first region can correspond to the position and size of the object corresponding to the information to be displayed in the environment in which the near-eye display device is located. The near-eye display device can display the information to be displayed in the environment in which the near-eye display device is located by determining the first position and first size of the processed first region in the environment in which the near-eye display device is located, thereby indicating the association between the information to be displayed and the object corresponding to the information to be displayed. Alternatively, the processed first region can also correspond to the position and size of the object corresponding to the information to be displayed in a first environmental image. The near-eye display device can display the information to be displayed in the first environmental image by determining the second position and second size of the processed first region in the first environmental image, thereby indicating the association between the information to be displayed and the object corresponding to the information to be displayed.

[0086] In some implementations, based on the first region and the first environmental image, a first position and a first size of the first region in the environment where the near-eye display device is located are determined; based on the first position and the first size, a first display position and a first display size of the information to be displayed are determined; and based on the first display position and the first display size of the information to be displayed, the information to be displayed is displayed.

[0087] Since a near-eye display device can display information to be displayed in the environment in which the near-eye display device is located, the near-eye display device needs to determine the first display position and the first display size of the information to be displayed in the environment in which the near-eye display device is located.

[0088] For example, a near-eye display device can display information to be displayed in the environment in which the information to be displayed is located, based on the position and size of the object corresponding to the information to be displayed within the environment in which the near-eye display device is located. Since the first region is associated with the object corresponding to the information to be displayed, the near-eye display device can determine the first display position and the first display size of the information to be displayed based on the first region and the first environmental image. For example, the near-eye display device can input the first region and the first environmental image into a preset size recognition algorithm, so that the size recognition algorithm determines the first size of the first region in the environment in which the near-eye display device is located, thus obtaining the first size of the first region. Correspondingly, the near-eye display device can determine the environmental size of the environment in which the near-eye display device is located, corresponding to the first environmental image, by combining the ratio between the first region and the first environmental image with the first size of the first region. As another example, the near-eye display device can input the first region and the first environmental image into a preset position recognition algorithm, so that the position recognition algorithm, combined with the imaging parameters of the near-eye display device, determines the first position of the first region in the environment in which the near-eye display device is located.

[0089] Given a first location and a first size of a first region, the near-eye display device can determine the first location of the first region as the first display location of the information to be displayed, and the near-eye display device can determine the first size of the first region as the first display size of the information to be displayed.

[0090] Accordingly, given the first display position and first display size of the information to be displayed, the near-eye display device can project the information to be displayed onto the environment in which the near-eye display device is located, thereby displaying the information to be displayed.

[0091] In this way, by displaying the information to be displayed in the first area of ​​the first environment image according to the object corresponding to the information to be displayed, the near-eye display device can project the information to be displayed onto the environment in which the near-eye display device is located. As a result, the user can view the information to be displayed in the environment in which the near-eye display device is located, which helps to improve the user's information viewing experience of the near-eye display device, thereby improving the display effect of the information to be displayed by the near-eye display device.

[0092] In some implementations, based on a first region and a first environmental image, a second position and a second size of the first region in the first environmental image are determined; based on the second position and the second size, a second display position and a second display size of the information to be displayed are determined; and based on the second display position and the second display size of the information to be displayed, the information to be displayed is displayed.

[0093] If a near-eye display device can display information to be displayed on a first ambient image, then the near-eye display device needs to determine the second display position and the second display size of the information to be displayed in the first ambient image.

[0094] For example, a near-eye display device can display information to be displayed in a first ambient image based on the position and size of the object corresponding to the information to be displayed within that first ambient image. Since the first region is associated with the object corresponding to the information to be displayed, the near-eye display device can determine the second display position and second display size of the information to be displayed based on the first region and the first ambient image. For example, the near-eye display device can obtain the size of the first ambient image. The near-eye display device can determine the second size of the first region in the first ambient image by combining the ratio between the first region and the first ambient image with the size of the first ambient image. Accordingly, the near-eye display device can determine the second position of the first region in the first ambient image by matching the local area of ​​the first region with that of the first ambient image.

[0095] Given a second position and a second size of the first region, the near-eye display device can determine the second position of the first region as the second display position of the information to be displayed, and the near-eye display device can determine the second size of the first region as the second display size of the information to be displayed.

[0096] Accordingly, when the second display position and the second display size of the information to be displayed are determined, the near-eye display device can display the information to be displayed in a first area of ​​the first environmental image based on the second display position and the second display size, thereby displaying the information to be displayed.

[0097] In this way, the near-eye display device can display the information to be displayed in the first area of ​​the first environment image according to the object corresponding to the information to be displayed. The user can view the information to be displayed in the first area of ​​the first environment image through the near-eye display device, which helps to improve the user's information viewing experience of the near-eye display device, thereby improving the display effect of the near-eye display device on the information to be displayed.

[0098] For example, when a near-eye display device displays information to be displayed, the near-eye display device can track and display the information. For instance, the near-eye display device can determine whether the user has a need to track and display the information based on the user's preset operations on the near-eye display device when acquiring the first environmental image. If the near-eye display device determines that the user has a need to track and display the information, the near-eye display device can anchor the display position of the information to be displayed to the position of a first area corresponding to that position. For example, the near-eye display device can anchor the first display position of the information to be displayed to the first position of the first area corresponding to the information to be displayed. Alternatively, the near-eye display device can anchor the second display position of the information to be displayed to the second position of the first area corresponding to the information to be displayed. Correspondingly, the near-eye display device can anchor the display size of the information to be displayed to the size of the first area corresponding to the information to be displayed. For example, the near-eye display device can anchor the first display size of the information to be displayed to the first size of the first area corresponding to the information to be displayed. Alternatively, the near-eye display device can anchor the second display size of the information to be displayed to the second size of the first area corresponding to the information to be displayed. Based on this, near-eye display devices can track and display information according to the positional and dimensional anchoring relationships between the information to be displayed and the first region corresponding to the information to be displayed.

[0099] In some implementations, when a preset operation meets preset information tracking conditions, a second environmental image is obtained by the near-eye display device taking another picture of the environment; the position change information and size change information of the information to be displayed in the second region corresponding to the second environmental image relative to the first region corresponding to the first environmental image are determined; and the information to be displayed is updated and displayed based on the position change information and size change information.

[0100] Near-eye display devices can be configured with preset information tracking conditions. For example, a near-eye display device can preset associations between different preset operations and tracking display demand tags. Accordingly, the preset information tracking conditions can be determined based on preset operations associated with tracking display demand tags that indicate to the user that they have a tracking display demand for the information to be displayed.

[0101] Taking preset operations including a near-eye display device maintaining a constant device pose for a duration greater than or equal to a preset duration threshold and button operations on the near-eye display device as examples, the association between different preset operations and tracking display requirement tags includes, for example: when the preset operation is a near-eye display device maintaining a constant device pose for a duration greater than or equal to the preset duration threshold, the tracking display requirement tag corresponding to this preset operation indicates that the user has a tracking display requirement for the information to be displayed; when the preset operation is a button operation on the near-eye display device, the tracking display requirement tag corresponding to this preset operation indicates that the user does not have a tracking display requirement for the information to be displayed. Based on this, when the preset operation corresponding to acquiring the first environmental image is a near-eye display device maintaining a constant device pose for a duration greater than or equal to the preset duration threshold, the near-eye display device can determine that the preset operation meets the preset information tracking conditions. Accordingly, the near-eye display device can acquire a second environmental image obtained by the near-eye display device re-capturing the surrounding environment for tracking display of the information to be displayed.

[0102] The preset information tracking conditions can be pre-set or user-defined, and there are no restrictions here.

[0103] For example, when a preset operation meets preset information tracking conditions, the near-eye display device can determine the position change information and size change information of the second region corresponding to the information to be displayed in the second environment image relative to the first region corresponding to the information to be displayed in the first environment image, based on the first environmental image and the second environmental image. For example, the near-eye display device can perform image matching processing on the first and second environmental images based on the positional and size anchoring relationships between the information to be displayed and the first region corresponding to the information to be displayed, combined with the object corresponding to the first region, to obtain a second region in the second environmental image that matches the first region. Accordingly, when the second region is determined, the near-eye display device can determine the third position and third size of the second region in the second environmental image. The method for determining the third position and third size of the second region in the second environmental image can refer to the aforementioned method for determining the second position and second size of the first region in the first environmental image, and will not be repeated here. The near-eye display device can determine the position change information of the second region relative to the first region by combining the second position and the third position of the first region. Near-eye display devices can determine the size change information of the second region relative to the first region by combining the second size of the first region and the third size of the second region.

[0104] When position change information and size change information are determined, the near-eye display device can update the target display information based on these information. For example, when the near-eye display device displays information to be displayed in its environment, it can determine the fourth position of the second region in the environment based on the position change information and the first position of the first region, and determine the fourth size of the second region in the environment based on the size change information and the first size of the first region. Accordingly, the near-eye display device can update the display of the information to be displayed based on the fourth position and fourth size of the second region. For example, when the near-eye display device displays information to be displayed on a first environmental image, it can update the display of the information to be displayed on a second environmental image based on the position change information and size change information.

[0105] In this way, when the preset operation meets the preset information tracking conditions, the near-eye display device can update the display information based on the second environmental image, which can adapt to the user's needs for tracking the display information and improve the convenience of information display and tracking, thereby improving the user's information tracking experience.

[0106] In some implementations, when the change in the first pose corresponding to the device pose of the near-eye display device is greater than or equal to the first pose change threshold, and / or the change in the second pose corresponding to the object pose of the object to be displayed is greater than or equal to the second pose change threshold, the display of the information to be displayed is stopped.

[0107] For example, during the display of information to be displayed, the near-eye display device can detect its own pose and determine the change in the first pose corresponding to that pose. If the near-eye display device detects a change in the first pose greater than or equal to a first pose change threshold, it can determine that the user's movement is significant and likely causes the object corresponding to the information to be displayed to gradually move away from the device's shooting range. Based on this, the near-eye display device can determine that there is no longer a need to display the information to be displayed, and thus can stop displaying the information. The first pose change threshold can be preset or set by the user; there is no restriction on this.

[0108] For example, during the display of information to be displayed, the near-eye display device can detect the object pose corresponding to the information to be displayed and determine the second pose change corresponding to the object pose. If the near-eye display device detects that the second pose change corresponding to the object pose is greater than or equal to a second pose change threshold, then the near-eye display device can determine that the movement of the object corresponding to the information to be displayed is large, and that the movement of the object is likely to cause the object to gradually deviate from the shooting range of the near-eye display device. Based on this, the near-eye display device can determine that there is no longer a need to display the information to be displayed, and thus the near-eye display device can stop displaying the information to be displayed. The second pose change threshold can be preset or set by the user, and there is no restriction here.

[0109] Thus, when displaying information to be displayed, the near-eye display device can determine whether to stop displaying the information based on at least one of the first pose change corresponding to the device pose of the near-eye display device and the second pose change corresponding to the object pose of the object to be displayed. This allows the near-eye display device to adapt its display of information to the user's information display needs, thereby improving the accuracy and effect of displaying information.

[0110] In one exemplary implementation, such as Figure 4 As shown, the display method of the near-eye display device includes steps S210 to S241.

[0111] S210, Detect the user's preset operations on the near-eye display device.

[0112] For example, preset operations may include the duration during which the near-eye display device's pose remains unchanged is greater than or equal to a preset duration threshold, button operations on the near-eye display device, etc., without limitation.

[0113] S220. Determine whether the preset operation meets the preset information tracking conditions; if yes, jump to S221; if no, jump to S240.

[0114] For example, a near-eye display device can have preset associations between different preset operations and tracking display demand tags. When a preset operation is detected, the near-eye display device can determine whether the tracking display demand tag associated with that preset operation indicates that the user has a tracking display demand for the information to be displayed. If the tracking display demand tag associated with the preset operation indicates that the user has a tracking display demand for the information to be displayed, the near-eye display device can determine that the preset operation meets the preset information tracking conditions. Conversely, if the tracking display demand tag associated with the preset operation indicates that the user does not have a tracking display demand for the information to be displayed, the near-eye display device can determine that the preset operation does not meet the preset information tracking conditions.

[0115] S221. Determine whether the IMU sensor is in a stable state; if yes, jump to S222; if no, jump to S230.

[0116] For example, in response to a user's need for tracking and displaying information, the near-eye display device needs to determine the information to be displayed. Based on this, the near-eye display device can determine whether the IMU sensor is in a stable state, so that, if the IMU sensor is in a stable state, it can acquire a first environmental image of the environment captured by the near-eye display device, and determine the information to be displayed based on the first environmental image.

[0117] S222. Determine whether the near-eye display device is currently displaying information to be displayed; if not, proceed to S223.

[0118] For example, when the IMU sensor is in a stable state, the near-eye display device needs to determine whether it needs to acquire the information to be displayed. For instance, if the near-eye display device is currently displaying information to be displayed, it does not need to acquire the first environmental image again to determine the information to be displayed; instead, it can maintain the display of the currently displayed information and track and display it. Conversely, if the near-eye display device is not currently displaying information to be displayed, it needs to acquire the first environmental image to determine the information to be displayed.

[0119] S223. Determine whether the near-eye display device is currently acquiring information to be displayed; if not, proceed to S224.

[0120] For example, if the near-eye display device is not currently displaying any information to be displayed, it needs to determine whether it needs to acquire that information. For instance, if the near-eye display device is currently acquiring information, it does not need to acquire the first ambient image again to determine the information to be displayed; in this case, the near-eye display device can continue acquiring the current information. Conversely, if the near-eye display device is not currently acquiring information, it needs to acquire the first ambient image to determine the information to be displayed.

[0121] S224. Obtain a first environmental image of the environment captured by the near-eye display device, and determine the information to be displayed based on the first environmental image.

[0122] For example, when a near-eye display device acquires a first ambient image, it can identify objects in the first ambient image, acquire preset information corresponding to the objects in the first ambient image, and determine that a portion of the preset information corresponding to the first ambient image is the information to be displayed.

[0123] S225. Determine whether the IMU sensor is in a stable state during the process of determining the information to be displayed by the near-eye display device; if yes, proceed to S226; otherwise, proceed to S227.

[0124] For example, during the process of determining the information to be displayed, the near-eye display device's pose still changes according to the user's behavior, such as changes in the user's head posture. When the near-eye display device's pose changes, the IMU sensor correspondingly changes from a stable state to an unstable state. When the IMU sensor is in an unstable state, the near-eye display device may be unable to acquire a clear first environmental image, or the user's display requirements may change. If a clear first environmental image cannot be acquired, the near-eye display device needs to pause acquiring the first environmental image. Consequently, if the user's display requirements change, the near-eye display device does not need to continue acquiring the information to be displayed corresponding to the current first environmental image. Based on this, the near-eye display device needs to detect whether the IMU sensor is in a stable state during the process of determining the information to be displayed, in order to determine whether it is necessary to continue determining the information to be displayed corresponding to the current first environmental image.

[0125] S226. Display the information to be displayed in the first region of the first environment image according to the object corresponding to the information to be displayed.

[0126] For example, when the IMU sensor is in a stable state, the near-eye display device can display the information to be displayed corresponding to the acquired first environmental image. For instance, the near-eye display device can project the information to be displayed onto the environment in which the near-eye display device is located, based on a first area corresponding to the information to be displayed. Or, for another example, the near-eye display device can display the information to be displayed on the first environmental image, based on the first area corresponding to the information to be displayed.

[0127] S227, pause acquiring at least one of the first environmental image and the second environmental image, and pause determining the information to be displayed, waiting for the IMU sensor to return to a stable state.

[0128] For example, when the IMU sensor is in an unstable state, the near-eye display device can pause acquiring at least one of the first and second environmental images, and pause determining the information to be displayed, to wait for the IMU sensor to return to a stable state. Once the IMU sensor returns to a stable state, the near-eye display device can resume acquiring at least one of the first and second environmental images, or resume determining the information to be displayed corresponding to each of the first and second environmental images. For example, the near-eye display device can resume acquiring at least one of the first and second environmental images, or resume determining the information to be displayed corresponding to each of the first and second environmental images, based on the fact that the change in the first pose corresponding to the device's pose is less than a first pose change threshold. The near-eye display device can also reacquire at least one of the first and second environmental images to determine the information to be displayed corresponding to each of the reacquired first and second environmental images. Again, for example, the near-eye display device can reacquire at least one of the first and second environmental images, based on the fact that the change in the first pose corresponding to the device's pose is greater than or equal to a first pose change threshold, to determine the information to be displayed corresponding to each of the reacquired first and second environmental images. The first pose change can be determined based on the device pose of the near-eye display device acquired by the IMU sensor.

[0129] S228. Obtain a second environmental image of the environment captured by the near-eye display device, and update the information to be displayed based on the first environmental image and the second environmental image.

[0130] For example, when a near-eye display device is displaying information to be displayed, since the preset operation meets the preset information tracking conditions, the near-eye display device needs to track and display the information to be displayed. Based on this, the near-eye display device can take another picture of the surrounding environment to obtain a second environmental image, and combine the first environmental image and the second environmental image to update and display the information to be displayed, so as to achieve the tracking and display of the information to be displayed.

[0131] S229. When the change in the first pose corresponding to the device pose of the near-eye display device is detected to be greater than or equal to the first pose change threshold, and / or the change in the second pose corresponding to the object pose of the object to be displayed is greater than or equal to the second pose change threshold, the display of the information to be displayed shall be stopped.

[0132] For example, during the tracking and display of information to be displayed, if the change in the first pose is greater than or equal to a first pose change threshold, and / or the change in the second pose is greater than or equal to a second pose change threshold, the near-eye display device can determine that at least one of the user's movement amplitude or the movement amplitude of the object corresponding to the information to be displayed is greater, causing the object corresponding to the information to be displayed to gradually move away from the shooting range of the near-eye display device. Based on this, the near-eye display device can determine that there is no longer a need to display the information to be displayed or to track and display the information to be displayed, and thus the near-eye display device can stop displaying the information to be displayed.

[0133] S230. Determine whether the near-eye display device is currently displaying information to be displayed; if so, proceed to S227.

[0134] For example, when the IMU sensor is in an unstable state, the near-eye display device can determine whether to display the information to be displayed, based on the preset operation meeting the preset information tracking conditions, and thus decide whether to pause the tracking and display of the information to be displayed. For instance, if the IMU sensor changes from a stable state to an unstable state during the tracking and display of the information to be displayed, the near-eye display device cannot acquire a clear second environmental image, thus adversely affecting the tracking and display process. Therefore, when the IMU sensor is in an unstable state, the acquisition of the second environmental image and the determination of the information to be displayed are paused, waiting for the IMU sensor to return to a stable state.

[0135] S240: Obtain a first environmental image of the environment captured by the near-eye display device, and determine the information to be displayed based on the first environmental image.

[0136] For example, in response to the user's lack of a need for tracking the information to be displayed, the near-eye display device can directly determine the information to be displayed. For instance, when the near-eye display device acquires a first ambient image, it can identify objects in the first ambient image, acquire preset information corresponding to the objects in the first ambient image, and determine a portion of the preset information corresponding to the first ambient image as the information to be displayed.

[0137] S241. Display the information to be displayed in the first region of the first environment image according to the object corresponding to the information to be displayed.

[0138] Accordingly, in response to the user's lack of need for tracking the information to be displayed, the near-eye display device can directly display the information to be displayed based on the object corresponding to the information in the first area of ​​the first environmental image, without tracking the information to be displayed.

[0139] The display method for a near-eye display device provided in the above embodiments includes: acquiring a first environmental image of the environment captured by the near-eye display device; determining objects in the first environmental image; acquiring preset information corresponding to the objects in the first environmental image; determining a portion of the preset information corresponding to the first environmental image as information to be displayed; and displaying the information to be displayed according to a first region in the first environmental image corresponding to the objects to be displayed.

[0140] Upon acquiring a first environmental image, the near-eye display device can identify objects within that image. Correspondingly, the near-eye display device can acquire preset information corresponding to those objects. As the number of objects in the first environmental image increases, the amount of preset information corresponding to those objects also increases. If the near-eye display device displays all preset information when the amount is excessive, it may experience slow display, stuttering, and overheating, resulting in poor display quality. Therefore, the near-eye display device can select a portion of the preset information from all available preset information as the information to be displayed, thus alleviating the information display burden on the device. Accordingly, the near-eye display device can display the information to be displayed based on a first region in the first environmental image corresponding to the object, allowing the user to clearly understand the relationship between the information to be displayed and its corresponding object. This reduction in information display burden and the improved intuitiveness of viewing the relationship between the information to be displayed and its corresponding object enhance the display effect of the near-eye display device.

[0141] Please see Figure 5 , Figure 5This is a schematic block diagram of a display device for a near-eye display device according to an embodiment of this application. The display device can be configured within a near-eye display device or a server to execute the aforementioned display method for the near-eye display device. The near-eye display device may include VR glasses, AR glasses, MR glasses, VR headsets, AR headsets, MR headsets, etc., and is not limited thereto. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0142] like Figure 5 As shown, the display device of the near-eye display includes an image acquisition module 110, a first determination module 120, an information acquisition module 130, a second determination module 140, and a display module 150.

[0143] The image acquisition module 110 is used to acquire a first environmental image obtained by the near-eye display device capturing the environment it is in.

[0144] The first determining module 120 is used to determine objects in the first environmental image.

[0145] The information acquisition module 130 is used to acquire preset information corresponding to the object in the first environment image.

[0146] The second determining module 140 is used to determine that a portion of the preset information corresponding to the first environmental image is information to be displayed.

[0147] Display module 150 is used to display the information to be displayed in a first area of ​​the first environment image based on the object corresponding to the information to be displayed.

[0148] For example, the second determining module 140 includes the first determining submodule.

[0149] The first determining submodule is used to determine, based on the size of the object corresponding to the preset information, a portion of the preset information corresponding to the first environment image as information to be displayed; the size of the object corresponding to the information to be displayed is larger than the size of the other objects corresponding to the preset information.

[0150] For example, the display module 150 includes a size expansion submodule and a first display submodule.

[0151] The size expansion submodule is used to perform size expansion processing on the first region when the size of the shortest side of the first region corresponding to the information to be displayed is less than a preset size threshold, so as to obtain the processed first region; wherein the size of the shortest side of the processed first region is greater than or equal to the preset size threshold.

[0152] The first display submodule is used to display the information to be displayed based on the processed first area.

[0153] For example, the second determining module 140 includes a second determining submodule.

[0154] The second determining submodule is used to determine that the preset information corresponding to the object within a preset range of the first environmental image is the information to be displayed, wherein the preset range is a local range of the first environmental image.

[0155] For example, the display module 150 includes a first information determining submodule, a second information determining submodule, and a second display submodule.

[0156] The first information determination submodule is used to determine the first position and first size of the first region in the environment where the near-eye display device is located, based on the first region and the first environmental image.

[0157] The second information determination submodule is used to determine the first display position and the first display size of the information to be displayed based on the first position and the first size.

[0158] The second display submodule is used to display the information to be displayed according to the first display position and the first display size of the information to be displayed.

[0159] For example, the display module 150 includes a third information determination submodule, a fourth information determination submodule, and a third display submodule.

[0160] The third information determination submodule is used to determine the second position and second size of the first region in the first environment image based on the first region and the first environment image.

[0161] The fourth information determination submodule is used to determine the second display position and the second display size of the information to be displayed based on the second position and the second size.

[0162] The third display submodule is used to display the information to be displayed according to the second display position and the second display size of the information to be displayed.

[0163] For example, the image acquisition module 110 includes a first acquisition submodule.

[0164] The first acquisition submodule is used to acquire a first environmental image of the environment captured by the near-eye display device when a user's preset operation on the near-eye display device is detected.

[0165] The display device also includes a second acquisition submodule, a change information determination submodule, and a display update submodule.

[0166] The second acquisition submodule is used to acquire a second environmental image obtained by the near-eye display device taking another picture of the environment when the preset operation meets the preset information tracking conditions.

[0167] The change information determination submodule is used to determine the position change information and size change information of the information to be displayed in the second region corresponding to the second environment image relative to the first region corresponding to the first environment image.

[0168] The display update submodule is used to update the display of the information to be displayed based on the position change information and the size change information.

[0169] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the above-described apparatus and its modules and units can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0170] The method of this application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0171] For example, the above-described method and apparatus can be implemented as a computer program that runs on a near-eye display device or a server to control the near-eye display device. For example, the near-eye display device may include VR glasses, AR glasses, MR glasses, VR headsets, AR headsets, MR headsets, etc., without limitation. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0172] Please see Figure 6 , Figure 6 This is a schematic block diagram of the structure of a near-eye display device provided in an embodiment of this application.

[0173] like Figure 6 As shown, the near-eye display device includes a memory and a processor. The memory and processor can be connected via a system bus, and the memory may include a storage medium and internal memory.

[0174] The storage medium can store the operating system and computer programs. When the computer program is executed, it enables the processor to perform any display method of the near-eye display device.

[0175] The processor provides computing and control capabilities to support the operation of the entire near-eye display device.

[0176] Internal memory provides an environment for the execution of computer programs stored in storage media. When these computer programs are executed by the processor, the processor can execute any display method of a near-eye display device.

[0177] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the near-eye display device to which the present application is applied. A specific near-eye display device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0178] It should be understood that a processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other convertible logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0179] In one embodiment, the processor is configured to execute a computer program and, when executing the computer program, perform the following steps:

[0180] Acquire a first environmental image of the surrounding environment captured by a near-eye display device;

[0181] Identify the objects in the first environment image;

[0182] Obtain preset information corresponding to the object in the first environmental image;

[0183] The preset information corresponding to the first environmental image is determined to be the information to be displayed.

[0184] The information to be displayed is displayed according to the first region of the object corresponding to the information to be displayed in the first environment image.

[0185] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the near-eye display device described above can be referred to the corresponding process in the aforementioned near-eye display device display method embodiments, and will not be repeated here.

[0186] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method implemented can be referred to in various embodiments of the display method of the near-eye display device of this application.

[0187] The computer-readable storage medium can be an internal storage unit of the near-eye display device described in the foregoing embodiments, such as the hard disk or memory of the near-eye display device. Alternatively, the computer-readable storage medium can be an external storage device of the near-eye display device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the near-eye display device.

[0188] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0189] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0190] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A display method for a near-eye display device, characterized in that, The display method includes: Acquire a first environmental image of the surrounding environment captured by a near-eye display device; Identify the objects in the first environment image; Obtain preset information corresponding to the object in the first environmental image; The preset information corresponding to the first environmental image is determined to be the information to be displayed. The information to be displayed is displayed according to the first region of the object corresponding to the information to be displayed in the first environment image; The step of determining that the preset information corresponding to the first environmental image is the information to be displayed includes: Based on the size of the object corresponding to the preset information, a portion of the preset information corresponding to the first environment image is determined to be information to be displayed; the size of the object corresponding to the information to be displayed is greater than the size of the other objects corresponding to the preset information; the size of the object corresponding to the preset information includes at least one of the width of the object corresponding to the preset information in the first environment image and the height of the object corresponding to the preset information in the first environment.

2. The display method according to claim 1, characterized in that, Displaying the information to be displayed in a first region of the first environment image based on the object corresponding to the information to be displayed includes: When the size of the shortest side of the first region corresponding to the information to be displayed is less than a preset size threshold, the size of the first region is expanded to obtain the processed first region; wherein the size of the shortest side of the processed first region is greater than or equal to the preset size threshold. The information to be displayed is then displayed based on the processed first area.

3. The display method according to claim 1, characterized in that, The step of determining that the preset information corresponding to the first environmental image is the information to be displayed includes: The preset information corresponding to the object within a preset range of the first environmental image is determined as the information to be displayed, where the preset range is a local range of the first environmental image.

4. The display method according to any one of claims 1 to 3, characterized in that, The step of displaying the information to be displayed according to the first region of the object corresponding to the information to be displayed in the first environment image includes: Based on the first region and the first environmental image, determine the first location and first size of the first region in the environment where the near-eye display device is located; Based on the first position and the first size, determine the first display position and the first display size of the information to be displayed; The information to be displayed is displayed according to the first display position and the first display size of the information to be displayed.

5. The display method according to any one of claims 1 to 3, characterized in that, The step of displaying the information to be displayed according to the first region of the object corresponding to the information to be displayed in the first environment image includes: Based on the first region and the first environment image, determine the second position and second size of the first region in the first environment image; Based on the second position and the second size, determine the second display position and the second display size of the information to be displayed; The information to be displayed is displayed according to the second display position and the second display size of the information to be displayed.

6. The display method according to any one of claims 1 to 3, characterized in that, The acquisition of the first environmental image obtained by the near-eye display device from the surrounding environment includes: When a user's preset operation on the near-eye display device is detected, a first environmental image of the environment captured by the near-eye display device is obtained. After displaying the information to be displayed in a first region of the first environment image based on the object corresponding to the information to be displayed, the method further includes: When the preset operation meets the preset information tracking conditions, a second environmental image is obtained by the near-eye display device taking another picture of the environment it is in. Determine the position change information and size change information of the information to be displayed in the second region corresponding to the second environment image relative to the first region corresponding to the first environment image; The information to be displayed is updated based on the position change information and the size change information.

7. A display device for a near-eye display system, characterized in that, The display device includes: The image acquisition module is used to acquire a first environmental image of the environment captured by the near-eye display device. The first determining module is used to determine objects in the first environmental image; The information acquisition module is used to acquire preset information corresponding to the object in the first environmental image; The second determining module is used to determine that a portion of the preset information corresponding to the first environmental image is information to be displayed. The display module is used to display the information to be displayed in a first area of ​​the first environment image according to the object corresponding to the information to be displayed. The step of determining that the preset information corresponding to the first environmental image is the information to be displayed includes: Based on the size of the object corresponding to the preset information, a portion of the preset information corresponding to the first environment image is determined to be information to be displayed; the size of the object corresponding to the information to be displayed is greater than the size of the other objects corresponding to the preset information; the size of the object corresponding to the preset information includes at least one of the width of the object corresponding to the preset information in the first environment image and the height of the object corresponding to the preset information in the first environment.

8. A near-eye display device, characterized in that, The near-eye display device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the display method of the near-eye display device as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the display method of the near-eye display device as described in any one of claims 1 to 6.

Images