Display method and device of near-to-eye display equipment, equipment and storage medium

Abstract

The invention relates to the technical field of display, and provides a display method and device of near-eye display equipment, equipment and a storage medium, and the method comprises the steps: obtaining a first environment image obtained by the environment where the near-eye display equipment shoots; determining an object in the first environment image; acquiring preset information corresponding to an object in the first environment image; determining partial preset information corresponding to the first environment image as to-be-displayed information; and displaying the to-be-displayed information according to the first area of the object corresponding to the to-be-displayed information in the first environment image so as to improve the display effect of the near-to-eye display device on the to-be-displayed information.

Description

Technical Field

[0001] The present application relates to the field of display technology, and in particular to a display method, apparatus, device and storage medium for a near-eye display device. Background Art

[0002] At present, when extracting and processing information, such as photo translation, object recognition processing, etc., it is generally necessary to preview and display the extracted information. The user can then perform corresponding processing on the previewed information, such as filtering information, selecting information, etc., for subsequent display of the processed information.

[0003] However, due to the limitations of battery size and heat dissipation of near-eye display devices, if the near-eye display device previews the extracted information, it is easy for the information display to become stuck, the information to be displayed slowly, the near-eye display device to heat up, etc., resulting in poor display of the information. Summary of the invention

[0004] The main purpose of the present application is to provide a display method, apparatus, device and storage medium for a near-eye display device, aiming to solve the technical problem of poor information display effect of the near-eye display device due to limitations of factors such as battery size and heat dissipation of the near-eye display device.

[0005] In a first aspect, the present application provides a display method of a near-eye display device, the display method comprising:

[0006] Acquire a first environment image obtained by photographing the environment in which the near-eye display device is located;

[0007] determining an object in the first environment image;

[0008] Acquire preset information corresponding to the object in the first environment image;

[0009] Determining that part of the preset information corresponding to the first environment image is information to be displayed;

[0010] The information to be displayed is displayed according to a first area of ​​the object corresponding to the information to be displayed in the first environment image.

[0011] In a second aspect, the present application provides a display device of a near-eye display device, the display device comprising:

[0012] An image acquisition module, used to acquire a first environment image obtained by photographing the environment in which the near-eye display device is located;

[0013] A first determining module, used to determine an object in the first environment image;

[0014] An information acquisition module, used to acquire preset information corresponding to the object in the first environment image;

[0015] A second determining module, used to determine that part of the preset information corresponding to the first environment image is information to be displayed;

[0016] The display module is used to display the information to be displayed according to a first area of ​​the object corresponding to the information to be displayed in the first environment image.

[0017] In a third aspect, the present application provides a near-eye display device, the near-eye display device comprising a memory and a processor;

[0018] The memory is used to store computer programs;

[0019] The processor is used to execute the computer program and implement the steps of the display method of the near-eye display device as described above when executing the computer program.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the display method of the near-eye display device as described above are implemented.

[0021] The present application provides a display method, apparatus, device and storage medium for a near-eye display device, the display method comprising: obtaining a first environment image obtained by shooting an environment in which the near-eye display device is located; determining an object in the first environment image; obtaining preset information corresponding to the object in the first environment image; determining that part of the preset information corresponding to the first environment image is information to be displayed; and displaying the information to be displayed according to a first area in the first environment image corresponding to the object to be displayed.

[0022] In the case of acquiring the first environment image, the near-eye display device can determine the object in the first environment image. Accordingly, the near-eye display device can acquire the preset information corresponding to the object. As the number of objects in the first environment image increases, the number of preset information corresponding to the object will also increase. In the case where the number of preset information corresponding to the object is too much, if the near-eye display device displays all the preset information, it is easy to have information display delay, information display freeze, near-eye display device heating, etc., resulting in poor display effect of the preset information. Based on this, the near-eye display device can select part of the preset information from all the preset information as the information to be displayed to alleviate the information display pressure of the near-eye display device. Accordingly, the near-eye display device can display the information to be displayed according to the first area of ​​the object corresponding to the information to be displayed in the first environment image, so that when the user views the information to be displayed, the association relationship between the information to be displayed and the object can be clearly understood. Based on the reduction of the information display pressure of the near-eye display device and the improvement of the viewing intuitiveness of the association relationship between the information to be displayed and the object, it is conducive to improving the display effect of the near-eye display device on the information to be displayed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 It is a flowchart of a display method of a near-eye display device provided in an embodiment of the present application;

[0025] Figure 2 is a schematic block diagram of a first environment image involved in an embodiment of the present application;

[0026] Figure 3 is a schematic block diagram of a first environment image involved in another embodiment of the present application;

[0027] Figure 4 is a flow chart of a display method of a near-eye display device according to an embodiment of the present application;

[0028] Figure 5 is a schematic block diagram of a display device of a near-eye display device provided in an embodiment of the present application;

[0029] Figure 6 It is a schematic block diagram of a near-eye display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0032] The embodiments of the present application provide a display method, device, equipment and storage medium for a near-eye display device. The display method of the near-eye display device can be applied to a near-eye display device. The near-eye display device may include virtual reality (VR) glasses, augmented reality (AR) glasses, mixed reality (MR) glasses, VR helmets, AR helmets, MR helmets, etc., without limitation. The display method of the near-eye display device can also be applied to a server, which can be a separate server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0033] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0034] See also Figure 1 , Figure 1 It is a flow chart of a display method of a near-eye display device provided in an embodiment of the present application. It should be noted that the display method of a near-eye display device provided in an embodiment of the present application can be used in a near-eye display device or in a server, and is not limited here.

[0035] like Figure 1 As shown, the display method of the near-eye display device includes steps S101 to S105.

[0036] S101: Acquire a first environment image obtained by photographing the environment in which a near-eye display device is located.

[0037] For example, the near-eye display device may have a shooting function. For example, the near-eye display device may be provided with a shooting module, and the near-eye display device may realize the shooting function of the near-eye display device through the shooting module, which is not limited here.

[0038] The near-eye display device can shoot the environment in which the near-eye display device is located to obtain a first environment image. Accordingly, the near-eye display device can obtain the first environment image. The first environment image may include one or more objects, which are not limited here. The objects in the first environment image may include objects, texts, etc., which are not limited here. The first environment image can be used by the near-eye display device to determine the information to be displayed. For example, the near-eye display device can perform object recognition on the first environment image to obtain object information corresponding to the object in the first environment image. The near-eye display device can determine the object information corresponding to the object as the preset information corresponding to the first environment image, so as to provide for subsequent determination of the information to be displayed. For another example, the near-eye display device can translate the text in the first environment image to obtain the translation information corresponding to the text in the first environment image. The near-eye display device can determine the translation information corresponding to the text as the preset information corresponding to the first environment image, so as to provide for subsequent determination of the information to be displayed, which is not limited here.

[0039] In some embodiments, when a preset operation of a user on a near-eye display device is detected, a first environment image obtained by photographing the environment in which the near-eye display device is located is acquired.

[0040] Exemplarily, the preset operation may include the time length for which the posture of the near-eye display device remains unchanged is greater than or equal to the preset time length threshold, key operation on the near-eye display device, touch operation on the near-eye display device, voice instruction information, gesture information, etc. The preset operation may be pre-set or user-set, which is not limited here.

[0041] Since the near-eye display device is worn on the user's head, the device posture of the near-eye display device remains unchanged when the user's head remains still. For example, if the user wearing the near-eye display device stares at an object in the environment where the near-eye display device is located for a time period greater than or equal to a preset time threshold, the near-eye display device can determine that the device posture of the near-eye display device remains unchanged for a time period greater than or equal to the preset time threshold, and further determine that a preset operation of the user on the near-eye display device is detected.

[0042] For example, the near-eye display device may be provided with a posture sensor for obtaining the device posture of the near-eye display device, such as an inertial measurement unit (IMU sensor), which is not limited here. The near-eye display device can determine whether the duration for which the device posture of the near-eye display device remains unchanged is greater than or equal to a preset duration threshold based on the posture data collected by the IMU sensor. For example, within a preset time period closest to the current time, the posture change between adjacent posture data is less than or equal to the preset posture change threshold, then the duration for which the device posture of the near-eye display device remains unchanged is greater than or equal to the preset duration threshold; wherein the duration of the preset time period is greater than or equal to the preset duration threshold. Accordingly, the near-eye display device can also determine that the IMU sensor is in a stable state.

[0043] In an exemplary embodiment, the near-eye display device can detect whether the IMU sensor is in a stable state. For example, the near-eye display device can detect whether the IMU sensor is in a stable state in real time, or detect whether the IMU sensor is in a stable state according to a preset time interval. The preset time interval may include 80 milliseconds, 90 milliseconds, 100 milliseconds, and so on, which are not limited here. In the process of the near-eye display device acquiring the first environmental image, if the IMU sensor changes from a stable state to an unstable state, the near-eye display device can stop acquiring the first environmental image. Correspondingly, if the IMU sensor changes from an unstable state to a stable state, the near-eye display device can resume acquiring the first environmental image.

[0044] For example, a preset button may be provided on the near-eye display device. The preset button may include a confirmation button, a tab button, etc., which are not limited here. When the near-eye display device detects a user's key operation on a preset button, the near-eye display device may determine that a preset operation of the user on the near-eye display device is detected.

[0045] For example, a touch area may be provided on the near-eye display device. When the near-eye display device detects a preset touch operation of the user on the touch area, the near-eye display device may determine that a preset operation of the user on the near-eye display device is detected. The preset touch operation may include a single touch operation, a double touch operation, a long touch operation, etc., which are not limited here.

[0046] For example, the near-eye display device may determine that a preset operation of the user on the near-eye display device is detected when the user speaks a preset voice message and / or the user makes a preset gesture. The preset voice message and the preset gesture may be pre-set or may be set by the user, which is not limited here.

[0047] When a preset operation of the user on the near-eye display device is detected, the near-eye display device may acquire the first environment image in response to the detected preset operation for subsequent determination of information to be displayed.

[0048] In this way, based on the settings of different types of preset operations, the near-eye display device can determine the user's display requirements for the information to be displayed according to the different types of preset operations performed by the user on the near-eye display device. Accordingly, the near-eye display device can acquire the first environment image in response to the user's display requirements for the information to be displayed, so as to subsequently determine the information to be displayed and display the information to be displayed, which is conducive to improving the convenience of the near-eye display device in acquiring the first environment image, and is conducive to improving the convenience of the near-eye display device in subsequently determining the information to be displayed.

[0049] S102: Determine an object in the first environment image.

[0050] When the first environment image is acquired, the near-eye display device may determine an object in the first environment image. The object may include an object, text, etc., which is not limited here.

[0051] Take the example of a supermarket in which the environment of the near-eye display device is located. Different types of objects may be displayed in the supermarket. Accordingly, the first environment image acquired by the near-eye display device may include different types of objects. The near-eye display device may determine the object in the first environment image as an object. For example, when the first environment image is acquired, the near-eye display device may determine that the user has an object recognition requirement for the first environment image. In response to the user's object recognition requirement for the first environment image, the near-eye display device may subsequently perform object recognition processing on the object in the first environment image so that the near-eye display device can subsequently determine the information to be displayed.

[0052] Take the example of an environment in which the near-eye display device is located, which includes a highway. Different types of traffic signs may be provided on the highway. Accordingly, the first environment image acquired by the near-eye display device may include different types of traffic signs. The near-eye display device may determine the different types of traffic signs included in the first environment image as objects. For example, text may be provided on the traffic sign. When the first environment image is acquired, the near-eye display device may determine that the user has a translation demand for the first environment image. In response to the user's translation demand for the first environment image, the near-eye display device may subsequently translate the text related to the traffic sign in the first environment image, so that the near-eye display device may subsequently determine the information to be displayed.

[0053] In this way, when the near-eye display device determines the object in the first environment image, the object can be used for subsequent determination of information to be displayed.

[0054] S103: Obtain preset information corresponding to the object in the first environment image.

[0055] In some embodiments, when an object in the first environment image is determined, the near-eye display device may perform information extraction processing on the object to determine preset information corresponding to the object. For example, the information extraction processing may include object recognition processing, optical character recognition (OCR) processing, natural language processing (NLP), etc., which are not limited here.

[0056] For example, the near-eye display device can perform object recognition processing on the object to obtain preset information corresponding to the object. The preset information includes, for example, the name of the object, a brief introduction of the object, instructions for use of the object, etc., which are not limited here.

[0057] For example, the near-eye display device can also perform OCR processing or NLP on the object to obtain preset information corresponding to the object. The preset information may include translation information of the object, text information of the object, etc., which are not limited here.

[0058] Exemplarily, the same object may correspond to one or more preset information.

[0059] For example, for objects in a supermarket, different texts may be set on different packaging areas of the object, such as the name text of the object, the introduction text of the object, etc. The near-eye display device may perform information extraction processing on the texts set on each packaging area of ​​the object to determine the preset information corresponding to the object. Based on this, different texts set on different packaging areas may correspond to multiple different preset information corresponding to the object, that is, the same object may correspond to multiple preset information.

[0060] For example, for a traffic sign, a traffic icon is provided on the traffic sign. The near-eye display device can perform information extraction processing on the traffic icon provided on the traffic sign to determine the preset information corresponding to the object. Based on this, a traffic icon provided on the traffic sign can correspond to a preset information of the traffic sign, that is, the same object can correspond to a preset information.

[0061] In this way, when obtaining the preset information corresponding to the object in the first environment image, the near-eye display device can use the preset information corresponding to the object as the basis for the near-eye display device to determine the information to be displayed. For example, the near-eye display device can select part of the preset information corresponding to the object as the information to be displayed. When the near-eye display device subsequently displays the information to be displayed, since the amount of information to be displayed is less than the amount of preset information, the display pressure of the near-eye display device on the information to be displayed can be alleviated, so as to avoid situations such as slow information display, information display freeze, and heating of the near-eye display device, thereby improving the display effect of the information to be displayed.

[0062] S104: Determine that part of the preset information corresponding to the first environment image is information to be displayed.

[0063] For example, in order to alleviate the information display pressure of the near-eye display device, the near-eye display device may select part of the preset information from all the preset information corresponding to the first environment image as the information to be displayed.

[0064] In some embodiments, based on the size of the object corresponding to the preset information, part of the preset information corresponding to the first environment image is determined as the information to be displayed; the size of the object corresponding to the information to be displayed is larger than the size of the object corresponding to the remaining preset information.

[0065] For example, the size of the object may include at least one of the width of the object in the first environment image and the height of the object in the first environment image, which is not limited here.

[0066] The near-eye display device can determine whether it has a display demand for the preset information corresponding to the corresponding object based on the size of the object corresponding to the preset information. For example, the closer the near-eye display device is to the object, the larger the size of the object in the first environmental image will be. The larger the size of the object in the first environmental image, the higher the possibility that the user can view the object, and the user's display demand for the preset information corresponding to the object will also increase accordingly. Based on this, the near-eye display device can determine that the user has a display demand for the preset information corresponding to the object based on the largest object in the first environmental image, and then determine the preset information corresponding to the largest object as the information to be displayed. Among them, the largest object can include one or more, and accordingly, the number of preset information corresponding to the largest object can include one or more, which is not limited here. In the case where the size of the object corresponding to the information to be displayed is the largest, the size of the object corresponding to the information to be displayed is larger than the size of the objects corresponding to the remaining preset information.

[0067] like Figure 2As shown, the objects in the first environment image include objects 1 to 3. In the first environment image, the size of object 1 is smaller than the size of object 2, and the size of object 1 is larger than the size of object 3. Based on this, the near-eye display device can determine that the preset information corresponding to object 2 is the information to be displayed based on the fact that the size of object 2 is larger than the other objects, such as the sizes of object 1 and object 3.

[0068] In this way, when determining the information to be displayed, the near-eye display device can judge whether the user has the display requirements for the preset information corresponding to different objects based on the size of the object corresponding to the preset information, and then select part of the preset information from the preset information corresponding to the object as the information to be displayed, which is beneficial to reduce the amount of information to be displayed, so as to alleviate the information display pressure of the near-eye display device, thereby helping to improve the display effect of the near-eye display device for the information to be displayed.

[0069] In some implementations, preset information corresponding to an object within a preset range of the first environment image is determined as information to be displayed, and the preset range is a local range of the first environment image.

[0070] The near-eye display device can determine whether the user has a display requirement for the preset information corresponding to the corresponding object based on the range of the object corresponding to the preset information in the first environment image.

[0071] For example, the near-eye display device may be provided with a preset range, and the preset range may be pre-set or may be set by the user, which is not limited here.

[0072] In an exemplary embodiment, the near-eye display device may output a range selection prompt before determining that part of the preset information corresponding to the first environment image is the information to be displayed; and determine the preset range according to the feedback information corresponding to the range selection prompt. For example, the near-eye display device may output a range selection prompt in the form of text, voice, etc. to prompt the user to determine the preset range. Accordingly, in response to the range selection prompt output by the near-eye display device, the user may select a local range as the preset range from the overall range of the first environment image through gestures, voice, touch, etc. For example, the range selection prompt information may be used to indicate the correspondence between each of the multiple local ranges included in the first environment image and different serial numbers, and the user may determine the feedback information of the range selection prompt information by making gestures indicating different serial numbers or speaking voices specifying different serial numbers. For another example, when the near-eye display device outputs the range selection prompt information, the user may select a local range from the first environment image as the preset range of the first environment image through a touch operation on the near-eye display device, such as a long touch and move operation. Of course, the method of determining the preset range is not limited to this, and is not limited here. The preset range can be used by the near-eye display device to select part of the preset information from all the preset information corresponding to the first environment image as the information to be displayed.

[0073] When there is an object within the preset range of the first environmental image, the near-eye display device can determine that the user has a display requirement for preset information corresponding to the object within the preset range of the first environmental image, and then determine the preset information corresponding to the object within the preset range as the information to be displayed.

[0074] like Figure 3 As shown, the objects include objects 1 to 3. In the first environment image, objects 1 and 2 are both within the preset range of the first environment image, and object 3 is not within the preset range of the first environment image, then the near-eye display device can determine that the preset information corresponding to each of objects 1 and 2 is the information to be displayed.

[0075] In this way, when determining the information to be displayed, the near-eye display device can judge whether the user has the display requirements for the preset information corresponding to different objects based on the preset range of the first environmental image, and then select part of the preset information from the preset information corresponding to the object as the information to be displayed, which is beneficial to reduce the amount of information to be displayed, so as to alleviate the information display pressure of the near-eye display device, thereby helping to improve the display effect of the near-eye display device for the information to be displayed.

[0076] Exemplarily, the near-eye display device may determine the information to be displayed based on the size of the object in the first environment image and the preset range of the first environment image.

[0077] In some implementations, within a preset range of the first environment image, preset information corresponding to an object with the largest size is determined as information to be displayed; the preset range is a local range of the first environment image.

[0078] comprehensive Figure 2 as well as Figure 3 It can be seen that the objects include objects 1 to 3. Since in the first environment image, the size of object 1 is smaller than the size of object 2, the size of object 1 is larger than the size of object 3, and both object 1 and object 2 are within the preset range of the first environment image, the near-eye display device can determine that the preset information corresponding to object 2 is the information to be displayed.

[0079] In this way, in the process of determining that part of the preset information corresponding to the first environmental image is the information to be displayed, the near-eye display device can flexibly select part of the information from all the preset information corresponding to the first environmental image as the information to be displayed based on the size of the object and at least one of the preset ranges of the first environmental image, which is conducive to improving the flexibility and convenience of determining the information to be displayed.

[0080] S105 . Display the information to be displayed according to the first area of ​​the object corresponding to the information to be displayed in the first environment image.

[0081] In the case of determining the information to be displayed, the near-eye display device may obtain the first area of ​​the object corresponding to the information to be displayed in the first environment image, and display the information to be displayed according to the first area of ​​the object corresponding to the information to be displayed in the first environment image. The first area may be used as a display basis for the information to be displayed. For example, the first area may correspond to the position and size of the object corresponding to the information to be displayed in the environment where the near-eye display device is located. The near-eye display device may display the information to be displayed in the environment where the near-eye display device is located by determining the first position and the first size of the first area in the environment where the near-eye display device is located, so as to indicate the association relationship between the information to be displayed and the object corresponding to the information to be displayed. For example, the first area may also correspond to the position and the size of the object corresponding to the information to be displayed in the first environment image. The near-eye display device may display the information to be displayed in the first environment image by determining the second position and the second size of the first area in the first environment image, so as to indicate the association relationship between the information to be displayed and the object corresponding to the information to be displayed.

[0082] Exemplarily, the near-eye display device may adjust the size of the first area of ​​the object acquired in the first environment image.

[0083] In some embodiments, when the size of the shortest side of the first area corresponding to the information to be displayed is smaller than a preset size threshold, the first area is size expanded to obtain a processed first area; wherein the size of the shortest side of the processed first area is greater than or equal to the preset size threshold; and the information to be displayed is displayed based on the processed first area.

[0084] For example, when the size of the shortest side of the first area is less than the preset size threshold, the first size of the first area in the environment where the near-eye display device is located, and the second size of the first area in the first environment image are generally also small. In the case of displaying the information to be displayed based on the first area later, it is easy to appear in a smaller range in the environment where the near-eye display device is located, or the information to be displayed is displayed in a smaller range in the first environment image, which may easily cause the user to be unable to see the information to be displayed, thereby causing the near-eye display device to display the information poorly. Based on this, the near-eye display device can perform a size expansion process on the first area when the size of the shortest side of the first area is less than the preset size threshold, until the size of the shortest side of the processed first area is greater than or equal to the preset size threshold. For example, when it is determined that part of the preset information corresponding to the first environment image is the information to be displayed, the near-eye display device can perform a size expansion process on the first area based on the size of the shortest side of the first area corresponding to the information to be displayed being less than the preset size threshold, so as to obtain the processed first area. For another example, when it is determined that the preset information corresponding to some objects within the preset range of the first environmental image is the information to be displayed, the near-eye display device can perform size expansion processing on the first area corresponding to the information to be displayed based on the fact that the size of the shortest side of the first area is less than the preset size threshold, so as to obtain the processed first area, which is not limited here. The preset size threshold can be pre-set or set by the user. For example, the preset size threshold can include 40 pixels, 45 pixels, 50 pixels, 55 pixels, 60 pixels, and so on. Of course, the preset size threshold is not limited to this, and the size unit of the preset size threshold can also include millimeters, centimeters, etc., which is not limited here.

[0085] The processed first area can be used by the near-eye display device as a basis for displaying the information to be displayed in the future, so as to improve the display effect of the information to be displayed. For example, the processed first area can correspond to the position and size of the object corresponding to the information to be displayed in the environment where the near-eye display device is located. The near-eye display device can display the information to be displayed in the environment where the near-eye display device is located by determining the first position and the first size of the processed first area in the environment where the near-eye display device is located, so as to indicate the association relationship between the information to be displayed and the object corresponding to the information to be displayed. For another example, the processed first area can also correspond to the position and the size of the object corresponding to the information to be displayed in the first environmental image. The near-eye display device can display the information to be displayed in the first environmental image by determining the second position and the second size of the processed first area in the first environmental image, so as to indicate the association relationship between the information to be displayed and the object corresponding to the information to be displayed.

[0086] In some embodiments, based on the first area and the first environmental image, a first position and a first size of the first area in the environment in which the near-eye display device is located are determined; based on the first position and the first size, a first display position and a first display size of the information to be displayed are determined; and based on the first display position and the first display size of the information to be displayed, the information to be displayed is displayed.

[0087] The near-eye display device can display the information to be displayed in the environment where the near-eye display device is located, so the near-eye display device needs to determine a first display position and a first display size of the information to be displayed in the environment where the near-eye display device is located.

[0088] For example, the near-eye display device can display the information to be displayed in the environment where the near-eye display device is located according to the position and size of the object corresponding to the information to be displayed in the environment where the near-eye display device is located. Since the first area is associated with the object corresponding to the information to be displayed, the near-eye display device can determine the first display position and the first display size of the information to be displayed according to the first area and the first environmental image. For example, the near-eye display device can input the first area and the first environmental image into a preset size recognition algorithm so that the size recognition algorithm determines the first size of the first area in the environment where the near-eye display device is located, and obtains the first size of the first area. Correspondingly, the near-eye display device can determine the environmental size of the environment where the near-eye display device is located corresponding to the first environmental image based on the ratio between the first area and the first environmental image, combined with the first size of the first area. For another example, the near-eye display device can input the first area and the first environmental image into a preset position recognition algorithm so that the position recognition algorithm determines the first position of the first area in the environment where the near-eye display device is located in combination with the imaging parameters of the near-eye display device.

[0089] When the first position and the first size of the first area are determined, the near-eye display device can determine the first position of the first area as the first display position of the information to be displayed, and the near-eye display device can determine the first size of the first area as the first display size of the information to be displayed.

[0090] Accordingly, when the first display position and the first display size of the information to be displayed are determined, the near-eye display device can project the information to be displayed into the environment where the near-eye display device is located according to the first display position and the first display size, thereby displaying the information to be displayed.

[0091] In this way, the near-eye display device displays the information to be displayed according to the first area of ​​the first environmental image corresponding to the object to be displayed, and can project the information to be displayed into the environment where the near-eye display device is located. The user can then view the information to be displayed in the environment where the near-eye display device is located, which is beneficial to improving the user's information viewing experience of the near-eye display device, thereby helping to improve the display effect of the near-eye display device on the information to be displayed.

[0092] In some embodiments, based on the first area and the first environmental image, the second position and the second size of the first area in the first environmental image are determined; based on the second position and the second size, the second display position and the second display size of the information to be displayed are determined; and based on the second display position and the second display size of the information to be displayed, the information to be displayed is displayed.

[0093] The near-eye display device can display the information to be displayed on the first environment image, and the near-eye display device needs to determine the second display position and the second display size of the information to be displayed in the first environment image.

[0094] For example, the near-eye display device can display the information to be displayed in the first environmental image according to the position and size of the object corresponding to the information to be displayed in the first environmental image. Since the first area is associated with the object corresponding to the information to be displayed, the near-eye display device can determine the second display position and the second display size of the information to be displayed according to the first area and the first environmental image. For example, the near-eye display device can obtain the size of the first environmental image. The near-eye display device can determine the second size of the first area in the first environmental image based on the ratio between the first area and the first environmental image and the size of the first environmental image. Accordingly, the near-eye display device can determine the second position of the first area in the first environmental image by matching the local range of the first area and the first environmental image.

[0095] When the second position and the second size of the first area are determined, the near-eye display device can determine the second position of the first area as the second display position of the information to be displayed, and the near-eye display device can determine the second size of the first area as the second display size of the information to be displayed.

[0096] Correspondingly, when the second display position and the second display size of the information to be displayed are determined, the near-eye display device can display the information to be displayed in the first area of ​​the first environmental image according to the second display position and the second display size, thereby displaying the information to be displayed.

[0097] In this way, the near-eye display device displays the information to be displayed in the first area of ​​the first environmental image according to the object to be displayed corresponding to the information to be displayed. The information to be displayed can be displayed in the first area of ​​the first environmental image. Then, the user can view the information to be displayed in the first area of ​​the first environmental image through the near-eye display device, which is beneficial to improving the user's information viewing experience of the near-eye display device, thereby helping to improve the display effect of the near-eye display device on the information to be displayed.

[0098] Exemplarily, when the near-eye display device displays the information to be displayed, the near-eye display device can track and display the information to be displayed. For example, the near-eye display device can determine whether the user has a tracking display requirement for the information to be displayed based on the preset operation of the user on the near-eye display device when acquiring the first environmental image. When the near-eye display device determines that the user has a tracking display requirement for the information to be displayed, the near-eye display device can anchor the display position of the information to be displayed with the position of the first area corresponding to the position to be displayed. For example, the near-eye display device can anchor the first display position of the information to be displayed with the first position of the first area corresponding to the information to be displayed. For another example, the near-eye display device can anchor the second display position of the information to be displayed with the second position of the first area corresponding to the information to be displayed. Accordingly, the near-eye display device can anchor the display size of the information to be displayed with the size of the first area corresponding to the information to be displayed. For example, the near-eye display device can anchor the first display size of the information to be displayed with the first size of the first area corresponding to the information to be displayed. For another example, the near-eye display device can anchor the second display size of the information to be displayed with the second size of the first area corresponding to the information to be displayed. Based on this, the near-eye display device can track and display the information to be displayed according to the position anchoring relationship and the size anchoring relationship between the information to be displayed and the first area corresponding to the information to be displayed.

[0099] In some embodiments, when a preset operation meets a preset information tracking condition, a second environment image is obtained by re-shooting the environment in which the near-eye display device is located; position change information and size change information of a second area corresponding to the information to be displayed in the second environment image relative to a first area corresponding to the information to be displayed in the first environment image are determined; and the information to be displayed is updated based on the position change information and the size change information.

[0100] The near-eye display device may be provided with preset information tracking conditions. For example, the near-eye display device may be provided with preset associations between different preset operations and tracking display requirement tags. Accordingly, the preset information tracking conditions may be determined according to the preset operations associated with the tracking display requirement tags indicating that the user has a tracking display requirement for the information to be displayed.

[0101] Take the preset operation including the device posture of the near-eye display device remaining unchanged for a time period greater than or equal to the preset time threshold and the key operation of the near-eye display device as an example. The association relationship between different preset operations and tracking display requirement labels includes, for example: when the preset operation is that the device posture of the near-eye display device remains unchanged for a time period greater than or equal to the preset time threshold, the tracking display requirement label corresponding to the preset operation is used to indicate that the user has a tracking display requirement for the information to be displayed; when the preset operation is a key operation of the near-eye display device, the tracking display requirement label corresponding to the preset operation is used to indicate that the user does not have a tracking display requirement for the information to be displayed. Based on this, when the corresponding preset operation is that the device posture of the near-eye display device remains unchanged for a time period greater than or equal to the preset time threshold when acquiring the first environmental image, the near-eye display device can determine that the preset operation meets the preset information tracking condition. Accordingly, the near-eye display device can obtain a second environmental image obtained by the near-eye display device taking another photo of the environment in which it is located, for tracking and displaying the information to be displayed.

[0102] The preset information tracking conditions may be pre-set or may be set by the user, and are not limited here.

[0103] For example, when it is determined that the preset operation meets the preset information tracking condition, the near-eye display device can determine the position change information and size change information of the second area corresponding to the information to be displayed in the second environment image relative to the first area corresponding to the information to be displayed in the first environment image based on the first environment image and the second environment image. For example, the near-eye display device can perform image matching processing on the first environment image and the second environment image based on the position anchoring relationship and size anchoring relationship between the information to be displayed and the first area corresponding to the information to be displayed, combined with the object corresponding to the first area, to obtain the second area matching the first area in the second environment image. Accordingly, when the second area is determined, the near-eye display device can determine the third position and third size of the second area in the second environment image. Among them, the method for determining the third position and third size of the second area in the second environment image can refer to the aforementioned method for determining the second position and second size of the first area in the first environment image, which will not be repeated here. The near-eye display device can determine the position change information of the second area relative to the first area in combination with the second position of the first area and the third position of the second area. The near-eye display device may determine size change information of the second area relative to the first area in combination with the second size of the first area and the third size of the second area.

[0104] In the case of determining the position change information and the size change information, the near-eye display device can update the display target display information based on the position change information and the size change information. For example, in the case where the near-eye display device displays the information to be displayed in the environment where the near-eye display device is located, the near-eye display device can determine the fourth position of the second area in the environment where the near-eye display device is located based on the position change information and the first position of the first area, and determine the fourth size of the second area in the environment where the near-eye display device is located based on the size change information and the first size of the first area. Accordingly, the near-eye display device can update and display the information to be displayed based on the fourth position and fourth size corresponding to the second area. For example, in the case where the near-eye display device displays the information to be displayed on the first environmental image, the near-eye display device can update and display the information to be displayed on the second environmental image based on the position change information and the size change information.

[0105] In this way, the near-eye display device can update and display the information to be displayed based on the second environmental image when the preset operation meets the preset information tracking conditions. It can adapt to the user's tracking and display needs for the displayed information, and improve the convenience of information display tracking of the displayed information, thereby helping to improve the user's information tracking experience for the displayed information.

[0106] In some embodiments, when it is detected that a first posture change corresponding to the device posture of the near-eye display device is greater than or equal to a first posture change threshold, and / or a second posture change corresponding to the object posture of the object corresponding to the information to be displayed is greater than or equal to a second posture change threshold, the information to be displayed is stopped from being displayed.

[0107] For example, in the process of displaying the information to be displayed, the near-eye display device can detect the device posture of the near-eye display device, and determine the first posture change amount corresponding to the device posture of the near-eye display device. If the near-eye display device detects that the first posture change amount corresponding to the device posture of the near-eye display device is greater than or equal to the first posture change threshold, the near-eye display device can determine that the user's movement amplitude is large, and the user's movement amplitude is likely to cause the object corresponding to the information to be displayed to gradually deviate from the shooting range of the near-eye display device. Based on this, the near-eye display device can determine that there is no longer a demand for displaying the information to be displayed, and then the near-eye display device can stop displaying the information to be displayed. The first posture change threshold can be pre-set or set by the user, and is not limited here.

[0108] For example, in the process of displaying the information to be displayed, the near-eye display device can detect the object posture of the object corresponding to the information to be displayed, and determine the second posture change amount corresponding to the object posture of the object corresponding to the information to be displayed. If the near-eye display device detects that the second posture change amount corresponding to the object posture of the object corresponding to the information to be displayed is greater than or equal to the second posture change threshold, the near-eye display device can determine that the movement amplitude of the object corresponding to the information to be displayed is large, and the movement amplitude of the object corresponding to the information to be displayed is likely to cause the object corresponding to the information to be displayed to gradually deviate from the shooting range of the near-eye display device. Based on this, the near-eye display device can determine that there is no longer a demand for displaying the information to be displayed, and then the near-eye display device can stop displaying the information to be displayed. The second posture change threshold can be pre-set or set by the user, and is not limited here.

[0109] In this way, when displaying information to be displayed, the near-eye display device can determine whether to stop displaying the information to be displayed based on at least one of the first posture change amount corresponding to the device posture of the near-eye display device and the second posture change amount corresponding to the object posture of the object corresponding to the information to be displayed. This can enable the near-eye display device to adapt whether to display the information to be displayed to the user's information display requirements, thereby helping to improve the display accuracy of the near-eye display device for the information to be displayed, as well as the display effect of the information to be displayed.

[0110] In an exemplary embodiment, Figure 4 As shown, the display method of the near-eye display device includes steps S210 to S241.

[0111] S210: Detecting a preset operation of a user on a near-eye display device.

[0112] For example, the preset operation may include that the device posture of the near-eye display device remains unchanged for a period greater than or equal to a preset time threshold, a key operation on the near-eye display device, etc., which is not limited here.

[0113] S220, determine whether the preset operation meets the preset information tracking condition; if so, jump to S221, if not, jump to S240.

[0114] For example, the near-eye display device may preset associations between different preset operations and tracking display requirement tags. Then, when the preset operation is detected, the near-eye display device may determine whether the tracking display requirement tag associated with the preset operation indicates that the user has a tracking display requirement for the information to be displayed. When the tracking display requirement tag associated with the preset operation indicates that the user has a tracking display requirement for the information to be displayed, the near-eye display device may determine that the preset operation meets the preset information tracking condition. Accordingly, when the tracking display requirement tag associated with the preset operation indicates that the user does not have a tracking display requirement for the information to be displayed, the near-eye display device may determine that the preset operation does not meet the preset information tracking condition.

[0115] S221, determine whether the IMU sensor is in a stable state; if so, jump to S222, if not, jump to S230.

[0116] For example, in response to the user's tracking and display requirements for information to be displayed, the near-eye display device needs to determine the information to be displayed. Based on this, the near-eye display device can determine whether the IMU sensor is in a stable state, so that when the IMU sensor is in a stable state, the near-eye display device can obtain a first environment image obtained by photographing the environment in which the near-eye display device is located, and determine the information to be displayed based on the first environment image.

[0117] S222, determine whether the near-eye display device currently displays information to be displayed; if not, jump to S223.

[0118] For example, when the IMU sensor is in a stable state, the near-eye display device needs to determine whether it needs to obtain the information to be displayed. For example, when the near-eye display device currently displays information to be displayed, the near-eye display device does not need to obtain the first environment image again to determine the information to be displayed, and the near-eye display device can keep displaying the currently displayed information to be displayed, and track and display the currently displayed information to be displayed. For another example, when the near-eye display device does not currently display the information to be displayed, the near-eye display device needs to obtain the first environment image to determine the information to be displayed.

[0119] S223, determine whether the near-eye display device is currently acquiring information to be displayed; if not, jump to S224.

[0120] For example, when the near-eye display device is not currently displaying information to be displayed, the near-eye display device needs to determine whether it needs to obtain the information to be displayed. For example, when the near-eye display device is currently obtaining information to be displayed, the near-eye display device does not need to obtain the first environment image again to determine the information to be displayed, and the near-eye display device can continue to obtain the current information to be displayed. For another example, when the near-eye display device is not currently obtaining information to be displayed, the near-eye display device needs to obtain the first environment image to determine the information to be displayed.

[0121] S224: Acquire a first environment image obtained by photographing the environment in which the near-eye display device is located, and determine information to be displayed according to the first environment image.

[0122] For example, when the near-eye display device obtains the first environmental image, it can determine the objects in the first environmental image, obtain preset information corresponding to the objects in the first environmental image, and determine part of the preset information corresponding to the first environmental image as information to be displayed.

[0123] S225. Determine whether the IMU sensor is in a stable state when the near-eye display device determines the information to be displayed; if so, jump to S226; if not, jump to S227.

[0124] For example, in the process of determining the information to be displayed by the near-eye display device, the device posture of the near-eye display device will still change with the user's behavior, such as changes in the user's head posture. In the case where the device posture of the near-eye display device changes, the IMU sensor will correspondingly change from a stable state to an unstable state. When the IMU sensor is in an unstable state, the near-eye display device, for example, cannot obtain a clear first environment image, or the user's display requirements for the displayed information may change. In the case where a clear first environment image cannot be obtained, the near-eye display device needs to suspend obtaining the first environment image. Accordingly, in the case where the user's display requirements for the displayed information change, the near-eye display device does not need to continue to obtain the information to be displayed corresponding to the current first environment image. Based on this, the near-eye display device needs to detect whether the IMU sensor is in a stable state during the process of the near-eye display device determining the information to be displayed, so as to determine whether it is necessary to continue to determine the information to be displayed corresponding to the current first environment image.

[0125] S226: Display the information to be displayed according to the first area of ​​the object corresponding to the information to be displayed in the first environment image.

[0126] For example, when the IMU sensor is in a stable state, the near-eye display device can display the information to be displayed corresponding to the acquired first environment image. For example, the near-eye display device can project the information to be displayed to the first area in the environment where the near-eye display device is located based on the first area corresponding to the information to be displayed. For another example, the near-eye display device can display the information to be displayed on the first environment image based on the first area corresponding to the information to be displayed.

[0127] S227, suspending the acquisition of at least one of the first environment image and the second environment image, and suspending the determination of the information to be displayed, and waiting for the IMU sensor to return to a stable state.

[0128] For example, when the IMU sensor is in an unstable state, the near-eye display device may suspend the acquisition of at least one of the first environment image and the second environment image, and suspend the determination of the information to be displayed, waiting for the IMU sensor to return to a stable state. When the IMU sensor returns to a stable state, the near-eye display device may resume the acquisition of at least one of the first environment image and the second environment image, or resume the determination of the information to be displayed corresponding to at least one of the first environment image and the second environment image. For example, the near-eye display device may resume the acquisition of at least one of the first environment image and the second environment image, or resume the determination of the information to be displayed corresponding to at least one of the first environment image and the second environment image, based on the first posture change amount corresponding to the device posture of the near-eye display device being less than the first posture change threshold. The near-eye display device may also reacquire at least one of the first environment image and the second environment image to determine the information to be displayed corresponding to at least one of the reacquired first environment image and the second environment image. For another example, the near-eye display device may reacquire at least one of the first environment image and the second environment image, based on the first posture change amount corresponding to the device posture of the near-eye display device being greater than or equal to the first posture change threshold, to determine the information to be displayed corresponding to at least one of the reacquired first environment image and the second environment image. The first posture change amount may be determined based on the device posture of the near-eye display device acquired by the IMU sensor.

[0129] S228. Obtain a second environment image obtained by the near-eye display device photographing the environment again, and update the information to be displayed according to the first environment image and the second environment image.

[0130] For example, when the near-eye display device displays information to be displayed, since the preset operation meets the preset information tracking condition, the near-eye display device needs to track and display the information to be displayed. Based on this, the near-eye display device can capture the environment again to obtain a second environment image, and combine the first environment image and the second environment image to update and display the information to be displayed, so as to achieve tracking display of the information to be displayed.

[0131] S229. When it is detected that the first posture change corresponding to the device posture of the near-eye display device is greater than or equal to the first posture change threshold, and / or the second posture change corresponding to the object posture of the object corresponding to the information to be displayed is greater than or equal to the second posture change threshold, stop displaying the information to be displayed.

[0132] For example, in the process of tracking and displaying the information to be displayed, if the first posture change is greater than or equal to the first posture change threshold, and / or the second posture change is greater than or equal to the second posture change threshold, the near-eye display device can determine that at least one of the user's movement amplitude and the movement amplitude of the object corresponding to the information to be displayed is larger, so that the object corresponding to the information to be displayed gradually deviates from the shooting range of the near-eye display device. Based on this, the near-eye display device can determine that there is no longer a demand for displaying the information to be displayed and no longer a demand for tracking and displaying the information to be displayed, and then the near-eye display device can stop displaying the information to be displayed.

[0133] S230, determining whether the near-eye display device is currently displaying information to be displayed; if so, jumping to S227.

[0134] For example, when the IMU sensor is in an unstable state, since the preset operation meets the preset information tracking conditions, the near-eye display device can determine whether the information to be displayed is currently displayed, so as to determine whether it is necessary to suspend the tracking and display of the information to be displayed. For example, during the display tracking of the information to be displayed, if the IMU sensor changes from a stable state to an unstable state, the near-eye display device cannot obtain a clear second environment image, thereby causing an adverse effect on the tracking and display process of the information to be displayed. Based on this, when the IMU sensor is in an unstable state, the acquisition of the second environment image is suspended, and the determination of the information to be displayed is suspended, waiting for the IMU sensor to return to a stable state.

[0135] S240: Acquire a first environment image obtained by photographing the environment in which the near-eye display device is located, and determine information to be displayed according to the first environment image.

[0136] For example, in response to the user not having a tracking display requirement for the information to be displayed, the near-eye display device may directly determine the information to be displayed. For example, when the near-eye display device obtains the first environment image, it may determine the object in the first environment image, obtain the preset information corresponding to the object in the first environment image, and determine part of the preset information corresponding to the first environment image as the information to be displayed.

[0137] S241: Display the information to be displayed according to a first area of ​​the object corresponding to the information to be displayed in the first environment image.

[0138] Accordingly, in response to the user not having a tracking and display requirement for the information to be displayed, the near-eye display device may directly display the information to be displayed according to the first area of ​​the object corresponding to the information to be displayed in the first environment image, without tracking and displaying the information to be displayed.

[0139] The display method of the near-eye display device provided in the above embodiment includes: obtaining a first environmental image obtained by shooting the environment in which the near-eye display device is located; determining an object in the first environmental image; obtaining preset information corresponding to the object in the first environmental image; determining that part of the preset information corresponding to the first environmental image is information to be displayed; and displaying the information to be displayed according to a first area in the first environmental image corresponding to the object to be displayed.

[0140] In the case of acquiring the first environment image, the near-eye display device can determine the object in the first environment image. Accordingly, the near-eye display device can acquire the preset information corresponding to the object. As the number of objects in the first environment image increases, the number of preset information corresponding to the object will also increase. In the case where the number of preset information corresponding to the object is too much, if the near-eye display device displays all the preset information, it is easy to have information display delay, information display freeze, near-eye display device heating, etc., resulting in poor display effect of the preset information. Based on this, the near-eye display device can select part of the preset information from all the preset information as the information to be displayed to relieve the information display pressure of the near-eye display device. Accordingly, the near-eye display device can display the information to be displayed according to the first area of ​​the object corresponding to the information to be displayed in the first environment image, so that when the user views the information to be displayed, the association relationship between the information to be displayed and the object corresponding to the information to be displayed can be clearly understood. Based on the reduction of the information display pressure of the near-eye display device and the improvement of the viewing intuitiveness of the association relationship between the information to be displayed and the object corresponding to the information to be displayed, it is conducive to improving the display effect of the near-eye display device on the information to be displayed.

[0141] See also Figure 5 , Figure 5It is a schematic block diagram of a display device of a near-eye display device provided in an embodiment of the present application. The display device of the near-eye display device can be configured in a near-eye display device or a server to execute the display method of the aforementioned near-eye display device. The near-eye display device may include VR glasses, AR glasses, MR glasses, VR helmets, AR helmets, MR helmets, and the like, without limitation. The server may be a separate server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms.

[0142] like Figure 5 As shown, the display device of the near-eye display apparatus includes an image acquisition module 110 , a first determination module 120 , an information acquisition module 130 , a second determination module 140 and a display module 150 .

[0143] The image acquisition module 110 is used to acquire a first environment image obtained by photographing the environment in which the near-eye display device is located.

[0144] The first determining module 120 is configured to determine an object in the first environment image.

[0145] The information acquisition module 130 is used to acquire preset information corresponding to the object in the first environment image.

[0146] The second determining module 140 is configured to determine that part of the preset information corresponding to the first environment image is information to be displayed.

[0147] The display module 150 is configured to display the information to be displayed according to a first area of ​​the object corresponding to the information to be displayed in the first environment image.

[0148] Exemplarily, the second determination module 140 includes a first determination submodule.

[0149] The first determination submodule is used to determine, based on the size of the object corresponding to the preset information, that part of the preset information corresponding to the first environment image is information to be displayed; the size of the object corresponding to the information to be displayed is larger than the size of the object corresponding to the rest of the preset information.

[0150] Exemplarily, the display module 150 includes a size expansion submodule and a first display submodule.

[0151] A size expansion submodule, configured to, when the size of the shortest side of the first area corresponding to the information to be displayed is smaller than a preset size threshold, perform size expansion processing on the first area to obtain a processed first area; wherein the size of the shortest side of the processed first area is greater than or equal to the preset size threshold;

[0152] The first display submodule is used to display the information to be displayed according to the processed first area.

[0153] Exemplarily, the second determination module 140 includes a second determination submodule.

[0154] The second determining submodule is used to determine that the preset information corresponding to the object within a preset range of the first environment image is the information to be displayed, and the preset range is a local range of the first environment image.

[0155] Exemplarily, the display module 150 includes a first information determining submodule, a second information determining submodule and a second display submodule.

[0156] The first information determination submodule is used to determine a first position and a first size of the first area in the environment where the near-eye display device is located according to the first area and the first environment image.

[0157] The second information determination submodule is used to determine a first display position and a first display size of the information to be displayed according to the first position and the first size.

[0158] The second display submodule is used to display the information to be displayed according to the first display position and the first display size of the information to be displayed.

[0159] Exemplarily, the display module 150 includes a third information determination submodule, a fourth information determination submodule and a third display submodule.

[0160] The third information determination submodule is used to determine a second position and a second size of the first area in the first environment image according to the first area and the first environment image.

[0161] The fourth information determination submodule is used to determine a second display position and a second display size of the information to be displayed according to the second position and the second size.

[0162] The third display submodule is used to display the information to be displayed according to the second display position and the second display size of the information to be displayed.

[0163] Exemplarily, the image acquisition module 110 includes a first acquisition submodule.

[0164] The first acquisition submodule is used to acquire a first environment image obtained by photographing the environment in which the near-eye display device is located when a preset operation of the user on the near-eye display device is detected.

[0165] The display device also includes a second acquisition submodule, a change information determination submodule and a display update submodule.

[0166] The second acquisition submodule is used to acquire a second environment image obtained by the near-eye display device photographing the environment again when the preset operation meets the preset information tracking condition.

[0167] The change information determination submodule is used to determine position change information and size change information of the second area corresponding to the second environment image of the information to be displayed relative to the first area corresponding to the first environment image of the information to be displayed.

[0168] The display update submodule is used to update and display the information to be displayed according to the position change information and the size change information.

[0169] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0170] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0171] Exemplarily, the above-mentioned method and apparatus may be implemented in the form of a computer program, which may be run on a near-eye display device or a server to control the near-eye display device. Exemplarily, the near-eye display device may include VR glasses, AR glasses, MR glasses, VR helmets, AR helmets, MR helmets, etc., without limitation. The server may be a separate server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms.

[0172] See also Figure 6 , Figure 6 It is a schematic block diagram of the structure of a near-eye display device provided in an embodiment of the present application.

[0173] like Figure 6 As shown, the near-eye display device includes a memory and a processor. The memory and the processor may be connected via a system bus, and the memory may include a storage medium and an internal memory.

[0174] The storage medium may store an operating system and a computer program. When the computer program is executed, the processor may execute any display method of a near-eye display device.

[0175] The processor is used to provide computing and control capabilities to support the operation of the entire near-eye display device.

[0176] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any display method of the near-eye display device.

[0177] Those skilled in the art will understand that Figure 6 The structure shown in the figure is merely a block diagram of a portion of the structure related to the scheme of the present application, and does not constitute a limitation on the near-eye display device to which the scheme of the present application is applied. The specific near-eye display device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0178] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0179] In one embodiment, the processor is used to execute a computer program and implement the following steps when executing the computer program:

[0180] Acquire a first environment image obtained by photographing the environment in which the near-eye display device is located;

[0181] determining an object in the first environment image;

[0182] Acquire preset information corresponding to the object in the first environment image;

[0183] Determining that part of the preset information corresponding to the first environment image is information to be displayed;

[0184] The information to be displayed is displayed according to a first area of ​​the object corresponding to the information to be displayed in the first environment image.

[0185] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the display of the near-eye display device described above can refer to the corresponding process in the display method embodiment of the aforementioned near-eye display device, and will not be repeated here.

[0186] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the display method of the near-eye display device of the present application.

[0187] The computer-readable storage medium may be an internal storage unit of the near-eye display device described in the aforementioned embodiment, such as a hard disk or memory of the near-eye display device. The computer-readable storage medium may also be an external storage device of the near-eye display device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the near-eye display device.

[0188] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0189] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0190] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A display method for a near-eye display device, characterized in that: The display method comprises: Acquire a first environment image obtained by photographing the environment in which the near-eye display device is located; determining an object in the first environment image; Acquire preset information corresponding to the object in the first environment image; Determining that part of the preset information corresponding to the first environment image is information to be displayed; The information to be displayed is displayed according to a first area of ​​the object corresponding to the information to be displayed in the first environment image.

2. The display method according to claim 1, characterized in that: The determining that part of the preset information corresponding to the first environment image is information to be displayed includes: According to the size of the object corresponding to the preset information, it is determined that part of the preset information corresponding to the first environment image is the information to be displayed; the size of the object corresponding to the information to be displayed is larger than the size of the object corresponding to the rest of the preset information.

3. The display method according to claim 2, characterized in that: Displaying the information to be displayed in a first area of ​​the first environment image according to the object corresponding to the information to be displayed includes: When the size of the shortest side of the first area corresponding to the information to be displayed is smaller than a preset size threshold, the first area is subjected to size expansion processing to obtain a processed first area; wherein the size of the shortest side of the processed first area is greater than or equal to the preset size threshold; The information to be displayed is displayed according to the processed first area.

4. The display method according to claim 1, characterized in that: The determining that part of the preset information corresponding to the first environment image is information to be displayed includes: It is determined that the preset information corresponding to the object within a preset range of the first environment image is the information to be displayed, and the preset range is a local range of the first environment image.

5. The display method according to any one of claims 1 to 4, characterized in that: The step of displaying the information to be displayed according to a first area of ​​the object corresponding to the information to be displayed in the first environment image includes: Determine, according to the first area and the first environment image, a first position and a first size of the first area in the environment where the near-eye display device is located; Determining a first display position and a first display size of the information to be displayed according to the first position and the first size; The information to be displayed is displayed according to a first display position and a first display size of the information to be displayed.

6. The display method according to any one of claims 1 to 4, characterized in that: The step of displaying the information to be displayed according to a first area of ​​the object corresponding to the information to be displayed in the first environment image includes: Determine a second position and a second size of the first area in the first environment image according to the first area and the first environment image; Determining a second display position and a second display size of the information to be displayed according to the second position and the second size; The information to be displayed is displayed according to the second display position and the second display size of the information to be displayed.

7. The display method according to any one of claims 1 to 4, characterized in that: The step of acquiring a first environment image obtained by photographing the environment in which the near-eye display device is located includes: When a preset operation of the user on the near-eye display device is detected, a first environment image obtained by photographing the environment in which the near-eye display device is located is acquired; After displaying the information to be displayed according to the first area of ​​the object corresponding to the information to be displayed in the first environment image, the method further includes: When the preset operation meets the preset information tracking condition, obtaining a second environment image obtained by the near-eye display device photographing the environment again; Determine position change information and size change information of the second area corresponding to the second environment image of the information to be displayed relative to the first area corresponding to the first environment image of the information to be displayed; The information to be displayed is updated and displayed according to the position change information and the size change information.

8. A display device of a near-eye display device, characterized in that: The display device comprises: An image acquisition module, used to acquire a first environment image obtained by photographing the environment in which the near-eye display device is located; A first determining module, used to determine an object in the first environment image; An information acquisition module, used to acquire preset information corresponding to the object in the first environment image; A second determining module, used to determine that part of the preset information corresponding to the first environment image is information to be displayed; The display module is used to display the information to be displayed according to a first area of ​​the object corresponding to the information to be displayed in the first environment image.

9. A near-eye display device, characterized in that: The near-eye display device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the steps of the display method of the near-eye display device according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the display method of the near-eye display device according to any one of claims 1 to 7 are implemented.

Images