Display method and device of near-to-eye display equipment, equipment and storage medium

Abstract

The invention relates to the technical field of display, and provides a display method, device and equipment of near-eye display equipment and a storage medium, the near-eye display equipment is provided with a first display module and a second display module, and the setting position of the first display module is different from that of the second display module; the display method comprises the following steps: acquiring to-be-displayed data of the near-to-eye display equipment; rendering the to-be-displayed data to obtain a to-be-displayed image corresponding to the to-be-displayed data; according to the display parameters of the first display module, displaying the to-be-displayed image on the first display module; and displaying the to-be-displayed image on the second display module according to the display parameter of the second display module so as to improve the display efficiency of the near-to-eye display device on the to-be-displayed image.

Description

Technical Field

[0001] This application relates to the field of display technologies, and in particular, to a display method, apparatus, device, and storage medium for a near-eye display device. Background Art

[0002] A near-eye display device has a display function. In related technologies, when a near-eye display device uses multiple display screens to cooperatively display the same display content, the near-eye display device often needs to repeatedly render the data corresponding to the same display content to determine the display content of each of the multiple display screens. In the case of repeatedly rendering the data corresponding to the same display content, it will additionally increase the rendering duration of the near-eye display device for the same display content, thereby resulting in poor display efficiency of the near-eye display device. Summary of the Invention

[0003] The main objective of this application is to provide a display method, apparatus, device, and storage medium for a near-eye display device, aiming to solve the technical problem of poor display efficiency of the near-eye display device caused by repeatedly rendering the data corresponding to the same display content when the near-eye display device uses multiple display screens to cooperatively display the same display content.

[0004] In a first aspect, this application provides a display method for a near-eye display device. A first display module and a second display module are provided on the near-eye display device, and the installation positions of the first display module and the second display module are different; The display method includes: Obtain the data to be displayed of the near-eye display device; Render the data to be displayed to obtain a to-be-displayed image corresponding to the data to be displayed; According to the display parameters of the first display module, display the to-be-displayed image on the first display module; According to the display parameters of the second display module, display the to-be-displayed image on the second display module.

[0005] In a second aspect, this application provides a display apparatus for a near-eye display device. The display apparatus includes: A data acquisition module, configured to acquire the data to be displayed of the near-eye display device; An image determination module, configured to render the data to be displayed to obtain a to-be-displayed image corresponding to the data to be displayed; A first display module, configured to display the to-be-displayed image on the first display module according to the display parameters of the first display module of the near-eye display device; A second display module, configured to display the to-be-displayed image on the second display module of the near-eye display device according to the display parameters of the second display module.

[0006] In a third aspect, the present application provides a near-eye display device, on which a first display module and a second display module are provided, and the installation positions of the first display module and the second display module are different; the near-eye display device includes a memory and a processor; The memory is configured to store a computer program; The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the display method of the near-eye display device as described above.

[0007] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the display method of the near-eye display device as described above are implemented.

[0008] The present application provides a display method, apparatus, device, and storage medium for a near-eye display device. A first display module and a second display module are provided on the near-eye display device, and the installation positions of the first display module and the second display module are different; the display method includes: obtaining to-be-displayed data of the near-eye display device; rendering the to-be-displayed data to obtain a to-be-displayed image corresponding to the to-be-displayed data; displaying the to-be-displayed image on the first display module according to the display parameters of the first display module; and displaying the to-be-displayed image on the second display module according to the display parameters of the second display module.

[0009] When a near-eye display device obtains data to be displayed, it can render the data to be displayed to obtain a to-be-displayed image corresponding to the data to be displayed. The to-be-displayed image can be used as the common display content of the first display module and the second display module of the near-eye display device. Then, the near-eye display device does not need to repeatedly render the same data to be displayed, so as to save the rendering process of the same data to be displayed by the near-eye display device, and further improve the rendering rate of the near-eye display device for the data to be displayed. Correspondingly, the near-eye display device can display the to-be-displayed image according to the respective display parameters of the first display module and the second display module. Since the respective display parameters of the first display module and the second display module can be different, when the near-eye display device collaboratively displays the same to-be-displayed image through the first display module and the second display module, the presentation effects of the to-be-displayed images respectively displayed by the first display module and the second display module on the user can be different, which is equivalent to the near-eye display device having a multi-screen different display function. When the to-be-displayed images respectively displayed by the first display module and the second display module are different, the to-be-displayed images respectively displayed by the first display module and the second display module can be adapted to the viewing requirements of the user's left eye and right eye for the to-be-displayed image, which is beneficial to improving the display effect of the near-eye display device for the to-be-displayed image. Based on the improvement of the rendering rate of the near-eye display device for the data to be displayed and the improvement of the display effect convenience of the near-eye display device for the to-be-displayed image, it is beneficial to improve the display efficiency of the near-eye display device for the to-be-displayed image. Description of the Drawings

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 It is a schematic flowchart of a display method of a near-eye display device provided by an embodiment of the present application; Figure 2 It is a schematic block diagram of a display device of a near-eye display device provided by an embodiment of the present application; Figure 3 It is a schematic block diagram of a near-eye display device provided by an embodiment of the present application. Detailed Embodiments

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0013] The flowcharts shown in the accompanying drawings are only illustrative, and do not necessarily include all the contents and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can be decomposed, combined, or partially merged, so the actual execution order may change according to the actual situation.

[0014] The embodiments of the present application provide a display method, device, equipment, and storage medium for a near-eye display device. The display method of the near-eye display device can be applied to the near-eye display device. The near-eye display device can include augmented reality (AR) glasses, mixed reality (MR) glasses, AR helmets, MR helmets, etc., which are not limited herein. The display method of the near-eye display device can also be applied to a server, which can be a single server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0015] The following will describe in detail some embodiments of the present application with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0016] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a display method for a near-eye display device provided by an embodiment of the present application. It should be noted that the display method for the near-eye display device provided by the embodiments of the present application can be used in the near-eye display device or in the server, which is not limited herein.

[0017] Exemplarily, a first display module and a second display module are provided on the near-eye display device, and the installation positions of the first display module and the second display module are different.

[0018] For example, the first display module can be disposed on the left side of the second display device so that the setting position of the first display module corresponds to the left eye of the user wearing the near-eye display device, and the setting position of the second display module corresponds to the right eye of the user. Of course, it is not limited thereto. The first display module can be disposed on the right side of the second display device so that the setting position of the first display module corresponds to the right eye of the user, and the setting position of the second display module corresponds to the left eye of the user. This is not restricted herein.

[0019] As Figure 1 shown, the display method of the near-eye display device includes steps S101 to S104.

[0020] S101. Obtain the data to be displayed of the near-eye display device.

[0021] Exemplarily, the data to be displayed of the near-eye display device may include navigation data to be displayed, user data to be displayed, etc. The data to be displayed is used to indicate the data for determining the display content of the near-eye display device.

[0022] The navigation data to be displayed may include road data corresponding to the near-eye display device, path planning data, traffic data, point of information (POI) data, weather data, etc., which are not restricted herein. The road data may include data such as road slope and road curvature that can be used to determine the three-dimensional information of the road, which is not restricted herein.

[0023] The user data to be displayed may include the physiological index data of the user, the interaction operation data of the user with the near-eye display device, etc., which are not restricted herein. The physiological index data may include heart rate, blood oxygen saturation, exercise data, etc., which are not restricted herein. The interaction operation data may include interaction operation type, interaction time data, etc., which are not restricted herein.

[0024] In some embodiments, the near-eye display device can obtain the data to be displayed of the near-eye display device through the application layer.

[0025] Taking the data to be displayed of the near-eye display device including the navigation data to be displayed as an example. The near-eye display device can obtain the navigation data to be displayed of the near-eye display device through a preset navigation data acquisition tool. The navigation data acquisition tool may include a software development kit (SDK) related to navigation data acquisition.

[0026] For example, the near-eye display device may have a positioning function. The near-eye display device can obtain the positioning information of the near-eye display device based on the positioning function. Correspondingly, when the near-eye display device obtains the navigation data to be displayed, it can combine the positioning information of the near-eye display device to obtain the navigation data to be displayed.

[0027] For example, in the case where the user needs to move from the current position of the near-eye display device to the destination, the near-eye display device can determine corresponding path planning data, traffic data, POI data, etc. based on its own positioning information and the positioning information of the destination, so as to determine the navigation data to be displayed by the near-eye display device. Of course, it is not limited to this, and no restrictions are made here.

[0028] In the case of obtaining the data to be displayed by the near-eye display device, the data to be displayed can be used to determine the display content of the near-eye display device.

[0029] S102. Render the data to be displayed to obtain a display image corresponding to the data to be displayed.

[0030] For example, in the case of obtaining the data to be displayed by the near-eye display device, the near-eye display device can render the data to be rendered to obtain a display image corresponding to the data to be displayed. The display image can be used as the display content of the near-eye display device for the near-eye display device to subsequently display the display image.

[0031] In some embodiments, when the near-eye display device obtains the data to be displayed through the application layer, the near-eye display device can transmit the data to be displayed to the synthesis layer of the near-eye display device to render the data to be displayed through the synthesis layer and obtain a display image corresponding to the data to be displayed. The synthesis layer of the near-eye display device can include a synthesizer, and no restrictions are made here.

[0032] Taking the data to be displayed including the navigation data to be displayed as an example. When the near-eye display device obtains the navigation data to be displayed, it can render the navigation data to be displayed to obtain a display image corresponding to the navigation data to be displayed. For example, when the navigation data to be displayed includes road data, the display image can include the road name corresponding to the road data, the buildings passed by the road, etc., and no restrictions are made here. When the navigation data to be displayed includes path planning data, the display image can include the navigation route corresponding to the path planning data, the length of the navigation route, etc., and no restrictions are made here. When the navigation data to be displayed includes traffic data, the display image can include traffic congestion conditions, traffic road recommendation information, etc., and no restrictions are made here. When the navigation data to be displayed includes POI data, the display image can include POI identifiers, such as the starting point identifier of the navigation route, the ending point identifier of the navigation route, navigation arrows, road condition labels, etc., and no restrictions are made here. When the navigation data to be displayed includes weather data, the display image can include weather special effects, weather information, etc., and no restrictions are made here.

[0033] When rendering the data to be displayed to obtain the image to be displayed corresponding to the data to be displayed, it is beneficial to improve the convenience of determining the image to be displayed by the near-eye display device. The image to be displayed can be used as the display content of the first display module or the second display module. Based on this, the near-eye display device can determine the common display content of the first display module and the second display module without repeatedly rendering the same data to be displayed, thus saving the rendering process of the near-eye display device for the same data to be displayed and further improving the rendering rate of the near-eye display device for the data to be displayed.

[0034] S103. Display the image to be displayed on the first display module according to the display parameters of the first display module.

[0035] For example, the near-eye display device can obtain the display parameters of the first display module. According to the first display parameters, display the image to be displayed on the first display module.

[0036] Taking the near-eye display device including an AR glasses and the first display module of the near-eye display device including the first lens of the AR glasses as an example. The AR glasses can display the image to be displayed on the first lens according to the display parameters of the first lens. If the setting position of the first display module on the near-eye display device corresponds to the user's left eye, for example, when the user views the environment where the near-eye display device is located through the AR glasses, the user can view the image to be displayed displayed by the first display module through the left eye.

[0037] S104. Display the image to be displayed on the second display module according to the display parameters of the second display module.

[0038] For example, the near-eye display device can obtain the display parameters of the second display module. According to the second display parameters, display the image to be displayed on the second display module.

[0039] Taking the near-eye display device including an AR glasses and the second display module of the near-eye display device including the second lens of the AR glasses as an example. The AR glasses can display the image to be displayed on the second lens according to the display parameters of the second lens. If the setting position of the second display module on the near-eye display device corresponds to the user's right eye, for example, when the user views the environment where the near-eye display device is located through the AR glasses, the user can view the image to be displayed displayed by the second display module through the right eye.

[0040] In some embodiments, during the process of determining the image to be displayed, the near-eye display device may adopt an asynchronous submission mechanism, and display the image to be displayed on the first display module and the second display module respectively according to the display parameters of the first display module and the second display module. For example, when determining the image to be displayed corresponding to the data to be displayed, the image to be displayed can be displayed on the first display module according to the display parameters of the first display module. Another example is that when determining the image to be displayed corresponding to the data to be displayed, the image to be displayed can be displayed on the second display module according to the display parameters of the second display module.

[0041] The display parameters of the first display module and the display parameters of the second display module can be pre-acquired by the near-eye display device. For example, the near-eye display device can acquire the display parameters of the first display module and the second display module respectively during the process of rendering the data to be displayed to obtain the image to be displayed corresponding to the data to be displayed. Based on this, when the near-eye display device displays the image to be displayed on the first display module according to the display parameters of the first display module and displays the image to be displayed on the second display module according to the display parameters of the second display module, the time difference between the display times of the image to be displayed by the first display module and the second display module can be reduced. Furthermore, the possibility of the user experiencing dizziness when viewing the image to be displayed through the first display module and the second display module respectively due to a large time difference between the display times of the first display module and the second display module can be reduced, thereby improving the user's viewing experience of the image to be displayed.

[0042] The display parameters of the first display module and the second display module can be different. For example, the display parameters of the first display module and the second display module can be determined according to the distance between the centers of the user's binocular pupils (Interpupillary Distance, IPD). The distance between the centers of the binocular pupils can be simply referred to as the pupillary distance. When the installation positions of the first display module and the second display module on the near-eye display device are different, the directions of the pupillary distances corresponding to the first display module and the second display module are different. Under the action of the pupillary distances in different directions, the to-be-displayed images respectively displayed by the first display module and the second display module can also be different. For example, the first display position of the to-be-displayed image on the first display module is different from the second display position of the to-be-displayed image on the second display module. Based on this, the near-eye display device can have the function of multi-screen different display. When the user views the display content of the near-eye display device, the user's left eye and right eye can respectively view the to-be-displayed images at different display positions through the first display module and the second display module, so as to adapt to the viewing needs of the user's left eye and right eye for the to-be-displayed images respectively. The to-be-displayed images at different display positions can present a three-dimensional effect, which is beneficial to improving the display effect of the near-eye display device on the to-be-displayed images. For example, the near-eye display device can present the slope of the road, the height of the bridge, etc. to the user through the to-be-displayed images respectively displayed by the first display module and the second display module, so that the user can intuitively feel the slope of the road, the height of the bridge, etc., and further improve the immersive experience of the user when viewing the to-be-displayed images.

[0043] In some embodiments, texture rendering processing is performed on the to-be-displayed data to obtain texture data corresponding to the to-be-displayed data; synthesis processing is performed on the texture data corresponding to multiple to-be-displayed data to obtain to-be-displayed images corresponding to the multiple to-be-displayed data.

[0044] For example, when the near-eye display device obtains the to-be-displayed data, texture rendering processing can be performed on the to-be-displayed data to obtain texture data corresponding to the to-be-displayed data. The texture data can be shared by the first display module and the second display module, and the texture data can also be referred to as shared texture data, for example.

[0045] Exemplarily, the near-eye display device can perform texture rendering processing on the to-be-displayed data through the application layer to obtain texture data corresponding to the to-be-displayed data, and transmit the texture data corresponding to the to-be-displayed data to the synthesis layer of the near-eye display device through the application layer. When the synthesis layer receives the texture data corresponding to multiple to-be-displayed data, synthesis processing can be performed on the multiple texture data to obtain to-be-displayed images corresponding to the multiple to-be-displayed data.

[0046] For example, during the process of the application layer of a near-eye display device transmitting the texture data corresponding to the data to be displayed to the composition layer of the near-eye display device, the application layer can create a texture manager, such as SurfaceTexture. Correspondingly, the application layer can set the texture data corresponding to the data to be displayed to be associated with SurfaceTexture, so that SurfaceTexture can manage the texture data, such as performing update callbacks on the texture data, which is not limited here. Another example is that the near-eye display device can use the composition layer to receive the texture data corresponding to the data to be displayed. The composition layer includes, for example, a texture synthesizer, such as SurfaceFlinger. When SurfaceFlinger receives the texture data, it can perform composition processing on multiple texture data according to the corresponding composition rules to obtain the images to be displayed corresponding to the multiple data to be displayed. Of course, it is not limited to this, and it is not restricted here.

[0047] Taking the image to be displayed including a navigation route as an example. The near-eye display device can obtain the data to be displayed corresponding to the navigation route. The data to be displayed corresponding to the navigation route can be determined based on the positioning information of the near-eye display device, the positioning information of the destination corresponding to the near-eye display device, and the preset navigation map of the near-eye display device. The data to be displayed corresponding to the navigation route can include data such as the geometry of the navigation route and the texture of the navigation route. The near-eye display device can perform texture rendering on the data to be displayed corresponding to the navigation route to obtain the texture data corresponding to the data to be displayed. For example, the near-eye display device can map the texture of the navigation route, such as the color, brightness, or other surface attribute data of the navigation route, into the geometry of the navigation route to bind the texture of the navigation route to the geometry of the navigation route and form the texture data of the navigation route.

[0048] Taking the image to be displayed including a navigation arrow for guiding the navigation route as an example. The near-eye display device can obtain the data to be displayed corresponding to the navigation arrow. The data to be displayed corresponding to the navigation arrow can include data such as the geometry of the navigation arrow and the texture of the navigation arrow. The near-eye display device can perform texture rendering on the data to be displayed corresponding to the navigation arrow to obtain the texture data corresponding to the data to be displayed. For example, the near-eye display device can map the texture of the navigation arrow, such as the color, brightness, or other surface attribute data of the navigation arrow, into the geometry of the navigation arrow to bind the texture of the navigation arrow to the geometry of the navigation arrow and form the texture data of the navigation arrow.

[0049] Of course, the data to be displayed and the texture data corresponding to the data to be displayed are not limited to this, and it is not restricted here.

[0050] For example, in the case where the near-eye display device determines the texture data corresponding to the data to be displayed, it can perform a synthesis process on the texture data corresponding to each of the multiple data to be displayed to obtain a to-be-displayed image corresponding to the multiple data to be displayed.

[0051] Taking the example that the to-be-displayed image can present navigation information to the user. The navigation information includes, for example, at least one of a navigation route and a navigation arrow guiding the navigation route. In the case where the near-eye display device determines the texture data of the navigation route and the texture data of the navigation arrow, the near-eye display device can jointly synthesize a to-be-displayed image based on the texture data of the navigation route and the texture data of the navigation arrow. For example, the near-eye display device can perform a synthesis process on the texture data of the navigation route and the texture data of the navigation arrow to obtain the corresponding to-be-displayed image. The to-be-displayed image can present both the navigation route and the navigation arrow to the user. Based on this, when the near-eye display device subsequently displays the to-be-displayed image, when the user views the to-be-displayed image through the near-eye display device, the user can act according to the guidance of the navigation route and the navigation arrow in the to-be-displayed image.

[0052] Of course, the to-be-displayed image is not limited to this, and there is no limitation here.

[0053] In the case where texture rendering processing is performed on the data to be displayed to obtain the texture data corresponding to the data to be displayed, the near-eye display device can perform a synthesis process on the texture data corresponding to each of the multiple data to be displayed to obtain a to-be-displayed image corresponding to the multiple data to be displayed, which is equivalent to that multiple data to be displayed can be used to determine the same to-be-displayed image. The to-be-displayed image can be displayed by the first display module and the second display module of the near-eye display device. Based on this, the near-eye display device does not need to perform repeated rendering on the same data to be displayed, and can determine the common display content of the first display module and the second display module, which is beneficial to improving the rendering rate of the near-eye display device for the data to be displayed, and further improving the convenience and speed of determining the to-be-displayed image.

[0054] In some embodiments, a synthesis process is performed on the layers corresponding to each of the multiple texture data to obtain a to-be-displayed image corresponding to the multiple data to be displayed.

[0055] For example, when the near-eye display device determines the to-be-displayed image corresponding to multiple data to be displayed, it can perform a synthesis process on the layers corresponding to each of the multiple texture data.

[0056] When the near-eye display device performs a synthesis process on the layers corresponding to each of the multiple texture data, it can perform a synthesis process on the vertices of each of the multiple layers to simplify the synthesis process of the multiple texture data.

[0057] In an exemplary embodiment, when the near-eye display device renders through the application layer to obtain texture data corresponding to certain data to be displayed, and the texture data includes 100,000 vertices, when the composition layer performs composition processing on the layer corresponding to the texture data, it only needs to perform composition processing on the vertices of the layer corresponding to the texture data to generate the corresponding image to be displayed. For example, when the layer corresponding to the texture data is a quadrilateral, the near-eye display device only needs to perform composition processing on the four vertices of the layer corresponding to the texture data through the composition layer. And so on.

[0058] Based on the simplification of the composition process of multiple texture data, it is beneficial to improve the composition rate of the near-eye display device for multiple texture data, reduce the rendering duration required to render the data to be displayed, and further improve the convenience and rate of determining the image to be displayed by the near-eye display device. When the convenience and rate of determining the image to be displayed are improved, it can effectively avoid performance bottlenecks due to the limitation of the computing resources of the near-eye display device during the process of displaying the image to be displayed, so that the near-eye display device can have sufficient computing resources to perform display rendering on the image to be displayed, and then display the image to be displayed on the first display module and the second display module respectively. Correspondingly, when the near-eye display device has sufficient computing resources to perform display rendering on the image to be displayed, it is possible to reduce the display rendering delay when the near-eye display device displays the image to be displayed on the first display module and the second display module respectively, which is beneficial to reducing the possibility of the user experiencing dizziness when viewing the image to be displayed through the first display module and the second display module respectively, thus improving the user's viewing experience of the image to be displayed.

[0059] In some embodiments, the display parameters of the first display module and the display parameters of the second display module are determined according to one or more of the pupil distance corresponding to the near-eye display device, the focal length of the near-eye display device, and the display size corresponding to the near-eye display device.

[0060] The near-eye display device can obtain one or more of the pupil distance corresponding to the near-eye display device, the focal length of the near-eye display device, and the display size corresponding to the near-eye display device, so as to determine the display parameters of the first display module and the display parameters of the second display module according to one or more of the pupil distance, the focal length, and the display size.

[0061] For example, the interpupillary distance corresponding to the near-eye display device can be changed. For example, in the case where the user wearing the near-eye display device changes, since the interpupillary distances of different users can be different, the interpupillary distance corresponding to the near-eye display device can be changed in response to the change of the user wearing the near-eye display device. For example, in the process of obtaining the interpupillary distance corresponding to the near-eye display device, the near-eye display device can capture the eyes of the user wearing the near-eye display device to obtain a corresponding eye image; based on the eye image of the user, determine the interpupillary distance of the user to determine the interpupillary distance corresponding to the near-eye display device. Of course, this is not limited to this. The near-eye display device can also store the interpupillary distances of different users respectively, so as to obtain the interpupillary distance of a certain user when detecting that the user wears the near-eye display device, so as to determine the interpupillary distance corresponding to the near-eye display device, which is not limited here. When the interpupillary distance corresponding to the near-eye display device can be changed, and the display parameters of the first display module and the second display module are determined according to the interpupillary distance corresponding to the near-eye display device, the near-eye display device can dynamically change the display parameters of the first display module and the second display module in response to the change of the interpupillary distance corresponding to the near-eye display device, so that the to-be-displayed image displayed by the near-eye display device through the first display module and the second display module can be adapted to the viewing needs of different users, thereby improving the viewing experience of different users for the to-be-displayed image.

[0062] The interpupillary distance corresponding to the near-eye display device can also be fixed. For example, the interpupillary distance corresponding to the near-eye display device can be fixed to one of 60mm, 65mm, 67mm, 70mm, etc., which is not limited here. When the interpupillary distance corresponding to the near-eye display device is fixed, and the display parameters of the first display module and the second display module are determined according to the interpupillary distance corresponding to the near-eye display device, the near-eye display device can directly obtain the interpupillary distance corresponding to the near-eye display device without dynamically changing the interpupillary distance corresponding to the near-eye display device, which is beneficial to improving the display rate of the near-eye display device for the to-be-displayed image.

[0063] For example, the focal length of the near-eye display device can be preset. For example, before leaving the factory, the near-eye display device can calibrate the camera parameters of the near-eye display device, and then determine the focal length of the near-eye display device. The near-eye display device can directly obtain the focal length of the near-eye display device.

[0064] For example, the near-eye display device can determine the display size of the near-eye display device according to the first display size of the first display module and the second display size of the second display module. Then the near-eye display device can obtain the first display size and the second display size to determine the display size of the near-eye display device.

[0065] When one or more of the pupil distance corresponding to the near-eye display device, the focal length of the near-eye display device, and the display size corresponding to the near-eye display device are obtained, the near-eye display device can, for example, determine the display parameters of the first display module and the display parameters of the second display module according to the pupil distance corresponding to the near-eye display device and the focal length of the near-eye display device, such as determining at least two of the first view matrix corresponding to the first display module, the second view matrix corresponding to the second display module, and the projection matrix corresponding to the near-eye display device, so as to determine the display parameters of the first display module and the display parameters of the second display module.

[0066] When determining the display parameters of the first display module and the display parameters of the second display module according to one or more of the pupil distance corresponding to the near-eye display device, the focal length of the near-eye display device, and the display size corresponding to the near-eye display device, the display parameters of the first display module can be used by the near-eye display device to subsequently display the image to be displayed on the first display module, and the display parameters of the second display module can be used by the near-eye display device to subsequently display the image to be displayed on the second display module, which is beneficial to improving the display convenience of the near-eye display device for the image to be displayed subsequently.

[0067] In some embodiments, the projection matrix corresponding to the near-eye display device is determined according to the focal length of the near-eye display device and the display size corresponding to the near-eye display device; the first view matrix corresponding to the first display module and the second view matrix corresponding to the second display module are determined according to the pupil distance of the near-eye display device and the focal length of the near-eye display device; the display parameters of the first display module are determined according to the projection matrix and the first view matrix; and the display parameters of the second display module are determined according to the projection matrix and the second view matrix.

[0068] For example, the near-eye display device can determine the projection matrix corresponding to the near-eye display device according to the focal length of the near-eye display device and the display size corresponding to the near-eye display device. The projection matrix corresponding to the near-eye display device can include a perspective matrix.

[0069] Exemplarily, when the near-eye display device displays the image to be displayed on the first display module and the second display module respectively according to the perspective matrix, the user can perceive the change in the size of the image to be displayed, with the object being larger when closer and smaller when farther.

[0070] Taking the image to be displayed including a navigation arrow as an example. When the navigation arrow can be used to guide the navigation route corresponding to the positioning information from the positioning information where the near-eye display device is located to the positioning information of the destination, if the user gradually approaches the destination, the size of the navigation arrow displayed on the first display module and the second display module can gradually increase. Correspondingly, if the user gradually moves away from the destination, the size of the navigation arrow displayed on the first display module and the second display module can gradually decrease.

[0071] Of course, the projection matrix corresponding to the near-eye display device is not limited to the perspective matrix, and no limitation is imposed here.

[0072] For example, the near-eye display device can determine the first view matrix corresponding to the first display module and the second view matrix corresponding to the second display module according to the pupil distance corresponding to the near-eye display device and the focal length corresponding to the near-eye display device.

[0073] Taking the setting position of the first display module corresponding to the left eye of the user and the setting position of the second display module corresponding to the right eye of the user in the near-eye display device as an example. When the near-eye display device obtains the pupil distance corresponding to the near-eye display device, the near-eye display device can determine the size and direction of the pupil distance included in the first view matrix and the second view matrix respectively according to the corresponding relationship between the first display module and the second display module and the user's eyes. For example, the pupil distance in the first view matrix can be expressed as -ipd / 2, and the pupil distance in the second view matrix can be expressed as ipd / 2, indicating that the sizes of the pupil distances included in the first view matrix and the second view matrix are the same and the directions are opposite. The difference between the first view matrix and the second view matrix is determined according to the pupil distance ipd corresponding to the near-eye display device. At this time, the first view matrix can also be called the left view matrix, and the second view matrix can also be called the right view matrix, and no limitation is imposed here.

[0074] When the near-eye display device determines the projection matrix corresponding to the near-eye display device, the first view matrix corresponding to the first display module, and the second view matrix corresponding to the second display module, the near-eye display device can determine the display parameters of the first display module according to the projection matrix and the first view matrix, and determine the display parameters of the second display module according to the projection matrix and the second view matrix, which is beneficial to improving the convenience of determining the display parameters of the first display module and the second display module. The display parameters of the first display module can be used by the near-eye display device to display the image to be displayed on the first display module subsequently, and the display parameters of the second display module can be used by the near-eye display device to display the image to be displayed on the second display module subsequently, which is beneficial to improving the display convenience of the near-eye display device for the image to be displayed subsequently.

[0075] In some embodiments, according to the display parameters of the first display module, determine the first display positions of the respective pixel points in the image to be displayed on the first display module; according to the first display positions, perform display rendering on the pixel points of the image to be displayed, so as to display the image to be displayed on the first display module.

[0076] For example, the near-eye display device can perform position conversion processing on the pixel positions of each pixel in the to-be-displayed image according to the display parameters of the first display module, so as to obtain the first display positions of each pixel on the first display module. When the near-eye display device determines the first display positions of the pixel positions of each pixel in the to-be-displayed image, it can perform display rendering on each pixel of the to-be-displayed image according to the first display positions, and then display the to-be-displayed image on the first display module.

[0077] When determining the first display positions of each pixel in the to-be-displayed image according to the display parameters of the first display module, since the display parameters of the first display module can change according to the change of the pupil distance corresponding to the near-eye display device, when the user wearing the near-eye display device changes, the near-eye display device can still determine the display parameters of the first display module according to the changed pupil distance, and then determine the first display positions of each pixel. Based on this, when the near-eye display device performs display rendering on each pixel of the to-be-displayed image according to the first display positions to display the to-be-displayed image on the first display module, it is beneficial to reduce the display error of the first display module for the to-be-displayed image.

[0078] In some embodiments, the second display positions of each pixel in the to-be-displayed image on the second display module are determined according to the display parameters of the second display module; display rendering is performed on each pixel of the to-be-displayed image according to the second display positions, so as to display the to-be-displayed image on the second display module.

[0079] For example, the near-eye display device can perform position conversion processing on the pixel positions of each pixel in the to-be-displayed image according to the display parameters of the second display module, so as to obtain the second display positions of each pixel on the second display module. When the near-eye display device determines the second display positions of the pixel positions of each pixel in the to-be-displayed image, it can perform display rendering on each pixel of the to-be-displayed image according to the second display positions, and then display the to-be-displayed image on the first display module.

[0080] When determining the second display positions of each pixel in the to-be-displayed image according to the display parameters of the second display module, since the display parameters of the second display module can change according to the change of the pupil distance corresponding to the near-eye display device, when the user wearing the near-eye display device changes, the near-eye display device can still determine the display parameters of the second display module according to the changed pupil distance, and then determine the second display positions of each pixel. Based on this, when the near-eye display device performs display rendering on each pixel of the to-be-displayed image according to the second display positions to display the to-be-displayed image on the second display module, it is beneficial to reduce the display error of the second display module for the to-be-displayed image.

[0081] When determining the image to be displayed, a near-eye display device can adopt an asynchronous submission mechanism to display the image to be displayed on a first display module and a second display module respectively. For example, the near-eye display device can determine the first display position of each pixel in the image to be displayed on the first display module according to the display parameters of the first display module. When the near-eye display device determines the first display position of each pixel, it can submit each first display position as display rendering content for the near-eye display device to display the image to be displayed on the first display module. Another example is that the near-eye display device can determine the second display position of each pixel in the image to be displayed on the second display module according to the display parameters of the second display module. When the near-eye display device determines the second display position of each pixel, it can submit each second display position as display rendering content for the near-eye display device to display the image to be displayed on the second display module.

[0082] Based on this, when the near-eye display device performs display rendering on each pixel of the image to be displayed according to the first display position of each pixel, it does not need to wait for the near-eye display device to determine the second display position of each pixel. Correspondingly, when the near-eye display device performs display rendering on each pixel of the image to be displayed according to the second display position of each pixel, it does not need to wait for the near-eye display device to determine the first display position of each pixel, which is beneficial to improving the display convenience of the near-eye display device for the image to be displayed.

[0083] In some embodiments, obtain the pose change amount of the near-eye display device after displaying the current frame of the image to be displayed and before displaying the next frame of the image to be displayed; update and display the current frame of the image to be displayed on the first display module and the second display module according to the pose change amount.

[0084] Exemplarily, the near-eye display device can have a device pose detection function. For example, sensors such as a gyroscope, an accelerometer, etc. can be provided on the near-eye display device for the near-eye display device to detect its own device pose, and then determine the pose change amount of the near-eye display device.

[0085] During the process of a near-eye display device displaying the images to be displayed through a first display module and a second display module respectively, the device pose of the near-eye display device can change in response to the user's head movement. After the near-eye display device displays the current frame of the image to be displayed and before displaying the next frame of the image to be displayed, when the near-eye display device detects a change in the device pose, since the near-eye display device has not completed the display rendering of the next frame of the image to be displayed, the near-eye display device cannot display the next frame of the image to be displayed. However, if the near-eye display device maintains the display of the current frame of the image to be displayed, it is likely that the current frame of the image to be displayed shown by the near-eye display device does not match the user's viewing requirements for the image to be displayed. Based on this, the near-eye display device can adopt the asynchronous time warping technology and use the amount of change in the device pose of the near-eye display device to update the display of the current frame of the image to be displayed by the near-eye display device.

[0086] For example, the near-eye display device can update the display of the current frame of the image to be displayed on the first display module and the second display module in response to the change in the device pose.

[0087] The near-eye display device can obtain the amount of change in the device pose after the near-eye display device displays the current frame of the image to be displayed and before displaying the next frame of the image to be displayed. The amount of change in the device pose can be used to indicate the corresponding change in the device pose of the near-eye display device. Accordingly, the near-eye display device can update the display of the current frame of the image to be displayed on the first display module and the second display module once according to each amount of change in the device pose after the near-eye display device displays the current frame of the image to be displayed and before displaying the next frame of the image to be displayed.

[0088] For example, according to the amount of change in the device pose, perform a position transformation process on the first display positions of each pixel point in the current frame of the image to be displayed on the first display module to obtain the processed first display positions; update the display of the current frame of the image to be displayed according to the processed first display positions. Accordingly, according to the amount of change in the device pose, perform a position transformation process on the second display positions of each pixel point in the current frame of the image to be displayed on the second display module to obtain the processed second display positions; update the display of the current frame of the image to be displayed according to the processed second display positions.

[0089] Taking the pose change amount of the near-eye display device from after displaying the current frame of the image to be displayed to before displaying the next frame of the image to be displayed, which includes pose change amount 1 and pose change amount 2, and the acquisition time corresponding to pose change amount 1 being earlier than the acquisition time corresponding to pose change amount 2 as an example. The near-eye display device can perform geometric transformation on the current frame of the image to be displayed that has been completed in display rendering according to pose change amount 1 and pose change amount 2. For example, when pose change amount 1 indicates that the device pose of the near-eye display device has a translation, the first display position and the second display position of each pixel point included in the current frame of the image to be displayed can be processed by a translation transformation according to pose change amount 1 to obtain the translated first display position and the second display position. Correspondingly, the near-eye display device can update and display the current frame of the image to be displayed on the first display module according to the translated first display position. The near-eye display device can update and display the current frame of the image to be displayed on the second display module according to the translated second display position. When the near-eye display device updates and displays the current frame of the image to be displayed, the updated and displayed current frame of the image to be displayed can be determined as the current frame of the image to be displayed. Also for example, when pose change amount 2 indicates that the device pose of the near-eye display device has a rotation, the first display position and the second display position of each pixel point included in the current frame of the image to be displayed can be processed by a rotation transformation according to pose change amount 2 to obtain the rotated first display position and the second display position. Correspondingly, the near-eye display device can update and display the current frame of the image to be displayed on the first display module according to the rotated first display position. The near-eye display device can update and display the current frame of the image to be displayed on the second display module according to the rotated second display position. And so on.

[0090] When acquiring the pose change amount of the near-eye display device from after displaying the current frame of the image to be displayed to before displaying the next frame of the image to be displayed, and updating and displaying the current frame of the image to be displayed on the first display module and the second display module according to the pose change amount, the near-eye display device can update and display the currently rendered and displayed current frame of the image to be displayed before displaying the next frame of the image to be displayed, so that the updated and displayed current frame of the image to be displayed can be closer to the user's perspective. When the updated and displayed current frame of the image to be displayed can be closer to the user's perspective, during the process of the device pose of the near-eye display device changing, the dizziness feeling generated by the user when viewing the image to be displayed due to the rendering delay of the image to be displayed can be reduced, which is beneficial to improving the display effect of the near-eye display device on the image to be displayed and enhancing the user experience of the near-eye display device.

[0091] In an exemplary embodiment, the rendering process of the near-eye display device for the data to be displayed and the display rendering process for the image to be displayed include: (1)Obtain the data to be displayed by the near-eye display device and determine the image to be displayed corresponding to the data to be displayed.

[0092] For example, the near-eye display device can obtain the navigation data to be displayed through a navigation data acquisition tool, such as road and traffic data. The road data can include three-dimensional information such as road slope and road curvature. The near-eye display device can determine the navigation data to be displayed as the data to be displayed.

[0093] The near-eye display device can render the data to be displayed to obtain the image to be displayed corresponding to the data to be displayed. For example, the near-eye display device can use a composition layer to perform a composition process on SurfaceTexture objects generated by elements such as navigation arrows and road condition labels to obtain the corresponding image to be displayed.

[0094] For example, the rendering architecture of the near-eye display device for the data to be displayed includes an application layer, a composition layer, SurfaceTexture, and SurfaceFlinger. When the near-eye display device renders the image to be displayed, the data to be displayed by the near-eye display device can be processed by the application layer, the composition layer, SurfaceTexture, and SurfaceFlinger in sequence to obtain the image to be rendered. The image to be rendered can be displayed by the first display module and the second display module of the near-eye display device.

[0095] (2)Calculate the dynamic MVP matrix of the near-eye display device.

[0096] The MVP matrix can be used to indicate the model matrix, view matrix, and projection matrix of the near-eye display device. Since one or more of the model matrix, view matrix, and projection matrix of the near-eye display device can change, the MVP matrix of the near-eye display device can also be called the dynamic MVP matrix.

[0097] The view matrix can include a first view matrix and a second view matrix. The code logic involved in determining the view matrix by the near-eye display device can be expressed, for example, as: float ipd = Settings.System.getFloat(context.getContentResolver(), "ipd_value", 0.065f); / / Interpupillary distance default 65mm Matrix.setLookAtM(V_left, 0, -ipd / 2, 0, 1.5f, 0, 0, 0, 0, 1, 0); / / Left-eye coordinate system Matrix.setLookAtM(V_right, 0, ipd / 2, 0, 1.5f, 0, 0, 0, 0, 1, 0); / / Right-eye coordinate system The near-eye display device can determine the view matrix in the left-eye coordinate system as one of the first view matrix and the second view matrix, and determine the view matrix in the right-eye coordinate system as the other of the first view matrix and the second view matrix. The coordinate system corresponding to the first view matrix is adapted to the set position of the first display module on the near-eye display device, and the coordinate system corresponding to the second view matrix is adapted to the set position of the second display module on the near-eye display device.

[0098] The code logic involved in the near-eye display device determining the projection matrix can be expressed, for example, as: Matrix.perspectiveM(P, 0, 60.0f, (float)width / height, 0.1f, 1000.0f); / / Perspective projection When calculating the dynamic MVP matrix of the near-eye display device, the dynamic MVP matrix can be used to determine the display parameters of the first display module and the second display module of the near-eye display device.

[0099] For example, during the process of the near-eye display device calculating the dynamic MVP matrix, it can obtain the interpupillary distance corresponding to the near-eye display device; determine the first view matrix and the second view matrix based on the interpupillary distance corresponding to the near-eye display device; determine the projection matrix corresponding to the near-eye display device; bind the first view matrix and the projection matrix to the first display module, and bind the second view matrix and the projection matrix to the second display module. Of course, it is not limited to this, and there is no restriction here.

[0100] (3) Display the image to be displayed on the first display module according to the display parameters of the first display module, and display the image to be displayed on the second display module according to the display parameters of the second display module.

[0101] Exemplarily, during the process of the near-eye display device determining the image to be displayed corresponding to the data to be displayed, it can create a layer corresponding to the texture data of the data to be displayed. The code logic involved in the near-eye display device creating a layer corresponding to the texture data can be expressed, for example, as: SurfaceControl.Builder builder = new SurfaceControl.Builder(display); builder.setLayerStack(layerStack); SurfaceControl leftSurface = builder.build(); SurfaceControl rightSurface = builder.build(); Accordingly, the near-eye display device can bind the texture data corresponding to the data to be displayed with the MVP matrix of the near-eye display device, so that the first display module and the second display module can share the texture data, and further share the data to be displayed. Accordingly, it can be ensured that the first display module and the second display module can display and render the same image to be displayed.

[0102] The code logic involved in binding the same texture data and different MVP matrices by the first display module and the second display module of the near-eye display device can be expressed as follows: leftSurface.getSurface().setTexture(navTexture); leftSurface.setMatrix(V_left, P); rightSurface.getSurface().setTexture(navTexture); rightSurface.setMatrix(V_right, P); The near-eye display device can also control the display and rendering order of the image to be displayed. For example, by using an asynchronous submission mechanism, the near-eye display device can be controlled to display and render the image to be displayed.

[0103] The code logic involved in the first display module and the second display module of the near-eye display device respectively displaying and rendering the image to be displayed can be expressed as follows: leftSurface.show(); rightSurface.show(); For example, when the display parameters of the first display module are determined, the first display positions of each pixel point in the image to be displayed on the first display module can be determined according to the display parameters of the first display module. When the first display positions of each pixel point are determined, each pixel point of the image to be displayed can be displayed and rendered according to the first display positions of each pixel point, and then the image to be displayed can be displayed on the first display module.

[0104] For another example, when the display parameters of the second display module are determined, the second display positions of the respective pixel points in the image to be displayed on the second display module can be determined according to the display parameters of the second display module. When the second display positions of the respective pixel points are determined, the respective pixel points of the image to be displayed can be displayed and rendered according to the second display positions of the respective pixel points, and then the image to be displayed can be displayed on the second display module.

[0105] When the near-eye display device renders the data to be displayed to obtain the image to be displayed, and respectively performs display rendering on the image to be displayed according to the respective display parameters of the first display module and the second display module, so as to display the image to be displayed on the first display module and the second display module respectively, the following can be achieved: ① The computing power of the Graphics Processing Unit (GPU) of the near-eye display device is saved by 50% (it can be reduced to the original value in the case of double load). When the computing power of the GPU is saved, it is beneficial to improve the battery life of the near-eye display device.

[0106] ② The three-dimensional dizziness is reduced by 60%. For example, the time difference between the display times of the image to be displayed by the first display module and the second display module of the near-eye display device can be reduced from 5 ms (which may be larger depending on the scene complexity) to less than 2 ms.

[0107] ③ The rendering latency corresponding to the display rendering of the image to be displayed by the first display module and the second display module respectively is less than 2 ms (the reduction rate exceeds 60%).

[0108] ④ Since the first display module and the second display module can share the data to be displayed and the image to be displayed, the memory occupancy of the near-eye display device is reduced by 50%.

[0109] ⑤ Since the composite layer of the near-eye display device, such as SurfaceFlinger, is closer to the display side, the display error of the near-eye display device can be reduced during the process from rendering the data to be displayed to obtaining the image to be displayed and then to displaying the image to be displayed. Taking the image to be displayed on the near-eye display device including navigation information, and the navigation information including the three-dimensional navigation height as an example, the three-dimensional navigation height error can be reduced from more than 10% to within 5%.

[0110] Of course, the rendering process of the near-eye display device for the data to be displayed and the display rendering process for the image to be displayed are not limited to this, and no restrictions are imposed here.

[0111] The display method of the near-eye display device provided by the above embodiment includes: obtaining the data to be displayed of the near-eye display device; rendering the data to be displayed to obtain a to-be-displayed image corresponding to the data to be displayed; displaying the to-be-displayed image on the first display module according to the display parameters of the first display module; and displaying the to-be-displayed image on the second display module according to the display parameters of the second display module.

[0112] When the near-eye display device obtains the data to be displayed, it can render the data to be displayed to obtain a to-be-displayed image corresponding to the data to be displayed. The to-be-displayed image can be used as the common display content of both the first display module and the second display module of the near-eye display device. Then, the near-eye display device does not need to repeatedly render the same data to be displayed, so as to save the rendering process of the near-eye display device for the same data to be displayed, and further improve the rendering rate of the near-eye display device for the data to be displayed. Correspondingly, the near-eye display device can display the to-be-displayed image according to the respective display parameters of the first display module and the second display module. Since the respective display parameters of the first display module and the second display module can be different, when the near-eye display device cooperatively displays the same to-be-displayed image through the first display module and the second display module, the presentation effects of the to-be-displayed images respectively displayed by the first display module and the second display module on the user can be different, which is equivalent to the near-eye display device having a multi-screen different display function. When the to-be-displayed images respectively displayed by the first display module and the second display module are different, the to-be-displayed images respectively displayed by the first display module and the second display module can be adapted to the viewing requirements of the user's left eye and right eye for the to-be-displayed image, which is beneficial to improving the display effect of the near-eye display device for the to-be-displayed image. Based on the improvement of the rendering rate of the near-eye display device for the data to be displayed and the improvement of the convenience of the display effect of the near-eye display device for the to-be-displayed image, it is beneficial to improve the display efficiency of the near-eye display device for the to-be-displayed image.

[0113] Please refer to Figure 2 , Figure 2 FIG. is a schematic block diagram of a display device of a near-eye display device provided by an embodiment of the present application. The display device of the near-eye display device can be configured in a near-eye display device or a server, and is used to execute the display method of the near-eye display device described above. The near-eye display device can include an AR glasses, an MR glasses, an AR helmet, an MR helmet, etc., which are not limited herein. The server can be a single server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0114] As shown in Figure 2As shown, the display device of the near-eye display device includes a data acquisition module 110, an image determination module 120, a first display module 130, and a second display module 140.

[0115] The data acquisition module 110 is configured to acquire the data to be displayed by the near-eye display device.

[0116] The image determination module 120 is configured to render the data to be displayed to obtain the image to be displayed corresponding to the data to be displayed.

[0117] The first display module 130 is configured to display the image to be displayed on the first display module according to the display parameters of the first display module.

[0118] The second display module 140 is configured to display the image to be displayed on the second display module according to the display parameters of the second display module.

[0119] Exemplarily, the image determination module 120 includes a texture rendering sub-module and an image synthesis sub-module.

[0120] The texture rendering sub-module is configured to perform texture rendering processing on the data to be displayed to obtain the texture data corresponding to the data to be displayed.

[0121] The image synthesis sub-module is configured to perform synthesis processing on the texture data corresponding to each of the multiple data to be displayed to obtain the image to be displayed corresponding to the multiple data to be displayed.

[0122] Exemplarily, the image synthesis sub-module includes a layer synthesis sub-module.

[0123] The layer synthesis sub-module is configured to perform synthesis processing on the layers corresponding to the texture data respectively to obtain the image to be displayed corresponding to the multiple data to be displayed.

[0124] Exemplarily, the display device further includes: a display parameter determination sub-module.

[0125] The display parameter determination sub-module is configured to determine the display parameters of the first display module and the display parameters of the second display module according to one or more of the pupil distance corresponding to the near-eye display device, the focal length of the near-eye display device, and the display size corresponding to the near-eye display device.

[0126] Exemplarily, the display parameter determination sub-module includes a first matrix determination sub-module, a second matrix determination sub-module, a first parameter determination sub-module, and a second parameter determination sub-module.

[0127] A first matrix determination sub-module, configured to determine a projection matrix corresponding to the near-eye display device according to the focal length of the near-eye display device and the display size corresponding to the near-eye display device.

[0128] A second matrix determination sub-module, configured to determine a first view matrix corresponding to the first display module and a second view matrix corresponding to the second display module according to the interpupillary distance of the near-eye display device and the focal length corresponding to the near-eye display device.

[0129] A first parameter determination sub-module, configured to determine display parameters of the first display module according to the projection matrix and the first view matrix.

[0130] A second parameter determination sub-module, configured to determine display parameters of the second display module according to the projection matrix and the second view matrix.

[0131] Exemplarily, the first display module 130 includes a first position determination sub-module and a first display rendering sub-module.

[0132] The first position determination sub-module is configured to determine a first display position of each pixel point in the to-be-displayed image on the first display module according to the display parameters of the first display module.

[0133] The first display rendering sub-module is configured to perform display rendering on each pixel point of the to-be-displayed image according to the first display position, so as to display the to-be-displayed image on the first display module.

[0134] The second display module 140 includes a second position determination sub-module and a second display rendering sub-module.

[0135] The second position determination sub-module is configured to determine a second display position of each pixel point in the to-be-displayed image on the second display module according to the display parameters of the second display module.

[0136] The second display rendering sub-module is configured to perform display rendering on each pixel point of the to-be-displayed image according to the second display position, so as to display the to-be-displayed image on the second display module.

[0137] Exemplarily, the display device further includes a pose acquisition sub-module and an image update sub-module.

[0138] The pose acquisition sub-module is configured to acquire a pose change amount of the near-eye display device after displaying the current frame of the to-be-displayed image and before displaying the next frame of the to-be-displayed image.

[0139] An image update sub-module, configured to update and display the current frame of the image to be displayed on the first display module and the second display module according to the pose change amount.

[0140] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described device and each module and unit can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0141] The method of the present application can be used in many general-purpose or special-purpose computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0142] Exemplarily, the above method and device can be implemented in the form of a computer program, which can run on a near-eye display device or a server to control the near-eye display device. Exemplarily, the near-eye display device can include AR glasses, MR glasses, AR helmets, MR helmets, etc., without limitation here. The server can be a single server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0143] Please refer to Figure 3 , Figure 3 which is a schematic block diagram of the structure of a near-eye display device provided by an embodiment of the present application.

[0144] As Figure 3 shown, the near-eye display device is provided with a first display module and a second display module, and the setting positions of the first display module and the second display module are different. The near-eye display device includes a memory and a processor. Among them, the memory and the processor can be connected through a system bus, and the memory can include a storage medium and an internal memory.

[0145] The storage medium can store an operating system and a computer program. When the computer program is executed, it can cause the processor to execute a display method for any near-eye display device.

[0146] The processor is used to provide computing and control capabilities to support the operation of the entire near-eye display device.

[0147] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, it can cause the processor to execute a display method for any near-eye display device.

[0148] Those skilled in the art can understand that Figure 3 the structure shown in

[0149] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0150] Among them, in one embodiment, the processor is used to execute the computer program and implement the following steps when executing the computer program: Obtain the data to be displayed of the near-eye display device; Render the data to be displayed to obtain a to-be-displayed image corresponding to the data to be displayed; According to the display parameters of the first display module, display the to-be-displayed image on the first display module; According to the display parameters of the second display module, display the to-be-displayed image on the second display module.

[0151] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the display of the near-eye display device described above can refer to the corresponding process in the embodiment of the display method of the near-eye display device described above, and will not be repeated here.

[0152] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor may refer to the various embodiments of the display method of the near-eye display device of the present application.

[0153] Among them, the computer-readable storage medium may be an internal storage unit of the near-eye display device described in the foregoing embodiments, such as the hard disk or memory of the near-eye display device. The computer-readable storage medium may also be an external storage device of the near-eye display device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the near-eye display device.

[0154] It should be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0155] It should also be understood that the term "and / or" used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. It should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article or system including the element.

[0156] The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages and disadvantages of the embodiments. The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A display method for a near-eye display device, characterized in that: The near-eye display device is provided with a first display module and a second display module, and the setting position of the first display module is different from the setting position of the second display module; The display method comprises: Acquiring data to be displayed of the near-eye display device; the data to be displayed includes navigation data to be displayed; the navigation data to be displayed includes three-dimensional information of road slope and / or road curvature; Obtaining texture data according to the data to be displayed; According to the pupil distance information corresponding to the near-eye display device, a first view matrix in a left-eye coordinate system and a second view matrix in a right-eye coordinate system are obtained, and according to the focal length of the near-eye display device, a projection matrix corresponding to the near-eye display device is obtained; Obtaining display parameters of the first display module according to the first view matrix and the projection matrix, and obtaining display parameters of the second display module according to the second view matrix and the projection matrix; The image to be displayed of the first display module is obtained according to the texture data and the display parameters of the first display module, and the image to be displayed of the second display module is obtained according to the texture data and the display parameters of the second display module. The image to be displayed of the first display module and the image to be displayed of the second display module are the same image to be displayed.

2. The display method according to claim 1, characterized in that: The obtaining of texture data according to the data to be displayed comprises: Texture rendering is performed on the data to be displayed to obtain texture data corresponding to the data to be displayed.

3. The display method according to claim 1, characterized in that: According to the pupil distance corresponding to the near-eye display device, a first view matrix in a left-eye coordinate system and a second view matrix in a right-eye coordinate system are obtained, including: Obtaining a first view matrix in a left-eye coordinate system and a second view matrix in a right-eye coordinate system according to a pupil distance corresponding to the near-eye display device and a focal length of the near-eye display device; The obtaining, according to the focal length of the near-eye display device, a projection matrix corresponding to the near-eye display device comprises: A projection matrix corresponding to the near-eye display device is determined according to a focal length of the near-eye display device and a display size corresponding to the near-eye display device.

4. The display method according to any one of claims 1 to 3, characterized in that: Also includes: Obtaining a position change amount of the near-eye display device after displaying a current frame of an image to be displayed and before displaying a next frame of an image to be displayed; According to the amount of change in posture, the current frame of the image to be displayed displayed on the first display module and the second display module is updated and displayed.

5. A display device of a near-eye display device, characterized in that: The display device comprises: A data acquisition module, used to acquire data to be displayed of the near-eye display device; the data to be displayed includes navigation data to be displayed; the navigation data to be displayed includes three-dimensional information of road slope and / or road curvature; A texture rendering module, used for obtaining texture data according to the data to be displayed; a matrix determination module, configured to obtain a first view matrix in a left-eye coordinate system and a second view matrix in a right-eye coordinate system according to pupil distance information corresponding to the near-eye display device, and obtain a projection matrix corresponding to the near-eye display device according to a focal length of the near-eye display device; a parameter determination module, configured to obtain display parameters of a first display module of the near-eye display device according to the first view matrix and the projection matrix, and to obtain display parameters of a second display module of the near-eye display device according to the second view matrix and the projection matrix; A display module is used to obtain an image to be displayed of the first display module according to the texture data and the display parameters of the first display module, and to obtain an image to be displayed of the second display module according to the texture data and the display parameters of the second display module, wherein the image to be displayed of the first display module and the image to be displayed of the second display module are the same image to be displayed.

6. A near-eye display device, characterized in that: The near-eye display device is provided with a first display module and a second display module, and the setting position of the first display module is different from the setting position of the second display module; the near-eye display device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the steps of the display method of the near-eye display device according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the display method of the near-eye display device according to any one of claims 1 to 4 are implemented.

Images