Terminal equipment, display rendering method thereof, display rendering device and storage medium

Abstract

The invention relates to the technical field of terminal equipment, and provides terminal equipment, a display rendering method thereof, a display rendering device and a storage medium, and the method comprises the steps: obtaining a to-be-rendered two-dimensional layer; obtaining a depth value corresponding to the two-dimensional layer; determining a three-dimensional coordinate of each pixel according to the depth value corresponding to the two-dimensional layer and the initial two-dimensional coordinate of each pixel in the two-dimensional layer; performing projection transformation on the three-dimensional coordinate of each pixel based on the projection matrix to obtain a target two-dimensional coordinate of each pixel in the two-dimensional layer; and rendering the two-dimensional layer according to the target two-dimensional coordinate of each pixel in the two-dimensional layer. The depth value is newly added to the two-dimensional layer, the depth value is combined with the original 2D information in the two-dimensional layer through perspective projection transformation to obtain the target two-dimensional coordinate of each pixel in the two-dimensional layer, and when the two-dimensional layer is rendered according to the target two-dimensional coordinate, an observer can have far and near visual perception visually. Therefore, the sense of reality of the 3D effect is enhanced.

Description

Technical Field

[0001] The present application relates to the field of display rendering technology, and in particular to a terminal device and a display rendering method thereof, a display rendering apparatus and a storage medium. Background Art

[0002] The rendering engines of terminal devices, such as Android phones and tablets, are mainly based on two-dimensional (2D) planes and cannot directly support system-level three-dimensional (3D) display effects. The limitations of 2D rendering engines restrict the implementation and performance of 3D effects in such terminal devices. Related technologies usually need to rely on additional libraries or applications to achieve system-level 3D effects, which increases the complexity of integration and use and limits the implementation of seamless 3D rendering in terminal devices such as Android systems. Summary of the invention

[0003] The main purpose of the present application is to provide a terminal device and a display rendering method, a display rendering device and a storage medium thereof, aiming to achieve a three-dimensional display effect based on a 2D rendering engine of the terminal device.

[0004] In a first aspect, the present application provides a display rendering method, the method comprising:

[0005] Obtain a two-dimensional layer to be rendered, wherein the two-dimensional layer includes a plurality of pixels;

[0006] Obtaining a depth value corresponding to the two-dimensional layer;

[0007] Determine the three-dimensional coordinates of each pixel in the two-dimensional layer according to the depth value corresponding to the two-dimensional layer and the initial two-dimensional coordinates of each pixel in the two-dimensional layer;

[0008] Based on the projection matrix, the three-dimensional coordinates of each pixel are projected and transformed to obtain the target two-dimensional coordinates of each pixel in the two-dimensional layer;

[0009] The two-dimensional layer is rendered according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer.

[0010] In a second aspect, the present application further provides a display rendering device, the display rendering device comprising:

[0011] A layer acquisition module, used to acquire a two-dimensional layer to be rendered, wherein the two-dimensional layer includes a number of pixels;

[0012] A depth acquisition module, used to acquire a depth value corresponding to the two-dimensional layer;

[0013] A three-dimensional determination module, used to determine the three-dimensional coordinates of each pixel in the two-dimensional layer according to the depth value corresponding to the two-dimensional layer and the initial two-dimensional coordinates of each pixel in the two-dimensional layer;

[0014] A projection transformation module, used for performing projection transformation on the three-dimensional coordinates of each pixel based on a projection matrix to obtain a target two-dimensional coordinate of each pixel in the two-dimensional layer;

[0015] The rendering module is used to render the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer.

[0016] In a third aspect, the present application also provides a wearable device, including a memory and a processor;

[0017] The memory is used to store computer programs;

[0018] The processor is used to execute the computer program and implement the steps of the aforementioned display rendering method when executing the computer program.

[0019] In a fourth aspect, the present application further provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements:

[0020] The aforementioned steps of the display rendering method.

[0021] The present application provides a terminal device and a display rendering method, a display rendering device and a storage medium thereof, wherein the method comprises: obtaining a two-dimensional layer to be rendered, the two-dimensional layer comprising a plurality of pixels; obtaining a depth value corresponding to the two-dimensional layer; determining the three-dimensional coordinates of each pixel according to the depth value corresponding to the two-dimensional layer and the initial two-dimensional coordinates of each pixel in the two-dimensional layer; performing a projection transformation on the three-dimensional coordinates of each pixel based on a projection matrix to obtain the target two-dimensional coordinates of each pixel in the two-dimensional layer; rendering the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer. The three-dimensional coordinates of each pixel are obtained by adding a depth value to the two-dimensional layer, and the target two-dimensional coordinates of each pixel in the two-dimensional layer are obtained by combining the added depth value with the original 2D information of each pixel in the two-dimensional layer through perspective projection transformation. When the two-dimensional layer is rendered according to the target two-dimensional coordinates, the observer can have a visual experience of near and far, so that the terminal device can achieve a three-dimensional display effect based on a 2D rendering engine without relying on additional libraries or applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 It is a flowchart of a display rendering method provided in an embodiment of the present application;

[0024] Figure 2 is a schematic diagram of performing perspective projection transformation in one implementation;

[0025] Figure 3 is a schematic block diagram of a display rendering device provided in an embodiment of the present application;

[0026] Figure 4 It is a schematic block diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0029] The embodiment of the present application provides a terminal device and a display rendering method, a display rendering device and a storage medium thereof. The display rendering method can be applied to the terminal device.

[0030] Exemplarily, the terminal device may include a mobile phone, a tablet computer, a laptop computer, a smart TV, a TV set-top box, VR (Virtual Reality) glasses, AR (Augmented Reality) glasses, VR helmets, AR helmets, etc., without limitation herein.

[0031] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0032] See also Figure 1, Figure 1 It is a flow chart of a display rendering method provided by an embodiment of the present application. It should be noted that the display rendering method provided by the embodiment of the present application can be used in a terminal device.

[0033] like Figure 1 As shown, the display rendering method includes steps S110 to S150.

[0034] Step S110: Obtain a two-dimensional layer to be rendered, where the two-dimensional layer includes a number of pixels.

[0035] The terminal device can render one or more two-dimensional layers to obtain a displayed image. A two-dimensional layer can be understood as a film containing elements such as text or graphics. A two-dimensional layer can be a patch composed of four or more or fewer vertex coordinates; each two-dimensional layer can include one or more elements, such as two non-intersecting rectangles; a two-dimensional layer can accurately locate the elements on the image; multiple two-dimensional layers are combined by stacking up and down to form the final effect of the image.

[0036] In some embodiments, the position of each pixel in the two-dimensional layer may be represented by a two-dimensional coordinate, for example, may be represented as vec2 pos=(x, y), where x represents the horizontal coordinate of the pixel, and y represents the vertical coordinate of the pixel.

[0037] Step S120: Obtain a depth value corresponding to the two-dimensional layer.

[0038] In some implementations, the depth value corresponding to the two-dimensional layer can be set by the designer. For example, a web designer can pre-set the elements included in each two-dimensional layer and the depth value of each two-dimensional layer when designing a web page.

[0039] In other implementations, the depth value of each two-dimensional layer may be determined according to a preset depth value determination logic.

[0040] Exemplarily, the two-dimensional layer to be rendered includes multiple two-dimensional layers, and obtaining the depth value corresponding to the two-dimensional layer includes: determining the depth value corresponding to each of the two-dimensional layers according to the arrangement order of the multiple two-dimensional layers.

[0041] The arrangement order of the two-dimensional layers indicates the order in which different two-dimensional layers are stacked up and down, or can be called a stacking order. Optionally, the arrangement order of the two-dimensional layers can be set by the designer. For example, a web designer can pre-set the arrangement order of different two-dimensional layers when designing a web page. Of course, it is not limited to this. For example, the arrangement order of different two-dimensional layers can also be determined by logic based on a preset arrangement order.

[0042] For example, the depth value corresponding to the two-dimensional layer in the upper arrangement order is smaller than the depth value corresponding to the two-dimensional layer in the lower arrangement order. By setting a smaller depth value for the two-dimensional layer in the upper arrangement order, when the processed two-dimensional layer is rendered to display an image in the embodiment of the present application, the elements of the two-dimensional layer in the upper arrangement order are visually closer to the observer, and the elements of the two-dimensional layer in the lower arrangement order are visually farther from the observer, resulting in visual perceptions of different depths, and can match the arrangement order of the unprocessed two-dimensional layer for display. For example, it can also prevent the occlusion relationship between different two-dimensional layers from being inconsistent with the occlusion relationship during design when the processed two-dimensional layer is rendered to display an image.

[0043] Step S130: Determine the three-dimensional coordinates of each pixel according to the depth value corresponding to the two-dimensional layer and the initial two-dimensional coordinates of each pixel in the two-dimensional layer.

[0044] For ease of explanation, the position of each pixel in the two-dimensional layer vec2 pos = (x, y) can be called the initial two-dimensional coordinate, the depth value corresponding to the two-dimensional layer can be represented as z, and the three-dimensional coordinate of each pixel can be represented as vec3 pos = (x, y, z). An attribute field z for representing the depth can be introduced into the layer data structure of the two-dimensional layer.

[0045] Step S140: Based on the projection matrix, perform projection transformation on the three-dimensional coordinates of each pixel to obtain the target two-dimensional coordinates of each pixel in the two-dimensional layer.

[0046] The projection matrix can play a role in mapping three-dimensional coordinates to two-dimensional coordinates. For example, based on the projection matrix, the three-dimensional coordinates of each pixel are projected and transformed, and the three-dimensional coordinates can be mapped to a two-dimensional plane with a horizontal coordinate range of [-1,1] and a vertical coordinate range of [-1,1]. The horizontal coordinate of each position on the two-dimensional plane multiplied by the width of the display interface can obtain the target horizontal coordinate of the corresponding pixel in the two-dimensional layer, and the vertical coordinate of each position on the two-dimensional plane multiplied by the height of the display interface can obtain the target vertical coordinate of the corresponding pixel in the two-dimensional layer. The target horizontal coordinate and target vertical coordinate of the same pixel constitute the target two-dimensional coordinate of the pixel.

[0047] The three-dimensional coordinates obtained in step S130 are introduced into the depth value z corresponding to the two-dimensional layer, and projection transformation can be performed in three-dimensional space. By performing perspective projection transformation, the newly added depth value can be combined with the original 2D information of each pixel in the two-dimensional layer; during perspective projection transformation, the depth value of the pixel will affect the two-dimensional coordinates projected onto the two-dimensional layer by the pixel; that is, for the same pixel, the two-dimensional coordinates obtained by perspective projection transformation are different when the pixel value is different.

[0048] Based on the projection matrix, the three-dimensional coordinates obtained in step S130 can be mapped to two-dimensional coordinates, which can be called target two-dimensional coordinates, for example, can be expressed as pos1=projection×vec3 pos=(x', y'), where projection represents the projection matrix, x' represents the horizontal coordinate of the pixel after projection transformation, and y' represents the vertical coordinate of the pixel after projection transformation.

[0049] In some embodiments, step S140 performs a projection transformation on the three-dimensional coordinates of each pixel based on a projection matrix to obtain a target two-dimensional coordinate of each pixel in the two-dimensional layer, including: determining the target two-dimensional coordinate of each pixel in the two-dimensional layer according to the product of the perspective projection matrix and the three-dimensional coordinates of each pixel.

[0050] Exemplarily, the projection matrix projection is a 4×4 matrix; the parameters of the projection matrix projection can be preset, and the projection matrix projection can be an orthogonal projection matrix or a perspective projection matrix. For example, the perspective projection matrix can be a projection matrix mat4, but it is certainly not limited thereto.

[0051] The three-dimensional coordinates (x, y, z) are transformed into homogeneous coordinates to obtain the homogeneous coordinates of the pixel (x, y, z, w). The perspective projection matrix projection multiplied by the homogeneous coordinates of the pixel can obtain a new vector (x1, y1, z1, w1); then the vector (x1, y1, z1, w1) of each pixel is subjected to perspective division, that is, each component in (x1, y1, z1, w1) is divided by w1, and then the first two elements (x' / w', y' / w') are taken to obtain the target two-dimensional coordinates (x', y') of each pixel, where x'=x' / w', y'=y' / w'. The perspective division process maps the points in the homogeneous coordinate space to the non-homogeneous standard 2D coordinate space, thereby converting the three-dimensional coordinates of each pixel into the target two-dimensional coordinates of each pixel.

[0052] See also Figure 2 In the illustrated window view, the three-dimensional coordinates of each pixel including the depth information can be transformed through perspective projection based on the perspective projection matrix to obtain the target two-dimensional coordinates of each pixel in the two-dimensional layer. The newly added depth value is combined with the original 2D information of each pixel in the two-dimensional layer, so that the observer can have a visual sense of distance when the two-dimensional layer is rendered and displayed, thereby enhancing the realism of the 3D effect.

[0053] It can make the elements of different layers in the space after perspective projection transformation larger the closer they are to the observer and smaller the farther they are from the observer, that is, a three-dimensional visual effect of larger near and smaller far away; subsequently, when the two-dimensional layer is rendered according to the target two-dimensional coordinates, it can ensure the correct presentation of the 3D scene, achieving a fascinating system-level 3D visual effect.

[0054] It should be noted that step S140 is not limited to performing projection transformation on the three-dimensional coordinates of each pixel through a perspective projection matrix, and other projection transformation matrices, such as an orthogonal projection matrix, may also be used to achieve a corresponding virtual reality effect.

[0055] Step S150: Rendering the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer.

[0056] The pixel value of each pixel may be the color information of the pixel, such as RGB (red, green, and blue) information, but is certainly not limited thereto, and may include, for example, YUV (where Y represents brightness, U represents image color, and V represents saturation) information.

[0057] An image to be displayed can be obtained by rendering one or more two-dimensional layers. The pixel values ​​of pixels at each two-dimensional coordinate in the image to be displayed are determined according to the pixel values ​​of pixels at corresponding target two-dimensional coordinates in the one or more two-dimensional layers, for example, according to the pixel values ​​of pixels at corresponding target two-dimensional coordinates in unobstructed two-dimensional layers.

[0058] By adding a depth value to the two-dimensional layer, the three-dimensional coordinates of each pixel including the newly added depth value and the original 2D information of each pixel in the two-dimensional layer are obtained, and by performing a perspective projection transformation on the three-dimensional coordinates, the corresponding target two-dimensional coordinates are obtained; when the terminal device renders the two-dimensional layer according to the target two-dimensional coordinates, the observer can visually experience the distance, thereby enhancing the realism of the 3D effect.

[0059] For example, the two-dimensional layer can be rendered based on the rendering engine or rendering pipeline of the terminal device system (such as the Android system) according to the target two-dimensional coordinates, so that the observer can have a visual sense of distance and nearness, providing a more realistic 3D visual experience. For example, based on the system rendering engine of the existing Android terminal device, a depth attribute can be added to each layer to enable it to support the rendering and display of 3D space effects, so as to achieve system-level 3D rendering effects, avoid dependence on external libraries or applications, facilitate seamless integration and performance advantages, and improve user experience.

[0060] The embodiments of the present application enrich the graphics rendering capabilities of at least the Android system, bringing a new immersive experience to users. For example, based on the embodiments of the present application, the Android system will be able to natively support system-level 3D rendering, and users do not need additional installation and settings. The newly added depth value and perspective projection technology make the graphics more realistic in 3D space while maintaining the performance of 2D rendering.

[0061] In some implementations, the method further includes: determining a rendering order for each of the two-dimensional layers according to the arrangement order of the multiple two-dimensional layers. Step S150 renders the two-dimensional layers, including: rendering the multiple two-dimensional layers according to the rendering order for each of the two-dimensional layers.

[0062] Exemplarily, two two-dimensional layers are rendered, and when the two two-dimensional layers have an overlapping area, the overlapping area of ​​the two-dimensional layer arranged in an upper order can obscure the overlapping area of ​​the two-dimensional layer arranged in a lower order; when displaying an image rendered by the multiple two-dimensional layers, the obscured overlapping area of ​​the two-dimensional layer arranged in a lower order may not be displayed.

[0063] For example, the rendering engine of the terminal device can render different two-dimensional layers in sequence according to the rendering order. By determining the rendering order of different two-dimensional layers, fine management of the rendering order of multiple two-dimensional layers can be achieved, and a balance between the display performance and quality of the terminal device can be taken into account.

[0064] Exemplarily, at least one of the two-dimensional layers arranged in an upper order is rendered before the two-dimensional layers arranged in a lower order. By rendering the two-dimensional layers arranged in an upper order first, a better display effect can be achieved, which conforms to the observer's observation habits and observation focus.

[0065] In some implementations, the method further includes: determining a rendering resolution of each of the two-dimensional layers when rendering according to the arrangement order of the multiple two-dimensional layers. Exemplarily, rendering the two-dimensional layers includes: rendering the two-dimensional layers according to the rendering resolution.

[0066] Exemplarily, the rendering resolution of at least one two-dimensional layer in the lower arrangement order is lower than the rendering resolution of the two-dimensional layer in the upper arrangement order; step S150 renders the two-dimensional layers, including: rendering each of the two-dimensional layers according to the rendering resolution when each of the two-dimensional layers is rendered.

[0067] For example, if the observation focus is on the 2D layer at the top of the arrangement order, it will be displayed at a higher resolution, so that the display effects of multiple 2D layers are in line with the observer's observation habits and observation focus. It can achieve fine management of the rendering resolution of the 2D layer, and can balance the display performance and quality of the terminal device.

[0068] In some embodiments, the method further includes: determining a target rendering area and / or a target rendering resolution of the two-dimensional layer arranged in the lower order according to an overlapping area between the two-dimensional layer arranged in the upper order and the two-dimensional layer arranged in the lower order, wherein the target rendering area includes at least a portion of the area except the overlapping area, and the target rendering resolution is smaller than the rendering resolution of the two-dimensional layer arranged in the upper order. Rendering the two-dimensional layer includes: rendering the two-dimensional layer according to the target rendering resolution and / or the target rendering area, wherein when rendering the two-dimensional layer according to the target rendering area, the target rendering area of ​​the two-dimensional layer is rendered.

[0069] Exemplarily, according to the overlapping area of ​​the two-dimensional layer arranged in the upper order and the two-dimensional layer arranged in the lower order, the target rendering area of ​​the two-dimensional layer arranged in the lower order is determined, and the target rendering area includes at least a part of the area except the overlapping area; wherein, rendering the two-dimensional layer includes: rendering the target rendering area of ​​the two-dimensional layer. The overlapping area is the area where the two-dimensional layer arranged in the lower order is blocked, and it is not necessary to render it, that is, only the target rendering area of ​​the two-dimensional layer is displayed; the amount of calculation and time spent on rendering the two-dimensional layer can be reduced, and the display efficiency can be improved.

[0070] Exemplarily, according to the overlapping area of ​​the two-dimensional layer arranged in the upper order and the two-dimensional layer arranged in the lower order, the target rendering resolution of the two-dimensional layer arranged in the lower order is determined, and the target rendering resolution is less than the rendering resolution of the two-dimensional layer arranged in the upper order; wherein, rendering the two-dimensional layer includes: rendering the two-dimensional layer according to the target rendering resolution. When there is an occluded area in the two-dimensional layer arranged in the lower order, the rendering resolution of the two-dimensional layer can be reduced to reduce the amount of calculation and time spent on rendering the two-dimensional layer, thereby improving display efficiency.

[0071] Exemplarily, according to the overlapping area of ​​the two-dimensional layer arranged in the upper order and the two-dimensional layer arranged in the lower order, the target rendering area and the target rendering resolution of the two-dimensional layer arranged in the lower order are determined, the target rendering area includes at least a part of the area except the overlapping area, and the target rendering resolution is smaller than the rendering resolution of the two-dimensional layer arranged in the upper order; wherein, rendering the two-dimensional layer includes: rendering the target rendering area of ​​the two-dimensional layer according to the target rendering resolution. The target rendering area can be rendered at a lower target rendering resolution, and the display efficiency can be further improved.

[0072] In some embodiments, the method further includes: determining, based on a viewport range of a terminal device used to display the two-dimensional layer, a layer area of ​​the two-dimensional layer within the viewport range as an area to be rendered.

[0073] Rendering the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer includes: rendering the area to be rendered of the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the area to be rendered of the two-dimensional layer and the pixel value of each pixel in the area to be rendered. The viewport range can be the area actually rendered on the screen. If part of the layer area of ​​a two-dimensional layer is within the viewport range and other layer areas are not within the viewport range, the layer area within the viewport range is determined as the area to be rendered of the two-dimensional layer; if all the layer areas of a two-dimensional layer are within the viewport range, all the layer areas of the two-dimensional layer are determined as the area to be rendered of the two-dimensional layer; if all the layer areas of a two-dimensional layer are not within the viewport range, it is determined that the two-dimensional layer has no area to be rendered.

[0074] For example, when rendering each two-dimensional layer, only the area to be rendered of each two-dimensional layer may be rendered, and the area of ​​each two-dimensional layer that is not within the viewport range may not be rendered, so as to improve the rendering efficiency of the layer within the viewport range; by finely managing the visibility of the two-dimensional layer, the balance between the display performance and quality of the terminal device can be taken into account.

[0075] The display rendering method provided by the embodiment of the present application includes: obtaining a two-dimensional layer to be rendered, the two-dimensional layer including a plurality of pixels; obtaining a depth value corresponding to the two-dimensional layer; determining the three-dimensional coordinates of each pixel according to the depth value corresponding to the two-dimensional layer and the initial two-dimensional coordinates of each pixel in the two-dimensional layer; performing a projection transformation on the three-dimensional coordinates of each pixel based on a projection matrix to obtain the target two-dimensional coordinates of each pixel in the two-dimensional layer; rendering the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer. The three-dimensional coordinates of each pixel are obtained by adding a depth value to the two-dimensional layer, and the target two-dimensional coordinates of each pixel in the two-dimensional layer are obtained by combining the added depth value with the original 2D information of each pixel in the two-dimensional layer through perspective projection transformation. When the two-dimensional layer is rendered according to the target two-dimensional coordinates, the observer can have a visual experience of near and far, so that the terminal device can achieve a three-dimensional display effect based on a 2D rendering engine; there is no need to rely on additional libraries or applications.

[0076] In some implementations, the embodiments of the present application also optimize the rendering pipeline, optimize the algorithm and rendering process in view of the complexity of 3D rendering, achieve efficient rendering, and provide a smoother user experience; for example, it can ensure that high-quality 3D rendering effects are achieved while retaining 2D rendering performance. For example, by managing the two-dimensional layers within the viewport range, it is determined that the areas where the two-dimensional layers are not within the viewport range are not rendered, ensuring that while achieving fascinating 3D effects, smooth performance can be maintained. For example, the depth value, rendering order, rendering resolution, and visibility of the two-dimensional layers can also be finely managed to ensure that the rendering of each frame can meet high-quality standards, achieve a balance between performance and quality, and bring high-level visual presentation to users.

[0077] Please refer to the above examples. Figure 3 , Figure 3 : is a schematic block diagram of a display rendering device provided in an embodiment of the present application. The display rendering device is configured in a terminal device to execute the aforementioned display rendering method. For example, the terminal device may include a mobile phone, a tablet computer, a laptop computer, a smart TV, a TV set-top box, VR (Virtual Reality) glasses, AR (Augmented Reality) glasses, VR helmets, AR helmets, etc., without limitation.

[0078] like Figure 3 As shown, the display rendering device includes:

[0079] The layer acquisition module 110 is used to acquire a two-dimensional layer to be rendered, wherein the two-dimensional layer includes a number of pixels;

[0080] A depth acquisition module 120, used to acquire a depth value corresponding to the two-dimensional layer;

[0081] A three-dimensional determination module 130, configured to determine the three-dimensional coordinates of each pixel in the two-dimensional layer according to the depth value corresponding to the two-dimensional layer and the initial two-dimensional coordinates of each pixel in the two-dimensional layer;

[0082] A projection transformation module 140, configured to perform projection transformation on the three-dimensional coordinates of each pixel based on a projection matrix to obtain a target two-dimensional coordinate of each pixel in the two-dimensional layer;

[0083] The rendering module 150 is used to render the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer.

[0084] Exemplarily, the projection transformation module 140 performs a projection transformation on the three-dimensional coordinates of each pixel based on the projection matrix to obtain the target two-dimensional coordinates of each pixel in the two-dimensional layer. It is used to: determine the target two-dimensional coordinates of each pixel in the two-dimensional layer according to the product of the perspective projection matrix and the three-dimensional coordinates of each pixel.

[0085] Exemplarily, the two-dimensional layer to be rendered includes multiple two-dimensional layers, and when the depth acquisition module 120 acquires the depth value corresponding to the two-dimensional layer, it is used to: determine the depth value corresponding to each of the two-dimensional layers according to the arrangement order of the multiple two-dimensional layers;

[0086] Among them, the depth value corresponding to the two-dimensional layer arranged in the upper order is less than the depth value corresponding to the two-dimensional layer arranged in the lower order.

[0087] Exemplarily, the device further comprises:

[0088] The order determination module is used to: determine the rendering order of each of the two-dimensional layers when rendering according to the arrangement order of the multiple two-dimensional layers; wherein the rendering order of at least one two-dimensional layer in the upper arrangement order is prior to the rendering order of the two-dimensional layer in the lower arrangement order.

[0089] When rendering the two-dimensional layers, the rendering module 150 renders the multiple two-dimensional layers according to the rendering order of the two-dimensional layers.

[0090] Exemplarily, the device further comprises:

[0091] The resolution determination module is used to: determine the rendering resolution of each of the two-dimensional layers when rendering according to the arrangement order of the multiple two-dimensional layers; wherein the rendering resolution of at least one two-dimensional layer in the lower arrangement order is lower than the rendering resolution of the two-dimensional layer in the upper arrangement order.

[0092] When rendering the two-dimensional layers, the rendering module 150 renders each of the two-dimensional layers according to the rendering resolution of each of the two-dimensional layers when rendering.

[0093] Exemplarily, the device further comprises:

[0094] a resolution determination module, used to: determine a target rendering area and / or a target rendering resolution of the two-dimensional layer arranged in the lower order according to an overlapping area between the two-dimensional layer arranged in the upper order and the two-dimensional layer arranged in the lower order, wherein the target rendering area includes at least a partial area except the overlapping area, and the target rendering resolution is smaller than the rendering resolution of the two-dimensional layer arranged in the upper order;

[0095] When rendering the two-dimensional layer, the rendering module 150 is used to: render the two-dimensional layer according to the target rendering resolution and / or the target rendering area, wherein when rendering the two-dimensional layer according to the target rendering area, the target rendering area of ​​the two-dimensional layer is rendered.

[0096] Exemplarily, the device further comprises:

[0097] The target layer determination module is used to: determine, according to the viewport range of the terminal device used to display the two-dimensional layer, a layer area of ​​the two-dimensional layer located within the viewport range as a to-be-rendered area;

[0098] When rendering the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer, the rendering module 150 is used to:

[0099] The area to be rendered of the two-dimensional layer is rendered according to the target two-dimensional coordinates of each pixel in the area to be rendered of the two-dimensional layer and the pixel value of each pixel in the area to be rendered.

[0100] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0101] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0102] Exemplarily, the above method and apparatus may be implemented in the form of a computer program, which may be run on a terminal device. Exemplarily, the terminal device may include VR glasses, AR glasses, VR helmets, AR helmets, etc., which are not limited here.

[0103] Please refer to the above examples. Figure 4 , Figure 4 It is a schematic block diagram of the structure of a terminal device provided in an embodiment of the present application.

[0104] like Figure 4 As shown, the terminal device includes a memory and a processor. The memory and the processor may be connected via a system bus, and the memory may include a storage medium and an internal memory.

[0105] The storage medium may store an operating system and a computer program. When the computer program is executed, the processor may execute any display rendering method.

[0106] The processor is used to provide computing and three-dimensional rendering capabilities to support the operation of the entire terminal device.

[0107] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any display rendering method.

[0108] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the terminal device to which the scheme of the present application is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0110] In one embodiment, the processor is used to execute a computer program and implement the following steps when executing the computer program:

[0111] Obtain a two-dimensional layer to be rendered, wherein the two-dimensional layer includes a plurality of pixels;

[0112] Obtaining a depth value corresponding to the two-dimensional layer;

[0113] Determine the three-dimensional coordinates of each pixel in the two-dimensional layer according to the depth value corresponding to the two-dimensional layer and the initial two-dimensional coordinates of each pixel in the two-dimensional layer;

[0114] Based on the projection matrix, the three-dimensional coordinates of each pixel are projected and transformed to obtain the target two-dimensional coordinates of each pixel in the two-dimensional layer;

[0115] The two-dimensional layer is rendered according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer.

[0116] Exemplarily, when the processor performs projection transformation on the three-dimensional coordinates of each pixel based on the projection matrix to obtain the target two-dimensional coordinates of each pixel in the two-dimensional layer, it is used to:

[0117] The target two-dimensional coordinates of each pixel in the two-dimensional layer are determined according to the product of the perspective projection matrix and the three-dimensional coordinates of each pixel.

[0118] Exemplarily, the two-dimensional layer to be rendered includes multiple two-dimensional layers, and when the processor obtains the depth value corresponding to the two-dimensional layer, it is used to:

[0119] Determining a depth value corresponding to each of the two-dimensional layers according to an arrangement order of the multiple two-dimensional layers;

[0120] Among them, the depth value corresponding to the two-dimensional layer arranged in the upper order is less than the depth value corresponding to the two-dimensional layer arranged in the lower order.

[0121] Exemplarily, the processor is further configured to:

[0122] Determining a rendering order of each of the two-dimensional layers according to the arrangement order of the multiple two-dimensional layers; wherein the rendering order of at least one two-dimensional layer arranged in an upper order is prior to the rendering order of a two-dimensional layer arranged in a lower order;

[0123] The rendering of the two-dimensional layers includes: rendering the multiple two-dimensional layers according to a rendering order when the two-dimensional layers are rendered.

[0124] Exemplarily, the processor is further configured to:

[0125] Determining, according to the arrangement order of the plurality of two-dimensional layers, a rendering resolution of each of the two-dimensional layers when rendering; wherein the rendering resolution of at least one two-dimensional layer in a lower arrangement order is lower than the rendering resolution of the two-dimensional layer in an upper arrangement order;

[0126] The rendering of the two-dimensional layers includes: rendering each of the two-dimensional layers according to a rendering resolution when each of the two-dimensional layers is rendered.

[0127] Exemplarily, the processor is further configured to:

[0128] Determine, according to the overlapping area of ​​the two-dimensional layer at the top and the two-dimensional layer at the bottom, a target rendering area and / or a target rendering resolution of the two-dimensional layer at the bottom, wherein the target rendering area includes at least a partial area except the overlapping area, and the target rendering resolution is smaller than the rendering resolution of the two-dimensional layer at the top;

[0129] Wherein, when rendering the two-dimensional layer, the processor is used to:

[0130] The two-dimensional layer is rendered according to the target rendering resolution and / or the target rendering area, wherein when the two-dimensional layer is rendered according to the target rendering area, the target rendering area of ​​the two-dimensional layer is rendered.

[0131] Exemplarily, the processor is further configured to:

[0132] According to the viewport range of the terminal device used to display the two-dimensional layer, determining a layer area of ​​the two-dimensional layer within the viewport range as a to-be-rendered area;

[0133] When rendering the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer, the processor is used to:

[0134] The area to be rendered of the two-dimensional layer is rendered according to the target two-dimensional coordinates of each pixel in the area to be rendered of the two-dimensional layer and the pixel value of each pixel in the area to be rendered.

[0135] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the vision test described above can refer to the corresponding process in the aforementioned display rendering method embodiment, and will not be repeated here.

[0136] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the display rendering method of the present application.

[0137] The computer-readable storage medium may be an internal storage unit of the terminal device described in the foregoing embodiment, such as a hard disk or memory of the terminal device. The computer-readable storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the terminal device.

[0138] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0139] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0140] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A display rendering method, characterized in that: The method comprises: Obtain a two-dimensional layer to be rendered, wherein the two-dimensional layer includes a plurality of pixels; Obtaining a depth value corresponding to the two-dimensional layer; Determine the three-dimensional coordinates of each pixel in the two-dimensional layer according to the depth value corresponding to the two-dimensional layer and the initial two-dimensional coordinates of each pixel in the two-dimensional layer; Based on the projection matrix, the three-dimensional coordinates of each pixel are projected and transformed to obtain the target two-dimensional coordinates of each pixel in the two-dimensional layer; The two-dimensional layer is rendered according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer.

2. The display rendering method according to claim 1, characterized in that: The projecting transformation of the three-dimensional coordinates of each pixel based on the projection matrix to obtain the target two-dimensional coordinates of each pixel in the two-dimensional layer includes: The target two-dimensional coordinates of each pixel in the two-dimensional layer are determined according to the product of the perspective projection matrix and the three-dimensional coordinates of each pixel.

3. The display rendering method according to claim 1, characterized in that: The two-dimensional layer to be rendered includes multiple two-dimensional layers, and obtaining the depth value corresponding to the two-dimensional layer includes: Determining a depth value corresponding to each of the two-dimensional layers according to an arrangement order of the multiple two-dimensional layers; Among them, the depth value corresponding to the two-dimensional layer arranged in the upper order is less than the depth value corresponding to the two-dimensional layer arranged in the lower order.

4. The display rendering method according to claim 3, characterized in that: The method further comprises: Determining a rendering order of each of the two-dimensional layers according to the arrangement order of the multiple two-dimensional layers; wherein the rendering order of at least one two-dimensional layer arranged in an upper order is prior to the rendering order of a two-dimensional layer arranged in a lower order; The rendering of the two-dimensional layers includes: rendering the multiple two-dimensional layers according to a rendering order when the two-dimensional layers are rendered.

5. The display rendering method according to claim 3, characterized in that: The method further comprises: Determining, according to the arrangement order of the plurality of two-dimensional layers, a rendering resolution of each of the two-dimensional layers when rendering; wherein the rendering resolution of at least one two-dimensional layer in a lower arrangement order is lower than the rendering resolution of the two-dimensional layer in an upper arrangement order; The rendering of the two-dimensional layers includes: rendering each of the two-dimensional layers according to a rendering resolution when each of the two-dimensional layers is rendered.

6. The display rendering method according to any one of claims 1 to 5, characterized in that: The method further includes: determining a target rendering area of ​​the two-dimensional layer arranged in the lower order according to an overlapping area of ​​the two-dimensional layer arranged in the upper order and the two-dimensional layer arranged in the lower order, wherein the target rendering area includes at least a partial area except the overlapping area; wherein rendering the two-dimensional layer includes: rendering the target rendering area of ​​the two-dimensional layer; or The method further includes: determining a target rendering resolution of the two-dimensional layer at the bottom according to an overlapping area of ​​the two-dimensional layer at the top and the two-dimensional layer at the bottom, wherein the target rendering resolution is smaller than a rendering resolution of the two-dimensional layer at the top; wherein rendering the two-dimensional layer includes: rendering the two-dimensional layer according to the target rendering resolution; or The method further includes: determining a target rendering area and a target rendering resolution of the two-dimensional layer in the lower arrangement order according to an overlapping area between the two-dimensional layer in the upper arrangement order and the two-dimensional layer in the lower arrangement order, wherein the target rendering area includes at least a partial area except the overlapping area, and the target rendering resolution is smaller than the rendering resolution of the two-dimensional layer in the upper arrangement order; wherein, rendering the two-dimensional layer includes: rendering the target rendering area of ​​the two-dimensional layer according to the target rendering resolution.

7. The display rendering method according to any one of claims 1 to 5, characterized in that: The method further comprises: According to the viewport range of the terminal device used to display the two-dimensional layer, determining a layer area of ​​the two-dimensional layer within the viewport range as a to-be-rendered area; Rendering the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer includes: The area to be rendered of the two-dimensional layer is rendered according to the target two-dimensional coordinates of each pixel in the area to be rendered of the two-dimensional layer and the pixel value of each pixel in the area to be rendered.

8. A display rendering device, characterized in that: The display rendering device comprises: A layer acquisition module, used to acquire a two-dimensional layer to be rendered, wherein the two-dimensional layer includes a number of pixels; A depth acquisition module, used to acquire a depth value corresponding to the two-dimensional layer; A three-dimensional determination module, used to determine the three-dimensional coordinates of each pixel in the two-dimensional layer according to the depth value corresponding to the two-dimensional layer and the initial two-dimensional coordinates of each pixel in the two-dimensional layer; A projection transformation module, used for performing projection transformation on the three-dimensional coordinates of each pixel based on a projection matrix to obtain a target two-dimensional coordinate of each pixel in the two-dimensional layer; The rendering module is used to render the two-dimensional layer according to the target two-dimensional coordinates of each pixel in the two-dimensional layer and the pixel value of each pixel in the two-dimensional layer.

9. A terminal device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the steps of the display rendering method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is enabled to implement: The steps of the display rendering method as described in any one of claims 1 to 7.

Images