An image rendering method, apparatus and electronic device

Abstract

Embodiments of this application disclose an image rendering method, apparatus, and electronic device. The method includes: writing an image captured by a target camera into a target rendering texture; obtaining the width and height values ​​of the target rendering texture in three-dimensional space based on the resolution, pixel-per-unit ratio, and scaling factor of the target rendering texture; constructing a cylindrical model based on the width and height values ​​of the target rendering texture in three-dimensional space, and obtaining the horizontal and vertical resolutions corresponding to the cylindrical model; constructing a three-dimensional mesh based on the horizontal and vertical resolutions corresponding to the cylindrical model; and rendering the three-dimensional mesh based on the texture in the target rendering texture. This application combines rendering textures for scene rendering, eliminating the need to draw scene content onto a buffer, thereby reducing an additional texture sampling, simplifying the entire rendering process, and improving rendering quality.

Description

Technical Field

[0001] This application relates to the field of image rendering technology; more specifically, it relates to an image rendering method, apparatus, and electronic device. Background Technology

[0002] With the rapid development of virtual reality technology, users have increasingly higher requirements for immersive experiences, especially in the design of user interface (UI). In a virtual reality environment, the UI not only needs to be aesthetically pleasing and practical, but also must be able to adapt to the characteristics of three-dimensional space to ensure good readability and ease of use from different perspectives.

[0003] Currently, when rendering UI images in a 3D engine, the camera first renders the captured scene content to a buffer, then the asynchronous time warp thread processes the images in the buffer through sampling and compositing, and finally renders the scene content to the display. This multiple image rendering reduces the rendering quality of the image. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application provide an image rendering method, apparatus, and electronic device, which solve the problem of low image rendering quality in related technologies.

[0005] According to one aspect of the embodiments of this application, an image rendering method is provided, the method comprising: constructing a target rendering texture according to the canvas size and scaling factor of a 3D engine; writing an image captured by a target camera into the target rendering texture; obtaining the width and height values ​​of the target rendering texture in three-dimensional space according to the resolution, pixel-per-unit ratio and scaling factor of the target rendering texture; constructing a cylindrical model according to the width and height values ​​of the target rendering texture in three-dimensional space, and obtaining the horizontal and vertical resolutions corresponding to the cylindrical model; constructing a 3D mesh according to the horizontal and vertical resolutions corresponding to the cylindrical model; and rendering the 3D mesh according to the texture in the target rendering texture.

[0006] Optionally, constructing a target rendering texture based on the canvas size and scaling factor of the 3D engine includes: obtaining the length and width values ​​of the canvas in 3D space based on the canvas size and scaling factor of the 3D engine; obtaining an initial resolution based on the length and width values ​​and a preset ratio per unit pixel; obtaining the resolution of the rendering texture based on the initial resolution and a preset rendering ratio coefficient; and constructing the target rendering texture based on the resolution of the rendering texture.

[0007] Optionally, writing the image captured by the target camera into the target rendering texture includes: obtaining the texture coordinates of each pixel in the image captured by the target camera in the target rendering texture; rendering the pixel value corresponding to each pixel onto the corresponding texture coordinates; obtaining the adjacent texture points corresponding to each sampling point in the target rendering texture; and updating the color value of the sampling point according to the color value of the adjacent texture points.

[0008] Optionally, constructing a cylindrical model based on the width and height values ​​of the target rendered texture in three-dimensional space, and obtaining the horizontal and vertical resolutions corresponding to the cylindrical model, includes: calculating the positive and negative boundary values ​​of the X-axis and Y-axis of the world coordinate system based on the width and height values ​​of the target rendered texture in three-dimensional space; placing the center point of the cylindrical model of the 3D engine at the origin of the world coordinate axis to obtain the actual radius of the cylindrical model; obtaining the number of vertices per unit length by multiplying the number of vertices per degree by the conversion coefficient from radians to degrees; obtaining the horizontal resolution corresponding to the cylindrical model by multiplying the number of vertices per unit length by the ratio of the positive boundary value of the X-axis to the actual radius of the cylindrical model; and obtaining the vertical resolution corresponding to the cylindrical model by multiplying the horizontal resolution by the ratio of the positive boundary value of the Y-axis to the positive boundary value of the X-axis.

[0009] Optionally, constructing a three-dimensional mesh based on the horizontal and vertical resolutions corresponding to the cylindrical model includes: calculating the UV coordinates corresponding to each pixel based on the horizontal and vertical resolutions; calculating the vertex coordinates corresponding to each UV coordinate on the cylindrical model based on each UV coordinate; iterating through the horizontal and vertical resolutions to generate triangle indices; and constructing a three-dimensional mesh based on the UV coordinates, vertex coordinates, and triangle indices.

[0010] Optionally, based on each UV coordinate, the corresponding vertex coordinates on the cylindrical model are calculated, including: obtaining the length value of the cylindrical model in the vertical direction by linear interpolation of the UV coordinate parameter U between the positive and negative boundary values ​​of the X-axis; obtaining the height value of the cylindrical model in the vertical direction by linear interpolation of the UV coordinate parameter V between the positive and negative boundary values ​​of the Y-axis; obtaining the angle from the center of the cylinder to the vertex by the ratio of the length value of the cylindrical model in the vertical direction to the actual radius of the cylindrical model; obtaining the X coordinate of the vertex by multiplying the sine of the angle by the actual radius of the cylindrical model; using the height value of the cylindrical model in the vertical direction as the Y coordinate of the vertex; and obtaining the Z coordinate of the vertex by subtracting the actual radius of the cylindrical model from the product of the cosine of the angle and the actual radius of the cylindrical model.

[0011] Optionally, constructing a 3D mesh based on the UV coordinates, vertex coordinates, and triangle indices includes: creating a Mesh object and assigning the vertex coordinates to the vertex set of the Mesh object; mapping textures onto the Mesh surface based on the UV coordinates; defining the Mesh surface based on the triangle indices; calculating axis-aligned bounding boxes based on the vertex positions of the Mesh; and calculating the normal of each vertex based on the vertex positions and triangle definitions of the Mesh to form the 3D mesh.

[0012] Optionally, rendering the 3D mesh according to the texture in the target rendering texture includes: converting the target rendering texture into a base texture map; creating a material sphere and assigning the material sphere to the mesh renderer of the cylinder model; setting the material properties of the material sphere to the base texture map; and applying the material properties to the mesh renderer of the cylinder model to achieve image rendering of the 3D mesh.

[0013] According to one aspect of the embodiments of this application, an image rendering apparatus is provided, the apparatus comprising: a rendering texture construction module, configured to construct a target rendering texture according to the canvas size and scaling factor of a 3D engine; a texture writing module, configured to write an image captured by a target camera into the target rendering texture; a width and height value acquisition module, configured to acquire the width and height values ​​of the target rendering texture in three-dimensional space according to the resolution, pixel-per-unit ratio and scaling factor of the target rendering texture; a resolution acquisition module, configured to construct a cylindrical model according to the width and height values ​​of the target rendering texture in three-dimensional space, and acquire the horizontal and vertical resolutions corresponding to the cylindrical model; a 3D mesh construction module, configured to construct a 3D mesh according to the horizontal and vertical resolutions corresponding to the cylindrical model; and an image rendering module, configured to perform image rendering on the 3D mesh according to the texture in the target rendering texture.

[0014] According to one aspect of the embodiments of this application, a computer-readable medium is provided, on which a computer program is stored, which, when executed by a processor, implements the image rendering method as described in the above technical solutions.

[0015] According to one aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to enable the electronic device to implement the image rendering method as described in the above technical solutions.

[0016] According to one aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the image rendering method as described in the above technical solutions.

[0017] The technical solution provided in this application includes at least the following beneficial effects:

[0018] 1. This application writes the scene content captured by the target camera into a rendering texture, constructs a cylindrical model according to the resolution of the rendering texture, and renders the image texture in the rendering texture onto the three-dimensional mesh corresponding to the cylindrical model. Therefore, this invention combines the rendering texture for scene rendering, eliminating the need to draw the scene content onto the buffer, thereby reducing an extra texture sampling, simplifying the entire rendering process, and improving the rendering quality.

[0019] 2. This application uses rendering textures in Unity to dynamically adjust the resolution, rendering images at high resolution and then scaling them down to the final display resolution, thereby making the edges smoother, reducing jagged edges, and also reducing moiré patterns caused by interference between pixel grids and texture details. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0021] Figure 1 The diagram shown is a flowchart of an image rendering method provided in an embodiment of this application;

[0022] Figure 2 As shown Figure 1 An exemplary flowchart of step S10;

[0023] Figure 3 As shown Figure 1 An exemplary flowchart of step S40;

[0024] Figure 4 As shown Figure 1 An exemplary flowchart of step S50;

[0025] Figure 5 As shown Figure 4 An exemplary flowchart of step S52

[0026] Figure 6 The diagram shown is a schematic representation of the relationship between UV coordinates and vertex coordinates according to an embodiment of this application.

[0027] Figure 7 The diagram shown is a structural schematic of an image rendering apparatus provided in an embodiment of this application;

[0028] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0030] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0031] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0032] It should also be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0033] Figure 1 The diagram shown is a flowchart illustrating an image rendering method provided in an embodiment of this application; as follows: Figure 1 As shown, the method specifically includes the following steps:

[0034] Step S10: Construct the target rendering texture based on the canvas size and scaling factor of the 3D engine.

[0035] like Figure 2 The specific steps for constructing the target rendering texture based on the canvas size and scaling factor of the 3D engine in this embodiment are as follows:

[0036] Step S11: Obtain the length and width values ​​of the canvas in three-dimensional space based on the canvas size and scaling factor of the 3D engine.

[0037] In this embodiment, the 3D engine includes the Unity game engine, and the canvas represents the container for all UI components in the 3D engine; multiple canvas objects can exist in a scene, and nesting between canvases is also allowed.

[0038] In this embodiment, a new script is created in the canvas of the Unity game engine. The script obtains the canvas size and scaling factor through the RectTransform component. The RectTransform component is a component in Unity used to control the position and size of UI elements, and the scaling factor is the scaling factor of the parent element. In Unity, the scaling of a UI element is affected not only by its own scaling settings but also by the scaling settings of all its parent elements. This is because Unity's UI system uses a mechanism called "relative layout," in which the transformations (including position, rotation, and scaling) of child elements are calculated relative to their parent elements. Therefore, the final size of the canvas is also affected by the scaling factor of its parent elements.

[0039] Optionally, the canvas size includes an X component (length) and a Y component (width), and the scaling factor includes a first scaling factor corresponding to the X component and a second scaling factor corresponding to the Y component. Each component of the canvas size is multiplied by the corresponding scaling factor to obtain the true length and width values ​​of the canvas in three-dimensional space. For example, if the canvas size is 1×1, the first scaling factor is 0.5, and the second scaling factor is 1, then the length and width values ​​of the canvas in three-dimensional space are 0.5×1.

[0040] Step S12: Obtain the initial resolution based on the length and width values ​​and the preset ratio per unit pixel.

[0041] In this embodiment, the per-pixel ratio of the driving texture size is defined by the script in the canvas. The per-pixel ratio is used to convert the unit size of the canvas into the pixel size (i.e., the resolution size). The pixel size, i.e., the initial resolution, is obtained by multiplying the actual length and width of the canvas in three-dimensional space by the per-pixel ratio. The API Mathf.RoundToInt (an API for converting floating-point numbers to integers) is used to round the converted floating-point number to an integer, thereby obtaining the initial resolution as an integer.

[0042] Step S13: Obtain the resolution of the rendered texture based on the initial resolution and the preset rendering scaling factor.

[0043] In this embodiment, a rendering scaling factor is defined in the script within the canvas. If additional resolution is needed, this scaling factor can be multiplied by the initial resolution to be used as the resolution of the final rendered texture. For example, a 1m x 1m canvas object has a pixel size of 100 x 100, but it is desired that this canvas can be rendered at a high resolution. This can be achieved by multiplying the initial resolution by the rendering scaling factor. This allows for dynamic adjustment of the rendered image resolution without changing the camera settings. By adjusting the integer rendering scaling factor, a balance can be found between image quality and rendering performance. For example, in scenes requiring a large amount of detail but with insufficient performance, the integer rendering scaling factor can be appropriately reduced to improve performance; while in scenes with ample performance but requiring higher image quality, the value of the integer rendering scaling factor can be increased.

[0044] Step S14: Construct the target rendering texture according to the resolution of the rendering texture.

[0045] The image rendering method in this embodiment is applied to a head-mounted display. The head-mounted display includes a UI camera and a main camera. The script sets the rendering level of the UI camera to the UI layer and sets the rendering level of the main camera to not render the UI through LayerMask. This step mainly uses rendering level management to prevent the VR main camera (user's view) from directly seeing the canvas and the UI elements on it.

[0046] Create new global variables for target rendering texture and camera parameters. Initialize the camera parameters as follows: Set the camera's projection mode to orthographic mode, meaning the camera has no perspective effect; limit the camera's viewing distance to save performance; set the camera's clear flag to a solid color, indicating that the camera is set to a solid color background; set the camera's background color to completely transparent to ensure that the background color will not affect the view of the UI; and check the UI layer in the camera rendering mask to ensure that this camera will only render UI elements.

[0047] In this embodiment, the width and height of the target rendering texture are equal to the size of the pixels to be rendered. The color format of the rendering texture is specified as ARGB32, which means that each pixel uses 32 bits of color, including 8 bits of red, 8 bits of green, 8 bits of blue and 8 bits of transparency. The read / write mode of the target rendering texture is specified as "sRGB". The target texture parameters of the UI camera are set to the newly created target rendering texture. In this way, the UI camera will not render the captured image content to the screen of the head-mounted display, but will write the content to the constructed target rendering texture.

[0048] Step S20: Write the image captured by the camera into the target rendering texture.

[0049] In this embodiment, writing the image captured by the target camera into the target rendering texture specifically includes: obtaining the texture coordinates of each pixel in the image captured by the target camera in the target rendering texture; rendering the pixel value corresponding to each pixel onto the corresponding texture coordinates; obtaining the adjacent texture points corresponding to each sampling point in the target rendering texture; and updating the color value of the sampling point according to the color value of the adjacent texture points.

[0050] It should be noted that the target camera is the UI camera, and the image captured by the target camera represents the UI image captured by the UI camera. When writing to the texture, an sRGB to linear color space conversion is applied, and the reverse conversion is applied when reading the texture to ensure correct color display on the display device. This is crucial for handling lighting and color mixing. After pixels are written to the texture, the texture pixels are sampled and interpolated to produce smoother results when scaling the texture. The specific writing process is as follows:

[0051] (1) Writing pixel values: In the process of texture mapping, it is first necessary to determine the texture coordinates corresponding to each pixel on the screen. Since screen pixels and texture pixels are usually not in one-to-one correspondence, it is necessary to find the texture coordinates corresponding to each pixel in the image on the target rendering texture through UV coordinate mapping and other methods; then the pixel value corresponding to each pixel point is rendered to the corresponding texture coordinates.

[0052] (2) Obtaining adjacent texture points: Take each texture point in the target rendering texture as a sampling point, and find the four nearest texture points around the sampling point on the texture. These four points are usually located at the upper left, upper right, lower left and lower right positions of the sampling point.

[0053] (3) Horizontal interpolation: Linear interpolation is performed on the texture points on both sides of the sampling point in the horizontal direction; specifically, the weighted sum of the color values ​​of the four pixels is calculated by using the ratio of the x coordinate (or U coordinate) of the sampling point to the x coordinate of the four adjacent texture points, so as to obtain the intermediate color value of the sampling point in the horizontal direction.

[0054] (4) Vertical interpolation: The weighted sum of the color values ​​of the four pixels is calculated by using the ratio of the y coordinate (or V coordinate) of the sampling point to the y coordinate of the four adjacent pixels in the vertical direction, so as to obtain the intermediate color value of the sampling point in the vertical direction.

[0055] (4) Color value update: Update the sampled color value by taking the middle color value of the sampling point in the horizontal direction and the middle color value in the vertical direction, thereby completing the texture rendering of the UI image.

[0056] Additionally, it should be noted that when setting the view height of the UI camera in orthographic mode, the distance from the center line to the top and bottom is half the height of the UI canvas. This is achieved by dividing the height of the pixel size by the pixel percentage (pixelsPerUnit) and then multiplying by 0.5 to ensure that the orthographic camera's view height can precisely render the UI.

[0057] Step S30: Based on the resolution of the target rendering texture, the pixel ratio, and the scaling factor, obtain the width and height values ​​of the target rendering texture in three-dimensional space.

[0058] It should be noted that the resolution and pixel ratio of the target rendering texture are obtained through the script under the canvas. The width value of the target rendering texture is obtained by dividing the column resolution of the target rendering texture by the pixel ratio, and the height value of the target rendering texture is obtained by dividing the row resolution of the target rendering texture by the pixel ratio. The width and height values ​​of the target rendering texture are then divided by the scaling factor to obtain the width and height values ​​of the target rendering texture in 3D space. The scaling factor represents the scaling ratio of the current object and all its parent objects in world space. If the canvas or any of its parent objects are scaled, the actual size of the rendering texture in world space will also change accordingly. Therefore, dividing by the scaling factor yields the true size of the target rendering texture without the effect of scaling.

[0059] Step S40: Construct a cylindrical model based on the width and height values ​​of the target rendering texture in three-dimensional space, and obtain the horizontal and vertical resolutions corresponding to the cylindrical model.

[0060] like Figure 3 As shown, this embodiment constructs a cylindrical model based on the width and height values ​​of the target rendering texture in three-dimensional space, and obtains the horizontal and vertical resolutions corresponding to the cylindrical model, specifically including the following steps:

[0061] Step S41: Calculate the positive and negative boundary values ​​of the X-axis and Y-axis of the world coordinate system based on the width and height values ​​of the target rendering texture in three-dimensional space.

[0062] It should be noted that the positive and negative boundary values ​​of the world coordinate system X and Y axes are calculated based on the width and height values ​​of the target rendered texture in three-dimensional space. Let the origin of the world coordinate system be (0, 0). The positive boundary value of the X axis is + half the width value, and the negative boundary value of the X axis is - half the width value. The positive boundary value of the Y axis is + half the height value, and the negative boundary value of the Y axis is - half the height value.

[0063] Step S42: Place the center point of the cylindrical model in the 3D engine at the origin of the world coordinate axis and obtain the actual radius of the cylindrical model.

[0064] It should be noted that the center point of the cylinder model built into the 3D engine is placed at the origin of the world coordinate system. However, the positive boundary values ​​of the X-axis and Y-axis of the world coordinate system represent the maximum dimensions of the cylinder model in the horizontal and vertical directions, respectively. The actual radius of the cylinder model can be obtained directly through the script under the canvas.

[0065] Step S43: Multiply the number of vertices per degree by the conversion coefficient from radians to degrees to obtain the number of vertices per unit length.

[0066] It should be noted that the number of vertices per degree is a preset parameter value, and the conversion factor from radians to degrees is equal to... The number of vertices per unit length can be obtained by multiplying the number of vertices per degree by the conversion factor from radians to degrees.

[0067] Step S44: Multiply the number of vertices per unit length by the ratio of the positive boundary value of the X-axis to the actual radius of the cylinder model to obtain the horizontal resolution corresponding to the cylinder model.

[0068] It should be noted that the number of vertices per unit length × The horizontal resolution corresponding to the cylindrical model is obtained; where the positive boundary value of the X-axis is the maximum size of the cylindrical model in the horizontal direction, which can be understood as the projection of the cylinder's radius in the X-axis direction, while the actual radius of the cylinder is the radius of the cylinder after scaling; this calculation method gives a preliminary estimate of the horizontal resolution based on the horizontal size of the cylinder and the number of vertices per degree.

[0069] Step S45: Multiply the horizontal resolution by the ratio of the positive boundary of the Y-axis to the positive boundary of the X-axis to obtain the vertical resolution corresponding to the cylindrical model.

[0070] It should be noted that vertical resolution is calculated based on horizontal resolution; multiplying the horizontal resolution by... This yields the vertical resolution; the calculation method adjusts the vertical resolution based on the relative dimensions of the cylinder in the vertical direction.

[0071] Step S50: Construct a three-dimensional mesh based on the horizontal and vertical resolutions corresponding to the cylindrical model.

[0072] In this embodiment, the UV coordinates are two-dimensional coordinates used to map a two-dimensional texture image onto the surface of a three-dimensional model. The UV coordinates are generated by calculating the relative position (u and v) of each vertex in the mesh and are typically normalized to the range of 0 to 1.

[0073] In this embodiment, a pixel is the basic unit of a texture image, representing a point in the image; when the texture is mapped onto the model, the UV coordinates determine which parts of the model's surface correspond to which pixels in the texture image.

[0074] In this embodiment, resolution refers to the width and height of the texture image (in pixels); the higher the resolution of the texture, the more pixels it contains, and the richer the details; the generation method of UV coordinates (based on mesh resolution) and the final texture mapping effect are closely related.

[0075] In this embodiment, vertices are the basic elements that constitute a three-dimensional mesh. The vertex list contains the three-dimensional coordinates of all vertices required to generate a cylindrical mesh. Each vertex has a corresponding UV coordinate to determine the vertex's position on the texture image.

[0076] In this embodiment, triangles are the basic polygons that make up a three-dimensional mesh. The triangle list contains the vertex indices of all the triangles that make up the cylindrical mesh. These triangles connect the vertices to form the surface of the mesh. When a texture is mapped onto the mesh, each triangle maps a portion of the texture image based on the UV coordinates of its vertices.

[0077] UV coordinates play a crucial role in mapping textures onto 3D meshes. They determine which pixels in the texture image will be mapped to which vertices of the mesh, thus affecting the overall visual appearance of the model. Vertices, triangles, and texture resolution are all important factors influencing the final rendering result.

[0078] like Figure 4 As shown, this embodiment constructs a 3D mesh based on the horizontal and vertical resolutions corresponding to the cylindrical model, specifically including the following steps:

[0079] Step S51: Calculate the UV coordinates corresponding to each pixel based on the horizontal resolution and the vertical resolution.

[0080] It should be noted that the horizontal and vertical resolutions for the cylindrical model represent the horizontal and vertical resolutions (or the number of columns and rows of the mesh), respectively. The algorithm iterates through the data, where X represents a constant (horizontal resolution) and x represents a variable ranging from 0 to X-1. Similarly, Y represents a constant (vertical resolution) and y represents a variable ranging from 0 to Y-1. The relative position of the current point x in its row (U coordinate) is determined by dividing x by X-1, ensuring that the value of u smoothly changes from 0 to 1 as x iterates from 0 to X-1. Likewise, the relative position of y in its column (V coordinate), also ranging from 0 to 1, is determined by dividing y by Y-1, ensuring that the value of v smoothly changes from 0 to 1 as y iterates from 0 to Y-1. This is done to match a common texture coordinate system, where the bottom left corner of the texture is (0,0) and the top right corner is (1,1), thus calculating the UV coordinates for each pixel.

[0081] Step S52: Calculate the corresponding vertex coordinates on the cylindrical model based on each UV coordinate;

[0082] like Figure 5 As shown, this embodiment calculates the corresponding vertex coordinates on the cylindrical model based on each UV coordinate, specifically including the following steps:

[0083] Step S521: Based on the linear interpolation of the UV coordinate parameter U between the positive and negative boundary values ​​of the X-axis, obtain the length value of the cylinder model in the vertical direction.

[0084] Step S522: Based on the linear interpolation of the UV coordinate parameter V between the positive and negative boundary values ​​of the Y-axis, obtain the height value of the cylinder model in the vertical direction.

[0085] Step S523: Based on the ratio of the length of the cylinder model in the vertical direction to the actual radius of the cylinder model, obtain the angle from the center of the cylinder to the vertex.

[0086] Step S524: Obtain the X coordinate of the vertex by multiplying the sine of the angle by the actual radius of the cylindrical model.

[0087] Step S525: Use the height value of the cylindrical model in the vertical direction as the Y coordinate of the vertex.

[0088] Step S526: The Z coordinate of the vertex is obtained by multiplying the cosine of the angle by the actual radius of the cylinder model and then subtracting the actual radius of the cylinder model.

[0089] It should be noted that vertex coordinates include X, Y, and Z coordinates, such as... Figure 6 As shown, Figure 6The rectangular texture image in the image consists of several UV coordinates. The arc surface formed by dashed lines represents part of the outer surface of the cylindrical model. Points on the outer surface of the cylindrical model are called vertices, and their corresponding coordinates are called vertex coordinates. In this embodiment, the corresponding vertex coordinates are calculated based on each UV coordinate. The specific steps are as follows:

[0090] (1) Parameter interpolation: The length of the cylinder model in the vertical direction is calculated by using the API interpolation function based on the linear interpolation of the parameter U of the UV coordinate between the positive and negative boundary values ​​of the X-axis, and is denoted as float.x; and the height of the cylinder model in the vertical direction is calculated by using the API interpolation function based on the linear interpolation of the parameter V between the positive and negative boundary values ​​of the Y-axis, and is denoted as float.y.

[0091] (2) Angle calculation: Calculate the angle from the center of the cylinder to the point according to the placement direction of the cylinder; for a vertically placed cylinder, the angle is calculated based on the length value float.x, that is, angle = float.x / scaledRadius; where scaledRadius represents the actual radius of the cylinder model, also known as the scaling radius of the cylinder.

[0092] (3) Coordinate calculation: Use trigonometric functions (sine and cosine) to convert the angle into the coordinates of a point on the surface of the cylinder, specifically:

[0093] The X-coordinate of the vertex is point.x = Mathf.Sin(angle)×scaledRadius, which lies on the circumference of the cylinder.

[0094] The y-coordinate of a vertex is point.y = float.y, which directly uses the height value as the y-coordinate of the point.

[0095] The Z-coordinate of the vertex is point.z = Mathf.Cos(angle) * scaledRadius - scaledRadius, where subtracting scaledRadius is to adjust the position of the cylinder's center on the z-axis (assuming the cylinder is placed along the z-axis and the bottom center is at the origin).

[0096] Step S53: Iterate through the horizontal resolution and the vertical resolution to generate a triangle index.

[0097] Note that the grid is based on a two-dimensional resolution, where resolution.x and resolution.y represent the number of vertices for the width (number of columns) and height (number of rows) of the grid respectively; each grid cell is traversed through a double loop, and two triangles are generated for each internal cell; the specific steps are as follows:

[0098] (1) Outer loop (y-axis): for (int y = 0; y<resolution.y - 1; y++) This loop traverses each row (y coordinate) of the grid, starting from 0 until resolution.y - 2 (since the last row does not need to be processed, it serves as the starting boundary for the next row).

[0099] (2) Inner loop (x-axis): for (int x = 0; x<resolution.x - 1; x++) This loop traverses each column (x coordinate) of each row, starting from 0 until resolution.x - 2 (since the last column does not need to be processed, it serves as the starting boundary for the next column).

[0100] (3) Vertex index calculation:

[0101] int v00 = x + y * resolution.x;: Calculate the vertex index (v00) of the lower left corner of the current cell.

[0102] int v10 = v00 + 1;: Calculate the vertex index (v10) of the lower right corner of the current cell.

[0103] int v01 = v00 + resolution.x;: Calculate the vertex index (v01) of the upper left corner of the current cell.

[0104] int v11 = v00 + 1 + resolution.x;: Calculate the vertex index (v11) of the upper right corner of the current cell.

[0105] These indices are based on the assumption that vertices are stored linearly from left to right and top to bottom. v00 is the lower left vertex of the currently processed cell, v10 is the lower right, v01 is the upper left (start of the next row), and v11 is the upper right (next vertex in the next row).

[0106] (4) Generate triangle indices:

[0107] `tris.Add(v00), tris.Add(v11), tris.Add(v10)` means adding the first triangle, consisting of v00 (bottom left), v11 (top right), and v10 (bottom right); `tris.Add(v00), tris.Add(v01), tris.Add(v11)` means adding the second triangle, consisting of v00 (bottom left), v01 (top left), and v11 (top right). For example, the first quadrilateral (consisting of vertices 0, 1, 100, 101) will generate two triangles (0, 101, 1) and (0, 1, 100).

[0108] Step S54: Construct a 3D mesh based on the UV coordinates, vertex coordinates, and triangle indices.

[0109] It should be noted that the specific method for constructing a 3D mesh is as follows: A Mesh object is created using the Unity engine's C# API. A Mesh is one of the core components in Unity used to represent the geometry of 3D objects. It contains key information such as vertices, normals, UV coordinates, and triangles. The specific steps are as follows:

[0110] (1) Create a Mesh object: Mesh mesh = new Mesh(), which creates an instance of the Mesh class using the new keyword. This instance is currently an empty Mesh, without any information such as vertices, triangles, UV coordinates, or normals.

[0111] (2) Set vertices: mesh.SetVertices(verts) calls the SetVertices method, assigning the verts array (expected to be an array of type Vector3[], where each Vector3 represents the 3D coordinates of a vertex) to the Mesh's vertex set. This is the foundation for building a Mesh, as vertices define the shape and size of the Mesh.

[0112] (3) Set UV coordinates: mesh.SetUVs(0, uvs) sets the first set of UV coordinates for the Mesh (usually called UV0). uvs is an array of type Vector2[], where each Vector2 represents the UV coordinates of a vertex. UV coordinates are used to map textures to the surface of the Mesh, so this set of coordinates is crucial for the texture rendering of the Mesh.

[0113] (4) Setting Triangles: `mesh.SetTriangles(tris, 0)` defines the set of triangles for the Mesh. `tris` is an array of type `int[]`, where each element is an index pointing to a vertex in the vertex array. Every three indices form a triangle, which defines the surface of the Mesh. The second parameter (0 here) is usually used to specify which set of UV coordinates to use, but in this example, since only one set of UV coordinates is set, the default value of 0 is used.

[0114] (5) Recalculate Bounds: mesh.RecalculateBounds(). The RecalculateBounds method automatically calculates the axis-aligned bounding box of the Mesh based on the vertex position. This is useful for determining the position, size and shape of the Mesh, especially in physical collision and rendering optimization.

[0115] (6) Recalculate normals: mesh.RecalculateNormals(). The RecalculateNormals method automatically calculates the normals of each vertex based on the vertex positions and triangle definitions of the mesh. Normals are crucial for determining the lighting and shadow directions of the mesh surface, as they determine how light interacts with the mesh surface.

[0116] Therefore, this embodiment creates a Mesh object and defines its shape and texture mapping by setting vertex, UV coordinates and triangles, and then refines the geometry and rendering properties of the Mesh by recalculating the boundaries and normals.

[0117] Step S60: Render the three-dimensional mesh according to the texture in the target rendering texture.

[0118] It should be noted that the target rendering texture is first converted into a base texture map. The base texture map represents a texture format that can be converted to and from the target rendering texture. The target rendering texture is usually used for advanced rendering techniques such as screen post-processing and off-screen rendering. It is connected to a FrameBufferObject, which allows the rendering result to be stored in a storage area of ​​the GPU for later use. The texture map is usually loaded from the hard drive into memory and then uploaded to the GPU for rendering.

[0119] Next, create a new material, obtain the MeshRenderer component of the cylinder, and assign the material to the MeshRenderer component of the cylinder.

[0120] Next, create a new MaterialPropertyBlock object and set its texture map to the base texture map Texture rendered by RenderTexture mentioned above, and set the clipping threshold of the alpha channel of its shader property to 1.

[0121] Finally, the configured MaterialPropertyBlock is applied to the MeshRenderer of the cylinder to achieve the image rendering of the 3D mesh. The MaterialPropertyBlock allows temporary changes to the material's properties without affecting the material instance itself. This is very useful for batch rendering and performance optimization. It ensures that the material's properties can be changed more efficiently, especially when frequent property updates are required. It maintains the flexibility of the material and reduces performance overhead.

[0122] Additionally, it's worth noting that after rendering the 3D mesh image, special effects processing can be applied. Specifically, this involves using Unity's ShaderLab syntax to set the shader's primary texture, specifying the receiving base texture map, and controlling the alpha channel's clipping threshold between 0 and 1 to make useless pixels (e.g., in a 10x10 pixel image, only 5x5 pixels are visible, and the rest are white by default). The SubShader (a key part of Unity Shaders used to define different stages or methods of rendering) is then rendered in the 'AlphaTest' rendering queue (typically used for objects requiring alpha testing). The projector is ignored to prevent shadows or lighting projections from affecting the object, excluding interference from the 3D scene, and the shader is marked as an opacity clipping type, allowing the rendering engine to perform specific optimizations or processing. The source color is used for blending, disregarding the target color, and the detail level is set to 100 for high-quality rendering.

[0123] Optionally, the core calculation of the shader is as follows: receiving vertex data as input, the texture coordinates passed from the vertex shader are sampled using the tex2D function on the main texture. The sampling result is a float4 type color value, which is then clipped using alpha. If the alpha is less than the clipping threshold, the current fragment is clipped. Finally, the final color value is returned for rendering. The standard Unity CG library file is imported and defined as CG code to enable the code to run on the GPU.

[0124] For example: Suppose a segment has sampled colors of (0.5, 0.2, 0.8, 0.4) (RGBA), and _Cutoff is set to 0.5. Calculating color.a(alpha value) - _Cutoff: 0.4 - 0.5 = -0.1, the clip function will discard the segment because the result is negative. Conversely, if the segment's sampled colors are (0.5, 0.2, 0.8, 0.6), calculating color.a(alpha value) - _Cutoff: 0.6 - 0.5 = 0.1, the result is positive, so the segment is retained, and the color (0.5, 0.2, 0.8, 0.6) is returned.

[0125] Optionally, in the LaterUpdate function (a lifecycle method in Unity that is called after all other Update methods, ensuring that the logic executed in this method is performed after all other frame update logic), the texture of the RenderTexture (target render texture) is mapped to the newly created cylindrical material sphere during texture mapping. The temporary render texture is obtained by using the API RenderTexture.GetTemporary (to obtain a temporary render texture of the same size and format), and the texture is copied using Graphics.CopyTexture (GPU-level, high-efficiency texture copying) to improve performance. After use, it is released by RenderTexture.ReleaseTemporary (returning the temporary render texture to Unity's temporary render texture pool) before the next frame update to avoid memory leaks. After processing the texture, if internal supersampling is set, the layer is submitted to the VR device. The layer has priority during rendering and can be directly rendered to the display, thus avoiding the image quality degradation caused by multiple texture sampling. This completes the optimization of aliasing and moiré.

[0126] In summary, this application writes the scene content captured by the target camera into a rendering texture, constructs a cylindrical model based on the resolution of the rendering texture, and renders the image texture in the rendering texture onto the 3D mesh corresponding to the cylindrical model. Therefore, this application combines the rendering texture for scene rendering, eliminating the need to draw the scene content onto the buffer, thereby reducing an extra texture sampling, simplifying the entire rendering process, and improving rendering quality.

[0127] In addition, this application uses rendering textures in Unity to dynamically adjust the resolution, rendering images at high resolution and then scaling them down to the final display resolution, thereby making the edges smoother, reducing jagged edges, and also reducing moiré patterns caused by interference between pixel grids and texture details.

[0128] Figure 7 The diagram shown is a structural schematic of an image rendering apparatus provided in an embodiment of this application; as follows: Figure 7 As shown, the device includes:

[0129] The rendering texture building module 710 is used to build the target rendering texture based on the canvas size and scaling factor of the 3D engine.

[0130] The texture writing module 720 is used to write the image captured by the target camera into the target rendering texture;

[0131] The width and height value acquisition module 730 is used to acquire the width and height values ​​of the target rendering texture in three-dimensional space based on the resolution of the target rendering texture, the ratio per unit pixel, and the scaling factor.

[0132] The resolution acquisition module 740 is used to construct a cylindrical model based on the width and height values ​​of the target rendering texture in three-dimensional space, and to acquire the horizontal and vertical resolutions corresponding to the cylindrical model.

[0133] The 3D mesh construction module 750 is used to construct a 3D mesh based on the horizontal and vertical resolutions corresponding to the cylindrical model.

[0134] The image rendering module 760 is used to perform image rendering on the three-dimensional mesh according to the texture in the target rendering texture.

[0135] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0136] It should be noted that, Figure 8 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0137] like Figure 8 As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 1003. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. An Input / Output (I / O) interface 1005 is also connected to bus 1004.

[0138] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0139] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0140] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0142] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0143] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. An image rendering method, characterized in that, The method includes: Construct the target rendering texture based on the canvas size and scaling factor of the 3D engine; The image captured by the target camera is written into the target rendering texture; Based on the resolution of the target rendering texture, the pixel-per-unit ratio, and the scaling factor, obtain the width and height values ​​of the target rendering texture in three-dimensional space. A cylindrical model is constructed based on the width and height values ​​of the target rendering texture in three-dimensional space, and the horizontal and vertical resolutions corresponding to the cylindrical model are obtained. A three-dimensional mesh is constructed based on the horizontal and vertical resolutions corresponding to the cylindrical model. The three-dimensional mesh is rendered based on the texture in the target rendering texture.

2. The method according to claim 1, characterized in that, Based on the canvas size and scaling factor of the 3D engine, construct the target rendering texture, including: Based on the canvas size and scaling factor of the 3D engine, the length and width values ​​of the canvas in 3D space are obtained. The initial resolution is obtained based on the length and width values ​​and the preset ratio per unit pixel. The resolution of the rendered texture is obtained based on the initial resolution and the preset rendering scaling factor; The target rendering texture is constructed based on the resolution of the rendering texture.

3. The method according to claim 1, characterized in that, Writing the image captured by the target camera into the target rendering texture includes: Obtain the texture coordinates of each pixel in the image captured by the target camera in the target rendered texture; Render the pixel value corresponding to each pixel onto the corresponding texture coordinates; Obtain the adjacent texture points corresponding to each sampling point in the target rendered texture; The color value of the sampled point is updated based on the color values ​​of the adjacent texture points.

4. The method according to claim 1, characterized in that, A cylindrical model is constructed based on the width and height values ​​of the target rendered texture in three-dimensional space, and the horizontal and vertical resolutions corresponding to the cylindrical model are obtained, including: Based on the width and height values ​​of the target rendered texture in three-dimensional space, calculate the positive and negative boundary values ​​of the X-axis and Y-axis of the world coordinate system; Place the center point of the cylindrical model in the 3D engine at the origin of the world coordinate axis to obtain the actual radius of the cylindrical model; The number of vertices per unit length is obtained by multiplying the number of vertices per degree by the conversion factor from radians to degrees. Multiply the number of vertices per unit length by the ratio of the positive boundary value of the X-axis to the actual radius of the cylinder model to obtain the horizontal resolution corresponding to the cylinder model; Multiply the horizontal resolution by the ratio of the positive boundary of the Y-axis to the positive boundary of the X-axis to obtain the vertical resolution corresponding to the cylindrical model.

5. The method according to claim 1, characterized in that, Based on the horizontal and vertical resolutions corresponding to the cylindrical model, a 3D mesh is constructed, including: Based on the horizontal resolution and the vertical resolution, calculate the UV coordinates corresponding to each pixel; Calculate the corresponding vertex coordinates on the cylindrical model based on each UV coordinate; The horizontal and vertical resolutions are iterated over to generate triangle indices; A 3D mesh is constructed based on the UV coordinates, vertex coordinates, and triangle indices.

6. The method according to claim 5, characterized in that, Based on each UV coordinate, calculate the corresponding vertex coordinates on the cylindrical model, including: The length of the cylindrical model in the vertical direction is obtained by linear interpolation of the parameter U of the UV coordinates between the positive and negative boundary values ​​of the X-axis. The height of the cylindrical model in the vertical direction is obtained by linear interpolation of the parameter V of the UV coordinate between the positive and negative boundary values ​​of the Y-axis. The angle from the center of the cylinder to the vertex is obtained by comparing the length of the cylinder model in the vertical direction with the actual radius of the cylinder model. The X coordinate of the vertex is obtained by multiplying the sine of the angle by the actual radius of the cylindrical model. The vertical height of the cylindrical model is used as the Y coordinate of the vertex. The Z-coordinate of the vertex is obtained by multiplying the cosine of the angle by the actual radius of the cylinder model, and then subtracting the actual radius of the cylinder model.

7. The method according to claim 5, characterized in that, Based on the UV coordinates, vertex coordinates, and triangle indices, a 3D mesh is constructed, including: Create a Mesh object and assign the vertex coordinates to the vertex set of the Mesh object; The texture is mapped onto the Mesh surface based on the UV coordinates; The Mesh surface is defined according to the triangle index; Calculate axis-aligned bounding boxes based on the vertex positions of the Mesh; The normal to each vertex is calculated based on the vertex positions and triangle definitions of the mesh to form the 3D mesh.

8. The method according to claim 4, characterized in that, Rendering the 3D mesh based on the texture in the target rendering texture includes: Convert the target rendering texture into a base texture map; Create a material sphere and assign the material sphere to the mesh renderer of the cylinder model; Set the material properties of the material ball to the base texture map; The material properties are applied to the mesh renderer of the cylindrical model to achieve image rendering of the three-dimensional mesh.

9. An image rendering apparatus, characterized in that, The device includes: The rendering texture building module is used to construct the target rendering texture based on the canvas size and scaling factor of the 3D engine; The texture writing module is used to write the image captured by the target camera into the target rendering texture; The width and height value acquisition module is used to acquire the width and height values ​​of the target rendering texture in three-dimensional space based on the resolution of the target rendering texture, the ratio per unit pixel, and the scaling factor. The resolution acquisition module is used to construct a cylindrical model based on the width and height values ​​of the target rendering texture in three-dimensional space, and to obtain the horizontal and vertical resolutions corresponding to the cylindrical model. A 3D mesh construction module is used to construct a 3D mesh based on the horizontal and vertical resolutions corresponding to the cylindrical model. An image rendering module is used to render the three-dimensional mesh based on the texture in the target rendering texture.

10. An electronic device, characterized in that, include: processor; as well as A memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to enable the electronic device to implement the image rendering method as described in any one of claims 1 to 8.

Images