Element rendering method and device, equipment, storage medium and program product

Abstract

The invention discloses an element rendering method and device, equipment, a storage medium and a program product, and relates to the technical field of computers. The method comprises the following steps: acquiring a first element corresponding to a first element in a virtual scene; acquiring a first camera parameter corresponding to the observation camera; mapping based on the first camera parameter and the first local coordinate to obtain a cutting coordinate of the first element; acquiring a second camera parameter corresponding to a rendering camera; the first element is rendered into a first scene area in the screen based on the clipping coordinates and the second camera parameters. Through the above mode, the first camera parameter can be fully utilized to accurately predict the cutting coordinate, so that when interface rendering is performed through the rendering camera, the efficient interface rendering process is executed with the assistance of the second camera parameter and the predicted cutting coordinate, the rendering process is greatly simplified, and the rendering overhead in the rendering process is reduced. The method can be applied to various scenes such as cloud technology, artificial intelligence and intelligent transportation.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computer technology, and in particular, to an element rendering method, apparatus, device, storage medium, and program product. Background Art

[0002] With the rapid development of computer technology, the demand for constructing three-dimensional scenes is applied to more and more fields, such as the game field, virtual reality, the architectural design field, etc. In the construction of three-dimensional scenes, diverse scene elements are beneficial to bringing rich visual effects to users and vividly presenting the detailed features of the three-dimensional scenes.

[0003] In the related art, the virtual scene is usually observed and rendered comprehensively and meticulously by means of a main scene camera and multiple sub-scene cameras to obtain a scene rendering result; then, the above scene rendering result is rendered through a user interface (UI) camera to present the interface rendering result after being rendered by the UI camera on the screen.

[0004] Although the above process combines the observation rendering process and the interface rendering process and can present a relatively realistic and detailed virtual scene on the interface displayed on the screen, after observing and rendering through multiple sub-scene cameras, it is still necessary to perform an interface rendering process through the UI camera, which easily results in a large rendering overhead during the process of rendering the virtual scene and poor rendering performance. Summary of the Invention

[0005] Embodiments of the present application provide an element rendering method, apparatus, device, storage medium, and program product, which can make full use of the first camera parameters to accurately predict the clipping coordinates, so that when performing interface rendering through a rendering camera, an efficient interface rendering process is executed with the assistance of the second camera parameters and the predicted clipping coordinates, greatly simplifying the rendering process and reducing the rendering overhead during the rendering process. The technical solutions are as follows.

[0006] On the one hand, an element rendering method is provided, and the method includes:

[0007] Obtain the first local coordinates corresponding to the first element in the virtual scene, where the first local coordinates are coordinate expressions determined based on the element coordinate system of the first element;

[0008] Obtain the first camera parameters corresponding to the observation camera, where the observation camera is configured in the virtual scene for scene observation, and the first element is within the observation viewport of the observation camera;

[0009] Map the first local coordinates based on the first camera parameters to obtain the clipping coordinates of the first element, where the clipping coordinates are coordinates representing position information based on the observation viewport range;

[0010] Obtain second camera parameters corresponding to a rendering camera, where the rendering camera is used to render a first scene area on a screen;

[0011] Based on the clipping coordinates and the second camera parameters, render the first element into the first scene area on the screen.

[0012] On the other hand, an element rendering device is provided, and the device includes:

[0013] An obtaining module, configured to obtain first local coordinates corresponding to a first element in a virtual scene, where the first local coordinates are coordinate expressions determined based on an element coordinate system of the first element;

[0014] The obtaining module is further configured to obtain first camera parameters corresponding to an observation camera, where the observation camera is disposed in the virtual scene for scene observation, and the first element is within an observation viewport range of the observation camera;

[0015] A mapping module, configured to map to obtain clipping coordinates of the first element based on the first camera parameters and the first local coordinates, where the clipping coordinates are coordinates representing position information based on the observation viewport range;

[0016] The obtaining module is further configured to obtain second camera parameters corresponding to a rendering camera, where the rendering camera is used to render a first scene area on a screen;

[0017] A rendering module, configured to render the first element into the first scene area on the screen based on the clipping coordinates and the second camera parameters.

[0018] On the other hand, a computer device is provided, where the computer device includes a processor and a memory, and at least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the element rendering method according to any one of the foregoing embodiments of the present application.

[0019] On the other hand, a computer-readable storage medium is provided, and at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the element rendering method according to any one of the foregoing embodiments of the present application.

[0020] On the other hand, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the element rendering method described in any one of the above embodiments.

[0021] The beneficial effects brought by the technical solutions provided in the embodiments of the present application at least include:

[0022] By observing the first camera parameters corresponding to the observation camera and the first local coordinates corresponding to the first element, the clipping coordinates of the first element located within the observation viewport range are determined. Then, with the help of the second camera parameters corresponding to the rendering camera and the clipping coordinates, the first element is rendered into the first scene area on the screen. By predicting the display situation of the first element within the local observation viewport range through the first camera parameters and the first local coordinates, the first element is displayed on the screen used for the interface rendering process through the predicted clipping coordinates obtained after prediction, avoiding the cumbersome process of needing to turn on the observation camera and perform scene rendering before the interface rendering process. The first camera parameters are fully utilized to accurately predict the clipping coordinates. Thus, when performing interface rendering through the rendering camera, an efficient interface rendering process is executed with the assistance of the second camera parameters and the predicted clipping coordinates, greatly simplifying the rendering process, reducing the rendering overhead during the rendering process, and fully enhancing the rendering performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0025] Figure 2 is a flowchart of an element rendering method provided by an exemplary embodiment of the present application;

[0026] Figure 3 is a flowchart of an element rendering method provided by another exemplary embodiment of the present application;

[0027] Figure 4 is a flowchart of an element rendering method provided by still another exemplary embodiment of the present application;

[0028] Figure 5It is a flowchart of an element rendering method provided by another exemplary embodiment of the present application;

[0029] Figure 6 It is a schematic diagram of a rendering interface provided by an exemplary embodiment of the present application;

[0030] Figure 7 It is a schematic diagram of a rendering interface provided by another exemplary embodiment of the present application;

[0031] Figure 8 It is a rendering flowchart of related technologies provided by an exemplary embodiment of the present application;

[0032] Figure 9 It is a rendering flowchart of an element rendering method provided by an exemplary embodiment of the present application;

[0033] Figure 10 It is a coordinate transformation flowchart in an element rendering method provided by an exemplary embodiment of the present application;

[0034] Figure 11 It is a flowchart of a method for executing element rendering by setting parameters provided by an exemplary embodiment of the present application;

[0035] Figure 12 It is a schematic diagram of a combined resource package provided by an exemplary embodiment of the present application;

[0036] Figure 13 It is a schematic diagram of scene settings provided by an exemplary embodiment of the present application;

[0037] Figure 14 It is a schematic diagram of the scene result of main scene rendering provided by an exemplary embodiment of the present application;

[0038] Figure 15 It is a schematic diagram of the object result of sub - camera rendering provided by an exemplary embodiment of the present application;

[0039] Figure 16 It is a schematic diagram of the interface of the first scene area provided by an exemplary embodiment of the present application;

[0040] Figure 17 It is a schematic diagram of the interface for setting script parameters provided by an exemplary embodiment of the present application;

[0041] Figure 18 It is an interface result diagram of the process of completing script parameter settings provided by an exemplary embodiment of the present application;

[0042] Figure 19 It is a schematic diagram of the interface for successfully rendering the first element into the first scene area provided by an exemplary embodiment of the present application;

[0043] Figure 20 It is a schematic diagram of an interface for viewing the rendering process through a frame regulator window provided by an exemplary embodiment of the present application;

[0044] Figure 21 It is a structural block diagram of an element rendering device provided by an exemplary embodiment of the present application;

[0045] Figure 22 It is a structural block diagram of a server provided by an exemplary embodiment of the present application. Detailed implementation manners

[0046] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0047] First, a brief introduction to the nouns involved in the embodiments of the present application is given.

[0048] Rendering optimization: There are many performance bottlenecks in graphics rendering. Finding these performance bottlenecks and making targeted optimizations to improve program efficiency. Common optimization methods include model optimization, culling, multi-threading, caching, etc.

[0049] Graphics Processing Unit (GPU): A dedicated chip for graphics image processing used in modern personal computers, servers, mobile devices, game consoles, etc.

[0050] Rendering pipeline: The graphics rendering process running in the GPU. Generally, attention is paid to its vertex shader, rasterization, and pixel shader. By writing code in the shader, it is possible to flexibly control the GPU to draw and render the rendering components.

[0051] Depth test: A link in the GPU rendering pipeline that can compare the depth of the current pixel with the depth buffer using a specified test method. If the test fails, the pixel rendering is abandoned.

[0052] Depth writing: A switch used to specify whether to write the depth of the current pixel to the depth buffer when the GPU renders pixels.

[0053] Vertex shader: A necessary link in the GPU rendering pipeline. The program will perform per-vertex calculations on the vertices of the model according to the code and output the results to the next stage.

[0054] Pixel shader: A necessary link in the GPU rendering pipeline. The program will perform shading calculations on the rasterized pixels according to the code and output them to the frame buffer after passing the test, completing a rendering pipeline process.

[0055] Coordinate transformation: The process of transforming coordinates in one spatial coordinate system to those in another spatial coordinate system. The transformation relationship between coordinate systems is usually represented by a matrix. Left-multiplying the matrix by the coordinate vector, the resulting value is the transformed coordinate.

[0056] Frame Buffer: A section of memory in the GPU used to store image space data, which can be specified as the drawing target by the rendering pipeline and data can be written into it through the rendering pipeline.

[0057] In the related art, the virtual scene is usually comprehensively and meticulously observed and rendered by means of a main scene camera and multiple sub-scene cameras to obtain the scene rendering result; then, the user interface (UI) camera performs the interface rendering process on the above scene rendering result to display the interface rendering result rendered by the UI camera on the screen. Although the above process combines the observation and rendering process and the interface rendering process and can present a relatively realistic and detailed virtual scene on the interface displayed on the screen, after observing and rendering through multiple sub-scene cameras, it is still necessary to perform the interface rendering process through the UI camera, which easily leads to a large rendering overhead in the process of rendering the virtual scene and results in poor rendering performance.

[0058] In the embodiments of the present application, an element rendering method is introduced, which can make full use of the first camera parameter to accurately predict the clipping coordinates, so that when performing interface rendering through the rendering camera, an efficient interface rendering process is executed with the assistance of the second camera parameter and the predicted clipping coordinates, greatly simplifying the rendering process and reducing the rendering overhead in the rendering process. The element rendering method provided by the embodiments of the present application can be applied to various element rendering scenarios such as game screen rendering scenarios, virtual reality game scenarios, product design scenarios, advertising development scenarios, film and television animation production scenarios, and architectural field scenarios, which are not limited here.

[0059] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards in the relevant regions. For example, the content such as the first element, the first camera parameter, and the second camera parameter involved in the present application is obtained under full authorization.

[0060] Secondly, the implementation environment involved in the embodiments of the present application is described. The element rendering method provided in the embodiments of the present application can be implemented by the terminal alone, or by the server, or by the terminal and the server through data interaction. The embodiments of the present application do not limit this. Optionally, taking the interaction between the terminal and the server to execute the element rendering method as an example for description.

[0061] Schematically, please refer to Figure 1 , in this implementation environment, the terminal 110 and the server 120 are involved, and the terminal 110 and the server 120 are connected through the communication network 130.

[0062] In some embodiments, the terminal 110 has a coordinate acquisition function for acquiring the first local coordinates corresponding to the first element in the virtual scene.

[0063] Schematically, the virtual scene is displayed on the screen corresponding to the terminal 110. Multiple elements can be displayed in the virtual scene. For any one of the elements that need to display the first element, the first local coordinates corresponding to the first element are acquired. The first local coordinates are coordinate expressions determined based on the element coordinate system of the first element.

[0064] In some embodiments, the terminal 110 also has a parameter acquisition function for acquiring the first camera parameters corresponding to the observation camera.

[0065] Schematically, the observation camera is configured in the virtual scene for scene observation, and the first element is within the observation viewport of the observation camera.

[0066] In some embodiments, the terminal 110 sends the first local coordinates and the first camera parameters to the server 120 through the communication network 130, and the server 120 maps the first camera parameters and the first local coordinates to obtain the clipping coordinates of the first element.

[0067] Among them, the clipping coordinates are coordinates representing position information based on the observation viewport range. Schematically, the first camera parameters, as the camera parameters corresponding to the observation camera, correspond to the observation viewport range in the virtual scene. When expressing the first element based on the first camera parameters and the first local coordinates, the position information corresponding to the first element will be determined within the observation viewport that the observation camera can observe, that is, the clipping coordinates of the first element are determined.

[0068] In some embodiments, the server 120 can also acquire the second camera parameters corresponding to the rendering camera.

[0069] Among them, the rendering camera is used to render the first scene area on the screen. Schematically, the observation camera and the rendering camera are used to implement different camera functions. Among them, the observation camera focuses on determining the position information of the first element within the observation viewport range, and the rendering camera focuses on determining the rendering display situation of the first element.

[0070] In some embodiments, the server 120 renders the first element into the first scene area on the screen based on the clipping coordinates and the second camera parameters.

[0071] Schematically, after knowing the clipping coordinates corresponding to the first element, in order to accurately understand the changes in the display of the first element, the second camera parameters are introduced to visually present the element display situation. Thus, by combining the clipping coordinates and the second camera element, the first element is rendered and displayed in the first scene area corresponding to the second camera.

[0072] Optionally, the server 120 generates data for rendering and displaying the first element on the screen based on the clipping coordinates and the second camera parameters. Thus, the server 120 sends the data to the terminal 110 through the communication network 130, so that the terminal 110 renders and displays the first element in the first scene area on the screen.

[0073] In some embodiments, the above-mentioned first scene area can be implemented as a scene area in a virtual scene with the first element, or can be implemented as a scene area in other virtual scenes, which is not limited here.

[0074] It should be noted that the above-mentioned terminal includes but is not limited to mobile terminals such as mobile phones, tablet computers, portable laptop computers, intelligent voice interaction devices, intelligent home appliances, vehicle-mounted terminals, etc., and can also be implemented as a desktop computer, etc.; the above-mentioned server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0075] Among them, cloud technology refers to a hosting technology that unifies a series of resources such as hardware, application programs, and networks within a wide area network or a local area network to achieve data calculation, storage, processing, and sharing. Cloud technology is the general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, be used on demand, and be flexible and convenient.

[0076] In some embodiments, the above server can also be implemented as a node in a blockchain system.

[0077] Combined with the above noun introduction and application scenarios, the element rendering method provided in this application will be described. Taking the application of this method to a server as an example, as Figure 2 shown, this method includes the following steps 210 to step 250.

[0078] Step 210, obtain the first local coordinates corresponding to the first element in the virtual scene.

[0079] Schematically, a virtual scene is a term in the fields of computer graphics and virtual reality, used to describe a simulated and fictional three-dimensional environment created through computer technology. Optionally, the virtual scene can be implemented as a simulated fictional scene or as a digital representation of a real scene; the virtual scene is usually presented through graphics rendering technology.

[0080] In some embodiments, the virtual scene is implemented as the scene presented in a game; or, the virtual scene is the scene presented during the production of a film or television; or, the virtual scene is implemented as the scene presented during the modeling process, etc.

[0081] Optionally, the virtual scene includes multiple three-dimensional models, and a three-dimensional model is a digital model obtained by depicting items, scenes, characters, etc. Schematically, the virtual scene includes multiple three-dimensional models such as virtual buildings, virtual trees, virtual puppies, etc.

[0082] Schematically, the first element is implemented as any one of the three-dimensional models in the virtual scene. For example: the first element is implemented as a virtual puppy in the virtual scene.

[0083] Among them, the first element corresponds to the first local coordinates, and the first local coordinates are a coordinate expression determined based on the element coordinate system of the first element.

[0084] Optionally, for any one three-dimensional model, a coordinate system called a local coordinate system can be created based on the three-dimensional model itself. Through this local coordinate system, the internal geometric structure and position information of the three-dimensional model can be described, so as to describe the geometric shape and local transformation of the three-dimensional model.

[0085] Schematically, for the first element, the local coordinate system created based on the first element itself is called the element coordinate system corresponding to the first element. For example: taking the center of the first element as the origin, and determining the horizontal axis (x-axis), vertical axis (y-axis), and longitudinal axis (z-axis) according to the axis directions of the first element itself, so as to establish the element coordinate system of the first element; or, taking any vertex of the first element as the origin, arbitrarily selecting the x-axis, y-axis, and z-axis based on this origin to establish the element coordinate system of the first element; or, taking any point in the first element as the origin, selecting the x-axis, y-axis, and z-axis to establish the element coordinate system of the first element, etc.

[0086] Optionally, when the first element object undergoes transformations such as moving, rotating, and scaling within the world represented by the virtual scene, these transformations are usually relative to the element coordinate system of the first element itself.

[0087] Schematically, the first element is implemented as a cuboid, and the corresponding element coordinate system is established with the center of the cuboid as the origin and along the three main axes of the cuboid as the coordinate axes; when rotating this cuboid, the rotation is relative to the element coordinate system corresponding to the cuboid, rather than relative to the world coordinate system corresponding to the virtual scene.

[0088] In some embodiments, after establishing the element coordinate system corresponding to the first element, determine the multiple element points that make up the first element, and determine the element point position coordinates of the multiple element points relative to the element coordinate system, so as to use the element point position coordinates corresponding to at least one element point among the multiple element points as the first local coordinates corresponding to the first element.

[0089] Schematically, assume that the first element is implemented as a cube, and the element coordinate system corresponding to the cube is established with the geometric center of the cube as the origin (0, 0, 0), the x-axis pointing to the right side of the cube, the y-axis pointing to the upper side of the cube, and the z-axis pointing to the front of the cube. Then, for any element point P in the first element, determine the element point position coordinates of the element point P relative to the element coordinate system. If the element point position coordinates are (1, 2, 3), it means that the element point P is offset by 1 unit on the x-axis, 2 units on the y-axis, and 3 units on the z-axis relative to the center of the first element, etc.

[0090] Step 220, obtain the first camera parameters corresponding to the observation camera.

[0091] Among them, the observation camera is configured in the virtual scene for scene observation. Schematically, the observation camera can observe the first element expressed in three-dimensional form.

[0092] Optionally, the observation camera can also be called a Subscene Camera, which is a camera that presents a local scene or a specific area in a larger scene.

[0093] Schematically, when observing a scene in a virtual scene through an observation camera, the area observable by the observation camera is called the observation viewport range, and the first element is within the observation viewport range of the observation camera.

[0094] In some embodiments, after determining the observation camera for observing the first element, the first camera parameters corresponding to the observation camera are obtained.

[0095] Optionally, the first camera parameters include the first view matrix (View Matrix) and the first projection matrix (Projection Matrix) of the observation camera. The view matrix defines the transformation from the world coordinate system to the camera coordinate system; the projection matrix defines the transformation from the camera coordinate system to the clip coordinate system, etc.

[0096] Schematically, the first camera parameters corresponding to the observation camera are used to assist the first local coordinates in the matrix transformation process.

[0097] Step 230, based on the first camera parameters and the first local coordinates, map to obtain the clip coordinates of the first element.

[0098] Schematically, after obtaining the first camera parameters, according to the meanings expressed by different parameters in the first camera parameters, perform targeted matrix adjustment based on the first local coordinates to map the first local coordinates to the observation viewport range corresponding to the observation camera to obtain the clip coordinates.

[0099] Among them, the clip coordinates are coordinates representing position information based on the observation viewport range.

[0100] Optionally, first, based on the first element and the first local coordinates, map the first element from the element coordinate system to the world coordinate system corresponding to the virtual scene; then, according to the coordinate expression of the first element in the world coordinate system and the first camera parameters, map the first element from the world coordinate system to the clip coordinate system corresponding to the observation camera, so that the coordinate expression of the first element in the clip coordinate system is called the clip coordinates corresponding to the first element.

[0101] In this process, the clip coordinates corresponding to the first element are determined by the first camera parameters corresponding to the first camera itself and the first local coordinates corresponding to the first element itself, that is, the clip coordinates used for analysis are the content obtained through the matrix transformation between the first camera parameters and the first local coordinates, so that it is possible to confirm the clip coordinates of the first element within the observation viewport range without actually observing through the first camera.

[0102] Step 240, obtain the second camera parameters corresponding to the rendering camera.

[0103] Among them, the rendering camera is used to render the first scene area on the screen. Schematically, the rendering camera is used to clearly display the content expressed in two-dimensional form.

[0104] Optionally, the rendering camera can also be called a UI camera. The UI camera is a camera used to render UI elements. The user interface usually includes elements such as buttons, text, and icons expressed in two-dimensional form; the UI camera processes UI elements through an independent rendering channel so that the UI elements are not affected by the depth and perspective of the three-dimensional scene.

[0105] Schematically, the rendering camera is usually targeted at a fixed screen space, rather than in a three-dimensional virtual scene; through the rendering camera, UI elements can be rendered to a specified position on the screen. The rendering camera and the above-mentioned observation camera are usually controlled separately to perform operations such as scaling, moving, and rotating on the UI elements without affecting the three-dimensional virtual scene.

[0106] In some embodiments, in order to display the scene change situation on a two-dimensional screen under the conditions of a three-dimensional virtual scene, both the rendering camera and the observation camera need to exist.

[0107] Schematically, when it is desired to display a Virtual Reality (VR) interface in a three-dimensional virtual scene, it is necessary to use the observation camera to determine the position of the first element to be displayed in the three-dimensional virtual scene, that is, to determine the above-mentioned clipping coordinates, and it is also necessary to use the rendering camera to render the two-dimensional user interface. That is: combining the clipping coordinates and the rendering camera to determine the change situation of the first element on the two-dimensional user interface corresponding to the screen.

[0108] In some embodiments, after determining the rendering camera for rendering and displaying the user interface, the second camera parameters corresponding to the rendering camera are obtained.

[0109] Optionally, the second camera parameters include the second view matrix and the second projection matrix of the rendering camera. Schematically, the second camera parameters corresponding to the rendering camera are used for the matrix transformation process in combination with the clipping coordinates.

[0110] Step 250, based on the clipping coordinates and the second camera parameters, render the first element into the first scene area on the screen.

[0111] Schematically, after obtaining the second camera parameters, according to the meanings expressed by different parameters in the second camera parameters, perform targeted matrix adjustment based on the clipping coordinates to render the first element into the first scene area on the screen.

[0112] Optionally, first map the first element to the world coordinate system corresponding to the user interface based on the cropping coordinates and the first scene area; then render the first element into the first scene area on the screen according to the coordinate expression of the first element in the world coordinate system and the second camera parameters.

[0113] The above-mentioned first scene area is the scene area determined during the user interface rendering process for the rendering camera, and the viewing viewport range corresponding to the virtual scene is the scene area determined for scene observation by the observation camera; based on the fact that the observation camera focuses on observing the three-dimensional scene and the rendering camera focuses on rendering the two-dimensional user interface, the observation camera and the rendering camera are used for different rendering pipelines.

[0114] That is: the first scene area and the viewing viewport range are different levels of content expressed by two different cameras. Separating the observation camera and the rendering camera can provide more flexible control and rendering process, so as to be able to independently manage the three-dimensional virtual scene and the two-dimensional user interface to achieve more complex visual effects and user interactions.

[0115] It should be noted that the above is only a schematic example, and the embodiments of the present application are not limited thereto.

[0116] In summary, the cropping coordinates of the first element located within the viewing viewport range are determined by the first camera parameters corresponding to the observation camera and the first local coordinates corresponding to the first element, so as to render the first element into the first scene area on the screen by means of the second camera parameters corresponding to the rendering camera and the cropping coordinates. The display situation of the first element within the local viewing viewport range is predicted through the first camera parameters and the first local coordinates, so that the first element is displayed on the screen for performing the interface rendering process through the predicted cropping coordinates, avoiding the cumbersome process of having to turn on the observation camera and perform scene rendering before the interface rendering process. The first camera parameters are fully utilized to accurately predict the cropping coordinates, so that when performing interface rendering through the rendering camera, the second camera parameters and the predicted cropping coordinates are used to perform an efficient interface rendering process, greatly simplifying the rendering process, reducing the rendering overhead during the rendering process, and fully improving the rendering performance.

[0117] In an optional embodiment, when mapping the cropping coordinates of the first element based on the first camera parameters and the first local coordinates, first obtain the first projection coordinates of projecting the first element onto the projection coordinate system by means of the first camera parameters and the first local coordinates, and then map the cropping coordinates corresponding to the first element based on the first projection coordinates. Schematically, as Figure 3 shown, the above-mentioned Figure 2 shown embodiment can also be implemented as the following steps 310 to 360; wherein, the above-mentioned Figure 2The step 230 shown can also be implemented as the following steps 330 to 340.

[0118] Step 310, obtain the first local coordinates corresponding to the first element in the virtual scene.

[0119] Schematically, the first element is implemented as any three-dimensional model in the virtual scene. For the first element, an element coordinate system corresponding to the first element is created based on the first element itself, and the first local coordinates corresponding to the first element can be determined through this element coordinate system.

[0120] Among them, the first local coordinates are coordinate expressions determined based on the element coordinate system of the first element. That is: the first local coordinates are coordinate information determined relative to the first element itself, which is convenient for describing the geometry and transformations inside the first element, such as rotation, scaling, etc.

[0121] Step 320, obtain the first camera parameters corresponding to the observation camera.

[0122] Among them, the observation camera is configured in the virtual scene for scene observation, and the first element is within the observation viewport range of the observation camera.

[0123] Schematically, in computer graphics, the virtual scene is usually divided into a main scene and a sub-scene. The camera is an important component for observing and rendering the virtual scene.

[0124] Among them, the main scene camera is the camera used to render the main scene in the virtual scene. The main scene usually contains the main content in the virtual scene, such as virtual characters, virtual environments, virtual props, etc. The main scene camera is responsible for capturing images in the main scene. Taking the virtual scene as the scene in the game as an example, the main scene camera is the main perspective in the game.

[0125] The sub-scene camera is a camera specifically used to render the sub-scene. In addition to showing the main content through the main scene camera, the virtual scene can also show various additional sub-scene contents through multiple sub-cameras, such as special effects, lighting, or other graphics unrelated to the main scene content. That is: different sub-scenes can independently render the contents contained in the corresponding sub-scene through the corresponding sub-scene cameras.

[0126] Optionally, the observation camera can also be called a sub-scene camera. As a camera participating in presenting the virtual scene, the observation camera is configured in the virtual scene for scene observation; the sub-scene observed by the observation camera can also be called the observation viewport range corresponding to the observation camera.

[0127] Based on the fact that the observation camera can observe the first element, the first element is within the observation viewport corresponding to the observation camera.

[0128] In some embodiments, although the observation camera can observe the first element, considering the first camera parameters of the observation camera itself, the observation process of the first element may not be executed by turning on the observation camera. Instead, the observation process of the first element can be predicted with the aid of the first camera parameters corresponding to the observation camera, so as to avoid the problem of a large amount of data analysis caused by continuously turning on the observation camera to execute the observation process.

[0129] That is: after determining the observation camera that can present the first element within the observation viewport, obtain the first camera parameters corresponding to the observation camera.

[0130] Optionally, the first camera parameters include the first view matrix and the first projection matrix of the observation camera; based on the view matrix defining the transformation from the world coordinate system to the camera coordinate system, the first view matrix represents the transformation from the world coordinate system corresponding to the observation camera to the camera coordinate system corresponding to the observation camera; based on the projection matrix defining the transformation from the camera coordinate system to the clip coordinate system, the first projection matrix represents the transformation from the camera coordinate system corresponding to the observation camera to the clip coordinate system corresponding to the observation camera.

[0131] Schematically, the first view matrix in the first camera parameters is denoted as V1, and the first projection matrix in the first camera parameters is denoted as P1. With the aid of the first camera parameters, an association relationship can be established between the first element and the observation camera to determine the position of the first element mapped within the observation viewport corresponding to the observation camera.

[0132] In some embodiments, the observation camera also has at least one of various camera parameter contents such as position parameters, direction parameters, field of view angle parameters, near clip plane parameters, far clip plane parameters, etc. The first local coordinates are analyzed with the aid of the above first camera parameters, and the situation of the observation camera is grasped more accurately with the aid of the above at least one camera parameter.

[0133] Among them, the position parameter of the observation camera is used to determine the position of the observation camera in the world coordinate system, which is the center point of the camera field of view of the observation camera and can determine the position observed by the observation camera.

[0134] The direction parameter is used to represent the direction in which the observation camera observes. This parameter defines the observation angle of the observation camera and its orientation in the virtual scene.

[0135] The Field of View parameter determines the field of view of the observation camera, that is, the angular range that the observation camera can observe. A larger field of view means that a wider scene can be displayed through the observation camera, while a smaller field of view means that the scene will appear more magnified through the observation camera.

[0136] The Near and Far Clipping Planes parameters belong to the clipping plane parameter content. The clipping plane parameters define the range of the viewing frustum of the observation camera, that is, only the objects between the clipping planes will be rendered, thereby improving the rendering performance and avoiding rendering objects that are too far or too close in the virtual scene, etc.

[0137] The above at least one camera parameter jointly defines the observation characteristics and rendering effects of the observation camera; by adjusting the camera parameters of the observation camera, different visual effects can be achieved to adapt to different scene requirements.

[0138] Step 330: Based on the first camera parameter and the first local coordinate, obtain the first projection coordinate for projecting the first element onto the projection coordinate system.

[0139] Schematically, the first camera parameter is used to determine the position of the first element within the observable viewport range of the observation camera; the first camera parameter can undergo a matrix transformation process based on the first local coordinate.

[0140] Schematically, the projection coordinate system is used to project the three-dimensional virtual scene onto a two-dimensional plane for subsequent final rendering onto the screen.

[0141] In some embodiments, the projection coordinate system is established according to the viewpoint position and the projection type.

[0142] Schematically, the viewpoint position is related to the camera position, the viewing direction, and the up direction of the observation camera; the camera position is the position of the observation camera in the virtual scene; the viewing direction is the direction that the observation camera points to, determining the viewing angle of the observable scene (viewport range); the up direction is the "upward" direction within the viewport range, usually related to the rotation of the observation camera.

[0143] Schematically, the projection type is usually implemented as perspective projection and orthographic projection. Perspective projection simulates the perspective effect of near and far elements in the real world, making distant elements appear smaller; orthographic projection maintains the same size of elements in all directions regardless of the distance between the elements and the observation camera.

[0144] Schematically, when the projection type is selected as perspective projection, the projection coordinate system is established based on the viewpoint position and the perspective projection type; among them, the first projection coordinate can be determined through the perspective projection matrix corresponding to the perspective projection.

[0145] That is: after obtaining the first camera parameters and the first local coordinates, the first local coordinates can be analyzed using the first camera parameters to project the first element onto the projection coordinate system, thereby obtaining the first projection coordinates of the first element in the projection coordinate system.

[0146] Optionally, first transform the first local coordinates corresponding to the first element to the world coordinate system to obtain the first world coordinates of the first element in the world coordinate system; then, transform the first world coordinates to the projection coordinate system to obtain the first projection coordinates of the first element in the projection coordinate system.

[0147] Schematically, the world coordinate system is a coordinate system used to describe the entire three-dimensional virtual scene; in computer graphics and three-dimensional computing, the world coordinate system is a common coordinate system that can represent the positions and orientations of all elements in the virtual scene, etc. That is: the world coordinate system is the basis for describing the entire three-dimensional virtual scene. By providing a consistent framework, the positions and orientations of all elements can be defined relative to a common reference point.

[0148] For example: select a point in the virtual scene as the origin of the world coordinate system, that is, the reference point of the virtual scene, select the horizontal direction as the x-axis, select the vertical direction as the y-axis, and select the depth or the front-back direction as the z-axis, thereby establishing the world coordinate system.

[0149] Optionally, in graphics rendering, information such as the shape and texture of an element is usually defined in the local coordinate system. Therefore, for the first element, after determining the first local coordinates, based on the fact that the first element is located in the virtual scene, the first local coordinates corresponding to the first element can be converted to the world coordinate system, that is, the first world coordinates corresponding to the first element are determined.

[0150] In an alternative embodiment, the first model matrix corresponding to the first element is obtained.

[0151] Schematically, the model matrix (Model Matrix) is a matrix parameter closely related to the attributes of the element (or model) itself, and is a matrix used to define the position, rotation, and scaling of the element in the world coordinate system. The role of the model matrix is to transform the element from the local coordinate system (or model space) to the world coordinate system; this process includes at least one of the operations of translating, rotating, and scaling the element to make the element adapt to the overall layout of the virtual scene.

[0152] Optionally, the model matrix contains transformation information such as the translation, rotation, and scaling of the element. When it is necessary to adjust the position of the element in the world coordinate system, the corresponding position transformation process can be achieved by adjusting the model matrix corresponding to the element.

[0153] In an alternative embodiment, the first local coordinates are adjusted by the first model matrix to obtain the first world coordinates of the first element in the world coordinate system.

[0154] Among them, the first model matrix is the model matrix corresponding to the first element, which is used to adjust the first local coordinates determined by the element coordinate system of the first element to determine the position of the first element in the world coordinate system.

[0155] For example: the first local coordinates corresponding to the first element are represented as v0, and the first model matrix corresponding to the first element is represented as M1. Then, by adjusting the first local coordinates v0 with the first model matrix M1, the position of the first element in the world coordinate system can be determined.

[0156] Schematically, the product of the first local coordinates and the first model matrix is called the first world coordinates corresponding to the first element, and the first world coordinates are the position of the first element in the world coordinate system. For example: the first world coordinates are represented as M1×v0.

[0157] In an alternative embodiment, the first camera parameters include the first view matrix and the first projection matrix corresponding to the observation camera; the first world coordinates are adjusted by the first view matrix to obtain the first view coordinates in the view coordinate system.

[0158] Schematically, the first view matrix is used to adjust the first world coordinates of the first element in the world coordinate system to determine the position of the first element in the view coordinate system, that is, to obtain the first view coordinates of the first element in the view coordinate system.

[0159] The view coordinate system (View Coordinate System) is the coordinate system where the camera is located, also known as the camera coordinate system. The view coordinate system describes the position, orientation, and perspective of the camera.

[0160] Schematically, the view coordinate system is established based on the position of the observation camera. For example: with the center of the observation camera as the origin, the right direction of the observation camera as the x-axis, the upper direction of the observation camera as the y-axis, and the observation direction of the observation camera as the z-axis, the view coordinate system corresponding to the observation camera is established. The view coordinate system is used to convert the first element in the virtual scene from the world coordinate system to the local coordinate system relative to the observation camera.

[0161] This process is achieved through the inverse transformation of the observation camera, ensuring that the observation camera is located at the origin of the view coordinate system, and the z-axis of the view coordinate system points to the virtual scene.

[0162] Optionally, the product of the first world coordinates corresponding to the first element and the first view matrix is called the first view coordinates corresponding to the first element, and the first view coordinates are the position of the first element in the view coordinate system.

[0163] For example: Given that the first view matrix is represented as V1 and the first world coordinates are represented as M1×v0, then the first view coordinates are represented as V1×M1×v0.

[0164] In an optional embodiment, the first view coordinates are adjusted by the first projection matrix to obtain the first projection coordinates in the projection coordinate system.

[0165] Optionally, when the projection type is implemented as a perspective projection, the first projection matrix is implemented as the perspective projection matrix corresponding to the perspective projection; when the projection type is implemented as an orthographic projection, the first projection matrix is implemented as the orthographic projection matrix corresponding to the orthographic projection.

[0166] Schematically, the product of the first view coordinates and the first projection matrix is called the first projection coordinates corresponding to the first element, and the first projection coordinates are the position of the first element in the projection coordinate system. For example: Given that the first projection matrix is represented as P1 and the first view coordinates are represented as V1×M1×v0, then the first projection coordinates are represented as P1×V1×M1×v0; if the first projection coordinates are represented as v1, then v1 = P1×V1×M1×v0.

[0167] Step 340, map the first projection coordinates to obtain the clip coordinates of the first element.

[0168] Schematically, the first local coordinates are four-dimensional vectors with homogeneous coordinate 1.

[0169] Among them, the homogeneous coordinate system is a coordinate system widely used in computer graphics and computer image processing. In the homogeneous coordinate system, the coordinates of a point are usually represented as four components (x, y, z, w), where (x, y, z) are the three-dimensional coordinates of the point respectively, and (w) represents the homogeneous coordinate.

[0170] For homogeneous coordinates, if (w) is not equal to 0, then the coordinates (x, y, z, w) can be converted to three-dimensional coordinates by dividing by the homogeneous coordinate (w). For example: For the point (x, y, z, w) where (w) is not equal to 0, its three-dimensional coordinates are (x / w, y / w, z / w). If (w) is 1, it usually means that the coordinates of the point are the result after homogeneous coordinate transformation. Considering that homogeneous coordinates have more convenient properties in perspective projection and matrix operations, the coordinates are preferably represented in the form with homogeneous coordinates during the graphics rendering process.

[0171] Schematically, the first local coordinates corresponding to the first element are represented in the form of three-dimensional components as (1, 3, 2), and the first local coordinates can be represented in the form of four-dimensional components as (1, 3, 2, 1). Among them, any value in the three-dimensional components divided by the homogeneous coordinate 1 is itself, that is, the first local coordinates expressed in the form of homogeneous coordinates can be determined without changing the meaning of the first local coordinates.

[0172] Optionally, the first model matrix, the first view matrix, and the first projection matrix used for matrix transformation of the first local coordinates are all four-dimensional matrices; the first projection coordinates obtained by mapping the first element through the first local coordinates are also four-dimensional vectors. For example, the first projection coordinate v1 is a four-dimensional homogeneous coordinate, but since the homogeneous coordinate 1 in the first local coordinates has already undergone a matrix transformation process, the homogeneous coordinate in the first projection coordinate v1 may not be 1.

[0173] In an optional embodiment, determine the first homogeneous coordinate in the first projection coordinates.

[0174] Among them, the first homogeneous coordinate is the coordinate dimension in the first projection coordinates used to characterize the projection information of the first element.

[0175] Schematically, the first local coordinates with homogeneous coordinate 1 are transformed by a matrix to obtain the first projection coordinates with the first homogeneous coordinate, and the first homogeneous coordinate may not be 1.

[0176] For example: Given that the first projection coordinates are represented as v1, the first homogeneous coordinate in the first projection coordinates can be represented as v 1.w .

[0177] In an optional embodiment, take the quotient of the first projection coordinates and the first homogeneous coordinate as the clipping coordinates corresponding to the first element.

[0178] Schematically, based on the homogeneous coordinate being 1 to represent the coordinates of this point is the result after homogeneous coordinate transformation, that is, it represents that this point has undergone a projection transformation and can be based on the mapping process of this point. Therefore, after determining the first homogeneous coordinate, in order to adjust the value of the first homogeneous coordinate to 1, a numerical adjustment process can be performed through the current first homogeneous coordinate, so as to achieve the process of projecting and adjusting the first projection coordinates.

[0179] For example: The first projection coordinate v1 can be represented in the form of four-dimensional components as (v 1.x , v 1.y , v 1.z , v 1.w ), that is, it represents that the first homogeneous coordinate in the first projection coordinate v1 is v 1.w . Taking each dimension component in the first projection coordinate v1 as the dividend and the first homogeneous coordinate v1.w As the divisor to obtain the clipping coordinates corresponding to the first element, the adjusted first homogeneous coordinate is 1; the clipping coordinates can be represented as v2, that is, v2 = v1 / v 1，w , and in the form of four-dimensional components, it can be expressed as v2 = (v 1，x / v 1，w , v 1，y / v 1，w , v 1，z / v 1，w , 1). Similarly, the clipping coordinates can also be represented in the form of three-dimensional components as v2 = (v 1，z / v 1，w , v 1，z / v 1，w , v 1，z / v 1，w ).

[0180] Step 350, obtain the second camera parameters corresponding to the rendering camera.

[0181] Among them, the rendering camera is used to render the first scene area on the screen. Schematically, the rendering camera is used to render the content expressed in two-dimensional form.

[0182] In some embodiments, in order to display the scene change situation through a two-dimensional screen under the condition of a three-dimensional virtual scene, the rendering camera and the observation camera need to exist simultaneously; after determining the rendering camera for rendering and displaying the user interface, obtain the second camera parameters corresponding to the rendering camera.

[0183] Optionally, the second camera parameters include the second view matrix and the second projection matrix of the rendering camera.

[0184] Schematically, the rendering camera is used to render and display the user interface, and the user interface includes the first scene area; based on the view matrix, the transformation from the world coordinate system to the camera coordinate system is defined, then the second view matrix represents the transformation from the world coordinate system corresponding to the rendering camera to the camera coordinate system corresponding to the rendering camera; based on the projection matrix, the transformation from the camera coordinate system to the clipping coordinate system is defined, then the second projection matrix represents the transformation from the camera coordinate system corresponding to the rendering camera to the clipping coordinate system corresponding to the rendering camera.

[0185] Schematically, the second view matrix in the second camera parameters is represented as V2, and the second projection matrix in the second camera parameters is represented as P2. With the help of the second camera parameters, an association relationship can be established between the first element and the rendering camera to determine the position situation of the second element mapped into the first scene area rendered by the rendering camera.

[0186] Among them, the rendering camera is used to render the first scene area on the screen.

[0187] Schematically, the rendering camera acts as a UI camera and is used to perform a rendering display process to display user interface content on the screen. The first scene area is an area in the user interface.

[0188] Step 360: Render the first element into the first scene area on the screen based on the clipping coordinates and the second camera parameters.

[0189] Schematically, since the clipping coordinates are the content obtained after homogeneous coordinate transformation, the clipping coordinates are a more standardized expression of the first projection coordinates. The first projection coordinates, as the content obtained after projection onto the projection coordinate system, involve the projection of the first element from a three-dimensional expression to a two-dimensional expression. Therefore, the position of the first element on the model can be determined through the first projection coordinates / clipping coordinates.

[0190] Although the first element can be initially presented on the screen based on the first projection coordinates / clipping coordinates, considering that the observation camera for observing the first element and the rendering camera for rendering and displaying the user interface are relatively independent, if the first element is directly presented in the user interface according to the first projection coordinates / clipping coordinates, there will be a problem that the first element cannot change correspondingly with the content displayed in the user interface, that is, the accurate user interface and the content changes within the user interface cannot be presented through the terminal. Therefore, after obtaining the clipping coordinates, it is necessary to adjust the clipping coordinates by integrating the second camera parameters corresponding to the rendering camera so that the process of displaying the first element can better adapt to the display situation of the user interface.

[0191] In some embodiments, the clipping coordinates are adjusted by the second camera parameters corresponding to the rendering camera. The rendering camera is a camera configured in the UI space and is used to capture and render the scene area in the UI space.

[0192] Among them, the UI space is a virtual coordinate system used to locate and layout interface contents such as buttons, text boxes, icons, etc. in the user interface. Different from the three-dimensional virtual scene, the UI space is usually two-dimensional, that is, the x-axis and the y-axis. In the UI space, units such as pixels or percentages are usually used instead of the traditional three-dimensional coordinates. The UI space is an expression introduced to facilitate the design and rendering of the user interface.

[0193] Although the UI space is usually implemented in a two-dimensional form, sometimes it is also necessary to express three-dimensional effects in the UI space, such as in games or certain special effect scenes.

[0194] Schematically, a pseudo-three-dimensional effect is set in the UI space, such as simulating a three-dimensional appearance by applying shadows, lighting, or other visual effects to UI elements. For example: A button may have a shadow at the bottom, thus showing a sense of protrusion, although the UI element itself is two-dimensional.

[0195] Schematically, perspective and projection are set in the UI space. For example, a perspective effect is applied to UI elements to make them look deeper or farther away. For example: Shrinking one side of a UI element or weakening its transparency to simulate the perspective effect.

[0196] Schematically, real three-dimensional elements (such as the first element above) are added in the UI space, that is, in some cases, real three-dimensional elements can be embedded in the UI space, which involves displaying the three-dimensional elements within the first scene area that needs to be presented.

[0197] In some embodiments, the role of the rendering camera is to convert the UI elements in the UI space to the screen space for correct presentation on the screen; when the first element needs to be projected into the first scene area of the user interface displayed on the screen, after determining the clipping coordinates for initially projecting the first element onto the screen, using the clipping coordinates as the parameter content to be adjusted, the clipping coordinates are specifically adjusted through the second camera parameters corresponding to the rendering camera, so as to adjust from the viewport space characterized by the viewing viewport range to the UI space.

[0198] That is: Through the process of adjusting the clipping coordinates by the second camera parameters, the first element is rendered into the first scene area on the screen.

[0199] In some embodiments, based on the clipping coordinates and the second camera parameters, screen coordinates are obtained; based on the screen coordinates, the first element is rendered into the first scene area on the screen.

[0200] Optionally, when this adjustment process is executed by a cloud server, the clipping coordinates can be adjusted through the second camera parameters to obtain screen coordinates; the cloud server renders the first element into the first scene area on the screen based on the screen coordinates and obtains rendering data; through the process of sending the rendering data to the terminal, the terminal renders and displays the content of the first element in the first scene area on the screen based on the rendering data.

[0201] Optionally, when this adjustment process is executed by a server, the server can adjust the clipping coordinates through the second camera parameters to obtain screen coordinates; the server sends the screen coordinates to the terminal, and the terminal renders and displays the content of the first element in the first scene area on the screen based on the screen data.

[0202] Optionally, when this adjustment process is executed by a terminal, the terminal can adjust the clipping coordinates through the second camera parameters to obtain screen coordinates; and then renders and displays the content of the first element in the first scene area on the screen based on the screen data.

[0203] It should be noted that the above are only schematic examples, and the embodiments of the present application are not limited thereto.

[0204] In summary, the display situation of the first element within the local observation viewport range is predicted through the first camera parameter and the first local coordinate, so that the first element is displayed on the screen used for performing the interface rendering process through the cropping coordinates obtained after prediction, avoiding the cumbersome process of starting the observation camera and performing scene rendering before the interface rendering process. The first camera parameter is fully utilized to accurately predict the cropping coordinates, so that when performing interface rendering through the rendering camera, the second camera parameter and the predicted cropping coordinates are used to perform an efficient interface rendering process, greatly simplifying the rendering process, reducing the rendering overhead during the rendering process, and fully improving the rendering performance.

[0205] In the embodiment of the present application, the content of mapping the first camera parameter and the first local coordinate to obtain the cropping coordinate is introduced. After determining the observation camera capable of observing the first element, the first camera parameter can be obtained without starting the observation camera, and the first local coordinate can be adjusted through the process of matrix transformation, and then the cropping coordinate is obtained after the matrix transformation; the cropping coordinate corresponding to the first element is predicted through the matrix transformation process, avoiding the problem of a large amount of rendering data caused by starting the observation camera for observation and rendering, which is beneficial to alleviating the rendering pressure.

[0206] In an alternative embodiment, the screen coordinate can be obtained based on the cropping coordinate and the second camera parameter, and then the first element is rendered into the first scene area on the screen based on the screen coordinate. Schematically, as Figure 4 shown, the above Figure 2 shown embodiment can also be implemented as the following steps 410 to step 460; wherein, the above Figure 2 shown step 250 can also be implemented as the following steps 450 to step 460.

[0207] Step 410, obtain the first local coordinate corresponding to the first element in the virtual scene.

[0208] In some embodiments, the virtual scene is implemented as the scene presented in the game; or, the virtual scene is the scene presented during the film and television production process in reality; or, the virtual scene is implemented as the scene presented during the modeling process, etc.

[0209] Among them, the first local coordinate is a coordinate expression determined based on the element coordinate system of the first element.

[0210] Step 410 has been described in the above steps 210 and 310, and will not be elaborated here.

[0211] Step 420, obtain the first camera parameter corresponding to the observation camera.

[0212] Among them, the observation camera is configured in the virtual scene for scene observation, and the first element is within the observation viewport range of the observation camera.

[0213] In some embodiments, after determining the observation camera for observing the first element, the first camera parameters corresponding to the observation camera are obtained.

[0214] Optionally, the first camera parameters include the first view matrix and the first projection matrix of the observation camera.

[0215] Step 420 has been described in the above steps 220 and 320, and will not be elaborated here.

[0216] Step 430, based on the first camera parameters and the first local coordinate mapping, obtains the clipping coordinates of the first element.

[0217] Among them, the clipping coordinates are coordinates representing position information based on the observation viewport range.

[0218] In an alternative embodiment, the first model matrix corresponding to the first element is obtained; the first local coordinates are adjusted by the first model matrix to obtain the first world coordinates of the first element in the world coordinate system.

[0219] Schematically, the first local coordinate is v0, the first model matrix is M1, and the first world coordinate is represented as M1×v0.

[0220] In an alternative embodiment, the first view matrix corresponding to the observation camera is included in the first camera parameters; the first world coordinates are adjusted by the first view matrix to obtain the first view coordinates in the view coordinate system.

[0221] Schematically, given that the first view matrix is represented as V1 and the first world coordinate is represented as M1×v0, then the first view coordinate is represented as V1×M1×v0.

[0222] In an alternative embodiment, the first projection matrix corresponding to the observation camera is further included in the first camera parameters; the first view coordinates are adjusted by the first projection matrix to obtain the first projection coordinates in the projection coordinate system.

[0223] Schematically, given that the first projection matrix is P1 and the first view coordinate is represented as V1×M1×v0, then the first projection coordinate is represented as P1×V1×M1×v0; if the first projection coordinate is represented as v1, then v1 = P1×V1×M1×v0.

[0224] In an alternative embodiment, the clipping coordinates of the first element are obtained based on the first projection coordinate mapping.

[0225] Schematically, determine the first homogeneous coordinates in the first projection coordinates; use the quotient of the first projection coordinates and the first homogeneous coordinates as the clipping coordinates corresponding to the first element.

[0226] Step 440: Obtain the second camera parameters corresponding to the rendering camera.

[0227] Among them, the rendering camera is used to render the first scene area on the screen.

[0228] In some embodiments, for the scene change situation to be displayed on a two-dimensional screen under the condition of a three-dimensional virtual scene, the rendering camera and the observation camera need to exist simultaneously; after determining the rendering camera for rendering and displaying the user interface, obtain the second camera parameters corresponding to the rendering camera.

[0229] Optionally, the second camera parameters include the second view matrix and the second projection matrix of the rendering camera.

[0230] Step 440 has been described in the above steps 240 and 350, and will not be elaborated here.

[0231] Step 450: Obtain the spatial transformation parameters corresponding to the first scene area.

[0232] Schematically, the first scene area is the area where the first element is presented in the user interface. To facilitate the analysis of the process of converting the first element to the UI space based on the clipping coordinates corresponding to the first element, after determining the first scene area in the user interface presenting the UI space, use the first scene area as the area to which the first element needs to be mapped, and share the two-dimensional area information of the first scene area in the user interface with the first element, that is, share the two-dimensional area information of the first scene area with the first element, so that after the first element is mapped to the first scene area, it can accurately follow the first scene area for graphic change processes such as scaling and rotation, improving the expression accuracy of the first element presented in the user interface.

[0233] That is: The spatial transformation parameters are used to share the two-dimensional area information of the first scene area with the first element when the first element is projected into the two-dimensional space; or it can be said that the spatial transformation parameters are used to present the first element expressed in three dimensions into the first scene area expressed in two dimensions.

[0234] Among them, the two-dimensional area information is used to characterize the area transformation situation of the first scene area presented on the screen. Based on the screen being two-dimensional expression, the first scene area presented on the screen is also two-dimensional expression, and the two-dimensional area information is implemented as, for example, scaling information characterizing size transformation, rotation information, movement information, etc. characterizing position transformation.

[0235] Schematically, the two-dimensional space is the screen space displayed on the screen, and the space conversion parameter is used to characterize the two-dimensional region information of the first scene region on the user interface displayed on the screen. Through the space conversion parameter, the change situation during the graphic change process such as scaling and rotation of the first scene region can be determined.

[0236] In some embodiments, the space conversion parameter includes a scaling parameter and an offset parameter, where the scaling parameter is used to characterize the change situation when the first scene region is scaled; the offset parameter is used to characterize the change situation when the first scene region is moved, rotated, etc.

[0237] Schematically, after determining the space conversion parameter corresponding to the first scene region, the space conversion parameter corresponding to the first scene region is shared with the first element, that is: by obtaining the space conversion parameter, the region information represented by the first scene region is shared to the first element.

[0238] In addition, the first scene region can also be adjusted by assigning a depth value to have a three-dimensional effect, and the relevant content will be introduced below.

[0239] Step 460, based on the clipping coordinates, the second camera parameter, and the space conversion parameter, render the first element into the first scene region on the screen.

[0240] Schematically, first, the first element is initially projected into the first scene region based on the clipping coordinates and the space conversion parameter; then, the second camera parameter is used to fine-tune the initial projection result to render the first element into the first scene region of the user interface on the screen.

[0241] In an alternative embodiment, based on the clipping coordinates, the second camera parameter, and the space conversion parameter, the screen coordinates corresponding to the first element are obtained.

[0242] Schematically, the clipping coordinates are sequentially adjusted by the space transformation parameter and the second camera parameter to obtain the screen coordinates corresponding to the first element. The screen coordinates not only contain the region information of the clipping coordinates in the first scene region but also contain the capture situation when the rendering camera captures and renders the first element. Thus, from the two perspectives of the first scene region and the rendering camera, the situation of the screen coordinates of the first element presented in the user interface is predicted. It not only realizes the purpose of converting the first element in the viewport space to the UI space but also more specifically projects the first element into the first scene region of the user interface represented by the UI space, that is: accurately determines the screen coordinates when the first element is transformed to the UI space according to the coordinate transformation process.

[0243] In some embodiments, the second local coordinates of the first element in the two-dimensional space are obtained through the clipping coordinates and the space conversion parameter.

[0244] Schematically, the UI space is referred to as a two-dimensional space; after obtaining the clipping coordinates and the space transformation parameters corresponding to the first scene area, the clipping coordinates are adjusted by the space transformation parameters to initially bind the first element to the first scene area, that is, sharing the space transformation parameters of the first scene area to the first element.

[0245] Optionally, the space transformation parameters, as the parameters corresponding to the first scene area, focus on expressing two-dimensional information; the clipping coordinates, as the content obtained after matrix transformation of the first local coordinates for three-dimensional expression, focus on expressing three-dimensional information, such as the above v2; therefore, when adjusting the clipping coordinates by the space transformation parameters, differential processing is performed on the expressions of the clipping coordinates in different dimensions, and then the second local coordinates are obtained.

[0246] Among them, the second local coordinates represent the local coordinates of the first element when performing matrix transformation in the UI space; the above first local coordinates represent the local coordinates of the first element when performing matrix transformation in the viewport space. That is: the first local coordinates and the second local coordinates are the local coordinate expressions respectively corresponding to the first element in different spaces.

[0247] Optionally, the first horizontal parameter and the first vertical parameter in the clipping coordinates are adjusted by the space transformation parameters, and, the first depth parameter in the clipping coordinates is adjusted by the first value to obtain the second local coordinates of the first element in the two-dimensional space.

[0248] Schematically, the first horizontal parameter is the x-axis data in the clipping coordinates, and the first vertical parameter is the y-axis data in the clipping coordinates. For example, if the clipping coordinates are expressed as v2, it can be expressed in the form of four-dimensional components as v2 = (v 1，x / v 1，w , v 1，y / v 1，w , v 1，z / v 1，w , 1), and it can be expressed in the form of three-dimensional components as v2 = (v 1，x / v 1，w , v 1，y / v 1，w , v 1，z / v 1，w ). Then the first horizontal parameter is v 1，x / v 1，w , and it can also be expressed as v 2.x ; the first vertical parameter is v 1，y / v 1，w , and it can also be expressed as v 2.y .

[0249] Optionally, the spatial transformation parameters include a scaling parameter and an offset parameter. When adjusting the first horizontal parameter and the first vertical parameter in the cropping coordinates through the spatial transformation parameters, the first horizontal parameter is adjusted by the scaling parameter and the offset parameter, and the first vertical parameter is adjusted by the scaling parameter and the offset parameter.

[0250] Schematically, the first horizontal parameter and the first vertical parameter can be collectively referred to as v 2.xy ; taking the scaling parameter in the spatial transformation parameters as S and the offset parameter in the spatial transformation parameters as O as an example, the process of adjusting the first horizontal parameter and the first vertical parameter in the cropping coordinates through the spatial transformation parameters can be expressed as v 2.xy ×S + O.

[0251] In addition, the cropping coordinates further include a first depth parameter representing the depth situation, and the depth situation is used to represent the distance between the first element and the observation camera.

[0252] Schematically, the first depth parameter is the z-axis data in the cropping coordinates. If the cropping coordinates are expressed as v2 above, then the first depth parameter is v 1，z / v 1，w and can also be expressed as v 2.z .

[0253] Optionally, to reduce the expression of the first depth parameter in the cropping coordinates in the two-dimensional space, the first depth parameter in the cropping coordinates is adjusted with a first value. That is, the first value is used to constrain the depth situation of the first element in the two-dimensional space.

[0254] Schematically, the first value is 0. Adjusting the first depth parameter in the cropping coordinates with the first value is regarded as temporarily ignoring the distance situation between the first element and the rendering camera when transformed into the two-dimensional space. For example: taking the first value as the depth parameter in the second local coordinates.

[0255] Optionally, to facilitate the transformation of the second local coordinates, homogeneous coordinates can also be set for the second local coordinates. For example, the homogeneous coordinate is 1, so as to obtain the adjusted second local coordinates of the cropping coordinates through the above process. The second local coordinates can be expressed as v3. Among them, the second local coordinates v3 can also be expressed as the following formula one.

[0256] Formula One:

[0257]

[0258] In some embodiments, based on the second local coordinates and the second camera parameters, the screen coordinates corresponding to the first element are obtained.

[0259] Schematically, after obtaining the second local coordinates of the first element transformed into the two-dimensional space, referring to the way of obtaining the clipped coordinates through the first local coordinates and the first camera parameters, the second local coordinates are adjusted by the second camera parameters to obtain the screen coordinates for displaying the first element in the two-dimensional space.

[0260] In an optional embodiment, based on the second local coordinates and the second camera parameters, the second projection coordinates of the first element in the two-dimensional space are obtained.

[0261] In some embodiments, a second model matrix corresponding to the first scene area is obtained.

[0262] Among them, the first scene area is a scene area delimited based on the two-dimensional space and is used for presenting the interface of the first element. Therefore, the first scene area can be regarded as the element content in the two-dimensional space, similar to the first element being in a virtual scene.

[0263] Based on the fact that the first scene area is the element content in the two-dimensional space, the second model matrix corresponding to the first scene area can be determined, similar to determining the first model matrix corresponding to the first element.

[0264] In some embodiments, the second local coordinates are adjusted by the second model matrix to obtain the second world coordinates of the first element in the world coordinate system corresponding to the two-dimensional space.

[0265] Among them, the second model matrix is the model matrix corresponding to the first scene area and is used to adjust the second local coordinates determined by the first element through the clipped coordinates to determine the position of the first element in the world coordinate system corresponding to the two-dimensional control.

[0266] For example: the first local coordinates corresponding to the first element are represented as v0, and the first model matrix corresponding to the first element is represented as M1. Then, by adjusting the first local coordinates v0 with the first model matrix M1, the position of the first element in the world coordinate system can be determined.

[0267] Schematically, the product of the second local coordinates and the second model matrix is called the second world coordinates corresponding to the first element. The second local coordinates corresponding to the first element are represented as v3, and the second model matrix corresponding to the first scene area is represented as M2; then the second world coordinates are represented as M2×v3.

[0268] In some embodiments, the second camera parameters include the second view matrix and the second projection matrix corresponding to the rendering camera; the second world coordinates are adjusted by the second view matrix to obtain the second view coordinates in the view coordinate system corresponding to the two-dimensional space.

[0269] For example: Given that the second view matrix is represented as V2 and the second world coordinate is represented as M2×v3, then the second view coordinate is represented as V2×M2×v3.

[0270] In some embodiments, the second view coordinate is adjusted by the second projection matrix to obtain the second projection coordinate in the projection coordinate system corresponding to the two-dimensional space.

[0271] Schematically, the product of the second view coordinate and the second projection matrix is called the second projection coordinate corresponding to the first element, and the second projection coordinate is the position of the first element in the projection coordinate system corresponding to the two-dimensional space. For example: Given that the second projection matrix is represented as P2 and the first view coordinate is represented as V2×M2×v3, then the second projection coordinate is represented as P2×V2×M2×v3; if the second projection coordinate is represented as v4, then v4 = P2×V2×M2×v3.

[0272] In an alternative embodiment, the screen coordinate corresponding to the first element is obtained through the second projection coordinate and the clip coordinate.

[0273] Schematically, the second projection coordinate includes a second horizontal parameter, a second vertical parameter, a second depth parameter, and a second homogeneous coordinate, and the second depth parameter is a first value.

[0274] Wherein, the second projection coordinate is obtained after matrix transformation based on the second local coordinate. Based on the fact that the first depth parameter in the second local coordinate is the first value 0, the value of the second depth parameter in the second projection coordinate remains the first value 0.

[0275] Optionally, when obtaining the screen coordinate, the second horizontal parameter, the second vertical parameter, and the second homogeneous coordinate in the second projection coordinate are respectively used as the coordinate parameters of the corresponding dimensions in the screen coordinate, that is, the first coordinate parameter in the screen coordinate is determined.

[0276] Schematically, the second projection coordinate is represented as v4, where the second horizontal parameter is represented as v 4.x , the second vertical parameter is represented as v 4.y , and the second homogeneous coordinate is represented as v 4.w ; the screen coordinate is represented as v5, then the first coordinate parameter in the screen coordinate can be represented as v 5.xyw = v 4.xyw .

[0277] Optionally, the first depth parameter in the clip coordinate is adjusted by the second homogeneous coordinate in the second projection coordinate to obtain the second coordinate parameter in the screen coordinate.

[0278] Wherein, the first depth parameter is used to characterize the distance of the first element relative to the observation camera; the second coordinate parameter is used to characterize the distance of the first element relative to the rendering camera.

[0279] That is: The second homogeneous coordinate is used to adjust the first depth parameter of the first element relative to the observation camera to the second depth parameter of the first element relative to the rendering camera.

[0280] Schematically, the obtained clipping coordinates are the content obtained by projecting the first element into the two-dimensional space. Although when processing the second local coordinates through the clipping coordinates, considering that the two-dimensional space focuses more on the planar position information, the distance between the first element and the observation camera is temporarily ignored through the first value. However, the clipping coordinates, as the depth determined when projected into the two-dimensional space, can more accurately display the distance between the first element in the three-dimensional space and the observation camera that needs to observe the first element. Therefore, while more accurately displaying the position information of the first element through the screen coordinates, it is also necessary to determine the distance between the first element and the rendering camera with the help of the first depth parameter in the clipping coordinates, so that when presenting the user interface on the screen, not only the position of the first element can be displayed, but also the distance between the first element and the rendering camera can be displayed, achieving the purpose of enabling the user to observe a three-dimensional picture through the screen.

[0281] Optionally, determine the second homogeneous coordinate in the second projection coordinate. The second homogeneous coordinate is also the homogeneous coordinate in the screen coordinate and can be expressed as v 4.w or v 5.w ; Based on the fact that the second homogeneous coordinate represents the projection information included after adjusting the second local coordinates through the second camera parameters corresponding to the rendering camera, the second homogeneous coordinate is associated with the projection situation of the rendering camera.

[0282] Schematically, take the product of the second homogeneous coordinate and the first depth parameter in the clipping coordinates as the second coordinate parameter in the screen coordinate; for example: the second coordinate parameter in the screen coordinate can be expressed as v 5.z = v 5.z × v 2.z .

[0283] Among them, combining the first coordinate parameter and the second coordinate parameter to obtain the screen coordinate corresponding to the first element.

[0284] Schematically, based on the above content, the screen coordinate v5 can be represented by the following formula two.

[0285] Formula Two:

[0286]

[0287] In an optional embodiment, render the first element into the first scene area on the screen based on the screen coordinate.

[0288] Schematically, after determining the screen coordinates, the first element is rendered into the first scene area in the user interface, that is, into the first scene area on the screen.

[0289] It should be noted that the above is only a schematic example, and the embodiments of the present application are not limited thereto.

[0290] In summary, by using the first camera parameters and the first local coordinates to predict the display situation of the first element within the local observation viewport range, the first element is then displayed on the screen used for the interface rendering process through the cropping coordinates obtained after prediction, avoiding the cumbersome process of having to turn on the observation camera and perform scene rendering before the interface rendering process. By making full use of the first camera parameters to accurately predict the cropping coordinates, when performing interface rendering through the rendering camera, the second camera parameters and the predicted cropping coordinates are used to perform an efficient interface rendering process, greatly simplifying the rendering process, reducing the rendering overhead during the rendering process, and fully improving the rendering performance.

[0291] In the embodiments of the present application, the content of obtaining the screen coordinates through the cropping coordinates and the second camera parameters is introduced. After obtaining the cropping coordinates, it is first necessary to map the cropping coordinates to the UI space to obtain the second local coordinates transformed based on the rendering camera, thereby retaining the depth information represented in the cropping coordinates while reflecting the two-dimensional position information, to obtain the screen coordinates for rendering the first element; by means of transforming from the viewport space to the UI space, while ensuring the rendering accuracy of the first element, a rendering process with a smaller amount of rendering data is performed with the aid of the predicted cropping coordinates, improving the rendering efficiency.

[0292] In an optional embodiment, to avoid the problem that the first element exceeds the first scene area when being rendered onto the screen, a masking method is used to cover the first scene area to obtain a masked scene area, so as to perform a more accurate rendering process on the first element based on the relationship between the multiple mapped pixel points after mapping of the first element and the masked scene area. Schematically, as Figure 5 shown, step 250 shown above can also be implemented as the following steps 510 to 550. Figure 2

[0293] Step 510, cover the first scene area with a preset mask value to obtain a masked scene area.

[0294] Schematically, the preset mask value is a mask value set in advance. The mask value (Mask Value) is a numerical value used for masking processing, usually implemented as a binary mask, which is used to represent the valid bits or valid areas of a certain specific information or operation. For example: the preset mask value is implemented as binary numerical values 0, 1, or numerical values such as 4, 7, etc.​

[0295] Among them, the first scene area is the effective scene for rendering the first element, and the first scene area is covered by a preset mask value; among them, the first scene area is the scene area corresponding to the UI space, which includes a plurality of pixels. When covering the first scene area with the preset mask value, the plurality of pixels in the first scene area are covered with the preset mask value, so as to synthesize the covered pixels to obtain the masked scene area.

[0296] Optionally, each pixel in the first scene area is covered with a fixed preset mask value to obtain the masked scene area.

[0297] Among them, the masked scene area is used to define the rendering area of the first element.

[0298] Schematically, the first scene area is the target area for rendering the first element. The masked scene area obtained by the preset mask value is used to limit the rendering situation of the first element, so as to prevent the first element from exceeding the masked scene area during rendering, that is, to prevent the first element from exceeding the first scene area during rendering.

[0299] Step 520, based on the cropping coordinates and the second camera parameters, obtain the screen coordinates.

[0300] Schematically, after obtaining the cropping coordinates, the cropping coordinates are matrix-transformed by the second camera parameters, so as to obtain the screen coordinates that can map the first element to the UI space.

[0301] Among them, step 520 can refer to the above step 460, which will not be elaborated here.

[0302] Step 530, map the first element based on the screen coordinates to obtain a plurality of mapped pixels.

[0303] Schematically, after obtaining the screen coordinates, first, the first element is not directly rendered based on the screen coordinates, but the first element is first mapped to the user interface based on the screen coordinates, and a plurality of mapped pixels are obtained.

[0304] Optionally, the process of rendering the first element is to write the information corresponding to the first element into the content of the frame buffer.

[0305] Among them, the frame buffer is used to store and manage the output image or image frame of the graphics rendering pipeline, and is a specific memory area for temporarily storing image data during the graphics rendering process. The frame buffer can also be regarded as the rendering target, and the image data to be rendered is written into the frame buffer; so as to perform the process of rendering the first element to the terminal based on the image input in the frame buffer subsequently.

[0306] Schematically, the frame buffer includes at least one of the following buffers.

[0307] A color buffer for storing color information of rendered pixels. For each pixel in the image data, this information may include RGB (Red, Green, Blue) color values, Alpha (transparency) values, etc.

[0308] A depth buffer (Z - Buffer) for storing the depth value (or Z - value) of each pixel in the image data. The depth buffer is used to resolve the occlusion relationship of objects in the image and ensure the correct rendering order.

[0309] A stencil buffer for achieving more complex rendering effects, such as applying different rendering methods in specific areas or restricting the rendering area, etc.

[0310] Optionally, after determining the screen coordinates, instead of directly writing the pixels in the image data corresponding to the first element into the frame buffer based on the screen coordinates, a mapping process is first performed based on the screen coordinates, so as to subsequently determine whether to write the pixels in the image data into the frame buffer, thereby avoiding the problem of exceeding the first scene area.

[0311] In some embodiments, after mapping the first element, multiple mapped pixel points are obtained, and the multiple mapped pixel points are used to characterize the display area of the first element on the user interface, that is, the display area is composed of multiple mapped pixel points.

[0312] Step 540: Match the multiple mapped pixel points with a preset mask value to obtain matching results corresponding to the multiple mapped pixel points respectively.

[0313] Schematically, after obtaining the multiple mapped pixel points, the multiple mapped pixel points are matched with a preset mask value.

[0314] In some embodiments, determine the mapped mask values corresponding to the multiple mapped pixel points respectively.

[0315] Wherein, the mapped mask value is the mask value generated by the mapped pixel point in the graphics rendering pipeline.

[0316] Schematically, the mapped mask value corresponding to the mapped pixel point is usually generated at a specific stage in the graphics rendering pipeline; the mask values corresponding to different pixels can be generated in various ways, and the generation method depends on the application requirements and the graphics programming tools used.

[0317] Optionally, the following introduces some common methods for generating the mask value corresponding to the current pixel.

[0318] (1) Manually set the mask value: In some cases, the mask value corresponding to a pixel may be manually set. Especially when precise control over drawing or processing in a specific area is required, a mask value can be directly specified for each pixel.

[0319] (2) Generate the mask value using image processing techniques: Such as using image processing techniques like edge detection, color thresholding, segmentation algorithms, etc. to generate the mask value. This method usually bases on the content of the image. For example, pixels of different colors, textures, or shapes may have different mask values.

[0320] (3) Generate the mask value based on the depth information of the pixel: Such as using the depth information cached in the depth buffer to generate the mask value; depth information is usually used to determine the relative distance of elements in the image, so the mask value can be generated according to the depth value.

[0321] (4) Generate the mask value based on conditional judgment: Such as testing the current pixel according to specific conditions and generating the mask value according to the test results; this includes making conditional judgments on at least one of multiple conditions such as color, normal direction, illumination, etc.

[0322] That is: The mask value of a pixel is the content determined according to requirements and application scenarios; in graphics programming, it is usually executed at different stages of the rendering pipeline. For example, the mask value is generated in the fragment shader for subsequent mask testing.

[0323] Optionally, the mapped mask value corresponding to the mapped pixel point is the mask value generated based on image processing techniques. After determining multiple mapped pixel points, the mapped mask values corresponding to the multiple mapped pixel points are determined according to the adopted image processing techniques.

[0324] In some embodiments, multiple mapped mask values are matched with a preset mask value to obtain matching results corresponding to the multiple mapped pixel points respectively.

[0325] Optionally, the matching result indicates that the mapped mask value is the same as the preset mask value; or, the matching result indicates that the mapped mask value is different from the preset mask value.

[0326] Step 550, based on the matching result, render the first element into the first scene area on the screen.

[0327] Schematically, according to the matching situation between the mapped mask value and the preset mask value indicated by the matching result, differential processing is performed on multiple mapped pixel points.

[0328] In an optional embodiment, in response to the first matching result indicating that the first mapped mask value is the same as the preset mask value, write the first mapped pixel point corresponding to the first mapped mask value into the frame buffer.

[0329] Schematically, the first matching result represents the matching result between the first mapped mask value corresponding to the first mapped pixel point and a preset mask value; the first mapped pixel point is any one of a plurality of mapped pixel points.

[0330] Among them, the frame buffer is used to store and manage the output image of the graphics rendering pipeline, and is a memory area for temporarily storing image data during the graphics rendering process.

[0331] In an optional embodiment, based on the mapped pixel points cached in the frame buffer, the first element is rendered into the first scene area.

[0332] Schematically, after determining the screen coordinates, first determine a plurality of mapped pixel points, and only when the mapped mask value corresponding to the mapped pixel point is the same as the preset mask value, will the mapped pixel point be written into the frame buffer for subsequent rendering processes, avoiding the problem that the first element exceeds the first scene area.

[0333] Such as Figure 6 As shown, it is an image directly obtained by writing the mapped pixel points corresponding to the first element based on the screen coordinates and rendering, where the first element is the virtual puppy 610, and the virtual puppy 610 needs to be rendered into the first scene area 620; if the virtual puppy 610 is directly rendered based on the screen coordinates, part of the content of the feet 611 of the virtual puppy will exceed the first scene area 620.

[0334] If the mapped mask values corresponding to the mapped pixel points of the first element are first numerically compared with the preset mask value, and then the writing process to the frame buffer is performed according to the matching result, it will control that only the mapped pixel points mapped into the first scene area will be written into the frame buffer that needs to perform subsequent rendering processes, and control that the mapped pixel points mapped outside the first scene area will not be written into the frame buffer, thereby effectively avoiding the problem that the first element exceeds the first scene area when rendering based on the frame buffer.

[0335] In an optional embodiment, the preset mask value covered in the masked scene area is erased to restore the first scene area.

[0336] Schematically, since the first scene area was previously covered with the preset mask value, in order to avoid the influence of the coverage of the preset mask value during rendering, first process the masked scene area covered with the preset mask value to erase the preset mask values respectively covered by multiple pixel points in the masked scene area, thereby restoring the first scene area.

[0337] In an optional embodiment, the depth nearest value corresponding to the depth buffer in the frame buffer is determined.

[0338] Schematically, in the depth buffer, depth values are typically mapped to a normalized range, and a typical interval range is implemented as [0.0, 1.0]. The interval range is usually used to represent the nearest distance to the farthest distance. The nearest distance is also called the nearest depth value and is used to represent the situation where an element is closest to the camera; the farthest distance is also called the farthest depth value and is used to represent the situation where an element is farthest from the camera.

[0339] Optionally, within the above interval range, the depth value 0.0 can be represented as the closest distance between the element and the camera, that is, 0.0 can be called the nearest depth value. Correspondingly, the depth value 1.0 can be represented as the farthest distance between the element and the camera, that is, 1.0 can be called the farthest depth value; or, the depth value 1.0 can also be represented as the closest distance between the element and the camera, that is, 1.0 can be called the nearest depth value. Correspondingly, the depth value 0.0 can be represented as the farthest distance between the element and the camera, that is, 0.0 can be called the farthest depth value.

[0340] That is: Based on the setting difference, when determining the nearest depth value, the nearest depth value is determined based on the setting of the depth buffer. This nearest depth value represents the closest distance between the element and the camera, and it may be implemented as 0.0 or 1.0.

[0341] In an optional embodiment, the depth values corresponding to multiple regional pixel points within the first scene area are respectively set to the nearest depth value.

[0342] Schematically, regional pixel points are used to represent the pixel points within the first scene area, that is, the pixel points within the first scene area are called regional pixel points. Among the multiple regional pixel points, it includes partial mapped pixel points of the first element mapped into the first scene area, and also includes other pixel points within the first scene area except the mapped pixel points.

[0343] Optionally, after determining the multiple regional pixel points within the first scene area, based on parameters such as the color value, brightness value, and depth value respectively corresponding to each pixel point, the depth values corresponding to the multiple regional pixel points can be adjusted with the nearest depth value, that is, the depth values corresponding to the multiple regional pixel points located within the first scene area are set to the nearest depth value, so that the first scene area has stronger coordination.

[0344] In an optional embodiment, based on multiple regional pixel points cached in the frame buffer, the first element is rendered into the first scene area.

[0345] Schematically, the process of writing mapped pixels corresponding to the first element among multiple regional pixels into the frame buffer, as well as the depth value determination process, can accurately render the first element; it can also accurately render the background other than the first element within the first scene area based on the setting process of color values, brightness values, etc. of other pixels among multiple regional pixels, and the above-mentioned depth value determination process. That is: realizing the process of rendering the first element to the first scene area.

[0346] As Figure 7 shown, based on the matching result and the depth value setting process, an interface schematic diagram of accurately rendering the first element into the first scene area is obtained. Among them, the first element is the virtual puppy 710, and it is necessary to render the virtual puppy 710 into the first scene area 720; compared with Figure 6 shown, the mapped pixels in the feet 611 of the virtual puppy that exceed the first scene area 620 are not written into the frame buffer, so that there will be no problem of exceeding the first scene area 720 in Figure 7 , improving the rendering accuracy.

[0347] It should be noted that the above is only a schematic example, and the embodiments of the present application are not limited thereto.

[0348] In summary, by predicting the display situation of the first element within the local observation viewport range through the first camera parameter and the first local coordinate, the first element is displayed on the screen used for the interface rendering process through the cropping coordinates obtained after prediction, avoiding the cumbersome process of having to turn on the observation camera and perform scene rendering before the interface rendering process. Making full use of the first camera parameter to accurately predict the cropping coordinates, so that when performing interface rendering through the rendering camera, the second camera parameter and the predicted cropping coordinates are used to perform an efficient interface rendering process, greatly simplifying the rendering process, reducing the rendering overhead during the rendering process, and fully improving the rendering performance.

[0349] In the embodiments of the present application, the content of restricting the rendering area of the first element by covering the preset mask value is introduced. To prevent the first element from being rendered outside the first scene area, the mapped mask values corresponding to the multiple mapped pixels after mapping the first element are respectively matched with the preset mask value, and based on the matching result, it is determined whether to write the mapped pixels into the frame buffer for the rendering process, making the rendering process based on the frame buffer more accurate and effectively achieving the purpose of rendering the first element into the first scene area.

[0350] In an alternative embodiment, the above-mentioned element rendering method can also be called a high-performance user interface sub-viewport rendering method, which is described through the following two parts.

[0351] (1) Technical content

[0352] Schematically, in the related art, the rendering process is as Figure 8 shown.

[0353] First, the main scene camera performs main scene rendering 810 to a frame buffer; then the sub-scene camera performs sub-scene rendering 820 to another frame buffer, and then this frame buffer is converted into a texture so as to read out the texture data and input it into the UI scene (UI space) rendering stage to be rendered into the frame buffer corresponding to the main scene rendering 810, and finally the rendering result is obtained and output.

[0354] In the method provided by the embodiments of the present application, the rendering process is changed through a script and a set of shaders on the technical side. As Figure 9 shown, it is the flowchart of the rendering process in the embodiments of the present application.

[0355] Schematically, compared with the related art, the sub-scene camera is turned off, and the sub-scene rendering 910 is completed between the main scene and the UI through the coordinate transformation process and the clipping process; thus, all the renderings occur in the same frame buffer, which can greatly improve the rendering efficiency of the GPU. Optionally, the technical details of the implementation process are described from three aspects of coordinate transformation, clipping, and rendering process as follows.

[0356] (1) Coordinate transformation

[0357] Schematically, in the process of coordinate transformation, the rendering coordinate transformation of the first element in the sub-scene is the most critical technology, and the sub-view (SubViewShader) shader is also implemented according to this technical detail, and its transformation process is as Figure 10 shown.

[0358] Let the local coordinate 1010 (the first local coordinate) of the first element be v0, the first model matrix be M1, the first view matrix and the first projection matrix of the sub-camera (the above-mentioned observation camera) be V1 and P1 respectively; the scaling of the UI element (the first scene area) be S, the offset be O, the second model matrix of the UI element be M2, and the second view matrix and the second projection matrix of the UI camera (the above-mentioned rendering camera) be V2 and P2 respectively. Then, the sub-viewport projection coordinate (the first projection coordinate 1020) of the first element is expressed as: v1 = P1 × V1 × M1 × v0.

[0359] Where v0 is a four-dimensional vector with homogeneous coordinate 1, and v1 is also a four-dimensional homogeneous coordinate, then the sub-viewport clipping coordinate v2 is expressed as: v2 = v1 / v1.w.

[0360] Next, it is necessary to transform into the UI space. Since the UI elements have scaling and offset in the layout, the local coordinates v3 of the UI elements need to be calculated first. Note that the transformation is only in the two-dimensional space, and the depth and homogeneous coordinates are set to 0 and 1, so as to obtain the above-mentioned second local coordinates v3.

[0361] After the setting is completed, the second local coordinates v3 are transformed into the projection space of the UI camera, and the second projection coordinates 1030v4 are obtained, which is expressed as: v4 = P2 × V2 × M2 × v3.

[0362] At this time, the depth information of the second projection coordinates 1030v4 is lost, and direct rendering will cause incorrect depth relationship of the model. Therefore, depth correction is required, and the output coordinates are obtained, that is, the above-mentioned screen coordinates v5 are obtained.

[0363] At this time, the screen coordinates v5 will obtain v2.z after perspective division, which is consistent with the sub-viewport rendering, and the correct depth relationship consistent with the direct rendering of the sub-camera can be obtained to ensure correct rendering.

[0364] (2) Clipping

[0365] Although the coordinate transformation process successfully transforms the model to be drawn into the projection coordinate system of the UI camera and solves the depth relationship of the first element itself. But there is still a problem when directly writing the first element to the frame buffer at this time. It may exceed the sub-viewport rendering area during rendering, as Figure 6 shown.

[0366] Therefore, two drawing processes are required before and after rendering the first element, corresponding to the implementation details before sub-view rendering (PreSubViewShader) and after sub-view rendering (PostSubViewShader) respectively.

[0367] In the PreSubViewShader, the shader will update the sub-viewport area in the screen with a fixed mask value (preset mask value) and output the color as the background color defined by the sub-camera to implement the process of sub-camera clear.

[0368] Next, the first element is drawn into the sub-viewport area. Note that the mask test is enabled here, and it is required that the mask value of the current pixel (the mapped mask value corresponding to the mapped pixel point) is the same as the just-fixed mask value to be able to perform pixel drawing. This step enables the first element to be drawn within a specific area (within the first scene area).

[0369] Then, use the PostSubViewShader to erase the mask value in the sub-viewport area of the screen and set the depth value of the current area to the nearest value of the buffer depth. This can ensure that UI elements will not affect this area during subsequent drawing processes. Thus, the correct drawing effect as shown in Figure 7 is obtained.

[0370] (3) Rendering process

[0371] The control of the rendering process is implemented in the script space transformation (SubViewToUI). The script will execute the flowchart as shown in Figure 11 during each frame update.

[0372] 1110. Set the aspect ratio of the sub-camera; 1120. Obtain the first view matrix and the first projection matrix of the sub-camera; 1130. Calculate the first projection coordinates and set the shader parameters; 1140. Set the shader parameters to S and O; 1150. Draw a unit rectangle using the pre-sub-view rendering process, that is, cover the first scene area with a preset mask value; 1160. Draw the sub-scene model using the post-sub-view rendering process; 1170. Draw a unit rectangle using the post-sub-view rendering process, that is, erase the preset mask value and restore the first scene area.

[0373] Schematically, calculate the matrices required for coordinate transformation and set multiple parameters for the shader. Then, through the three draws described in 1150, 1160, and 1170 above, the most real-time rendering effect can be ensured during drawing.

[0374] (2) Content on the interface side

[0375] Schematically, executing the above technical content in the form of a Unity resource package can be quickly deployed to products with both 3D scene and 2D user interface rendering. The following is the deployment method.

[0376] First, as shown in Figure 12 , introduce the Unity resource package to obtain three shader resources 1210 and one script resource 1220.

[0377] After that, as shown in Figure 13As shown in the figure, a scene is set up, which includes a directional light 1310, a main scene 1320 with a main camera and multiple scene rendering objects; in addition, there is a sub-scene, which includes a sub-camera Sub Camera and several objects to be rendered. In this scene, the name of the object is Object (such as the first element). Finally, there is a UI Camera responsible for rendering the user interface (UI) scene and the layout (Canvas) of the UI scene.

[0378] As Figure 14 shown, the scene result 1410 rendered for the main scene; as Figure 15 shown, the object result rendered by the sub-camera is a virtual puppy 1510, that is, the first element is the virtual puppy 1510.

[0379] As Figure 16 shown, there is a UI element 1610 reserved in the UI layout to display the rendering result of the sub-perspective. The black area is the border, and the white area is the actual filling area of the sub-perspective. The UI element 1610 can be regarded as the first scene area.

[0380] Next, it is necessary to render the content of the sub-perspective onto the reserved UI element and ensure that the rendering content of the sub-perspective does not exceed the white area in the UI element; first, create a UI element SubView aligned with the white area, add a script SubViewToUI, and set the parameters of the script, as Figure 17 shown.

[0381] Among them, the in-object parameter 1710 binds the object to be rendered to the sub-viewport, the sub-camera parameter 1720 binds the sub-camera, and the user interface camera (UI Camera) parameter binds the UI camera. The subsequent three shaders have been automatically set and do not need to be manually bound. Since the SubViewToUI script will submit the in-object parameter 1710 to the UI camera for rendering when running, the sub-camera does not need to be rendered separately, and it can be selected to be turned off in the scene to save rendering overhead.

[0382] As Figure 18 shown, it is the interface result diagram 1810 of the process of completing the script parameter setting; after the setting is completed and the scene is run, as Figure 19 shown, it can be seen that the virtual puppy 1910 that the sub-camera needs to render has been successfully rendered into the white area in the UI interface, that is: the schematic diagram of successfully rendering the first element into the first scene area.

[0383] At this time, modifying the camera parameters of the sub-camera (such as the above first camera parameters) or modifying the transformation parameters of the elements in the sub-scene (such as the above first model matrix), it can be seen that the rendering result of the sub-viewport can be updated in real time, meeting the real-time rendering requirements of the sub-viewport.

[0384] In addition, view the rendering process through the Frame Debugger window provided by Unity as Figure 20 shown. It can be seen that when drawing the objects in the sub-scene, the render target 2010 "Nameless" - Noname is the same as the render targets corresponding to the main camera and the UI camera. This shows that the sub-viewport rendering is not achieved by creating additional render targets and can meet the requirements with extremely low rendering overhead.

[0385] In the above process, when deploying the interface content, compared with the unoptimized rendering process, it can have smaller GPU rendering overhead, GPU memory occupancy, and GPU bandwidth read and write, can improve the overall rendering performance, and is convenient to deploy, flexible and controllable.

[0386] To sum up, the display situation of the first element within the local observation viewport range is predicted by the first camera parameters and the first local coordinates, so that the first element is displayed on the screen used to execute the interface rendering process through the cropping coordinates obtained after prediction, avoiding the cumbersome process of having to turn on the observation camera and perform scene rendering before the interface rendering process. The first camera parameters are fully utilized to accurately predict the cropping coordinates. Thus, when performing interface rendering through the rendering camera, the second camera parameters and the predicted cropping coordinates are used to execute an efficient interface rendering process, greatly simplifying the rendering process, reducing the rendering overhead during the rendering process, and fully improving the rendering performance.

[0387] In the embodiment of the present application, a high-performance user interface sub-viewport rendering method is proposed. Based on coordinate transformation and mask cropping functions, the first element of the sub-scene is transformed into the UI space according to the first camera parameters of the observation camera and is rendered through the rendering camera. Compared with the related technology, it has extremely low rendering overhead, saves a large amount of reading and writing of GPU render targets, saves the memory of the render target, and improves the rendering efficiency. And it can ensure the rendering real-time and rendering quality under the condition of the same rendering effect.

[0388] In addition, in addition to the above rendering process for a single sub-viewport, the multi-viewport rendering commonly seen in information visualization applications can also apply the method mentioned above. In multi-viewport rendering, it is rendered in an instantiated manner, and has a good rendering optimization effect.

[0389] In addition, the first scene area in the user interface required in the application scenario can be implemented as a rectangular area or a non-rectangular area, and only a small amount of adaptation work is required to complete the deployment. Therefore, the above method can be widely applied to various scenarios, bringing considerable performance optimization effects with almost no additional overhead, having high flexibility and compatibility, and being able to be extended to run in various platform environments.

[0390] Figure 21 is a structural block diagram of an element rendering device provided by an exemplary embodiment of the present application. As Figure 21 shown, the device includes the following parts:

[0391] An acquisition module 2110, configured to acquire a first local coordinate corresponding to a first element in a virtual scene, where the first local coordinate is a coordinate expression determined based on an element coordinate system of the first element;

[0392] The acquisition module 2110 is further configured to acquire first camera parameters corresponding to an observation camera, where the observation camera is configured in the virtual scene for scene observation, and the first element is within an observation viewport range of the observation camera;

[0393] A mapping module 2120, configured to map, based on the first camera parameters and the first local coordinate, to obtain a clipping coordinate of the first element, where the clipping coordinate is a coordinate representing position information based on the observation viewport range;

[0394] The acquisition module 2110 is further configured to acquire second camera parameters corresponding to a rendering camera, where the rendering camera is used to render a first scene area on a screen;

[0395] A rendering module 2130, configured to render the first element into the first scene area on the screen based on the clipping coordinate and the second camera parameters.

[0396] In an optional embodiment, the mapping module 2120 is further configured to obtain, based on the first camera parameters and the first local coordinate, a first projection coordinate for projecting the first element onto a projection coordinate system, where the projection coordinate system is a coordinate system established when projecting the observation viewport range onto a two-dimensional plane; and map based on the first projection coordinate to obtain the clipping coordinate of the first element.

[0397] In an optional embodiment, the mapping module 2120 is further configured to determine a first homogeneous coordinate in the first projection coordinate, where the first homogeneous coordinate is a coordinate dimension in the first projection coordinate used to represent the projection information of the first element; and use the quotient of the first projection coordinate and the first homogeneous coordinate as the clipping coordinate corresponding to the first element.

[0398] In an alternative embodiment, the rendering module 2130 is further configured to obtain spatial transformation parameters corresponding to the first scene area, where the spatial transformation parameters are used to share the two-dimensional area information of the first scene area with the first element when the first element is projected onto the two-dimensional space; based on the clipping coordinates, the second camera parameters, and the spatial transformation parameters, render the first element into the first scene area on the screen.

[0399] In an alternative embodiment, the rendering module 2130 is further configured to obtain the screen coordinates corresponding to the first element based on the clipping coordinates, the second camera parameters, and the spatial transformation parameters; and render the first element into the first scene area on the screen based on the screen coordinates.

[0400] In an alternative embodiment, the rendering module 2130 is further configured to obtain second local coordinates of the first element in the two-dimensional space through the clipping coordinates and the spatial transformation parameters; and obtain the screen coordinates corresponding to the first element based on the second local coordinates and the second camera parameters.

[0401] In an alternative embodiment, the rendering module 2130 is further configured to adjust a first horizontal parameter and a first vertical parameter in the clipping coordinates through the spatial transformation parameters, and adjust a first depth parameter in the clipping coordinates through a first value, to obtain second local coordinates of the first element in the two-dimensional space; where the first value is used to constrain the depth condition of the first element in the two-dimensional space.

[0402] In an alternative embodiment, the rendering module 2130 is further configured to obtain second projection coordinates of the first element in the two-dimensional space based on the second local coordinates and the second camera parameters; and obtain the screen coordinates corresponding to the first element through the second projection coordinates and the clipping coordinates.

[0403] In an alternative embodiment, the second projection coordinates include a second horizontal parameter, a second vertical parameter, a second depth parameter, and a second homogeneous coordinate, and the second depth parameter is the first value;

[0404] The rendering module 2130 is further configured to use the second horizontal parameter, the second vertical parameter, and the second homogeneous coordinate in the second projection coordinates as the coordinate parameters of the corresponding dimensions in the screen coordinates, respectively, to obtain the first coordinate parameters in the screen coordinates; adjust the first depth parameter in the clipping coordinates through the second homogeneous coordinate in the second projection coordinates to obtain the second coordinate parameters in the screen coordinates, where the first depth parameter is used to characterize the distance of the first element relative to the observation camera, and the second coordinate parameters are used to characterize the distance of the first element relative to the rendering camera; synthesize the first coordinate parameters and the second coordinate parameters to obtain the screen coordinates corresponding to the first element.

[0405] In an alternative embodiment, the rendering module 2130 is further configured to cover the first scene area with a preset mask value to obtain a masked scene area, where the masked scene area is used to define the rendering area of the first element; obtain screen coordinates based on the clipping coordinates and the second camera parameters; map the first element based on the screen coordinates to obtain a plurality of mapped pixel points; match the plurality of mapped pixel points with the preset mask value to obtain matching results respectively corresponding to the plurality of mapped pixel points; and render the first element into the first scene area in the screen based on the matching results.

[0406] In an alternative embodiment, the rendering module 2130 is further configured to determine the mapped mask values respectively corresponding to the plurality of mapped pixel points, where the mapped mask value is the mask value generated by the mapped pixel point in the graphics rendering pipeline; and match the plurality of mapped mask values with the preset mask value to obtain the matching results respectively corresponding to the plurality of mapped pixel points.

[0407] In an alternative embodiment, the rendering module 2130 is further configured to, in response to a first matching result indicating that a first mapped mask value is the same as the preset mask value, write the first mapped pixel point corresponding to the first mapped mask value into a frame buffer, where the frame buffer is a memory area for temporarily storing image data; and render the first element into the first scene area based on the mapped pixel points cached in the frame buffer.

[0408] In an optional embodiment, the rendering module 2130 is further configured to erase the preset mask value covered in the mask scene area to restore the first scene area; determine the depth nearest value corresponding to the depth buffer in the frame buffer, where the depth nearest value is determined based on the setting of the depth buffer; set the depth values corresponding to multiple regional pixel points in the first scene area to the depth nearest value, where the regional pixel points are used to represent the pixel points in the first scene area; and render the first element into the first scene area based on the multiple regional pixel points cached in the frame buffer.

[0409] In summary, the display situation of the first element within the local observation viewport range is predicted through the first camera parameters and the first local coordinates, so that the first element is displayed on the screen for performing the interface rendering process through the obtained clipping coordinates after prediction, avoiding the cumbersome process of having to turn on the observation camera and perform scene rendering before the interface rendering process. The first camera parameters are fully utilized to accurately predict the clipping coordinates, so that when performing interface rendering through the rendering camera, the second camera parameters and the predicted clipping coordinates are used to perform an efficient interface rendering process, greatly simplifying the rendering process, reducing the rendering overhead during the rendering process, and fully improving the rendering performance.

[0410] It should be noted that: for the element rendering device provided in the above embodiment, only the above division of each functional module is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the element rendering device provided in the above embodiment and the element rendering method embodiment belong to the same concept, and the specific implementation process can be seen in the method embodiment, which will not be elaborated here.

[0411] Figure 22 The structural schematic diagram of a server provided by an exemplary embodiment of the present application is shown. The server 2200 includes a central processing unit (CPU) 2201, a system memory 2204 including a random access memory (RAM) 2202 and a read only memory (ROM) 2203, and a system bus 2205 connecting the system memory 2204 and the central processing unit 2201. The server 2200 further includes a mass storage device 2206 for storing an operating system 2213, an application program 2214, and other program modules 2215.

[0412] The mass storage device 2206 is connected to the central processing unit 2201 through a mass storage controller (not shown) connected to the system bus 2205. The mass storage device 2206 and its associated computer-readable medium provide non-volatile storage for the server 2200. That is to say, the mass storage device 2206 may include a computer-readable medium (not shown) such as a hard disk or a Compact Disc Read Only Memory (CD-ROM) drive.

[0413] Without loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. The above-mentioned system memory 2204 and mass storage device 2206 may be collectively referred to as memory.

[0414] According to various embodiments of the present application, the server 2200 may also be run by connecting to a remote computer on the network through a network such as the Internet. That is, the server 2200 may be connected to the network 2212 through the network interface unit 2211 connected to the system bus 2205, or in other words, the network interface unit 2211 may also be used to connect to other types of networks or remote computer systems (not shown).

[0415] The above-mentioned memory further includes one or more programs, and one or more programs are stored in the memory and are configured to be executed by the CPU.

[0416] Embodiments of the present application also provide a computer device, which includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory, and at least one instruction, at least one program, a code set, or an instruction set is loaded and executed by the processor to implement the element rendering method provided by the above-mentioned method embodiments.

[0417] Embodiments of the present application also provide a computer-readable storage medium, on which at least one instruction, at least one program, a code set, or an instruction set is stored, and at least one instruction, at least one program, a code set, or an instruction set is loaded and executed by the processor to implement the element rendering method provided by the above-mentioned method embodiments.

[0418] Embodiments of the present application also provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the element rendering method described in any one of the foregoing embodiments.

[0419] The foregoing are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An element rendering method, characterized in that, The method includes: Obtaining a first local coordinate corresponding to a first element in a virtual scene, where the first local coordinate is a coordinate expression determined based on the element coordinate system of the first element; Obtaining first camera parameters corresponding to an observation camera, where the observation camera is configured in the virtual scene for scene observation, and the first element is within the observation viewport of the observation camera; Mapping to obtain a clipping coordinate of the first element based on the first camera parameters and the first local coordinate, where the clipping coordinate is a coordinate representing position information based on the observation viewport range; Obtaining second camera parameters corresponding to a rendering camera, where the rendering camera is used to render a first scene area on a screen; Rendering the first element into the first scene area on the screen based on the clipping coordinate and the second camera parameters.

2. The method according to claim 1, characterized in that The mapping to obtain the clipping coordinate of the first element based on the first camera parameters and the first local coordinate includes: Based on the first camera parameters and the first local coordinate, obtaining a first projection coordinate for projecting the first element onto a projection coordinate system, where the projection coordinate system is a coordinate system established when projecting the observation viewport range onto a two-dimensional plane; Mapping to obtain the clipping coordinate of the first element based on the first projection coordinate.

3. The method according to claim 2, wherein The mapping to obtain the clipping coordinate of the first element based on the first projection coordinate includes: Determining a first homogeneous coordinate in the first projection coordinate, where the first homogeneous coordinate is the coordinate dimension in the first projection coordinate used to represent the projection information of the first element; Taking the quotient of the first projection coordinate and the first homogeneous coordinate as the clipping coordinate corresponding to the first element.

4. The method according to any one of claims 1 to 3, characterized in that The rendering of the first element into the first scene area on the screen based on the clipping coordinate and the second camera parameters includes: Obtaining a spatial transformation parameter corresponding to the first scene area, where the spatial transformation parameter is used to share the two-dimensional area information of the first scene area with the first element when the first element is projected onto a two-dimensional space; Rendering the first element into the first scene area on the screen based on the clipping coordinate, the second camera parameters, and the spatial transformation parameter.

5. The method according to claim 4, characterized in that, The rendering of the first element into the first scene area on the screen based on the clipping coordinate, the second camera parameters, and the spatial transformation parameter includes: Based on the clipping coordinate, the second camera parameters, and the spatial transformation parameter, obtaining the screen coordinate corresponding to the first element; Rendering the first element into the first scene area on the screen based on the screen coordinate.

6. The method according to claim 5, characterized in that, The obtaining of the screen coordinate corresponding to the first element based on the clipping coordinate, the second camera parameters, and the spatial transformation parameter includes: Obtaining a second local coordinate of the first element in the two-dimensional space through the clipping coordinate and the spatial transformation parameter; Based on the second local coordinate and the second camera parameters, obtaining the screen coordinate corresponding to the first element.

7. The method according to claim 6, characterized in that Obtaining the second local coordinates of the first element in the two-dimensional space through the clipping coordinates and the spatial transformation parameters includes: Adjusting the first horizontal parameter and the first vertical parameter in the clipping coordinates through the spatial transformation parameters, and adjusting the first depth parameter in the clipping coordinates through a first value to obtain the second local coordinates of the first element in the two-dimensional space; wherein, the first value is used to constrain the depth condition of the first element in the two-dimensional space.

8. The method according to claim 6, wherein Obtaining the screen coordinates corresponding to the first element based on the second local coordinates and the second camera parameters includes: Obtaining the second projection coordinates of the first element in the two-dimensional space based on the second local coordinates and the second camera parameters; Obtaining the screen coordinates corresponding to the first element through the second projection coordinates and the clipping coordinates.

9. The method according to claim 8, wherein The second projection coordinates include a second horizontal parameter, a second vertical parameter, a second depth parameter, and a second homogeneous coordinate, and the second depth parameter is the first value; Obtaining the screen coordinates corresponding to the first element through the second projection coordinates and the clipping coordinates includes: Taking the second horizontal parameter, the second vertical parameter, and the second homogeneous coordinate in the second projection coordinates as the coordinate parameters of the corresponding dimensions in the screen coordinates respectively to obtain the first coordinate parameters in the screen coordinates; Adjusting the first depth parameter in the clipping coordinates through the second homogeneous coordinate in the second projection coordinates to obtain the second coordinate parameters in the screen coordinates, where the first depth parameter is used to represent the distance condition of the first element relative to the observation camera, and the second coordinate parameter is used to represent the distance condition of the first element relative to the rendering camera; Combining the first coordinate parameters and the second coordinate parameters to obtain the screen coordinates corresponding to the first element.

10. The method according to any one of claims 1 to 3, characterized in that, Rendering the first element into the first scene area on the screen based on the clipping coordinates and the second camera parameters includes: Covering the first scene area with a preset mask value to obtain a masked scene area, and the masked scene area is used to define the rendering area of the first element; Obtaining screen coordinates based on the clipping coordinates and the second camera parameters; Mapping the first element based on the screen coordinates to obtain a plurality of mapped pixel points; Matching the plurality of mapped pixel points with the preset mask value to obtain matching results corresponding to the plurality of mapped pixel points respectively; Rendering the first element into the first scene area on the screen based on the matching results.

11. The method according to claim 10, wherein Matching the plurality of mapped pixel points with the preset mask value to obtain matching results corresponding to the plurality of mapped pixel points respectively includes: Determining the mapped mask values corresponding to the plurality of mapped pixel points respectively, where the mapped mask value is the mask value generated by the mapped pixel point in the graphics rendering pipeline; Match multiple mapped mask values with the preset mask value to obtain the matching results respectively corresponding to the multiple mapped pixel points.

12. The method according to claim 10, wherein Based on the matching results, rendering the first element into the first scene area on the screen includes: In response to the first matching result indicating that the first mapped mask value is the same as the preset mask value, writing the first mapped pixel point corresponding to the first mapped mask value into a frame buffer, where the frame buffer is a memory area for temporarily storing image data; Based on the mapped pixel points cached in the frame buffer, rendering the first element into the first scene area.

13. The method according to claim 12, wherein Based on the mapped pixel points cached in the frame buffer, rendering the first element into the first scene area includes: Erase the preset mask value covered in the mask scene area to restore the first scene area; Determine the depth nearest value corresponding to the depth buffer in the frame buffer, where the depth nearest value is determined based on the setting of the depth buffer; Set the depth values respectively corresponding to multiple area pixel points in the first scene area to the depth nearest value, where the area pixel points are used to represent the pixel points in the first scene area; Based on the multiple area pixel points cached in the frame buffer, render the first element into the first scene area.

14. An element rendering device, characterized in that, The device includes: An acquisition module, configured to acquire a first local coordinate corresponding to a first element in a virtual scene, where the first local coordinate is a coordinate expression determined based on the element coordinate system of the first element; The acquisition module is further configured to acquire first camera parameters corresponding to an observation camera, where the observation camera is configured in the virtual scene for scene observation, and the first element is within the observation viewport of the observation camera; A mapping module, configured to map the first element to obtain a clipped coordinate based on the first camera parameters and the first local coordinate, where the clipped coordinate is a coordinate representing position information based on the observation viewport range; The acquisition module is further configured to acquire second camera parameters corresponding to a rendering camera, where the rendering camera is used to render a first scene area on the screen; A rendering module, configured to render the first element into the first scene area on the screen based on the clipped coordinate and the second camera parameters.

15. A computer device, characterized in that, The computer device includes a processor and a memory, where at least one program is stored in the memory, and the at least one program is loaded and executed by the processor to implement the element rendering method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, At least one program is stored in the storage medium, and the at least one program is loaded and executed by a processor to implement the element rendering method according to any one of claims 1 to 13.

17. A computer program product, characterized in that, Including computer instructions, where the computer instructions, when executed by a processor, implement the element rendering method according to any one of claims 1 to 13.

Images