A method, apparatus, device, medium, and program product for occlusion culling

Abstract

Embodiments of the present application provide a kind of occlusion culling method, device, equipment, medium and program product, for efficiently and accurately the vertex in grid model is carried out occlusion culling, to reduce rendering amount.Therefore, it can be applied to computer, game rendering, video rendering and other fields.It includes: obtaining the grid model to be processed;Obtain the graphical feature representation of each vertex in the grid model to be processed;Based on multiple vertices and the connection relationship of multiple vertices, construct graph structure, wherein the vertex is as the node of graph structure, the connection relationship of multiple vertices is as the edge of graph structure, and the graphical feature representation is as the initial node representation of each node in graph structure;Based on graph neural network, the graph structure is carried out multiple rounds of node aggregation iterative processing, to obtain the first node representation of each node in graph structure;Based on the first node representation, the occlusion prediction of each node in graph structure is carried out, to obtain the prediction probability value;According to the prediction probability value, the grid model to be processed is rendered.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to an occlusion removal method, apparatus, device, medium, and program product. Background Technology

[0002] With the development of computer graphics technology, traditional rendering techniques have become very mature and can render complex scenes with small and complex features. However, for large scenes, how to efficiently utilize the computing resources of graphics processing units (GPUs) and render complex scenes with fewer GPU computing resources is one of the main research directions in computer graphics.

[0003] Rendering complex scenes often presents the problem of objects farther from the viewpoint being rendered first, followed by those closer. This results in objects closer to the viewpoint being rendered later and overwriting previously rendered objects, causing objects that are not visible from the viewpoint to still be rendered—a phenomenon known as overdraw—severely impacting scene rendering efficiency. Occlusion culling, as a technique to accelerate the rendering of large scenes, plays a significant role in resolving overdraw and improving rendering efficiency. Occlusion culling can determine the occlusion relationships of objects in a virtual scene, discarding a large number of unnecessary objects from the current viewpoint, reducing the complexity of 3D scene rendering, and ultimately achieving rapid rendering of virtual scenes.

[0004] However, how to perform efficient occlusion culling is a problem that urgently needs to be solved. Summary of the Invention

[0005] This application provides an occlusion culling method, apparatus, device, medium, and program product for efficiently and accurately culling occlusions of vertices in a mesh model, thereby reducing rendering overhead.

[0006] In view of this, this application provides an occlusion culling method, comprising: obtaining a mesh model to be processed, the mesh model being composed of multiple connected vertices; obtaining the graphical feature representation of each vertex in the mesh model to be processed; constructing a graph structure based on the multiple vertices and their connection relationships, wherein the vertex serves as a node of the graph structure, the connection relationships of the multiple vertices serve as edges of the graph structure, and the graphical feature representation serves as the initial node representation of each node in the graph structure; performing multi-round node aggregation iteration processing on the graph structure based on a graph neural network to obtain a first node representation of each node in the graph structure; performing occlusion prediction on each node in the graph structure based on the first node representation to obtain a predicted probability value, the predicted probability value being used to indicate the occlusion probability of each node in the graph structure; and rendering the mesh model to be processed according to the predicted probability value.

[0007] Another aspect of this application provides an occlusion culling device, comprising: an acquisition module for acquiring a mesh model to be processed, the mesh model being composed of multiple connected vertices; and acquiring the graphical feature representation of each vertex in the mesh model to be processed.

[0008] The processing module is used to construct a graph structure based on the multiple vertices and their connections, where each vertex serves as a node in the graph structure, the connections between the multiple vertices serve as edges in the graph structure, and the graph feature representation serves as the initial node representation for each node in the graph structure. It then performs multiple rounds of node aggregation iteration on the graph structure using a graph neural network to obtain the first node representation for each node in the graph structure. Based on the first node representation, it performs occlusion prediction on each node in the graph structure to obtain a predicted probability value, which indicates the occlusion probability of each node in the graph structure. Finally, it renders the mesh model to be processed based on the predicted probability value.

[0009] In one possible design, in another implementation of another aspect of the embodiments of this application, the acquisition module is used to acquire the normal feature vector, texture feature vector, and material feature vector of each vertex in the mesh model to be processed. The normal feature vector is used to describe the surface orientation of each vertex of the mesh model to be processed, the texture feature vector is used to describe the surface texture or pattern information of each vertex of the mesh model to be processed, and the material feature vector is used to describe the surface lighting information of each vertex of the mesh model to be processed.

[0010] The normal feature vector, the texture feature vector, and the material feature vector are concatenated to obtain the graphical feature representation of each vertex.

[0011] In one possible design, in another implementation of another aspect of the embodiments of this application, the acquisition module is used to acquire the first coordinate component, the second coordinate component and the third coordinate component of each vertex in the mesh model to be processed based on the normal in three-dimensional space, and to use the first coordinate component, the second coordinate component and the third coordinate component as the normal feature vector.

[0012] Obtain the fourth and fifth coordinate components of each vertex in the mesh model to be processed in two-dimensional space based on the texture, and use the fourth and fifth coordinate components as the texture coordinates;

[0013] The texture coordinates are 3D mapped to obtain transformed texture coordinates, and the texture coordinates and the transformed texture coordinates are used as the texture feature vector;

[0014] Obtain the lighting simulation vector of each vertex in the mesh model to be processed. The lighting simulation vector includes the diffuse reflection vector, specular reflection vector, and smoothness vector.

[0015] Based on the diffuse reflection vector, the specular reflection vector, and the smoothness vector, the color vector of each vertex in the style model to be processed is determined, and the lighting simulation vector and the color vector are used as the material feature vector.

[0016] In one possible design, in another implementation of another aspect of the embodiments of this application, the processing module is used to perform neighbor node aggregation on each node in the graph structure based on the first graph convolutional layer of the graph neural network, so as to update the initial node representation of each node in the graph structure to the second node representation.

[0017] Based on the second graph convolutional layer of the graph neural network, the neighbor nodes of each node in the graph structure are aggregated to update the second node representation of each node in the graph structure to the third node representation.

[0018] The entire graph convolutional layer of the graph neural network is executed in this manner to obtain the representation of the first node.

[0019] In one possible design, in another implementation of another aspect of the embodiments of this application, the processing module is used to obtain an occlusion threshold, which is used to determine whether the vertices of the mesh model to be processed are occluded.

[0020] When the predicted probability value is greater than or equal to the occlusion threshold, it is determined that the vertex corresponding to the predicted probability value is occluded, and the vertex corresponding to the predicted probability value is removed.

[0021] When the predicted probability value is less than the occlusion threshold, it is determined that the vertex corresponding to the predicted probability value is not occluded, and the vertex corresponding to the predicted probability value is rendered.

[0022] In one possible design, in another implementation of another aspect of the embodiments of this application, the processing module is used to obtain the usage information of the current rendering resources;

[0023] When the usage information indicates that the current rendering resource is greater than the first threshold, the occlusion threshold is reduced.

[0024] When the usage information indicates that the current rendering resource is less than or equal to the first threshold, the occlusion threshold is increased.

[0025] In one possible design, in another implementation of another aspect of the embodiments of this application, the processing module is used to sort the vertices of the mesh model to be processed according to the predicted probability value to obtain a rendering queue.

[0026] Within the constraints of the current rendering resources, render each vertex in the mesh model to be processed sequentially according to the rendering queue.

[0027] This application also provides a computer device, including: a memory, a processor, and a bus system;

[0028] The memory is used to store programs;

[0029] The processor is used to execute programs in memory, and the processor is used to execute the methods mentioned above according to the instructions in the program code;

[0030] Bus systems are used to connect memory and processor to enable communication between them.

[0031] Another aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0032] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the above aspects.

[0033] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: the mesh structure of the mesh model is represented as a graph structure, where the nodes of the graph structure are the vertices of the mesh model, the edges of the graph structure are the connections between the vertices of the mesh model, and the nodes in the graph structure are aggregated by the graph neural network, thereby capturing the local spatial relationship and global context information of the vertices of the mesh model, thereby more accurately predicting the occlusion of each vertex in the mesh model, and then performing corresponding culling rendering to reduce rendering resource consumption. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of an application system for occlusion culling in the embodiments of this application;

[0035] Figure 2 This is an optimized flowchart of screen rendering in a game scene according to an embodiment of this application;

[0036] Figure 3 This is a schematic diagram of one embodiment of occlusion culling in this application.

[0037] Figure 4 This is a schematic diagram of a mesh model in an embodiment of this application;

[0038] Figure 5 This is a schematic diagram of an occlusion culling process in an embodiment of this application;

[0039] Figure 6 This is a schematic diagram of one embodiment of the occlusion removal device in this application.

[0040] Figure 7 This is a schematic diagram of one embodiment of the server in this application;

[0041] Figure 8 This is a schematic diagram of one embodiment of the terminal in this application. Detailed Implementation

[0042] This application provides an occlusion culling method, apparatus, device, medium, and program product for efficiently and accurately culling occlusions of vertices in a mesh model, thereby reducing rendering overhead.

[0043] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0044] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0045] With the development of computer graphics technology, traditional rendering techniques have become very mature, capable of rendering complex scenes with small but varied elements. However, for large scenes, one of the main research directions in computer graphics is how to efficiently utilize the computing resources of the graphics processing unit (GPU) to render complex scenes with fewer GPU resources. Rendering complex scenes often presents the problem of objects farther from the viewpoint being rendered first, followed by objects closer to the viewpoint. This results in objects closer to the viewpoint being rendered later and covering previously rendered objects, leading to overdrawing of objects that are not visible to the viewpoint, severely impacting rendering efficiency. Occlusion culling, as a technique to accelerate the rendering of large scenes, plays a significant role in solving overdrawing and improving rendering efficiency. Occlusion culling can determine the occlusion relationships of objects in a virtual scene, discarding a large number of unnecessary objects from the current viewpoint, reducing the complexity of 3D scene rendering, and ultimately achieving fast rendering of virtual scenes. However, how to perform efficient occlusion culling remains a problem that urgently needs to be solved.

[0046] To address this technical problem, this application provides the following technical solution: Obtaining a mesh model to be processed, which is composed of multiple connected vertices; obtaining the graphical feature representation of each vertex in the mesh model to be processed; constructing a graph structure based on the multiple vertices and their connections, wherein each vertex serves as a node in the graph structure, the connections between the multiple vertices serve as edges in the graph structure, and the graphical feature representation serves as the initial node representation of each node in the graph structure; performing multi-round node aggregation iteration processing on the graph structure based on a graph neural network to obtain the first node representation of each node in the graph structure; performing occlusion prediction on each node in the graph structure based on the first node representation to obtain a prediction probability value, which indicates the occlusion probability of each node in the graph structure; and rendering the mesh model to be processed according to the prediction probability value. This approach represents the mesh structure of the mesh model as a graph structure, where the nodes of the graph structure are the vertices of the mesh model, and the edges of the graph structure are the connections between the vertices of the mesh model. By aggregating the nodes in the graph structure through a graph neural network, the local spatial relationships and global context information of the vertices of the mesh model can be captured, thereby more accurately predicting the occlusion of each vertex in the mesh model and performing corresponding culling rendering to reduce rendering resource consumption.

[0047] The Graph Neural Network (GNN) used in this application involves deep learning of graph-structured data in the field of artificial intelligence. Graph Neural Networks are commonly applied in areas such as social networks, knowledge graphs and recommender systems, bioinformatics, natural language processing, image and computer vision, the Internet of Things, dynamic system modeling, and scientific computing.

[0048] In the context of social networks, nodes in a graph structure represent users, and edges represent relationships between users (such as friend relationships, follow relationships, etc.). Generative Neural Networks (GNNs) can extract deep-level information about users and their relationships from social networks. Specifically, this can be used to predict user interests, preferences, or whether connections can be established between users (such as friend recommendations).

[0049] In the field of knowledge graphs, a knowledge graph is a graph structure composed of entities (nodes) and relationships (edges) used to represent knowledge in the real world. Therefore, graph neural networks can be used to infer new relationships or facts from existing knowledge graphs. For example, predicting a relationship like "A is B's tutor." Alternatively, they can be used for question-and-answer applications based on existing knowledge graphs, such as knowledge retrieval through search engines.

[0050] In the field of recommender systems, the problem can often be represented as a graph structure, such as a user-item interaction graph (users and items as nodes, and interaction behaviors as edges). Therefore, graph neural networks can recommend potentially interesting items based on the user's interaction history with items. For example, recommending music that a user might be interested in within a music app.

[0051] When applied to the field of bioinformatics, much of the data in bioinformatics naturally possesses a graph structure, such as molecular structures and biological networks. Therefore, this GNN can be used to model the atoms and bonds in protein molecules and predict their three-dimensional structure. Alternatively, molecular properties (such as toxicity and water solubility) can be predicted based on the graph structure of molecules, leading to the design of new drugs. Furthermore, gene expression networks can be analyzed to discover relationships between genes.

[0052] When applied to natural language processing techniques, text can be represented by constructing syntactic trees or semantic graphs, and GNNs can capture complex relationships between words or sentences. For example, text can be constructed as a dependency graph between words, and GNNs can be used to extract high-level semantic features, thereby achieving sentiment analysis or news classification, etc.

[0053] When applied to image and computer vision, although image data is primarily a grid structure, images or videos can be modeled as graphs in tasks such as object detection and scene understanding. For example, objects in an image can be modeled as nodes in a graph, and the spatial relationships between objects can be modeled as edges. Alternatively, objects and relationships in complex scenes (such as images or videos) can be represented as graphs.

[0054] When applied to modeling dynamic systems, dynamic systems (such as the changes in social networks over time) can be modeled as time series graphs, and GNNs can capture dynamic changes in the time dimension. For example, they can be used to analyze changes in user relationships in social networks, such as the formation of new connections or the disappearance of old connections. Alternatively, they can be used to simulate dynamic interactions in physical systems, such as particle motion and fluid dynamics.

[0055] When applied to the Internet of Things (IoT), the connections between devices can be modeled as graphs. For example, in a device network, graph modeling can be used to predict potential failures by establishing collaborative relationships between devices. Alternatively, modeling nodes (power plants, substations) and edges (power transmission) in a power grid can optimize power distribution.

[0056] In scientific computing, many problems in the scientific field can be represented as graphs, such as particle systems and molecular dynamics. For example, GNNs can be used to simulate complex physical systems, such as fluid dynamics and elastic body deformation. Alternatively, celestial systems in the universe (such as galaxies and black holes) can be modeled as graphs to simulate their interactions.

[0057] In this embodiment, the graph neural network can be based on a Graph Convolutional Network (GCN), a Graph Attention Network (GAT), or a Graph Isomorphism Network (GIN). The core idea of ​​a GCN-based graph neural network is to replace the convolution of a regular neural network with graph convolution operations, aggregating features of neighboring nodes while preserving the node's own features. This structure makes the graph neural network simple and efficient, suitable for processing static graphs, and thus suitable for tasks such as node classification and graph classification. The core idea of ​​a GAT-based graph neural network is to introduce an attention mechanism, assigning different weights to different neighboring nodes and dynamically adjusting the contribution of feature aggregation. This captures the importance relationships between nodes, making it suitable for processing heterogeneous graphs or complex graphs with uneven weights. The core idea of ​​a GIN-based graph neural network is to provide a more powerful feature aggregation method, enabling the model to distinguish different graph structures. This results in strong expressive power, able to distinguish between isomorphic and non-isomorphic graphs, making it suitable for structure-sensitive graph classification tasks.

[0058] Optionally, the occlusion culling method provided in the embodiments of this application can also be implemented based on cloud technology. For example, various calculations involved in the online application of graph neural networks can be implemented using cloud computing technology, the graph neural network and the mesh model to be processed can be stored in the cloud, and the data transmission of the mesh model to be processed can be transmitted in the cloud.

[0059] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0060] The occlusion removal method provided in this application embodiment can be applied to, for example, Figure 1 The system shown includes a terminal 100, a server 200, a database 300, and a network 400. The terminal 100, server 200, and database 300 are connected via the network 400. The database 300 provides the system with the mesh model to be processed and its various parameters. Figure 1 The number of terminals, servers, and databases in the system shown is merely an example. For instance, there may be multiple terminals, servers, and databases. This application does not limit the number of terminals, servers, and databases.

[0061] The aforementioned system is used to provide model rendering functions for game scenes, film and television scenes, or other fields, and this application embodiment does not limit this. During the model rendering process, the server 200 can perform occlusion culling on each vertex of the received mesh model to be processed, and finally perform model rendering based on the result of occlusion culling. It should be noted that the mesh model to be processed and its various parameters are authorized by the user or fully authorized by all parties.

[0062] In this process, terminal 100 communicates with server 200 via a network. Database 300 can be integrated on server 200 or located in the cloud or on other servers. During the occlusion removal process, interaction can occur between terminal 100 and server 200. For example, a user generates a mesh model to be processed through terminal 100, and terminal 100 sends the mesh model to server 200.

[0063] Terminal 100 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, smart voice interaction device, smart home appliance, or in-vehicle terminal, but is not limited to these. Server 200 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), big data, and artificial intelligence platforms.

[0064] In short, a database can be viewed as an electronic filing cabinet—a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, shared by multiple users, with minimal redundancy, and independent of application programs. A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or Extensible Markup Language (XML); or according to the type of computer they support, such as server clusters or mobile phones; or according to the query language used, such as Structured Query Language (SQL) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, supporting multiple query languages ​​simultaneously.

[0065] Based on the above description, this embodiment uses the rendering of game screens to illustrate the rendering process of the mesh model. In an exemplary solution, the specific process can be as follows: Figure 2 As shown:

[0066] Initially, the game's individual frames were debugged and analyzed to identify key performance issues. Then, relevant models (such as...) were extracted from this debugging process. Figure 2 The texture data shown is used to prepare for subsequent optimization work. Then, intelligent occlusion culling is performed on the model to achieve the corresponding optimization. For example, ... Figure 2The illustrated scheme uses a texture optimization algorithm to process the exported textures, reducing texture resource consumption or improving texture loading efficiency. The optimized textures are then reapplied to the game, and single-frame debugging is performed again to verify the optimization effect. During debugging, single-frame performance data is recorded to compare the performance differences before and after optimization. Based on the recorded performance data, it is determined whether the texture optimization algorithm has brought about a performance improvement. If there is optimization: the process ends, and optimization is complete. If there is no optimization: return to the texture optimization algorithm step, adjust or adopt a new optimization strategy, and repeat the process.

[0067] It is understood that in the specific embodiments of this application, data related to the mesh model to be processed and its parameters, graph neural network and its parameters are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0068] Based on the above introduction, the occlusion culling method in this application will be described below, taking the server as the execution subject. Please refer to [link / reference needed]. Figure 3 One embodiment of the occlusion culling method in this application includes:

[0069] 301. Obtain the mesh model to be processed, which consists of multiple connected vertices.

[0070] In this embodiment, a mesh model is a common method for representing three-dimensional geometry, widely used in fields such as computer graphics, 3D modeling, computer vision, game development, engineering design, and scientific computing. A mesh model uses vertices, edges, and faces as its basic building blocks, representing the surface of an object with discrete geometric data.

[0071] In this context, a vertex is a point in three-dimensional space, representing a key location or shape of an object. Vertices are typically represented using three-dimensional coordinates, such as (x, y, z). Vertices may carry other information, including: Normal: Describes the vertex's orientation and is used for lighting calculations; Texture coordinates (UV): Defines how textures are mapped onto the model's surface; Color: The vertex's color attribute, used for rendering.

[0072] An edge is a connection between two vertices, used to describe the relationship between them. Edges are typically represented by two vertex indices, such as (v1, v2). Edges are an important component of the mesh's topology. In a triangular mesh, an edge usually belongs to one or more faces.

[0073] A face is a two-dimensional polygon composed of vertices and edges, used to form the surface of an object. The most common face is a triangle, defined by three vertices, such as (v1, v2, v3). It can also be a quadrilateral or other polygons. Information a face may carry includes: Face normals: used to calculate lighting and shadows. Material: the face may be bound to a specific material for rendering. Face texture map: defines the texture coordinates of a specific area.

[0074] Based on the above description, according to the organization of vertices, edges, and faces, mesh models can be divided into the following common types: triangular meshes, quadrilateral meshes, polygonal meshes, and sparse meshes. Triangular meshes consist of triangular faces, each defined by three vertices. Quadrilateral meshes consist of quadrilateral faces, each defined by four vertices. Polygonal meshes consist of arbitrary polygonal faces, not limited to triangles or quadrilaterals. Sparse meshes are used to represent low-resolution geometry and contain fewer vertices and faces.

[0075] In one exemplary scenario, during game development, mesh models form the foundation for constructing 3D scenes and characters. All 3D objects in the game (such as characters, buildings, props, terrain, etc.) can be represented using mesh models. The quality, complexity, and optimization level of the mesh model directly affect the game's visual effects, runtime efficiency, and player experience.

[0076] like Figure 4 The mesh model shown includes 8 vertices and 12 edges, forming a star-shaped object with connections in the center, surrounded by a square frame.

[0077] 302. Obtain the graphical feature representation of each vertex in the mesh model to be processed.

[0078] In this embodiment, based on the representation of the mesh model to be processed, the graphical feature representation of each vertex in the mesh model can be obtained. In an exemplary scheme, the graphical feature representation of each vertex in the mesh model to be processed includes the normal feature vector, texture feature vector, and material feature vector of each vertex. The normal feature vector is used to describe the surface orientation of each vertex in the mesh model to be processed, the texture feature vector is used to describe the surface texture or pattern information of each vertex in the mesh model to be processed, and the material feature vector is used to describe the surface lighting information of each vertex in the mesh model to be processed. The normal feature vector, the texture feature vector, and the material feature vector are concatenated to obtain the graphical feature representation of each vertex.

[0079] In an exemplary scheme, the normal eigenvector can be represented as follows: N = (Nx, Ny, Nz), where N indicates the normal eigenvector, Nx indicates the coordinate component of the normal eigenvector on the X-axis, Ny indicates a coordinate component of the normal eigenvector on the Y-axis, and Nz indicates a coordinate component of the normal eigenvector on the Z-axis. In practical applications, the normal eigenvector can be a unit vector representing the normal direction of a vertex. The values ​​in (Nx, Ny, Nz) can be set to the range [-1, 1], typically normalized to a unit length (|N| = 1). For example, if the normal eigenvector of vertex 1 is represented as N1 = (0, 1, 0), it indicates that the normal of vertex 1 is vertically upward. If the normal eigenvector of vertex 2 is represented as N2 = (1, 0, 0), it indicates that the normal of vertex 2 points to the right. The eigenvector of the normal of vertex 3 is represented as N3 = (-0.707, 0, 0.707), which means that the normal of vertex 3 points in the diagonal direction.

[0080] The texture feature vector can include texture coordinates and their transformed texture coordinates. The texture coordinates can be represented as T = (Tu, Tv), where Tu and Tv are texture coordinate values. This defines the sampling position of the two-dimensional texture image. The values ​​of Tu and Tv can be set to [0, 1], thus representing the normalized coordinate system of the texture image. When calculating the transformed texture coordinates, a transformation matrix M can be used to calculate the transformed texture coordinates.

[0081] That is, T′(P) = M·T(P), where T′(P) is used to indicate the transformed texture coordinates and P is used to indicate the vertex.

[0082] In an exemplary scheme, assuming the transformed texture coordinates are used to indicate color values, then M can be used to map the texture coordinates to color values. For example, texture coordinates (u,v) = (0.1, 0.2): corresponding color value (r,g,b) = (255, 0, 0) (red). Texture coordinates (u,v) = (0.5, 0.5): corresponding color value (r,g,b) = (0, 255, 0) (green). Texture coordinates (u,v) = (0.9, 0.8): corresponding color value (r,g,b) = (0, 0, 255) (blue).

[0083] The material feature vector can include a lighting simulation vector and a color vector. The lighting simulation vector can be represented as: M = (Kd, Ks, α), where Kd is the diffuse reflection coefficient, Ks is the specular reflection coefficient, and α is the smoothness. Based on this lighting simulation vector, several pieces of information can be calculated.

[0084] For example, the diffuse reflection term: Id = Kd·(L·N);

[0085] Specular reflection term: Is = Ks·(V·R)α

[0086] Final color: C = Id + Is.

[0087] Based on the above description, the image feature representation of this vertex can be expressed as follows:

[0088] Fi = [N, T, T', M, C]

[0089] 303. Construct a graph structure based on the multiple vertices and their connection relationships, wherein the vertex serves as a node of the graph structure, the connection relationships of the multiple vertices serve as edges of the graph structure, and the graph feature representation serves as the initial node representation of each node in the graph structure.

[0090] In this embodiment, the server constructs the graph structure based on the vertices and edges of the mesh model to be processed, that is, the vertices of the mesh model to be processed are used as nodes of the graph structure, and the edges of the mesh model to be processed are used as edges of the graph structure; the graphical feature representation of each vertex of the mesh model to be processed is used as the initial node representation of each node in the graph structure.

[0091] In one exemplary scheme, the graph structure can be represented as G = (V, E), where V indicates the set of nodes, representing the vertices in the graph structure (such as the mesh vertices in a mesh model). E indicates the set of edges, representing the connections between nodes in the graph structure (such as the edges between adjacent vertices in a mesh model). An adjacency matrix can also be used in this graph structure to represent the connections between nodes, such as A... ij This is used to indicate that there is a connection between node i and node j. Then, X is used to indicate the node representation of each node. X can also be represented as V. i .

[0092] In this graph structure, for node i, we can calculate the average representation of its neighboring nodes as follows:

[0093] V i ′ =1 / |N(i)|∑ j∈N(i) V j

[0094] Where N(i) indicates the set of neighbors of node i, and V j Used to indicate the neighboring nodes of node i.

[0095] It can also calculate the representation of all nodes in the graph structure, and its calculation information can be represented as follows:

[0096] h = ∑ i V i

[0097] It should be understood that semantic features can be introduced to learn more graphical feature representations of each vertex. For example, a pre-trained visual Transformer (such as ViT) can be used to extract high-level semantic information (such as object category, scene background, etc.) from the texture of the mesh model to be processed; then, a cross-modal graph neural network (Cross-modal GNN, CM-GNN) can be used to combine the above geometric features (i.e., normals, textures, materials, etc.) and semantic information to capture more complex occlusion relationships.

[0098] 304. Based on the graph neural network, perform multiple rounds of node aggregation and iteration processing on the graph structure to obtain the first node representation of each node in the graph structure.

[0099] In this embodiment, the server can perform multi-round iterative node aggregation processing on each node in the graph structure through a pre-trained graph neural network to obtain the final node representation of each node in the graph structure, namely the first node representation.

[0100] During execution in this graph neural network, the features of its input layer are the initial node representations of each node in the graph structure, i.e., the graphical feature representations corresponding to the vertices of the grid model to be processed. Then, the graph convolutional layers in the graph neural network perform neighbor aggregation processing on each node in the graph structure to update the feature representation of each node. Finally, through the fully connected layer and activation function of the output layer, the output of each vertex is mapped to the range [0, 1].

[0101] The update process of multiple graph convolutional layers can be as follows: the first graph convolutional layer based on the graph neural network performs neighbor node aggregation on each node in the graph structure to update the initial node representation of each node in the graph structure to the second node representation; the second graph convolutional layer based on the graph neural network performs neighbor node aggregation on each node in the graph structure to update the second node representation of each node in the graph structure to the third node representation; and so on, all graph convolutional layers of the graph neural network are executed to obtain the first node representation.

[0102] It should be understood that when performing neighbor aggregation across multiple graph convolutional layers, the specific calculation process can be as follows: for each vertex i, its updated feature representation is:

[0103] H i ′ =σ(∑ j W·H j / |N(i)|

[0104] Where, H i ′ It is the updated feature representation of vertex i. H jσ is the feature representation of vertex j (the neighbors of i). W is the weight matrix (learnable parameters). σ is the activation function (such as ReLU). |N(i)| is the number of neighbors of node i, used for normalization.

[0105] It should be understood that, to improve the flexibility and efficiency of GCNs, avoid computational waste in simple scenarios using deep networks, and provide stronger global feature capture capabilities for complex mesh models, Adaptive Graph Neural Networks (GCNs) can be introduced. This involves dynamically selecting the depth of the graph neural network and the feature aggregation method based on the complexity of the mesh model. In other words, the server can automatically adjust the number of GCN layers in the graph neural network. For example, fewer GCN layers can be set for simple scenarios, while more GCN layers can be set for complex scenarios.

[0106] Furthermore, the feature aggregation method for each node in the graph structure can be dynamically switched in different scenarios. For example, weighted average aggregation can be used for simple scenarios, while attention mechanisms (GraphAttention, GAT) can be used for aggregation in complex scenarios.

[0107] In its specific implementation, a multi-path GCN framework (with shallow and deep paths coexisting) can be constructed; then the optimal path can be dynamically selected to adjust the network depth and aggregation method, thereby reducing the computational overhead in the prediction process.

[0108] 305. Based on the representation of the first node, perform occlusion prediction on each node in the graph structure to obtain a prediction probability value, which is used to indicate the occlusion probability of each node in the graph structure.

[0109] In this embodiment, after the server obtains the first node representation output by the last graph convolutional layer in the graph neural network, it performs occlusion prediction and mapping on the first node representation based on the output layer of the graph neural network to obtain the predicted probability value. In an exemplary scheme, the server calls a fully connected layer and a sigmoid activation function of the graph neural network to map the output of each vertex to the range [0,1] to obtain the predicted probability value. Its calculation method can be represented as follows:

[0110] P i =sigmoid(W out ·H i )

[0111] Among them, the P i The W is used to indicate the occlusion probability of vertex i. out H is the weight matrix of the output layer. i Used to indicate the representation of the first node (i.e., the output of the last graph convolutional layer).

[0112] In this embodiment, the server can also train the graph neural network, and the training process can be as follows:

[0113] Suppose we have a 3D model containing the following information: Vertex data (V): The model has 5000 vertices, each with geometric, normal, and texture features. Edge data (E): Connections exist between vertices, forming a mesh structure; for example, edges between vertices represent adjacent vertices. Occlusion labels: In the training data, some vertices are labeled as "occluded" (1) or "visible" (0). A graph neural network (GNN) is used to learn the occlusion relationships of vertices, training a model to predict the occlusion probability of each vertex in an unknown mesh model. The node representation of the input layer of this graph neural network can be represented as the initial feature matrix H. i (Size is N×F, where N is the number of vertices and F is the feature dimension, e.g., 10); then a multi-layer Graph Convolutional Network (GCN) is used to extract high-level features of the vertices. This graph neural network can include stacked multi-layer GCNs to progressively capture information from more distant neighbors. Assume we use a 3-layer GCN: the first layer captures information from directly adjacent vertices. The second layer combines information from two-hop neighbors. The third layer captures global context information. Then, the predicted probability of each vertex is output through the output layer; finally, a loss function is calculated based on this predicted probability and the occlusion label to obtain the loss value; then, the parameters of the graph neural network are trained based on this loss value to obtain the graph neural network used in practical applications in this embodiment.

[0114] 306. Render the grid model to be processed based on the predicted probability value.

[0115] After obtaining the predicted probability value, the server renders each vertex of the processed mesh model accordingly.

[0116] Specifically, the server compares the predicted probability value with an occlusion threshold. If the predicted probability value is greater than or equal to the occlusion threshold, the vertex corresponding to the predicted probability value is determined to be occluded, and the vertex can be removed and no longer rendered; if the predicted probability value is less than the occlusion threshold, the vertex corresponding to the predicted probability value is determined to be unoccluded, and the vertex can be rendered.

[0117] To ensure that the server can achieve a balance between device performance, rendering resources, and rendering quality during actual operation, the occlusion threshold can be dynamically adjusted. That is, when device performance is good or rendering resources are sufficient, the occlusion threshold can be lowered, allowing more vertices to be rendered; when device performance is low or rendering resources are insufficient, the occlusion threshold can be raised to reduce the rendering burden.

[0118] On the other hand, in order to achieve the best possible rendering effect, the server can sort the vertices of the mesh model to be processed in descending order based on the predicted probability value to obtain a rendering queue; then, within the constraints of the current rendering resources, the server renders each vertex of the mesh model to be processed in the order of the rendering queue from back to front.

[0119] Alternatively, the server can sort the vertices of the mesh model to be processed in ascending order based on the predicted probability value to obtain a rendering queue; then, within the constraints of the current rendering resources, the server can render each vertex of the mesh model to be processed in the order of the rendering queue from front to back.

[0120] In this embodiment, to more accurately implement dynamic adjustment of the occlusion culling strategy, reinforcement learning (RL) can be introduced into the occlusion culling decision-making process. A reinforcement learning agent is used to monitor rendering performance in real time and dynamically adjust the occlusion culling strategy; combined with a temporal difference (TD) reinforcement learning algorithm, the effectiveness of the occlusion culling decision is gradually optimized. The process can be as follows: obtain the real-time rendering state (e.g., current frame rate, occlusion probability, vertex rendering); then adjust the occlusion culling threshold τ or the dynamically sorted strategy based on the rendering state; then obtain the rendering effect and determine the feedback for strategy adjustment based on the rendering effect. This allows the optimal culling strategy to be automatically learned based on the dynamic rendering environment (e.g., GPU load). The introduction of reinforcement learning makes the system more adaptive and reduces the complexity of manually adjusting parameters.

[0121] Based on the above description, it can be seen that the occlusion culling in this application can be performed as follows: Figure 5 The process shown:

[0122] Will as Figure 4 The graph model is shown to obtain its graphical feature representation; then, the graph structure is constructed based on the graphical feature representation and the graph model; the trained graph neural network is used to perform neighbor aggregation on the graph structure to obtain updated node representations; finally, occlusion prediction is performed on each vertex of the graph model based on the node representation to obtain the prediction probability value; finally, vertex culling and rendering are performed based on the prediction probability value.

[0123] The occlusion removal device in this application is described in detail below. Please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of one embodiment of the occlusion removal device in this application. The occlusion removal device 20 includes:

[0124] The acquisition module 201 is used to acquire the mesh model to be processed, which is composed of multiple vertices connected together; and to acquire the graphical feature representation of each vertex in the mesh model to be processed.

[0125] Processing module 202 is used to construct a graph structure based on the multiple vertices and their connection relationships, wherein the vertex is used as a node of the graph structure, the connection relationships of the multiple vertices are used as edges of the graph structure, and the graph feature representation is used as the initial node representation of each node in the graph structure; the graph structure is subjected to multiple rounds of node aggregation iteration processing based on a graph neural network to obtain the first node representation of each node in the graph structure; occlusion prediction is performed on each node in the graph structure based on the first node representation to obtain a prediction probability value, which is used to indicate the occlusion probability of each node in the graph structure; and the mesh model to be processed is rendered according to the prediction probability value.

[0126] This application provides an occlusion culling device. Using this device, the mesh structure of a mesh model is represented as a graph structure, where the nodes of the graph structure are the vertices of the mesh model, and the edges of the graph structure are the connections between the vertices of the mesh model. A graph neural network is used to aggregate the nodes in the graph structure, thereby capturing the local spatial relationships and global context information of the mesh model vertices. This allows for more accurate prediction of occlusion at each vertex in the mesh model, enabling corresponding culling rendering and reducing rendering resource consumption.

[0127] Optionally, in the above Figure 6 Based on the corresponding embodiments, in another embodiment of the occlusion removal device 20 provided in this application,

[0128] The acquisition module 201 is used to acquire the normal feature vector, texture feature vector, and material feature vector of each vertex in the mesh model to be processed. The normal feature vector is used to describe the surface orientation of each vertex of the mesh model to be processed. The texture feature vector is used to describe the surface texture or pattern information of each vertex of the mesh model to be processed. The material feature vector is used to describe the surface lighting information of each vertex of the mesh model to be processed.

[0129] The normal feature vector, the texture feature vector, and the material feature vector are concatenated to obtain the graphical feature representation of each vertex.

[0130] In this embodiment, an occlusion culling device is provided. Using this device, feature representations of various information such as vertex normals, textures, and materials are fused into node representations of each node in the graph structure, thereby enabling a more comprehensive description of vertex characteristics. Simultaneously, multi-layer node aggregation processing is performed on the graph structure based on a graph neural network, thereby capturing the local spatial relationships and global contextual information of the vertices in the mesh model, and thus more accurately predicting the occlusion of each vertex in the mesh model.

[0131] Optionally, in the above Figure 6Based on the corresponding embodiments, in another embodiment of the occlusion culling device 20 provided in this application, the acquisition module 201 is used to acquire the first coordinate component, the second coordinate component and the third coordinate component of each vertex in the mesh model to be processed based on the normal in the three-dimensional space, and to use the first coordinate component, the second coordinate component and the third coordinate component as the normal feature vector;

[0132] Obtain the fourth and fifth coordinate components of each vertex in the mesh model to be processed in two-dimensional space based on the texture, and use the fourth and fifth coordinate components as the texture coordinates;

[0133] The texture coordinates are 3D mapped to obtain transformed texture coordinates, and the texture coordinates and the transformed texture coordinates are used as the texture feature vector;

[0134] Obtain the lighting simulation vector of each vertex in the mesh model to be processed. The lighting simulation vector includes the diffuse reflection vector, specular reflection vector, and smoothness vector.

[0135] Based on the diffuse reflection vector, the specular reflection vector, and the smoothness vector, the color vector of each vertex in the style model to be processed is determined, and the lighting simulation vector and the color vector are used as the material feature vector.

[0136] In this embodiment, an occlusion culling device is provided. Using this device, feature representations of various information such as vertex normals, textures, and materials are fused into node representations of each node in the graph structure, thereby enabling a more comprehensive description of vertex characteristics. Simultaneously, multi-layer node aggregation processing is performed on the graph structure based on a graph neural network, thereby capturing the local spatial relationships and global contextual information of the vertices in the mesh model, and thus more accurately predicting the occlusion of each vertex in the mesh model.

[0137] Optionally, in the above Figure 6 Based on the corresponding embodiments, in another embodiment of the occlusion removal device 20 provided in this application,

[0138] The processing module 202 is used to perform neighbor node aggregation on each node in the graph structure based on the first graph convolutional layer of the graph neural network, so as to update the initial node representation of each node in the graph structure to the second node representation.

[0139] Based on the second graph convolutional layer of the graph neural network, the neighbor nodes of each node in the graph structure are aggregated to update the second node representation of each node in the graph structure to the third node representation.

[0140] The entire graph convolutional layer of the graph neural network is executed in this manner to obtain the representation of the first node.

[0141] This application provides an occlusion culling device. Using this device, different convolutional layers in a graph neural network perform neighbor node aggregation operations on each node in the graph structure to update the node representation of each node. Specifically, a single-layer graph convolutional layer can capture the local occlusion relationships of the mesh model through the aggregation of adjacent vertices. The operation of multiple layers of graph convolutional layers can progressively pass and integrate contextual information to capture the overall occlusion pattern of the mesh model. This fusion of local and global features makes occlusion prediction more accurate.

[0142] Optionally, in the above Figure 6 Based on the corresponding embodiments, in another embodiment of the occlusion removal device 20 provided in this application,

[0143] The processing module 202 is used to obtain an occlusion threshold, which is used to determine whether the vertices of the mesh model to be processed are occluded;

[0144] When the predicted probability value is greater than or equal to the occlusion threshold, it is determined that the vertex corresponding to the predicted probability value is occluded, and the vertex corresponding to the predicted probability value is removed.

[0145] When the predicted probability value is less than the occlusion threshold, it is determined that the vertex corresponding to the predicted probability value is not occluded, and the vertex corresponding to the predicted probability value is rendered.

[0146] This application provides an occlusion culling device. Using this device, an occlusion threshold is set to determine whether each vertex in the mesh model is occluded. This allows for adjustments to the rendering effect based on actual conditions, achieving the best rendering effect within limited rendering resources, thereby improving the overall rendering quality and enhancing the user experience.

[0147] Optionally, in the above Figure 6 Based on the corresponding embodiments, in another embodiment of the occlusion culling device 20 provided in this application, the processing module 202 is used to obtain the usage information of the current rendering resources;

[0148] When the usage information indicates that the current rendering resource is greater than the first threshold, the occlusion threshold is reduced.

[0149] When the usage information indicates that the current rendering resource is less than or equal to the first threshold, the occlusion threshold is increased.

[0150] This application provides an occlusion culling device. Using this device, the occlusion threshold is dynamically adjusted, enabling intelligent control of the rendering range when rendering resources are limited.

[0151] Optionally, in the above Figure 6Based on the corresponding embodiments, in another embodiment of the occlusion culling device 20 provided in this application, the processing module 202 is used to sort each vertex of the mesh model to be processed according to the predicted probability value to obtain a rendering queue.

[0152] Within the constraints of the current rendering resources, render each vertex in the mesh model to be processed sequentially according to the rendering queue.

[0153] This application provides an occlusion culling device. Using this device, during the rendering process, mesh vertices are sorted according to occlusion probability values, and vertices predicted to be important are rendered first. This allows for the best rendering effect within limited rendering resources, thereby improving the overall rendering quality and enhancing the user experience.

[0154] The occlusion removal device provided in this application can be used on servers. Please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 322 (e.g., one or more processors) and memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 342 or data 344. The memory 332 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 322 may be configured to communicate with the storage media 330 and execute the series of instruction operations stored in the storage media 330 on the server 300.

[0155] Server 300 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0156] The steps performed by the server in the above embodiments can be based on this Figure 7 The server structure shown.

[0157] The occlusion removal device provided in this application can be used in terminal devices. Please refer to [link / reference]. Figure 8 For ease of explanation, only the parts relevant to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. In the embodiments of this application, a smartphone is used as an example for illustration:

[0158] Figure 8 This is a block diagram illustrating a portion of the structure of a smartphone related to the terminal device provided in the embodiments of this application. (Reference) Figure 8 The smartphone includes components such as a radio frequency (RF) circuit 410, a memory 420, an input unit 430, a display unit 440, a sensor 450, an audio circuit 460, a wireless fidelity (WiFi) module 470, a processor 480, and a power supply 490. Those skilled in the art will understand that... Figure 8 The smartphone structure shown does not constitute a limitation on smartphones and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0159] The following is combined with Figure 8 A detailed introduction to the various components of a smartphone:

[0160] RF circuit 410 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 480; additionally, it transmits uplink data to the base station. Typically, RF circuit 410 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 410 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Message Service (SMS), etc.

[0161] The memory 420 can be used to store software programs and modules. The processor 480 executes various functions and data processing of the smartphone by running the software programs and modules stored in the memory 420. The memory 420 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the smartphone (such as audio data, phonebook, etc.). In addition, the memory 420 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0162] The input unit 430 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the smartphone. Specifically, the input unit 430 may include a touch panel 431 and other input devices 432. The touch panel 431, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 431), and drive the corresponding connected devices according to a pre-set program. Optionally, the touch panel 431 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 480, and can also receive and execute commands sent by the processor 480. In addition, the touch panel 431 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 431, the input unit 430 may also include other input devices 432. Specifically, other input devices 432 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0163] Display unit 440 can be used to display information input by the user or information provided to the user, as well as various menus of the smartphone. Display unit 440 may include display panel 441, optionally configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar form. Further, touch panel 431 may cover display panel 441. When touch panel 431 detects a touch operation on or near it, it transmits the information to processor 480 to determine the type of touch event. Subsequently, processor 480 provides corresponding visual output on display panel 441 based on the type of touch event. Although in Figure 8 In this embodiment, the touch panel 431 and the display panel 441 are two separate components to realize the input and output functions of the smartphone. However, in some embodiments, the touch panel 431 and the display panel 441 can be integrated to realize the input and output functions of the smartphone.

[0164] The smartphone may also include at least one sensor 450, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 441 according to the ambient light level, and the proximity sensor can turn off the display panel 441 and / or the backlight when the smartphone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the smartphone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the smartphone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0165] Audio circuit 460, speaker 461, and microphone 462 provide an audio interface between the user and the smartphone. Audio circuit 460 converts received audio data into electrical signals and transmits them to speaker 461, where speaker 461 converts them into sound signals for output. On the other hand, microphone 462 converts collected sound signals into electrical signals, which are received by audio circuit 460, converted into audio data, and then processed by processor 480 before being transmitted via RF circuit 410 to, for example, another smartphone, or the audio data can be output to memory 420 for further processing.

[0166] WiFi is a short-range wireless transmission technology. Smartphones, through their WiFi modules (470), can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 8 WiFi module 470 is shown, but it is understood that it is not an essential component of a smartphone and can be omitted as needed without changing the nature of the invention.

[0167] The processor 480 is the control center of the smartphone, connecting various parts of the smartphone through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 420, and by calling data stored in the memory 420, thereby providing overall monitoring of the smartphone. Optionally, the processor 480 may include one or more processing units; optionally, the processor 480 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 480.

[0168] The smartphone also includes a power supply 490 (such as a battery) that supplies power to various components. Optionally, the power supply can be logically connected to the processor 480 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0169] Although not shown, smartphones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0170] The steps performed by the terminal device in the above embodiments can be based on this Figure 8 The terminal device structure is shown.

[0171] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.

[0172] This application also provides a computer program product including a program, which, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.

[0173] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0174] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0175] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0176] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0177] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An occlusion removal method, characterized in that, include: Obtain the mesh model to be processed, which is composed of multiple connected vertices; Obtain the graphical feature representation of each vertex in the mesh model to be processed; A graph structure is constructed based on the plurality of vertices and their connection relationships, wherein the vertices serve as nodes of the graph structure, the connection relationships of the plurality of vertices serve as edges of the graph structure, and the graph feature representation serves as the initial node representation of each node in the graph structure; The graph structure is subjected to multiple rounds of node aggregation and iteration based on a graph neural network to obtain the first node representation of each node in the graph structure; Based on the representation of the first node, occlusion prediction is performed on each node in the graph structure to obtain a prediction probability value, which is used to indicate the occlusion probability of each node in the graph structure. The mesh model to be processed is rendered based on the predicted probability value.

2. The method according to claim 1, characterized in that, The process of obtaining the graphical feature representation of each vertex in the mesh model to be processed includes: Obtain the normal feature vector, texture feature vector, and material feature vector of each vertex in the mesh model to be processed. The normal feature vector is used to describe the surface orientation of each vertex of the mesh model to be processed. The texture feature vector is used to describe the surface texture or pattern information of each vertex of the mesh model to be processed. The material feature vector is used to describe the surface lighting information of each vertex of the mesh model to be processed. The normal feature vector, the texture feature vector, and the material feature vector are concatenated to obtain the graphical feature representation of each vertex.

3. The method according to claim 2, characterized in that, Obtaining the normal feature vector, texture feature vector, and material feature vector of each vertex in the mesh model to be processed includes: Obtain the first coordinate component, second coordinate component, and third coordinate component of each vertex in the mesh model to be processed in three-dimensional space based on the normal, and use the first coordinate component, second coordinate component, and third coordinate component as the normal feature vector; Obtain the fourth and fifth coordinate components of each vertex in the mesh model to be processed in two-dimensional space based on the texture, and use the fourth and fifth coordinate components as the texture coordinates; The texture coordinates are 3D mapped to obtain transformed texture coordinates, and the texture coordinates and the transformed texture coordinates are used as the texture feature vector; Obtain the lighting simulation vector of each vertex in the mesh model to be processed, wherein the lighting simulation vector includes diffuse reflection vector, specular reflection vector and smoothness vector; The color vector of each vertex in the style model to be processed is determined based on the diffuse reflection vector, the specular reflection vector, and the smoothness vector, and the lighting simulation vector and the color vector are used as the material feature vector.

4. The method according to any one of claims 1 to 3, characterized in that, The process of performing multi-round node aggregation iteration on the graph structure based on a graph neural network to obtain the first node representation of each node in the graph structure includes: The first graph convolutional layer of the graph neural network performs neighbor node aggregation on each node in the graph structure to update the initial node representation of each node in the graph structure to the second node representation; Based on the second graph convolutional layer of the graph neural network, the neighbor nodes of each node in the graph structure are aggregated to update the second node representation of each node in the graph structure to the third node representation; The graph neural network is then executed through all its graph convolutional layers to obtain the representation of the first node.

5. The method according to any one of claims 1 to 3, characterized in that, The step of rendering the mesh model to be processed based on the predicted probability value includes: Obtain an occlusion threshold, which is used to determine whether the vertices of the mesh model to be processed are occluded; When the predicted probability value is greater than or equal to the occlusion threshold, it is determined that the vertex corresponding to the predicted probability value is occluded, and the vertex corresponding to the predicted probability value is removed. When the predicted probability value is less than the occlusion threshold, it is determined that the vertex corresponding to the predicted probability value is not occluded, and the vertex corresponding to the predicted probability value is rendered.

6. The method according to claim 5, characterized in that, The occlusion threshold acquisition includes: Get the usage information of the currently rendered resources; When the usage information indicates that the current rendering resource is greater than a first threshold, the occlusion threshold is reduced; When the usage information indicates that the current rendering resource is less than or equal to the first threshold, the occlusion threshold is increased.

7. The method according to any one of claims 1 to 3, characterized in that, The step of rendering the mesh model to be processed based on the predicted probability value includes: The vertices of the mesh model to be processed are sorted according to the predicted probability values ​​to obtain a rendering queue; Within the limitations of the current rendering resources, each vertex in the mesh model to be processed is rendered sequentially according to the rendering queue.

8. A device for removing obstructions, characterized in that, include: The acquisition module is used to acquire a mesh model to be processed, which is composed of multiple connected vertices; and to acquire the graphical feature representation of each vertex in the mesh model to be processed. The processing module is used to construct a graph structure based on the plurality of vertices and their connection relationships, wherein the vertices are nodes of the graph structure, the connection relationships of the plurality of vertices are edges of the graph structure, and the graph feature representation is the initial node representation of each node in the graph structure; perform multi-round node aggregation iteration processing on the graph structure based on a graph neural network to obtain the first node representation of each node in the graph structure; perform occlusion prediction on each node in the graph structure based on the first node representation to obtain a prediction probability value, the prediction probability value being used to indicate the occlusion probability of each node in the graph structure; and render the mesh model to be processed according to the prediction probability value.

9. A computer device, characterized in that, include: Memory, processor, and bus system; The memory is used to store programs; The processor is configured to execute a program in the memory, and the processor is configured to execute the method of any one of claims 1 to 7 according to instructions in the program code; The bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

10. A computer-readable storage medium comprising instructions, when executed on a computer, causing the computer to perform the method as claimed in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor using the method as described in any one of claims 1 to 7.

Images