Real-time mesh outline vectorization and rendering system based on GPU

Abstract

The present invention discloses a real-time grid outline vectorization and rendering system based on GPU, including: a grid import module, used to extract grid data from a local grid model file and perform preprocessing; a rasterization module, which extracts grid outlines using outline definition, converts grid outlines into fragments using scan conversion, performs occlusion culling on fragments using depth buffer and depth test, and integrates unoccluded fragments into a group of contour pixels; a vectorization module, which obtains a group of closed paths around contour pixels using a parallelized Potrace algorithm; a line extraction module, which is used to segment closed paths according to geometric information of grid outlines, and extract a group of line paths that can reflect scene occlusion relationships; and a line rendering module, which is used to render a group of line paths into a group of two-dimensional lines with controllable styles. The present invention combines GPU parallel computing technology with grid outline properties and image vectorization algorithms to improve the visual effect and performance of real-time contour rendering.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time rendering and stylized drawing, and in particular to a real-time grid outline vectorization and rendering system based on a GPU. Background Art

[0002] Contours play an important role in artistic creation: contours reveal the basic shape of the drawn object, and stylized contour lines can achieve expressive visual effects. In the field of stylized drawing, extracting contours from 3D meshes and drawing lines has always been an important technical point. Most contour rendering systems in real-time applications are fully implemented on GPUs. One reason is that GPUs generally have better graphics performance than CPUs. Another reason is that the geometric topology of mesh contours changes dynamically under camera and object motion. If contours are generated on the CPU side, it is necessary to update contour data from memory to video memory for GPU rendering every frame, and the resulting delay is unbearable for real-time performance. However, existing real-time contour drawing systems based on GPUs cannot link contour edges / pixels into stroke curves. The linking process usually starts from one element (pixel or mesh edge) and then connects to its neighboring elements. The linking will be repeated until a singularity is reached, at which the visibility of the mesh contour changes. When the linking occurs in image space (each contour element is treated as a 2D object), the process is called vectorization. The iterative nature of vectorization makes it difficult to parallelize on GPUs.

[0003] At present, real-time GPU contour rendering systems are mainly divided into three categories: 1) Image filter-based systems: Use fragment shaders to perform edge detection on geometry cache textures to calculate the gradient value of geometric attributes (such as depth, normal, object label) in image space. If the gradient of a texture pixel is higher than a specified threshold, it is considered a pixel on the contour. 2) Three-dimensional mesh-based systems: If two adjacent faces of an edge of a triangular mesh have opposite viewpoint orientations, the mesh is a contour edge. 3) Vertex extrusion-based systems: Offset vertices according to vertex normals; draw outlines based on the offset vertices. In general, most existing GPU-based contour rendering systems can only output scattered contour elements and cannot connect contour elements into line curves. This means that the current system cannot fully extract the geometric and topological properties of the mesh contour, resulting in greatly limited visual effects of contour rendering. On the other hand, even if there are systems on the CPU side that can extract lines from contours, their performance cannot provide effective support for increasingly complex real-time rendering scenes. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and proposes a real-time grid contour vectorization and rendering system based on GPU, breaking through the limitation that the traditional GPU grid contour rendering system cannot extract line paths, making full use of the performance of GPU and the properties of grid contours, and achieving real-time contour line extraction through contour extraction based on three-dimensional grids and vectorization algorithm based on image space.

[0005] To achieve the above object, the technical solution provided by the present invention is: a real-time grid outline vectorization and rendering system based on GPU, comprising:

[0006] Mesh import module, used to extract mesh data from local mesh model files and perform preprocessing;

[0007] The rasterization module extracts the mesh outline using the outline definition, converts the mesh outline into fragments using scan conversion, removes occlusions from the fragments using the depth buffer and depth test, and integrates the unoccluded fragments into a set of outline pixels;

[0008] The vectorization module uses the parallelized Potrace algorithm to obtain a set of closed paths around the contour pixels;

[0009] The line extraction module is used to segment the closed path according to the geometric information of the grid outline and extract a set of line paths that can reflect the occlusion relationship of the scene;

[0010] The line rendering module is used to render a set of line paths into a set of two-dimensional lines with controllable styles.

[0011] Furthermore, the grid import module specifically performs the following operations:

[0012] 1) Read the required mesh data from the local computer, including obj and fbx formats, where the mesh data includes the face to vertex index array F v , vertex coordinate array P v And the vertex normal array N v ;

[0013] 2) Preprocess the mesh data imported in step 1) and output the edge-to-face index array E f , edge-to-vertex index array E v , face normal array N f .

[0014] Further, the rasterization module includes a contour extraction module, a scan conversion module and a depth testing module, wherein:

[0015] The GPU threads process all mesh edges in parallel and determine whether a mesh edge is a contour based on the following definition:

[0016] (f0 ,f 1 )=E f [e]

[0017] (N f [f 0 ]·V)(N f [f 1 ]·V)<0

[0018] Where e is the mesh edge, from the index array E f The triangle f obtained in 0 ,f 1 Adjacent to e, from the face normal array N f N f [f 0 ],N f [f 1 ] are f 0 ,f 1 The surface normal of , V is the vector from any point on e to the viewpoint;

[0019] Use the Stream Compaction algorithm to remove non-contour edges from the mesh edges;

[0020] The scan conversion module is used to convert the contour edge output by the contour extraction module into a set of fragments, each fragment being a two-dimensional object with a pixel size and contour geometric data, including the following steps:

[0021] Each contour edge e is transformed into c Scan and convert into a set of fragments;

[0022] The attributes of the fragment are calculated by performing perspective interpolation on the two vertices of the contour edge, where the attributes of each fragment include the screen tangent vector of the contour edge to which the fragment belongs, the interpolated vertex normal and depth information;

[0023] The depth test module performs occlusion culling on the fragments converted by the scan conversion module and extracts the outline pixels, including the following steps:

[0024] The first visibility test is performed on the fragment using the scene depth buffer. If more than two pixels in the fragment's 3×3 neighborhood have depth values ​​greater than the fragment, the fragment passes the first visibility test.

[0025] The fragments that pass the first visibility test are tested a second time using hardware depth sorting, and the fragment with the smallest depth is converted to a contour pixel in the image.

[0026] Further, the vectorization module includes a path decomposition module and a path linking module, wherein:

[0027] The path decomposition module extracts pixel edges using the contour pixels output by the rasterization module, establishes adjacency relationships between pixel edges using the Potrace algorithm, and outputs a set of circular linked lists according to the adjacency relationships between pixel edges, including the following steps:

[0028] Each GPU thread corresponds to one or more contour pixels and observes its 3×3 neighborhood. For each background pixel adjacent to it in the neighborhood, the adjacent edges are marked as pixel edges.

[0029] Use the Potrace algorithm to determine the direction of each pixel edge, ensuring that it has contour pixels on the left and background pixels on the right, or background pixels on the left and contour pixels on the right;

[0030] The Potrace algorithm is used to establish an adjacency relationship for each pair of pixel edges that share pixel vertices, and a set of circular linked lists are output according to the adjacency relationship. Each node in the linked list corresponds to a pixel edge.

[0031] The path linking module converts a set of circular linked lists output by the path decomposition module into a set of linear lists using a parallel List Ranking algorithm, including the following steps:

[0032] Use List Ranking algorithm to select the head node for the circular linked list output by the path decomposition module;

[0033] The List Ranking algorithm is used to serialize the circular linked list. The pixel edges in each linked list are arranged into a list in order, and the obtained head node is the starting pixel edge of the list.

[0034] Furthermore, the line extraction module extracts the line path from the linear list output by the vectorization module, comprising the following steps:

[0035] 1) Use local curve fitting to calculate the approximate tangent directions for all pixel edges;

[0036] 2) Use the contour pixels output by the rasterization module to analyze all pixel edges. If the tangent of the pixel edge has the same direction as the weighted average tangent of the contour pixels around it, then mark the pixel edge as 1, otherwise 0;

[0037] 3) According to the marks processed in step 2), use the GPU segmented scanning algorithm to split each list output by the vectorization module into a group of sub-lists, use the midpoint of each pixel edge in each sub-list marked as 1 as the path vertex, and arrange the points in the order of the pixel edges in the list as the line path.

[0038] Furthermore, the line rendering module renders the line path to the screen, comprising the following steps:

[0039] 1) Calculate the line grid using the line path output by the line extraction module, where the line grid can be calculated by placing a brush instance at each path point, or by expanding the line path;

[0040] 2) Rendering the line mesh calculated in step 1) to the screen through the GPU shader. The style of the screen lines is determined by the form of the line mesh output in step 1) and the implementation of the GPU shader. The GPU shader receives the vertex data of the line mesh and converts the vertices into screen pixels according to its implementation logic. The input vertex data includes its screen coordinates and other vertex attributes used for line drawing, including brush texture coordinates, line color and transparency.

[0041] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0042] 1. Output contour lines that reflect occlusion relationships, providing technical support for stylized line drawing.

[0043] 2. By utilizing the parallel computing capability of GPU, the entire system achieves real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of the relationship between various modules of the system of the present invention.

[0045] Figure 2 This is a flowchart of the rasterization module.

[0046] Figure 3 The figure is a flowchart of the vectorization module.

[0047] Figure 4 This is a flow chart of the line extraction module. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with specific embodiments.

[0049] This embodiment provides a real-time mesh outline vectorization and rendering system based on GPU, which is a real-time mesh outline vectorization and rendering system developed using C# and HLSL languages ​​and can be run on Windows devices. The relationship between the modules of the system is as follows: Figure 1 As shown in the system flow chart Figure 2 , 3, 4. It includes the following functional modules:

[0050] Mesh import module, used to extract mesh data from local mesh model files and perform preprocessing;

[0051] The rasterization module extracts the mesh outline using the outline definition, converts the mesh outline into fragments using scan conversion, removes occlusions from the fragments using the depth buffer and depth test, and integrates the unoccluded fragments into a set of outline pixels;

[0052] The vectorization module uses the parallelized Potrace algorithm to obtain a set of closed paths around the contour pixels;

[0053] The line extraction module is used to segment the closed path according to the geometric information of the grid outline and extract a set of line paths that can reflect the occlusion relationship of the scene;

[0054] The line rendering module is used to render a set of line paths into a set of two-dimensional lines with controllable styles.

[0055] Furthermore, the grid import module specifically performs the following operations:

[0056] 1) Read the required mesh data from the local computer, including obj and fbx formats, where the mesh data includes the face to vertex index array F v , vertex coordinate array P v And the vertex normal array N v ;

[0057] 2) Preprocess the mesh data imported in step 1) and output the edge-to-face index array E f , edge-to-vertex index array E v , face normal array N f .

[0058] Further, the rasterization module extracts contour pixels from the grid loaded by the grid import module, such as Figure 2 As shown, it includes a contour extraction module, a scan conversion module and a depth test module, wherein:

[0059] The contour extraction module uses the mesh data obtained by the mesh import module to extract a group of mesh edges as contours, including the following steps:

[0060] The GPU threads process all mesh edges in parallel and determine whether a mesh edge is a contour based on the following definition:

[0061] (f 0 ,f 1 )=E f [e]

[0062] (N f [f 0 ]·V)(N f [f 1 ]·V)<0

[0063] Where e is the mesh edge, from the index array E f The triangle f obtained in 0 ,f 1 Adjacent to e, from the face normal array N f N f [f 0 ],N f [f 1 ] are f 0 ,f 1 The surface normal of , V is the vector from any point on e to the viewpoint;

[0064] Use the Stream Compaction algorithm to remove non-contour edges from the mesh edges;

[0065] The scan conversion module is used to convert the contour edge output by the contour extraction module into a set of fragments, each fragment being a two-dimensional object with a pixel size and contour geometric data, including the following steps:

[0066] Each contour edge e is transformed into c Scan and convert into a set of fragments;

[0067] The attributes of the fragment are calculated by performing perspective interpolation on the two vertices of the contour edge, where the attributes of each fragment include the screen tangent vector of the contour edge to which the fragment belongs, the interpolated vertex normal and depth information;

[0068] The depth test module performs occlusion culling on the fragments converted by the scan conversion module and extracts the outline pixels, including the following steps:

[0069] The first visibility test is performed on the fragment using the scene depth buffer. If more than two pixels in the fragment's 3×3 neighborhood have depth values ​​greater than the fragment, the fragment passes the first visibility test.

[0070] The fragments that pass the first visibility test are tested a second time using hardware depth sorting, and the fragment with the smallest depth is converted to a contour pixel in the image.

[0071] like Figure 3 As shown, the vectorization module includes a path decomposition module and a path linking module, wherein:

[0072] The path decomposition module extracts pixel edges using the contour pixels output by the rasterization module, establishes adjacency relationships between pixel edges using the Potrace algorithm, and outputs a set of circular linked lists according to the adjacency relationships between pixel edges, including the following steps:

[0073] Each GPU thread corresponds to one or more contour pixels and observes its 3×3 neighborhood. For each background pixel adjacent to it in the neighborhood, the adjacent edges are marked as pixel edges.

[0074] Use the Potrace algorithm to determine the direction of each pixel edge, ensuring that it has contour pixels on the left and background pixels on the right, or background pixels on the left and contour pixels on the right;

[0075] The Potrace algorithm is used to establish an adjacency relationship for each pair of pixel edges that share pixel vertices, and a set of circular linked lists are output according to the adjacency relationship. Each node in the linked list corresponds to a pixel edge.

[0076] The path linking module converts a set of circular linked lists output by the path decomposition module into a set of linear lists using a parallel List Ranking algorithm, including the following steps:

[0077] Use List Ranking algorithm to select the head node for the circular linked list output by the path decomposition module;

[0078] The List Ranking algorithm is used to serialize the circular linked list. The pixel edges in each linked list are arranged into a list in order, and the obtained head node is the starting pixel edge of the list.

[0079] The line extraction module extracts the line path from the linear list output by the vectorization module, such as Figure 4 As shown in the figure, the hollow arrows represent the tangent direction of the contour pixels, and the solid arrow lines represent the linear list output from the vectorization module, which includes the following steps:

[0080] 1) Use local curve fitting to calculate the approximate tangent directions for all pixel edges;

[0081] 2) Use the contour pixels output by the rasterization module to analyze all pixel edges. If the tangent of the pixel edge has the same direction as the weighted average tangent of the contour pixels around it, then mark the pixel edge as 1, otherwise 0;

[0082] 3) According to the marks processed in step 2), use the GPU segmented scanning algorithm to split each list output by the vectorization module into a group of sub-lists, use the midpoint of each pixel edge in each sub-list marked as 1 as the path vertex, and arrange the points in the order of the pixel edges in the list as the line path.

[0083] The line rendering module renders the line path to the screen, comprising the following steps:

[0084] 1) Calculate the line grid using the line path output by the line extraction module, where the line grid can be calculated by placing a brush instance at each path point, or by expanding the line path;

[0085] 2) Rendering the line mesh calculated in step 1) to the screen through the GPU shader. The style of the screen lines is determined by the form of the line mesh output in step 1) and the implementation of the GPU shader. The GPU shader receives the vertex data of the line mesh and converts the vertices into screen pixels according to its implementation logic. The input vertex data includes its screen coordinates and other vertex attributes used for line drawing, including brush texture coordinates, line color and transparency.

[0086] The above-described embodiments are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A real-time grid outline vectorization and rendering system based on GPU, characterized by: include: Mesh import module, used to extract mesh data from local mesh model files and perform preprocessing; The rasterization module extracts the mesh outline using the outline definition, converts the mesh outline into fragments using scan conversion, removes occlusions from the fragments using the depth buffer and depth test, and integrates the unoccluded fragments into a set of outline pixels; The vectorization module uses the parallelized Potrace algorithm to obtain a set of closed paths around the contour pixels; The line extraction module is used to segment the closed path according to the geometric information of the grid outline and extract a set of line paths that can reflect the occlusion relationship of the scene; A line rendering module, used for rendering a set of line paths into a set of two-dimensional lines with controllable styles; The rasterization module includes a contour extraction module, a scan conversion module and a depth testing module, wherein: The contour extraction module uses the mesh data obtained by the mesh import module to extract a group of mesh edges as contours, including the following steps: The GPU threads process all mesh edges in parallel and determine whether a mesh edge is a contour based on the following definition: (f0,f1)=AND f [and] (N f [f0]·V)(N f [f1]·V)<0 Where e is the mesh edge, from the index array E f The triangular faces f0 and f1 obtained in the above are adjacent to e. From the face normal array N f N f [f0],N f [f1] are the surface normals of f0 and f1 respectively, and V is the vector from any point on e to the viewpoint; Use the Stream Compaction algorithm to remove non-contour edges from the mesh edges; The scan conversion module is used to convert the contour edge output by the contour extraction module into a set of fragments, each fragment being a two-dimensional object with a pixel size and contour geometric data, including the following steps: Each contour edge e is transformed into c Scan and convert into a set of fragments; The attributes of the fragment are calculated by performing perspective interpolation on the two vertices of the contour edge, where the attributes of each fragment include the screen tangent vector of the contour edge to which the fragment belongs, the interpolated vertex normal and depth information; The depth test module performs occlusion culling on the fragments converted by the scan conversion module and extracts the outline pixels, including the following steps: The first visibility test is performed on the fragment using the scene depth buffer. If the depth values ​​of more than two pixels in the fragment's 3×3 neighborhood are greater than the fragment, the fragment passes the first visibility test. The fragments that pass the first visibility test are tested for the second time using hardware depth sorting, and the fragments with the smallest depth are converted into outline pixels in the image; The vectorization module includes a path decomposition module and a path linking module, wherein: The path decomposition module extracts pixel edges using the contour pixels output by the rasterization module, establishes adjacency relationships between pixel edges using the Potrace algorithm, and outputs a set of circular linked lists according to the adjacency relationships between pixel edges, including the following steps: Each GPU thread corresponds to one or more contour pixels and observes its 3×3 neighborhood. For each background pixel adjacent to it in the neighborhood, the adjacent edges are marked as pixel edges. Use the Potrace algorithm to determine the direction of each pixel edge, ensuring that it has contour pixels on the left and background pixels on the right, or background pixels on the left and contour pixels on the right; The Potrace algorithm is used to establish an adjacency relationship for each pair of pixel edges that share pixel vertices, and a set of circular linked lists are output according to the adjacency relationship. Each node in the linked list corresponds to a pixel edge. The path linking module converts a set of circular linked lists output by the path decomposition module into a set of linear lists using a parallel List Ranking algorithm, including the following steps: Use List Ranking algorithm to select the head node for the circular linked list output by the path decomposition module; The List Ranking algorithm is used to serialize the circular linked list. The pixel edges in each linked list are arranged into a list in order, and the obtained head node is the starting pixel edge of the list.

2. The GPU-based real-time mesh outline vectorization and rendering system according to claim 1, characterized in that: The grid import module specifically performs the following operations: 1) Read the required mesh data from the local computer, including obj and fbx formats, where the mesh data includes the face to vertex index array F v , vertex coordinate array P v And the vertex normal array N v ; 2) Preprocess the mesh data imported in step 1) and output the edge-to-face index array E f , edge-to-vertex index array E v 、Surface normal array N f .

3. The GPU-based real-time mesh outline vectorization and rendering system according to claim 1, characterized in that: The line extraction module extracts the line path from the linear list output by the vectorization module, comprising the following steps: 1) Use local curve fitting to calculate the approximate tangent directions for all pixel edges; 2) Use the contour pixels output by the rasterization module to analyze all pixel edges. If the tangent of the pixel edge has the same direction as the weighted average tangent of the contour pixels around it, then mark the pixel edge as 1, otherwise 0; 3) According to the marks processed in step 2), use the GPU segmented scanning algorithm to split each list output by the vectorization module into a group of sub-lists, use the midpoint of each pixel edge in each sub-list marked as 1 as the path vertex, and arrange the points in the order of the pixel edges in the list as the line path.

4. The GPU-based real-time mesh outline vectorization and rendering system according to claim 1, characterized in that: The line rendering module renders the line path to the screen, comprising the following steps: 1) Calculate the line grid using the line path output by the line extraction module, where the line grid can be calculated by placing a brush instance at each path point, or by expanding the line path; 2) Rendering the line mesh calculated in step 1) to the screen through the GPU shader. The style of the screen lines is determined by the form of the line mesh output in step 1) and the implementation of the GPU shader. The GPU shader receives the vertex data of the line mesh and converts the vertices into screen pixels according to its implementation logic. The input vertex data includes its screen coordinates and other vertex attributes used for line drawing, including brush texture coordinates, line color and transparency.

Images