Sketch segmentation system and method based on graph neural network
Through the sketch segmentation system based on graph neural network, the geometric and topological information of the sketch is used for segmentation, which solves the problem of the inability to efficiently utilize the geometric information of CAD sketches in existing technologies, and realizes efficient and accurate sketch segmentation and model generation.
Patent Information
- Application Number
- CN202510874568.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-27
- Publication Date
- 2025-10-17
AI Technical Summary
Existing sketch segmentation methods cannot efficiently utilize the geometric information in CAD sketches, resulting in poor segmentation performance of sketch reconstruction models.
A sketch segmentation system based on graph neural network is adopted. The geometric attribute information and topological association relationship are extracted through the sketch stroke preprocessing module. The sketch is segmented by combining the multi-branch graph attention network architecture, and the geometric and topological information of the sketch strokes is used for segmentation.
It significantly improves the accuracy and robustness of sketch segmentation, can adapt to strokes of different directions and scales, lowers the professional threshold, enables non-professional users to quickly generate high-quality model sketches, and promotes the popularization of CAD technology.
Smart Images
Figure CN120807918A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of computer graphics and computer vision, and particularly relates to a sketch segmentation system and method based on a graph neural network. BACKGROUND
[0002] Sketch-Based Modeling (SBM) research automatically generates a model from a sketch, which can quickly shape the design concept of a user. Automatically generating a CAD model based on a sketch helps to reduce the software learning cost of a novice and the understanding and segmentation requirements for model generation steps, and is conducive to the daily use of CAD software and the promotion of the CAD software among non-professional user groups. Sketch understanding and segmentation are important steps for sketch reconstruction model. Existing sketch segmentation methods are often based on sketch data in the image or stroke sequence modal, and use CNN, RNN, graph neural network or multi-modal network to perform component-level segmentation on abstract sketches. However, these methods cannot efficiently utilize the geometric information in the CAD sketch. In view of this situation, a novel sketch segmentation method based on a graph neural network is proposed to extract the geometric information and topological information in the sketch and construct a graph structure to perform sketch segmentation, thereby achieving better sketch segmentation performance. SUMMARY
[0003] The application aims to provide a sketch segmentation method based on a graph neural network to solve the above technical problems.
[0004] To solve the above technical problems, the specific technical scheme of the sketch segmentation system and method based on a graph neural network is as follows:
[0005] A sketch segmentation system based on a graph neural network comprises a sketch stroke preprocessing module and a sketch segmentation module based on modeling features. The sketch stroke preprocessing module is used to receive a stroke sequence input by a user, independently store each stroke, extract geometric attribute information of each stroke, calculate and store topological correlation between strokes in the stroke sequence, and encode the geometric attribute information of the strokes extracted by the sketch stroke preprocessing module into embedding vectors as representations of nodes in the graph attention network, and construct an adjacency structure of the graph attention network according to the topological correlation determined by the sketch stroke preprocessing module. The sketch segmentation module based on modeling features adopts a graph attention network architecture comprising multiple parallel branches, and each branch of the multiple parallel branches corresponds to a different stroke spacing relationship.
[0006] The application further discloses a sketch segmentation method of the sketch segmentation system based on a graph neural network, which comprises the following steps:
[0007] Step 1: sketch stroke preprocessing,
[0008] Step 2: Sketch segmentation based on modeling features.
[0009] Furthermore, the step 1 specifically includes the following steps:
[0010] Step 1.1: The sketch stroke preprocessing module extracts the geometric attribute information of each stroke, specifically including: identifying the starting point and ending point of the stroke;
[0011] Step 1.2: Sampling the strokes;
[0012] Step 1.3: Apply the geometric classification algorithm to distinguish between straight line strokes and curved line strokes;
[0013] Step 1.4: Generate a minimum bounding rectangle for each stroke; calculate the size of the minimum bounding rectangle and the angle between its diagonal and the coordinate axis to represent the direction and scale attributes of the stroke;
[0014] Step 1.5: The sketch stroke preprocessing module calculates the topological association relationship between strokes, specifically including: determining the shortest Euclidean distance between the stroke endpoints and other stroke sampling points; when the shortest Euclidean distance is less than a preset connection threshold, confirming that there is a topological connection between the corresponding strokes.
[0015] Furthermore, the step 2 specifically includes the following steps:
[0016] Step 2.1: Generate node input features using a dedicated encoder designed based on stroke geometric attribute information;
[0017] Step 2.2: defining graph nodes using the embedded vectors encoded with the stroke geometric attribute information, and constructing an adjacency matrix between the nodes based on the topological association relationship;
[0018] Step 2.3: The multi-branch structure is designed based on the differences in topological distances between nodes. This structure enables any node to absorb contextual information from its multiple topological neighborhoods through a multi-branch attention mechanism.
[0019] Furthermore, the step 2.1 specifically includes the following steps:
[0020] Step 2.1.1: First, the geometric features of each edge obtained by the sketch stroke preprocessing module are used as the basic input information of each node, including: the sampling coordinates of each stroke, the start and end coordinates, the geometric type of the stroke, which is encoded as a one-hot vector, the diagonal tilt angle α of the bounding box, the diagonal length l, and the size w and h of the bounding box, as shown below:
[0021] f i =[{(x,y) n＝1,2,...,10}, (x,y)1,(x,y) 10 , type, α, l, w, h] (9)
[0022] Step 2.1.2: The initial node information is obtained by a stroke encoding unit E containing a multi-layer perceptron machine MLP s Encoding as f i , all vertex features are stacked as f i 0 ∈R N*8 , where N represents the embedding dimension of the encoding block, 8 represents the number of input features, each vertex has 8 initial input information, and the input features of each vertex are as follows:
[0023] f i 0 = Concat(E s (f i )) (10)
[0024] Further, the step 2.2 specifically comprises the following steps:
[0025] Step 2.2.1: After obtaining the vertex feature V, the initial node feature is first input into a graph attention module GAT head, and the feature is enhanced by the vertex feature connected thereto;
[0026] Step 2.2.2: A ResGAT main body is subsequently input, which is used to update the vertex feature and is composed of 3 residual connected graph attention units, each of which uses a residual connection to enhance stability, so as to achieve more reliable performance;
[0027] Step 2.2.3: The graph of the l-th layer is G l = (V l ,E l ,F l ), where V l is the node feature of the graph, E l is the edge of the graph, is a set composed of each node feature of the l-th layer of the graph, and each GAT module updates the node feature of the graph as follows: the next layer node feature of a node is aggregated from the node features of the nodes adjacent to the node according to the attention weight, and a residual connection is used:
[0028]
[0029] f i l represents the node feature of the previous layer, f i l+1 represents the updated node feature of the next layer, W v is a learnable linear transformation matrix, and αij is the attention weight between node i and node j, which is calculated as follows:
[0030]
[0031] W q is a learnable linear transformation matrix, is an attention mechanism that calculates the importance of neighboring node j to node i, and the softmax operation is used to normalize the coefficients of different neighboring nodes, and within a single branch, the features of each node are updated through GAT calculation, and the output of each branch is the superposition of the node feature output of each GAT module:
[0032] f branch =Concat(f head ,f1,f2,f3)。 (13)
[0033] Further, the step 2.3 specifically comprises the following steps:
[0034] The sketch feature segmentation network designs a multi-branch network framework, each branch extracts node features of different distances from the node and updates the features of the node, and the basic branch focuses on the feature update between the directly connected stroke nodes, and the stroke features with a topological distance of 2 from the node are used as the input of another branch; the global stroke features not directly connected to the node are encoded and fused through the global branch, and the final output of the multi-branch network is as follows, which fuses the node features from different topological distances:
[0035] F=Concat(f D1 ,f D2 ,f global ) (14)
[0036] The fused features will be input into a multi-layer MLP block, and the modeling category result corresponding to each edge will be output:
[0037] Y=Softmax(MLP(F)) (15)
[0038] The loss of segmentation is calculated using cross entropy:
[0039] loss segmentation =CrossEntropy(Y,Y gt )。 (16)
[0040] The sketch segmentation system and method based on the graph neural network have the following advantages:
[0041] 1. Efficiently utilize geometric and topological information
[0042] The present application extracts the geometric attribute information (such as starting point, ending point, geometric category, minimum circumscribed rectangle, etc.) of each stroke through the sketch stroke preprocessing module, and calculates the topological correlation between strokes, fully exploiting the geometric and topological features in the sketch. The efficient use of this information significantly improves the accuracy and robustness of sketch segmentation.
[0043] 2. Multi-branch graph attention network architecture
[0044] A graph attention network (GAT) containing multiple parallel branches is adopted, and each branch extracts and fuses node features for different topological distances. This design enables the network to capture both local and global context information, enhancing the model's understanding of complex sketch structures.
[0045] 3. Dynamic feature aggregation and update
[0046] Through the residual connection graph attention unit (ResGAT) and multi-layer perceptron (MLP), dynamic updating and enhancement of node features are achieved. The attention mechanism can adaptively weight the importance of neighboring nodes, more accurately aggregate key information, and improve the reliability of the segmentation results.
[0047] 4. Strong adaptability
[0048] The present application can handle both straight and curved strokes, and adapt to strokes of different directions and scales through geometric properties such as minimum circumscribed rectangle. The multi-branch structure further enhances the model's adaptability to different topological distance stroke relationships, making it suitable for a variety of sketch inputs.
[0049] 5. Lower professional threshold
[0050] Through automated sketch segmentation and modeling feature extraction, the present application reduces the user's operation threshold for professional CAD software, enabling non-professional users to quickly generate high-quality model sketches, promoting the popularization and application of CAD technology.
[0051] 6. Superior performance
[0052] Experiments show that the present application performs well in the sketch segmentation task, and the use of cross-entropy loss function further optimizes the model training process, ensuring high precision and stability of the segmentation results.
[0053] In summary, the present application effectively combines innovative graph neural network architecture and geometric and topological features to achieve efficient and accurate sketch segmentation, providing strong technical support for the field of sketch modeling. BRIEF DESCRIPTION OF DRAWINGS
[0054] Figure 1 The network framework diagram of the segmentation network provided for the embodiments of the present application. DETAILED DESCRIPTION
[0055] In order to better understand the purpose, structure and function of the present application, the following will be further described in detail in combination with the drawings.
[0056] The sketch stroke preprocessing module is used for receiving a stroke sequence input by a user, independently storing each stroke, extracting geometric attribute information of each stroke, the geometric attribute information at least including a geometric category, and calculating and storing a topological correlation relationship between strokes in the stroke sequence. The sketch stroke preprocessing module extracts stroke geometric attribute information, and encodes the stroke geometric attribute information into an embedding vector as a representation of each node in the graph attention network, and constructs an adjacency structure of the graph attention network according to the topological correlation relationship determined by the sketch stroke preprocessing module. The plurality of parallel branches each correspond to a different stroke spacing relationship. The network architecture enables any node to interact with other nodes based on different spacing relationships and aggregate stroke features from different scale contexts.
[0057] As shown in Figure 1 The present embodiment discloses a sketch segmentation method based on a graph neural network, specifically including the following steps:
[0058] Step 1: a sketch stroke preprocessing module,
[0059] Step 1.1: the sketch stroke preprocessing module extracts geometric attribute information of each stroke, specifically including identifying a starting point and an ending point of a stroke;
[0060] Step 1.2: sampling a stroke;
[0061] Step 1.3: applying a geometric category discrimination algorithm to distinguish a straight line type stroke from a curve type stroke;
[0062] Step 1.4: generating a minimum circumscribed rectangle for each stroke; calculating a size of the minimum circumscribed rectangle and an included angle of a diagonal line of the minimum circumscribed rectangle with a coordinate axis to represent a direction and a scale attribute of the stroke;
[0063] Step 1.5: the sketch stroke preprocessing module calculates a topological correlation relationship between strokes, specifically including determining a shortest Euclidean distance between a stroke endpoint and a sampling point of another stroke; when the shortest Euclidean distance is less than a preset connection threshold, confirming that there is a topological connection between the corresponding strokes.
[0064] Step 2: a sketch segmentation module based on modeling features.
[0065] Step 2.1: generating node input features using a special encoder designed based on stroke geometric attribute information;
[0066] 2.1.1 Firstly, the geometric features of each edge obtained by the sketch stroke preprocessing module are taken as the basic input information of each node, including: the sampling coordinates of each stroke (10 points are sampled for each stroke), the start point and end point coordinates, the geometric type of the stroke, which is encoded into a one-hot vector, the diagonal line inclination angle a of the bounding box, the diagonal line length l, and the size w, h of the bounding box, as shown below:
[0067] f i = [{(x,y) n＝1,2,...,10}, (x,y)1,(x,y) 10 , type, a, l, w, h] (17)
[0068] 2.1.2 The initial node information is encoded into f s by a stroke encoding unit E i containing multiple layers of perceptron machines (MLP). All vertex features are stacked into f i 0 ∈R N*8 , where N represents the embedding dimension of the encoding block, and 8 represents the number of input features, according to the description in the previous section, each vertex has 8 initial input information. The input features of each vertex are as follows:
[0069] f i 0 = Concat(E s (f i )) (18)
[0070] Step 2.2: defining the graph nodes with the embedding vectors formed by encoding the stroke geometric attribute information, and constructing the adjacency matrix between the nodes according to the topological association relationship;
[0071] Step 2.2.1: After obtaining the vertex feature V l , the initial node feature is first input into a graph attention module (GAT) head, and its feature is enhanced by the vertex feature connected thereto;
[0072] Step 2.2.2: a ResGAT backbone is subsequently input, which is used to update the vertex feature and is composed of 3 residual connected graph attention units, each unit uses residual connection to enhance stability, so as to achieve more reliable performance.
[0073] Step 2.2.3: the graph of the l-th layer is G l = (V l , E l , Fl ), where V l is the node feature of the graph, E l is the edge of the graph, is the set of node features of the l-th layer of the graph. Each GAT module updates the node features of the graph as follows: the node features of the next layer of a node are aggregated from the node features of its neighbors with attention weights and a residual connection:
[0074]
[0075] f i l represents the node features of the previous layer, f i l+1 represents the updated node features of the next layer, W v is a learnable linear transformation matrix, a ij is the attention weight between node i and node j, which is calculated as follows:
[0076]
[0077] W q is a learnable linear transformation matrix, is the attention mechanism used to calculate the importance of neighboring node j to node i, and the softmax operation is used to normalize the coefficients of different neighboring nodes. Within a single branch, the features of each node are updated by GAT calculation, and the output of each branch is the superposition of the node feature outputs of each GAT module:
[0078] f branch = Concat(f head ,f1,f2,f3) (21)
[0079] Step 2.3: The multi-branch structure is designed according to the difference in the topological distance between nodes. This structure enables any node to absorb context information from its multi-topological neighborhood range through the multi-branch attention mechanism.
[0080] The sketch feature segmentation network designs a multi-branch network framework, each branch extracts node features of different distances from the node and updates the node features. The basic branch focuses on feature updating between directly connected stroke nodes, and stroke features with a topological distance of 2 from the node are used as input to another branch. In addition, the method also encodes global stroke features that are not directly connected to the node and fuses them through a global branch. The final output of the multi-branch network is as follows, which fuses node features from different topological distances:
[0081] F = Concat(f D1 ,fD2 f global ) (22)
[0082] The fusion features will be input into a multi-layer MLP block, and the modeling category result corresponding to each edge will be output:
[0083] Y = Softmax (MLP (F)) (23)
[0084] The loss of segmentation is calculated using cross entropy:
[0085] loss segmentation = CrossEntropy (Y, Y gt ) (24)
[0086] To sum up, the method of the present application proposes a sketch segmentation method based on graph neural network, including a sketch stroke preprocessing module and a sketch segmentation module based on modeling features. A feature extraction module based on geometric information and a topological information construction module and a multi-branch graph attention segmentation network are designed, so as to efficiently fuse the geometric information in the sketch and obtain better segmentation performance.
[0087] It can be understood that the present application is described through some embodiments, and those skilled in the art know that various changes or equivalent replacements can be made to these features and embodiments without departing from the spirit and scope of the present application. In addition, under the guidance of the present application, these features and embodiments can be modified to adapt to specific conditions and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application are within the scope of protection of the present application.
Claims
1. A sketch segmentation system based on graph neural network, characterized by: It includes a sketch stroke preprocessing module and a sketch segmentation module based on modeling features. The sketch stroke preprocessing module is used to receive a stroke sequence input by a user, independently store each stroke, extract geometric attribute information of each stroke, wherein the geometric attribute information at least includes a geometric category, and calculate and store the topological association relationship between each stroke in the stroke sequence; the sketch segmentation module based on modeling features adopts a graph attention network architecture containing multiple parallel branches, encodes the stroke geometric attribute information extracted by the sketch stroke preprocessing module into an embedded vector as a representation of each node in the graph attention network, and constructs the adjacency structure of the graph attention network according to the topological association relationship determined by the sketch stroke preprocessing module; each branch of the multiple parallel branches corresponds to a different stroke spacing relationship.
2. A sketch segmentation method based on a graph neural network sketch segmentation system according to claim 1, characterized in that: The steps include: Step 1: Sketch stroke preprocessing, Step 2: Sketch segmentation based on modeling features.
3. The sketch segmentation method according to claim 2, characterized in that: The step 1 specifically includes the following steps: Step 1.1: The sketch stroke preprocessing module extracts the geometric attribute information of each stroke, specifically including: identifying the starting point and ending point of the stroke; Step 1.2: Sampling the strokes; Step 1.3: Apply the geometric classification algorithm to distinguish between straight line strokes and curved line strokes; Step 1.4: Generate a minimum bounding rectangle for each stroke; calculate the size of the minimum bounding rectangle and the angle between its diagonal and the coordinate axis to represent the direction and scale attributes of the stroke; Step 1.5: The sketch stroke preprocessing module calculates the topological association relationship between strokes, specifically including: determining the shortest Euclidean distance between the stroke endpoints and other stroke sampling points; when the shortest Euclidean distance is less than a preset connection threshold, confirming that there is a topological connection between the corresponding strokes.
4. The sketch segmentation method according to claim 3, characterized in that: The step 2 specifically includes the following steps: Step 2.1: Generate node input features using a dedicated encoder designed based on stroke geometric attribute information; Step 2.2: defining graph nodes using the embedded vectors encoded with the stroke geometric attribute information, and constructing an adjacency matrix between the nodes based on the topological association relationship; Step 2.3: The multi-branch structure is designed based on the differences in topological distances between nodes. This structure enables any node to absorb contextual information from its multiple topological neighborhoods through a multi-branch attention mechanism.
5. The sketch segmentation method according to claim 4, characterized in that: The step 2.1 specifically includes the following steps: Step 2.1.1: First, the geometric features of each edge obtained by the sketch stroke preprocessing module are used as the basic input information of each node, including: the sampling coordinates of each stroke, the start and end coordinates, the geometric type of the stroke, which is encoded as a one-hot vector, the diagonal tilt angle α of the bounding box, the diagonal length l, and the size w and h of the bounding box, as shown below: f i =[{(x,y) n＝1,2,...,10 }, (x,y)1 ,(x,y) 10 , type, α, l, w, h] (1) Step 2.1.2: The initial node information is passed through the stroke encoding unit E containing the multi-layer perceptron machine MLP s Coded as f i , all vertex features are stacked as Where N represents the embedding dimension of the encoding block, 8 represents the number of input features, each vertex has 8 initial input information, and the input features of each vertex are as follows:
6. The sketch segmentation method according to claim 4, characterized in that: The step 2.2 specifically includes the following steps: Step 2.2.1: After obtaining the vertex features Vi, the initial node features are first input into a graph attention module GAT head, and their features are enhanced by the vertex features connected to it; Step 2.2.2: A ResGAT backbone is then fed into the network to update vertex features. It consists of three residually connected graph attention units, each of which uses residual connections to enhance stability and achieve more reliable performance. Step 2.2.3: The graph of the lth layer is G l =(V l ,E l ,F l ), where V l is the node feature of the graph, E l is the edge of the graph, The set of node features of each node in the lth layer of the graph is composed of each GAT module. Each GAT module updates the node features of the graph as follows: the node features of the next layer of a node are aggregated with the node features adjacent to the node according to the attention weight, and residual connections are used: Represents the node features of the previous layer, Represents the updated node features of the next layer, W v is a learnable linear transformation matrix, α ij is the attention weight between node i and node j, which is calculated as follows: W q is a learnable linear transformation matrix, The attention mechanism is used to calculate the importance of neighboring node j to node i. The softmax operation is used to normalize the coefficients of different neighboring nodes. Within a single branch, the features of each node will be updated through GAT calculation. The output of each branch is the superposition of the feature outputs of each GAT module node: f branch =Concat(f head ,f1,f2,f3)。 (5) 7. The sketch segmentation method according to claim 4, characterized in that: The step 2.3 specifically includes the following steps: A multi-branch network framework was designed for the sketch feature segmentation network. Each branch extracts node features at different distances from a node and uses them to update the node features. The basic branch focuses on updating features between directly connected stroke nodes, and stroke features with a topological distance of 2 from the node serve as input to another branch. Global stroke features that are not directly connected to the node are encoded and fused through the global branch. The final output of the multi-branch network is as follows, fusing node features from different topological distances: F=Concat(f D1 ,f D2 ,f global ) (6) The fused features will be input into a multi-layer MLP block, and the modeling category results corresponding to each edge will be output: Y=Softmax(MLP(F)) (7) The loss of segmentation is calculated using cross entropy: loss segmentation =CrossEntropy(Y,Y gt ) (8)。