End-to-end shape recognition method and system based on complex network
By employing an end-to-end shape recognition method based on complex networks, which automatically learns shape features using graph convolutional networks and fully connected layers, this method solves the problem of limited accuracy in manual feature extraction in existing methods and achieves efficient and accurate shape classification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NAT SPACE SCI CENT CAS
- Filing Date
- 2024-11-21
- Publication Date
- 2026-05-22
AI Technical Summary
Existing shape classification methods rely on manually extracted topological features when processing shape information, which limits classification accuracy, makes them sensitive to small changes in boundary points, has high computational complexity, and ignores local feature interactions, thus affecting classification performance.
An end-to-end shape recognition method based on complex networks is adopted. By densely discretizing the image edges, local features of key points are extracted, a multi-level complex network is constructed, and a graph convolutional network and a fully connected layer are combined for shape classification, automatically learning the most suitable features.
It improves the accuracy and robustness of shape classification, enhances adaptability to different shapes, and achieves efficient shape description and classification.
Smart Images
Figure CN122073015A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision technology, and in particular relates to an end-to-end shape recognition method and system based on complex networks. Background Technology
[0002] With the widespread application of Convolutional Neural Networks (CNNs) in computer vision, numerous applications have benefited greatly. However, CNNs tend to focus on capturing lighting, color, and texture information in images, while relatively neglecting shape information. In specific scenarios such as part recognition in industrial automation or leaf classification in botany, shape information is particularly important.
[0003] Therefore, shape classification has become an important research direction in computer vision. Existing shape classification methods can be broadly divided into two categories: methods based on ordered point sets and methods based on unordered point sets. Ordered point set methods trace the shape boundary to form a parametric curve, and then further analyze its geometric properties. For example, the Fourier description method extracts shape features by performing a Fourier transform on the curve of the shape boundary; while the curvature scale space (CSS) method extracts shape features by extracting the curvature parameters of the shape boundary. However, these methods rely on the order of the shape boundary points and are very sensitive to small changes or noise at the boundary points, which may lead to inaccurate extraction of shape features. At the same time, in order to ensure the correct order of the shape boundary points, additional preprocessing steps are usually required on the original data, such as smoothing and sorting the boundary points, which increases computational complexity.
[0004] In contrast, unordered point set methods directly extract topological features from the shape boundary point set, such as geometric invariant moments, Zernike moments, distance sets, and complex networks. Compared to ordered point set-based methods, unordered point set-based methods overcome the limitation of only handling contour shapes and do not require sorting of boundary points, thus avoiding the instability caused by sorting operations. However, existing unordered point set-based methods focus too much on extracting local features and neglect the interaction of local features at different locations. Secondly, most of these methods use manual feature extraction methods to extract shape features, which can lead to the extracted features not being optimal for the target task, resulting in a decrease in algorithm performance.
[0005] Methods based on complex networks classify shapes by modeling the network's topological features. This approach can flexibly handle imperfect shape contours and possesses good rotation invariance. However, most existing methods rely on manually extracted topological features, which may limit classification accuracy. Summary of the Invention
[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and propose an end-to-end shape recognition method based on complex networks. This invention also discloses an end-to-end shape recognition system based on complex networks. This method combines the excellent shape representation capabilities of complex network models with the powerful topology extraction capabilities of GNNs, achieving efficient shape description. Furthermore, the dynamic evolution process of complex networks provides diverse representations of the topology of point sets, enhancing the robustness of the method.
[0007] To achieve the above objectives, this invention proposes an end-to-end shape recognition method based on complex networks, comprising:
[0008] Step 1) Densely discretize the edges of the image to be identified to obtain a set of edge description points;
[0009] Step 2) Select a series of key points at equal intervals on the obtained edge description point set;
[0010] Step 3) Extract local features of the point set by describing the edges around the keypoints using shape context;
[0011] Step 4) Treat each key point as a node, define the edges in the network according to the spatial correlation between key points, and use a dynamic evolution strategy to construct a multi-layered complex network.
[0012] Step 5) Process local features and topological features of complex networks through graph convolutional networks to obtain a robust representation of the shape classification task;
[0013] Step 6) Classify the data using a fully connected layer network to obtain the classification results and achieve end-to-end shape recognition.
[0014] Preferably, step 1) includes:
[0015] The edge description point set D, containing N points, is obtained based on the shape and contour of the image to be identified.
[0016] Preferably, step 2) includes:
[0017] Select a key point every z points to obtain a point set C containing O key points.
[0018] Preferably, step 3) includes:
[0019] For each key point in the point set C, a polar coordinate system is defined with each key point as the center, where the polar radius and polar angle are divided in a uniform manner, and the coordinate system is divided into M×H grids;
[0020] Count the number of points in the edge description point set D that fall into each cell, and normalize the number of points in each cell;
[0021] Unfold the normalized histogram, encode the position information of the key points, and obtain the local feature F = {f1, f2, …, f
[0029] } corresponding to each key point.
[0022] Preferably, in step 4), each key point is taken as a node, and the edges in the network are defined according to the spatial correlation between key points, including:
[0023] For N' key points, an N'×N' distance matrix B is obtained, where B(i, j) represents the Euclidean distance between key points p' i and p' j ;
[0024] According to the set threshold t, when B(i, j) < t, there is an edge between key points p' i and p' j , which is represented using the adjacency matrix A:
[0025]
[0026] Thus, an undirected graph model G T =(V, E, w, A, X) of the complex network is obtained; where, V represents the set of nodes, E represents the set of edges, w is the weight, and X is the feature matrix.
[0027] Preferably, in step 4), a multi-level complex network is constructed using a dynamic evolution strategy; including:
[0028] By selecting L gradually increasing distance thresholds such that T = {t1, t2, …, t L}, a series of network models
[0029] Preferably, step 5) includes:
[0030] Extract node features from the network model G using a graph convolutional neural network, and update each node feature vector iteratively;
[0031] Adopt a hierarchical graph pooling operation to adaptively select a subset of nodes to form an induced subgraph of the next layer;
[0032] Use an institutional learning mechanism to reconstruct the edges and weights in the graph model;
[0033] Fuse the multi-scale graph representations after multiple pooling and convolution operations to form a comprehensive graph-level feature representation. <00The graph-level feature representation is classified through a multilayer perceptron and a softmax layer to obtain the probability distribution of the categories, thereby achieving end-to-end shape recognition.
[0036] On the other hand, the present invention also provides an end-to-end shape recognition system based on complex networks, comprising:
[0037] The dense discretization module is used to densely discretize the edges of the image to be recognized to obtain the edge description point set. The key point extraction module is used to select a series of key points at equal intervals on the shape contour of the image to be recognized.
[0038] The local feature extraction module is used to extract local features of the edge description point set around the keypoint using the shape context;
[0039] The complex network construction module is used to treat each key point as a node, define the edges in the network based on the spatial correlation between key points, and construct multi-level complex networks using a dynamic evolution strategy.
[0040] The graph-level feature representation output module is used to process local features and topological features of complex networks through graph convolutional networks to obtain a robust representation for shape classification tasks.
[0041] The classification and recognition module is used to classify shapes through a fully connected layer network, obtain classification results, and realize end-to-end shape recognition.
[0042] Compared with the prior art, the advantages of the present invention are:
[0043] 1. This invention proposes a novel method that combines global and local topological features of shape. It extracts local features of the contour from the shape context and constructs a topological model of these contexts using a complex network. This method improves the robustness of feature representation.
[0044] 2. By integrating Complex Networks (CN) and Graph Convolutional Neural Networks (GCN), an end-to-end shape classification method was implemented. This method enables the model to automatically learn the features best suited for the shape classification task, improving classification accuracy.
[0045] 3. A multi-layered complex network is formed through a dynamic evolution strategy to comprehensively describe the topological information of the shape. This not only provides richer feature descriptions but also enhances the adaptability and robustness of the method to different shape classification tasks. Attached Figure Description
[0046] Figure 1 This is a flowchart of the end-to-end shape recognition method based on complex networks according to the present invention. Detailed Implementation
[0047] In this method, to reduce the number of nodes in the complex network and ensure effective training of the subsequent GNN network, we first select a series of key points at equal intervals on the shape contour. Next, we extract local features around these key points using the shape context. Then, we treat each key point as a node and construct a multi-layered complex network model. By processing these local features and the topological features of the complex network through a Graph Convolutional Network (GCN), we obtain a robust representation for the shape classification task. Finally, classification is performed using a fully connected layer network. We name this approach "Learnable Multi-Layered Complex Network Model".
[0048] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.
[0049] Example 1
[0050] Embodiment 1 of this invention proposes an end-to-end shape recognition method based on complex networks. After extracting the edge point set of an image, the large number of points leads to a significant waste of computational resources when performing GNN operations after constructing a complex network. Reducing the number of points results in the loss of point set information, leading to a decrease in classification accuracy. To address this issue, we employ shape context to describe local features, thereby obtaining more complete global topological information with minimal loss of point set location information.
[0051] Includes the following steps:
[0052] Step 1) Densely discretize the edges of the image to be identified to obtain a set of edge description points;
[0053] Step 2) Select a series of key points at equal intervals on the obtained edge description point set;
[0054] Step 3) Extract local features of the point set by describing the edges around the keypoints using shape context;
[0055] Step 4) Treat each key point as a node, define the edges in the network according to the spatial correlation between key points, and use a dynamic evolution strategy to construct a multi-layered complex network.
[0056] Step 5) Process local features and topological features of complex networks through graph convolutional networks to obtain a robust representation of the shape classification task;
[0057] Step 6) Classify the data using a fully connected layer network to obtain the classification results and achieve end-to-end shape recognition.
[0058] The core steps are explained in detail below:
[0059] 1. Local Feature Extraction
[0060] After extracting the edge point set of the image, the large number of point sets will cause a lot of computational resources to be wasted when performing GNN operations after building a complex network. To address this issue, we use shape context to describe local features, thereby obtaining more complete global topological information with less loss of point set position information.
[0061] A shape context is a local shape descriptor that can finely characterize the neighborhood environment of each point in a point set. For a point set D = {p1, p2, ..., p...} containing N points... N We select a series of key points. For each key point p i =(x i ,y i ), whose shape context C(p i It is defined as a histogram of the spatial distribution of other points relative to the key point.
[0062] Specifically, for the obtained set of shape edge points D, we select one keypoint every z points, for a total of O keypoints, and define these keypoints as B = {b1, b2, ..., b...} O Then, a polar coordinate system is defined centered on each keypoint, where the polar radius and polar angle are divided in a uniform manner, thus dividing the coordinate system into M×H grids. For keypoint b... i Each cell C jk (b i (where j represents the polar radius index and k represents the polar angle index), we count the number of points n that fall into this cell. jk (b i Then, normalize the value in each cell.
[0063]
[0064] Finally, the normalized histogram is expanded and encoded with the keypoint location information through an encoder to obtain the keypoint set B = {b1, b2, ..., b...}. o The local features F = {f1, f2, ..., f} corresponding to each point in the matrix are: N′ The overall flowchart is as follows: Figure 1 As shown in (a).
[0065] In this way, we obtained the local topological information of each key point, providing rich and discriminative information for subsequent complex network construction and feature learning.
[0066] 2. Construction of Complex Network Models
[0067] Constructing a complex network model is one of the core steps of this method, aiming to capture the global topological structure of the shape. This section will introduce in detail how to construct a complex network model based on the extracted key points and their local features, and generate a multi-level network model through a dynamic evolution strategy to provide a more comprehensive shape description.
[0068] First, each key point is regarded as a node in the complex network. Then, the edges in the network are defined according to the spatial correlation between the key points.
[0069] Specifically, for a set of N' key points, an N'×N' distance matrix B can be obtained, where B(i,j) represents the Euclidean distance between key points p' i and p' j .
[0070]
[0071] Select a threshold t. If B(i,j) < t, there is an edge between key points p' i and p' j . It is represented using the adjacency matrix A.
[0072]
[0073] To sum up, the undirected graph model G of the complex network constructed in this way T =(V, E, w, A, X), where t is the threshold, V is the set of nodes V = {v1, v2,..., v N′}. E is the set of edges, and each edge connects two vertices in the graph E = {(v i , v j ) | B(i,j) < t}. w: E → R is a weight function that maps each edge to a value in the real number set R, and this value is the weight of the edge; A is the adjacency matrix. X is the feature matrix, X = {x1, x2,..., x N′}, here, V is the mapping of D', p' i = v i , X is the mapping of F, x i = f i .
[0074] The undirected graph of the complex network constructed above not only captures the local details of the shape but also shows the overall structure of the shape through the connection of edges.
[0075] To capture the shape characteristics more comprehensively, we also adopt a dynamic evolution strategy to construct a multi-level complex network model. By selecting L gradually increasing distance thresholds such that T = {t1, t2,..., t L}, a series of network models are generated These graphical models capture shape features at different scales, providing rich topological information for subsequent graph neural network learning. By adjusting the distance threshold T, the connection density of the network can be controlled, thus affecting the network's ability to describe shapes. Figure 1 As shown in (b), this flexibility allows our method to adapt to the needs of different shape classification tasks.
[0076] As the threshold T increases, we can generate a series of complex network models, each reflecting shape characteristics at different connection densities. Lower thresholds generate sparser networks, while higher thresholds produce denser networks, which helps to understand the topology of the shape from multiple perspectives.
[0077] 3. Using graph neural networks for graph classification
[0078] After obtaining multi-layered complex network graph models, we use Graph Convolutional Neural Networks (GCNs) to classify these graph models. This section details how to use HGP-SL (Hierarchical Graph Pooling with Structure Learning), an advanced graph classification method, to process structured graph data and achieve graph classification. HGP-SL is an innovative graph pooling operator that combines graph pooling with structure learning mechanisms. It can generate hierarchical representations of graphs and can be integrated into various GCN architectures. This method first extracts features from nodes in the graph using GCNs, and then adaptively selects a subset of nodes to form the induced subgraph for the next layer using hierarchical graph pooling operations. To maintain the integrity of the graph's topological information, a structure learning mechanism is introduced to learn an optimized graph structure for each pooled layer. Finally, the graph after multiple pooling and convolution operations is compressed into a global feature vector. The feature vectors obtained from multiple layers are fused and graph classification is performed using a Multilayer Perceptron (MLP) and a softmax layer. The specific steps are as follows:
[0079] Feature extraction: Node features are extracted from the multi-layered complex network graph model G obtained above using GCN. These features integrate local and global information about the nodes. For each layer of the graph in G... Each node v i Its initial eigenvector is H (0) =X. GCN captures local and global structural information of a graph by iteratively updating node representations. Specifically, in the k-th layer GCN, the hidden representation H of a node... (k) Update using the following formula:
[0080]
[0081] Where σ(·) is the activation function. It is an adjacency matrix with added self-connections. yes The degree matrix, W (k) It is the trainable weight matrix of the k-th layer.
[0082] Hierarchical Graph Pooling and Structure Learning: After updating the feature vector of each node, hierarchical graph pooling using HGP-SL progressively compresses each graph while learning an optimized graph structure to preserve key substructures. The pooling operation adaptively selects a subset of nodes to form the induced subgraph for the next layer. An information score for each node in the graph is calculated using the following formula to evaluate the information contained in each node within its neighborhood. Typically, if a node's representation can be reconstructed from its neighborhood representations, it means that the node can be removed from the pooled graph with minimal information loss.
[0083] p = |(I (k) -(D (k) ) -1 A (k) )H (k) |1
[0084] Subsequently, we choose to retain nodes that cannot be well represented by their neighbors, that is, nodes with relatively high node information scores will be retained in the construction of the pooling graph because they can provide more information.
[0085] After deleting some nodes, the edges and weights in the graph model are reconstructed using a structure learning mechanism. The structure learning mechanism learns the graph structure through sparse attention, calculating v as follows: q With v p The weight S between nodes i (p,q):
[0086] S pq =sparsemax(a T [H (k) (p,:)||H (k) (q,:)] T +λA pq )
[0087] Where 'a' is the attention weight vector, λ is the tradeoff parameter, and sparsemax(x) = [x - τ(x)] + The function is used to generate sparse graph structures. Where [x] + =max(0,x), where τ(·) is the threshold function. In subsequent convolution operations, S is used... pq The adjacency matrix A between the original nodes is used for calculation.
[0088] Feature fusion and graph representation learning: The multi-scale graph representations after multiple pooling and convolution operations are fused to form a comprehensive graph-level feature representation. This step ensures that graph features are comprehensively considered at different levels.
[0089]
[0090] Finally, the features from different levels are fused to obtain:
[0091] z = r 1 +r 2 +…+r K
[0092] Classification: Perform the above calculations on all graphs in G to obtain... The final graph-level feature representation is fed into an MLP and softmax layer for classification. The final feature vector Z is then processed by the MLP and classification layer to obtain the class probability distribution:
[0093]
[0094] MLP stands for Multilayer Perceptron. It is the predicted class probability.
[0095] like Figure 1 As shown in (c), this method allows us to fully leverage the advantages of graph neural networks to achieve an efficient end-to-end graph classification solution.
[0096] This paper proposes an end-to-end shape recognition method based on complex networks and graph neural networks. The method first utilizes shape context to describe the local features of the shape, and then constructs a multi-level complex network model to capture the global topological structure of the shape. The topological features of these complex networks are learned through graph convolutional neural networks, and these features are fused through deep neural networks, achieving a robust representation for shape classification tasks. Experimental results show that the proposed method achieves state-of-the-art classification accuracy on multiple general datasets and exhibits good robustness to rotation and scaling. Furthermore, this study demonstrates the robustness of the proposed method in parameter selection, providing flexibility for practical applications.
[0097] Example 2
[0098] Embodiment 2 of the present invention provides an end-to-end shape recognition system based on complex networks, implemented based on the method of Embodiment 1, including:
[0099] The dense discretization module is used to densely discretize the edges of the image to be recognized to obtain the edge description point set. The key point extraction module is used to select a series of key points at equal intervals on the shape contour of the image to be recognized.
[0100] The local feature extraction module is used to extract local features of the edge description point set around the keypoint using shape context;
[0101] The complex network construction module is used to treat each key point as a node, define the edges in the network based on the spatial correlation between key points, and construct multi-level complex networks using a dynamic evolution strategy.
[0102] The graph-level feature representation output module is used to process local features and topological features of complex networks through graph convolutional networks to obtain a robust representation for shape classification tasks.
[0103] The classification and recognition module is used to classify shapes through a fully connected layer network, obtain classification results, and realize end-to-end shape recognition.
[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. An end-to-end shape recognition method based on complex networks, comprising: Step 1) Densely discretize the edges of the image to be identified to obtain a set of edge description points; Step 2) Select a series of key points at equal intervals on the obtained edge description point set; Step 3) Extract local features of the point set by describing the edges around the keypoints using shape context; Step 4) Treat each key point as a node, define the edges in the network according to the spatial correlation between key points, and use a dynamic evolution strategy to construct a multi-layered complex network. Step 5) Process local features and topological features of complex networks through graph convolutional networks to obtain a robust representation of the shape classification task; Step 6) Classify the data using a fully connected layer network to obtain the classification results and achieve end-to-end shape recognition.
2. The end-to-end shape recognition method based on complex networks according to claim 1, characterized in that, Step 1) includes: The edge description point set D, containing N points, is obtained based on the shape and contour of the image to be identified.
3. The end-to-end shape recognition method based on complex networks according to claim 2, characterized in that, Step 2) includes: Select a key point every z points to obtain a point set C containing O key points.
4. The end-to-end shape recognition method based on complex networks according to claim 3, characterized in that, Step 3) includes: For each key point in the point set C, a polar coordinate system is defined with each key point as the center, where the polar radius and polar angle are divided in a uniform manner, and the coordinate system is divided into M×H grids; Count the number of points in the edge description point set D that fall into each cell, and normalize the number of points in each cell; The normalized histogram is expanded, and the location information of the key points is encoded to obtain the local feature F = {f1, f2, ..., f...} corresponding to each key point. N′ } 5. The end-to-end shape recognition method based on complex networks according to claim 4, characterized in that, In step 4), each key point is treated as a node, and the edges in the network are defined based on the spatial correlation between key points, including: For N′ keypoints, we obtain an N′×N′ distance matrix B, where B(i,j) represents the distance between keypoints p. i ′ and p j The Euclidean distance between them; According to the set threshold t, when B(i, j) < t, there is an edge between the key points p' i and p' j which is represented by the adjacency matrix A: Thus, the undirected graph model G of complex networks is obtained. T = (V,E,w,A,X); where V represents the set of nodes, E represents the set of edges, w is the weight, and X is the feature matrix.
6. The end-to-end shape recognition method based on complex networks according to claim 5, characterized in that, Step 4) employs a dynamic evolution strategy to construct a multi-layered complex network; including: By selecting L gradually increasing distance thresholds, T = {t1, t2, ..., t...} L }, generating a series of network models 7. The end-to-end shape recognition method based on complex networks according to claim 6, characterized in that, Step 5) includes: A graph convolutional neural network is used to extract node features from the network model G, and the feature vector of each node is updated iteratively. Hierarchical graph pooling operations are used to adaptively select a subset of nodes to form the induced subgraph of the next layer; Reconstruct the edges and weights in the graph model using mechanism learning; The multi-scale graph representations after multiple pooling and convolution operations are fused to form a comprehensive graph-level feature representation.
8. The end-to-end shape recognition method based on complex networks according to claim 7, characterized in that, Step 6) includes: The graph-level feature representation is classified through a multilayer perceptron and a softmax layer to obtain the probability distribution of the categories, thereby achieving end-to-end shape recognition.
9. An end-to-end shape recognition system based on complex networks, characterized in that, include: The dense discretization module is used to densely discretize the edges of the image to be recognized to obtain the edge description point set. The key point extraction module is used to select a series of key points at equal intervals on the shape contour of the image to be recognized. The local feature extraction module is used to extract local features of the edge description point set around the keypoint using the shape context; The complex network construction module is used to treat each key point as a node, define the edges in the network based on the spatial correlation between key points, and construct multi-level complex networks using a dynamic evolution strategy. The graph-level feature representation output module is used to process local features and topological features of complex networks through graph convolutional networks to obtain a robust representation for shape classification tasks; and The classification and recognition module is used to classify shapes through a fully connected layer network, obtain classification results, and realize end-to-end shape recognition.